big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

HADOOP-18982. Fix doc about loading native libraries. (#6281). Contributed by Shuyan Zhang.

Browse Source 
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
This commit is contained in:
zhangshuyan 2023-12-06 21:24:14 +08:00 committed by GitHub
parent 931fa49363
commit 809ae58e71
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 4 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
hadoop-common-project/hadoop-common/src/site/markdown/NativeLibraries.md.vm
View File

@ -126,10 +126,10 @@ Native Shared Libraries
You can load any native shared library using DistributedCache for distributing and symlinking the library files.
This example shows you how to distribute a shared library, mylib.so, and load it from a MapReduce task.
This example shows you how to distribute a shared library in Unix-like systems, mylib.so, and load it from a MapReduce task.
1. First copy the library to the HDFS: `bin/hadoop fs -copyFromLocal mylib.so.1 /libraries/mylib.so.1`
2. The job launching program should contain the following: `DistributedCache.createSymlink(conf);` `DistributedCache.addCacheFile("hdfs://host:port/libraries/mylib.so. 1#mylib.so", conf);`
3. The MapReduce task can contain: `System.loadLibrary("mylib.so");`
1. First copy the library to the HDFS: `bin/hadoop fs -copyFromLocal libmyexample.so.1 /libraries/libmyexample.so.1`
2. The job launching program should contain the following: `DistributedCache.createSymlink(conf);` `DistributedCache.addCacheFile("hdfs://host:port/libraries/libmyexample.so.1#libmyexample.so", conf);`
3. The MapReduce task can contain: `System.loadLibrary("myexample");`
Note: If you downloaded or built the native hadoop library, you don’t need to use DistibutedCache to make the library available to your MapReduce tasks.