big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

HADOOP-18764. fs.azure.buffer.dir to be under Yarn container path on yarn applications (#5737)

Browse Source 
Changing fs.azure.buffer.dir for azure so things clean up better in long-lived yarn clusters.

Contributed by: Mehakmeet Singh
This commit is contained in:
Mehakmeet Singh 2023-06-27 20:22:00 +05:30 committed by GitHub
parent 0d057e27c3
commit 5db7107b77
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 4 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
View File

@ -2165,9 +2165,11 @@ The switch to turn S3A auditing on or off.
<property> <property>
<name>fs.azure.buffer.dir</name> <name>fs.azure.buffer.dir</name>
<value>${hadoop.tmp.dir}/abfs</value> <value>${env.LOCAL_DIRS:-${hadoop.tmp.dir}}/abfs</value>
<description>Directory path for buffer files needed to upload data blocks <description>Directory path for buffer files needed to upload data blocks
in AbfsOutputStream.</description> in AbfsOutputStream.
Yarn container path will be used as default value on yarn applications,
otherwise fall back to hadoop.tmp.dir </description>
</property> </property>
<property> <property>