From 5db7107b773fc799d2a92e2a388816352ba40204 Mon Sep 17 00:00:00 2001 From: Mehakmeet Singh Date: Tue, 27 Jun 2023 20:22:00 +0530 Subject: [PATCH] HADOOP-18764. fs.azure.buffer.dir to be under Yarn container path on yarn applications (#5737) Changing fs.azure.buffer.dir for azure so things clean up better in long-lived yarn clusters. Contributed by: Mehakmeet Singh --- .../hadoop-common/src/main/resources/core-default.xml | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml b/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml index 34dadf3160..438e2df137 100644 --- a/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml +++ b/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml @@ -2165,9 +2165,11 @@ The switch to turn S3A auditing on or off. fs.azure.buffer.dir - ${hadoop.tmp.dir}/abfs + ${env.LOCAL_DIRS:-${hadoop.tmp.dir}}/abfs Directory path for buffer files needed to upload data blocks - in AbfsOutputStream. + in AbfsOutputStream. + Yarn container path will be used as default value on yarn applications, + otherwise fall back to hadoop.tmp.dir