big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

YARN-2682. Updated WindowsSecureContainerExecutor to not use DefaultContainerExecutor#getFirstApplicationDir and use getWorkingDir() instead. Contributed by Zhihai Xu

Browse Source
This commit is contained in:
Jian He 2014-10-16 18:14:34 -07:00
parent b0d6ac92fe
commit 0fd0ebae64
3 changed files with 7 additions and 9 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
hadoop-yarn-project/CHANGES.txt
View File

@ -360,6 +360,10 @@ Release 2.6.0 - UNRELEASED
YARN-2621. Simplify the output when the user doesn't have the access for YARN-2621. Simplify the output when the user doesn't have the access for
getDomain(s). (Zhijie Shen via jianhe) getDomain(s). (Zhijie Shen via jianhe)
YARN-2682. Updated WindowsSecureContainerExecutor to not use
DefaultContainerExecutor#getFirstApplicationDir and use getWorkingDir()
instead. (Zhihai Xu via jianhe)
OPTIMIZATIONS OPTIMIZATIONS
BUG FIXES BUG FIXES

7
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DefaultContainerExecutor.java
View File

@ -462,11 +462,6 @@ public void deleteAsUser(String user, Path subDir, Path... baseDirs)
* $logdir/$user/$appId */ * $logdir/$user/$appId */
static final short LOGDIR_PERM = (short)0710; static final short LOGDIR_PERM = (short)0710;
protected Path getFirstApplicationDir(List<String> localDirs, String user,
String appId) {
return getApplicationDir(new Path(localDirs.get(0)), user, appId);
}
private long getDiskFreeSpace(Path base) throws IOException { private long getDiskFreeSpace(Path base) throws IOException {
return lfs.getFsStatus(base).getRemaining(); return lfs.getFsStatus(base).getRemaining();
} }
@ -489,7 +484,7 @@ private Path getFileCacheDir(Path base, String user) {
ContainerLocalizer.FILECACHE); ContainerLocalizer.FILECACHE);
} }
private Path getWorkingDir(List<String> localDirs, String user, protected Path getWorkingDir(List<String> localDirs, String user,
String appId) throws IOException { String appId) throws IOException {
Path appStorageDir = null; Path appStorageDir = null;
long totalAvailable = 0L; long totalAvailable = 0L;

5
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/WindowsSecureContainerExecutor.java
View File

@ -115,9 +115,8 @@ public void startLocalizer(Path nmPrivateContainerTokens,
createUserCacheDirs(localDirs, user); createUserCacheDirs(localDirs, user);
createAppDirs(localDirs, user, appId); createAppDirs(localDirs, user, appId);
createAppLogDirs(appId, logDirs, user); createAppLogDirs(appId, logDirs, user);
// TODO: Why pick first app dir. The same in LCE why not random? Path appStorageDir = getWorkingDir(localDirs, user, appId);
Path appStorageDir = getFirstApplicationDir(localDirs, user, appId);
String tokenFn = String.format(ContainerLocalizer.TOKEN_FILE_NAME_FMT, locId); String tokenFn = String.format(ContainerLocalizer.TOKEN_FILE_NAME_FMT, locId);
Path tokenDst = new Path(appStorageDir, tokenFn); Path tokenDst = new Path(appStorageDir, tokenFn);