big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Make 3.2.0 aware to other branches

Browse Source
This commit is contained in:
Sunil G 2019-01-21 21:24:51 +05:30
parent abde1e1f58
commit 2e2508b8e3
2 changed files with 397 additions and 18 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

315
hadoop-common-project/hadoop-common/src/site/markdown/release/3.2.0/CHANGELOG.3.2.0.md
View File

@ -18,16 +18,18 @@
-->
# Apache Hadoop Changelog
## Release 3.2.0 - Unreleased (as of 2018-09-02)
## Release 3.2.0 - 2019-01-16
### INCOMPATIBLE CHANGES:
| JIRA | Summary | Priority | Component | Reporter | Contributor |
|:---- |:---- | :--- |:---- |:---- |:---- |
| [YARN-6257](https://issues.apache.org/jira/browse/YARN-6257) | CapacityScheduler REST API produces incorrect JSON - JSON object operationsInfo contains deplicate key | Minor | capacityscheduler | Tao Yang | Tao Yang |
| [YARN-6257](https://issues.apache.org/jira/browse/YARN-6257) | CapacityScheduler REST API produces incorrect JSON - JSON object operationsInfo contains duplicate keys | Minor | capacityscheduler | Tao Yang | Tao Yang |
| [HADOOP-15146](https://issues.apache.org/jira/browse/HADOOP-15146) | Remove DataOutputByteBuffer | Minor | common | BELUGA BEHR | BELUGA BEHR |
| [YARN-8191](https://issues.apache.org/jira/browse/YARN-8191) | Fair scheduler: queue deletion without RM restart | Major | fairscheduler | Gergo Repas | Gergo Repas |
| [HADOOP-15495](https://issues.apache.org/jira/browse/HADOOP-15495) | Upgrade common-lang version to 3.7 in hadoop-common-project and hadoop-tools | Major | . | Takanobu Asanuma | Takanobu Asanuma |
| [HDFS-13806](https://issues.apache.org/jira/browse/HDFS-13806) | EC: No error message for unsetting EC policy of the directory inherits the erasure coding policy from an ancestor directory | Minor | erasure-coding | Souryakanta Dwivedy | Ayush Saxena |
| [HADOOP-14833](https://issues.apache.org/jira/browse/HADOOP-14833) | Remove s3a user:secret authentication | Major | fs/s3, security | Steve Loughran | Steve Loughran |
### IMPORTANT ISSUES:
@ -45,7 +47,14 @@
| [HDFS-13056](https://issues.apache.org/jira/browse/HDFS-13056) | Expose file-level composite CRCs in HDFS which are comparable across different instances/layouts | Major | datanode, distcp, erasure-coding, federation, hdfs | Dennis Huo | Dennis Huo |
| [HDFS-13283](https://issues.apache.org/jira/browse/HDFS-13283) | Percentage based Reserved Space Calculation for DataNode | Major | datanode, hdfs | Lukas Majercak | Lukas Majercak |
| [HDFS-13448](https://issues.apache.org/jira/browse/HDFS-13448) | HDFS Block Placement - Ignore Locality for First Block Replica | Minor | block placement, hdfs-client | BELUGA BEHR | BELUGA BEHR |
| [YARN-7812](https://issues.apache.org/jira/browse/YARN-7812) | Improvements to Rich Placement Constraints in YARN | Major | . | Arun Suresh | |
| [HADOOP-15707](https://issues.apache.org/jira/browse/HADOOP-15707) | Add IsActiveServlet to be used for Load Balancers | Major | common | Lukas Majercak | Lukas Majercak |
| [YARN-3409](https://issues.apache.org/jira/browse/YARN-3409) | Support Node Attribute functionality | Major | api, client, RM | Wangda Tan | Naganarasimha G R |
| [HDFS-10285](https://issues.apache.org/jira/browse/HDFS-10285) | Storage Policy Satisfier in HDFS | Major | datanode, namenode | Uma Maheswara Rao G | Uma Maheswara Rao G |
| [HADOOP-15407](https://issues.apache.org/jira/browse/HADOOP-15407) | Support Windows Azure Storage - Blob file system in Hadoop | Blocker | fs/azure | Esfandiar Manii | Da Zhou |
| [YARN-8734](https://issues.apache.org/jira/browse/YARN-8734) | Readiness check for remote service belonging to the same user | Major | yarn-native-services | Eric Yang | Eric Yang |
| [HDFS-8707](https://issues.apache.org/jira/browse/HDFS-8707) | Implement an async pure c++ HDFS client | Major | hdfs-client | Owen O'Malley | James Clampffer |
| [HADOOP-14365](https://issues.apache.org/jira/browse/HADOOP-14365) | Stabilise FileSystem builder-based create API | Major | fs | Steve Loughran | Lei (Eddy) Xu |
| [HADOOP-15996](https://issues.apache.org/jira/browse/HADOOP-15996) | Plugin interface to support more complex usernames in Hadoop | Major | security | Eric Yang | Bolke de Bruin |
### IMPROVEMENTS:
@ -56,7 +65,6 @@
| [HADOOP-15332](https://issues.apache.org/jira/browse/HADOOP-15332) | Fix typos in hadoop-aws markdown docs | Minor | . | Gabor Bota | Gabor Bota |
| [HADOOP-15330](https://issues.apache.org/jira/browse/HADOOP-15330) | Remove jdk1.7 profile from hadoop-annotations module | Minor | . | Akira Ajisaka | fang zhenyi |
| [HADOOP-15295](https://issues.apache.org/jira/browse/HADOOP-15295) | Remove redundant logging related to tags from Configuration | Major | . | Ajay Kumar | Ajay Kumar |
| [HADOOP-15339](https://issues.apache.org/jira/browse/HADOOP-15339) | Support additional key/value propereties in JMX bean registration | Major | common | Elek, Marton | Elek, Marton |
| [YARN-8077](https://issues.apache.org/jira/browse/YARN-8077) | The vmemLimit parameter in ContainersMonitorImpl#isProcessTreeOverLimit is confusing | Trivial | nodemanager | Sen Zhao | Sen Zhao |
| [HDFS-13357](https://issues.apache.org/jira/browse/HDFS-13357) | Improve AclException message "Invalid ACL: only directories may have a default ACL." | Minor | . | Wei-Chiu Chuang | Gabor Bota |
| [HADOOP-15342](https://issues.apache.org/jira/browse/HADOOP-15342) | Update ADLS connector to use the current SDK version (2.2.7) | Major | fs/adl | Atul Sikaria | Atul Sikaria |
@ -75,6 +83,7 @@
| [YARN-8140](https://issues.apache.org/jira/browse/YARN-8140) | Improve log message when launch cmd is ran for stopped yarn service | Major | yarn-native-services | Yesha Vora | Eric Yang |
| [MAPREDUCE-7086](https://issues.apache.org/jira/browse/MAPREDUCE-7086) | Add config to allow FileInputFormat to ignore directories when recursive=false | Major | . | Sergey Shelukhin | Sergey Shelukhin |
| [HADOOP-15377](https://issues.apache.org/jira/browse/HADOOP-15377) | Improve debug messages in MetricsConfig.java | Minor | common | BELUGA BEHR | BELUGA BEHR |
| [YARN-8226](https://issues.apache.org/jira/browse/YARN-8226) | Improve anti-affinity section description in YARN Service API doc | Major | docs, documentation | Charan Hebri | Gour Saha |
| [HADOOP-15382](https://issues.apache.org/jira/browse/HADOOP-15382) | Log kinit output in credential renewal thread | Minor | security | Wei-Chiu Chuang | Gabor Bota |
| [YARN-8163](https://issues.apache.org/jira/browse/YARN-8163) | Add support for Node Labels in opportunistic scheduling. | Major | . | Abhishek Modi | Abhishek Modi |
| [HDFS-5926](https://issues.apache.org/jira/browse/HDFS-5926) | Documentation should clarify dfs.datanode.du.reserved impact from reserved disk capacity | Minor | documentation | Alexander Fahlke | Gabor Bota |
@ -82,6 +91,7 @@
| [HDFS-12981](https://issues.apache.org/jira/browse/HDFS-12981) | renameSnapshot a Non-Existent snapshot to itself should throw error | Minor | hdfs | Sailesh Patel | Kitti Nanasi |
| [YARN-8239](https://issues.apache.org/jira/browse/YARN-8239) | [UI2] Clicking on Node Manager UI under AM container info / App Attempt page goes to old RM UI | Major | yarn-ui-v2 | Sumana Sathish | Sunil Govindan |
| [YARN-8260](https://issues.apache.org/jira/browse/YARN-8260) | [UI2] Per-application tracking URL is no longer available in YARN UI2 | Major | yarn-ui-v2 | Sunil Govindan | Sunil Govindan |
| [HADOOP-15356](https://issues.apache.org/jira/browse/HADOOP-15356) | Make HTTP timeout configurable in ADLS Connector | Major | fs/adl | Atul Sikaria | Atul Sikaria |
| [YARN-8201](https://issues.apache.org/jira/browse/YARN-8201) | Skip stacktrace of few exception from ClientRMService | Minor | . | Bibin A Chundatt | Bilwa S T |
| [HADOOP-15354](https://issues.apache.org/jira/browse/HADOOP-15354) | hadoop-aliyun & hadoop-azure modules to mark hadoop-common as provided | Major | build, fs/azure, fs/oss | Steve Loughran | Steve Loughran |
| [YARN-3610](https://issues.apache.org/jira/browse/YARN-3610) | FairScheduler: Add steady-fair-shares to the REST API documentation | Major | documentation, fairscheduler | Karthik Kambatla | Ray Chiang |
@ -142,7 +152,7 @@
| [HDFS-13714](https://issues.apache.org/jira/browse/HDFS-13714) | Fix TestNameNodePrunesMissingStorages test failures on Windows | Major | hdfs, namenode, test | Lukas Majercak | Lukas Majercak |
| [HDFS-13712](https://issues.apache.org/jira/browse/HDFS-13712) | BlockReaderRemote.read() logging improvement | Minor | hdfs-client | Gergo Repas | Gergo Repas |
| [YARN-8302](https://issues.apache.org/jira/browse/YARN-8302) | ATS v2 should handle HBase connection issue properly | Major | ATSv2 | Yesha Vora | Billie Rinaldi |
| [HDFS-13719](https://issues.apache.org/jira/browse/HDFS-13719) | Docs around dfs.image.transfer.timeout are misleading | Major | . | Kitti Nanasi | Kitti Nanasi |
| [HDFS-13719](https://issues.apache.org/jira/browse/HDFS-13719) | Docs around dfs.image.transfer.timeout are misleading | Major | documentation | Kitti Nanasi | Kitti Nanasi |
| [HADOOP-15591](https://issues.apache.org/jira/browse/HADOOP-15591) | KMSClientProvider should log KMS DT acquisition at INFO level | Minor | kms | Kitti Nanasi | Kitti Nanasi |
| [HADOOP-15581](https://issues.apache.org/jira/browse/HADOOP-15581) | Set default jetty log level to INFO in KMS | Major | . | Kitti Nanasi | Kitti Nanasi |
| [HADOOP-15568](https://issues.apache.org/jira/browse/HADOOP-15568) | fix some typos in the .sh comments | Trivial | bin | Steve Loughran | Steve Loughran |
@ -155,6 +165,7 @@
| [YARN-8501](https://issues.apache.org/jira/browse/YARN-8501) | Reduce complexity of RMWebServices' getApps method | Major | restapi | Szilard Nemeth | Szilard Nemeth |
| [YARN-7300](https://issues.apache.org/jira/browse/YARN-7300) | DiskValidator is not used in LocalDirAllocator | Major | . | Haibo Chen | Szilard Nemeth |
| [HADOOP-15596](https://issues.apache.org/jira/browse/HADOOP-15596) | Stack trace should not be printed out when running hadoop key commands | Minor | common | Kitti Nanasi | Kitti Nanasi |
| [HADOOP-15586](https://issues.apache.org/jira/browse/HADOOP-15586) | Fix wrong log statement in AbstractService | Minor | util | Szilard Nemeth | Szilard Nemeth |
| [YARN-7133](https://issues.apache.org/jira/browse/YARN-7133) | Clean up lock-try order in fair scheduler | Major | fairscheduler | Daniel Templeton | Szilard Nemeth |
| [HDFS-13761](https://issues.apache.org/jira/browse/HDFS-13761) | Add toString Method to AclFeature Class | Minor | . | Shweta | Shweta |
| [HADOOP-15609](https://issues.apache.org/jira/browse/HADOOP-15609) | Retry KMS calls when SSLHandshakeException occurs | Major | common, kms | Kitti Nanasi | Kitti Nanasi |
@ -169,7 +180,6 @@
| [HADOOP-15476](https://issues.apache.org/jira/browse/HADOOP-15476) | fix logging for split-dns multihome | Major | . | Ajay Kumar | Ajay Kumar |
| [YARN-7948](https://issues.apache.org/jira/browse/YARN-7948) | Enable fair scheduler to refresh maximum allocation for multiple resource types | Major | fairscheduler | Yufei Gu | Szilard Nemeth |
| [HDFS-13796](https://issues.apache.org/jira/browse/HDFS-13796) | Allow verbosity of InMemoryLevelDBAliasMapServer to be configurable | Trivial | . | Virajith Jalaparti | Virajith Jalaparti |
| [YARN-8626](https://issues.apache.org/jira/browse/YARN-8626) | Create HomePolicyManager that sends all the requests to the home subcluster | Minor | . | Giovanni Matteo Fumarola | Íñigo Goiri |
| [HDFS-13728](https://issues.apache.org/jira/browse/HDFS-13728) | Disk Balancer should not fail if volume usage is greater than capacity | Minor | diskbalancer | Stephen O'Donnell | Stephen O'Donnell |
| [HDFS-13447](https://issues.apache.org/jira/browse/HDFS-13447) | Fix Typos - Node Not Chosen | Trivial | namenode | BELUGA BEHR | BELUGA BEHR |
| [YARN-8601](https://issues.apache.org/jira/browse/YARN-8601) | Print ExecutionType in Container report CLI | Minor | . | Bilwa S T | Bilwa S T |
@ -182,7 +192,6 @@
| [HDFS-13813](https://issues.apache.org/jira/browse/HDFS-13813) | Exit NameNode if dangling child inode is detected when saving FsImage | Major | hdfs, namenode | Siyao Meng | Siyao Meng |
| [HADOOP-14212](https://issues.apache.org/jira/browse/HADOOP-14212) | Expose SecurityEnabled boolean field in JMX for other services besides NameNode | Minor | . | Ray Burgemeestre | Adam Antal |
| [HDFS-13217](https://issues.apache.org/jira/browse/HDFS-13217) | Audit log all EC policy names during addErasureCodingPolicies | Major | erasure-coding | liaoyuxiangqin | liaoyuxiangqin |
| [HDFS-13732](https://issues.apache.org/jira/browse/HDFS-13732) | ECAdmin should print the policy name when an EC policy is set | Trivial | erasure-coding, tools | Soumyapn | Zsolt Venczel |
| [HDFS-13829](https://issues.apache.org/jira/browse/HDFS-13829) | Remove redundant condition judgement in DirectoryScanner#scan | Minor | datanode | liaoyuxiangqin | liaoyuxiangqin |
| [HDFS-13822](https://issues.apache.org/jira/browse/HDFS-13822) | speedup libhdfs++ build (enable parallel build) | Minor | . | Pradeep Ambati | Allen Wittenauer |
| [HADOOP-9214](https://issues.apache.org/jira/browse/HADOOP-9214) | Create a new touch command to allow modifying atime and mtime | Minor | tools | Brian Burton | Hrishikesh Gadre |
@ -192,6 +201,36 @@
| [HDFS-13861](https://issues.apache.org/jira/browse/HDFS-13861) | RBF: Illegal Router Admin command leads to printing usage for all commands | Major | . | Ayush Saxena | Ayush Saxena |
| [HDFS-13831](https://issues.apache.org/jira/browse/HDFS-13831) | Make block increment deletion number configurable | Major | . | Yiqun Lin | Ryan Wu |
| [YARN-8051](https://issues.apache.org/jira/browse/YARN-8051) | TestRMEmbeddedElector#testCallbackSynchronization is flakey | Major | test | Robert Kanter | Robert Kanter |
| [HDFS-13885](https://issues.apache.org/jira/browse/HDFS-13885) | Add debug logs in dfsclient around decrypting EDEK | Major | hdfs-client | Kitti Nanasi | Kitti Nanasi |
| [HDFS-13857](https://issues.apache.org/jira/browse/HDFS-13857) | RBF: Choose to enable the default nameservice to read/write files | Major | federation, hdfs | yanghuafeng | yanghuafeng |
| [HDFS-13812](https://issues.apache.org/jira/browse/HDFS-13812) | Fix the inconsistent default refresh interval on Caching documentation | Trivial | documentation | BELUGA BEHR | Hrishikesh Gadre |
| [YARN-8638](https://issues.apache.org/jira/browse/YARN-8638) | Allow linux container runtimes to be pluggable | Minor | nodemanager | Craig Condit | Craig Condit |
| [HADOOP-15657](https://issues.apache.org/jira/browse/HADOOP-15657) | Registering MutableQuantiles via Metric annotation | Major | metrics | Sushil Ks | Sushil Ks |
| [HDFS-13820](https://issues.apache.org/jira/browse/HDFS-13820) | Add an ability to disable CacheReplicationMonitor | Minor | caching | BELUGA BEHR | Hrishikesh Gadre |
| [HDFS-13884](https://issues.apache.org/jira/browse/HDFS-13884) | Improve the description of the setting dfs.image.compress | Minor | documentation | Yiqun Lin | Ryan Wu |
| [HADOOP-14671](https://issues.apache.org/jira/browse/HADOOP-14671) | Upgrade to Apache Yetus 0.8.0 | Major | build, documentation, test | Allen Wittenauer | Allen Wittenauer |
| [HDFS-12615](https://issues.apache.org/jira/browse/HDFS-12615) | Router-based HDFS federation phase 2 | Major | . | Íñigo Goiri | Íñigo Goiri |
| [HDFS-13902](https://issues.apache.org/jira/browse/HDFS-13902) | Add JMX, conf and stacks menus to the datanode page | Minor | datanode | fengchuang | fengchuang |
| [HADOOP-15584](https://issues.apache.org/jira/browse/HADOOP-15584) | move httpcomponents version in pom.xml | Minor | build | Brandon Scheller | Brandon Scheller |
| [HDFS-13744](https://issues.apache.org/jira/browse/HDFS-13744) | OIV tool should better handle control characters present in file or directory names | Critical | hdfs, tools | Zsolt Venczel | Zsolt Venczel |
| [YARN-8680](https://issues.apache.org/jira/browse/YARN-8680) | YARN NM: Implement Iterable Abstraction for LocalResourceTracker state | Critical | yarn | Pradeep Ambati | Pradeep Ambati |
| [YARN-8045](https://issues.apache.org/jira/browse/YARN-8045) | Reduce log output from container status calls | Major | . | Shane Kumpf | Craig Condit |
| [MAPREDUCE-7140](https://issues.apache.org/jira/browse/MAPREDUCE-7140) | Refactoring TaskAttemptInfo to separate Map and Reduce tasks | Minor | . | Oleksandr Shevchenko | Oleksandr Shevchenko |
| [HADOOP-15226](https://issues.apache.org/jira/browse/HADOOP-15226) | Über-JIRA: S3Guard Phase III: Hadoop 3.2 features | Major | fs/s3 | Steve Loughran | |
| [HADOOP-15742](https://issues.apache.org/jira/browse/HADOOP-15742) | Log if ipc backoff is enabled in CallQueueManager | Minor | . | Yiqun Lin | Ryan Wu |
| [HADOOP-15726](https://issues.apache.org/jira/browse/HADOOP-15726) | Create utility to limit frequency of log statements | Major | common, util | Erik Krogen | Erik Krogen |
| [HDFS-13927](https://issues.apache.org/jira/browse/HDFS-13927) | Improve TestDataNodeMultipleRegistrations#testDNWithInvalidStorageWithHA wait | Minor | . | Ayush Saxena | Ayush Saxena |
| [YARN-8616](https://issues.apache.org/jira/browse/YARN-8616) | systemClock should be used in RMAppImpl instead of System.currentTimeMills() to be consistent | Major | . | Szilard Nemeth | Szilard Nemeth |
| [YARN-7974](https://issues.apache.org/jira/browse/YARN-7974) | Allow updating application tracking url after registration | Major | . | Jonathan Hung | Jonathan Hung |
| [HDFS-8196](https://issues.apache.org/jira/browse/HDFS-8196) | Post enabled Erasure Coding Policies on NameNode UI | Major | namenode | Kai Sasaki | Kitti Nanasi |
| [YARN-8758](https://issues.apache.org/jira/browse/YARN-8758) | Support getting PreemptionMessage when using AMRMClientAsync | Major | yarn | Krishna Kishore | Zian Chen |
| [HDFS-13956](https://issues.apache.org/jira/browse/HDFS-13956) | iNotify should include information to identify a file as either replicated or erasure coded | Minor | . | Hrishikesh Gadre | Hrishikesh Gadre |
| [HDFS-13156](https://issues.apache.org/jira/browse/HDFS-13156) | HDFS Block Placement Policy - Client Local Rack | Minor | documentation | BELUGA BEHR | Ayush Saxena |
| [YARN-8468](https://issues.apache.org/jira/browse/YARN-8468) | Enable the use of queue based maximum container allocation limit and implement it in FairScheduler | Critical | fairscheduler, scheduler | Antal Bálint Steinbach | Antal Bálint Steinbach |
| [HADOOP-15339](https://issues.apache.org/jira/browse/HADOOP-15339) | Support additional key/value propereties in JMX bean registration | Major | common | Elek, Marton | Elek, Marton |
| [HDFS-14053](https://issues.apache.org/jira/browse/HDFS-14053) | Provide ability for NN to re-replicate based on topology changes. | Major | . | ellen johansen | Hrishikesh Gadre |
| [HADOOP-15957](https://issues.apache.org/jira/browse/HADOOP-15957) | WASB: Add asterisk wildcard support for PageBlobDirSet | Major | fs/azure | Da Zhou | Da Zhou |
| [HADOOP-15959](https://issues.apache.org/jira/browse/HADOOP-15959) | revert HADOOP-12751 | Minor | security | Steve Loughran | Steve Loughran |
### BUG FIXES:
@ -213,6 +252,7 @@
| [HADOOP-15352](https://issues.apache.org/jira/browse/HADOOP-15352) | Fix default local maven repository path in create-release script | Minor | scripts | Elek, Marton | Elek, Marton |
| [HADOOP-15317](https://issues.apache.org/jira/browse/HADOOP-15317) | Improve NetworkTopology chooseRandom's loop | Major | . | Xiao Chen | Xiao Chen |
| [HADOOP-15355](https://issues.apache.org/jira/browse/HADOOP-15355) | TestCommonConfigurationFields is broken by HADOOP-15312 | Major | test | Konstantin Shvachko | LiXin Ge |
| [YARN-8035](https://issues.apache.org/jira/browse/YARN-8035) | Uncaught exception in ContainersMonitorImpl during relaunch due to the process ID changing | Major | . | Shane Kumpf | Shane Kumpf |
| [YARN-7764](https://issues.apache.org/jira/browse/YARN-7764) | Findbugs warning: Resource#getResources may expose internal representation | Major | api | Weiwei Yang | Weiwei Yang |
| [YARN-8115](https://issues.apache.org/jira/browse/YARN-8115) | [UI2] URL data like nodeHTTPAddress must be encoded in UI before using to access NM | Major | yarn-ui-v2 | Sunil Govindan | Sreenath Somarajapuram |
| [HADOOP-14855](https://issues.apache.org/jira/browse/HADOOP-14855) | Hadoop scripts may errantly believe a daemon is still running, preventing it from starting | Major | scripts | Aaron T. Myers | Robert Kanter |
@ -223,6 +263,7 @@
| [HDFS-13292](https://issues.apache.org/jira/browse/HDFS-13292) | Crypto command should give proper exception when trying to set key on existing EZ directory | Major | hdfs, kms | Harshakiran Reddy | Ranith Sardar |
| [HADOOP-15366](https://issues.apache.org/jira/browse/HADOOP-15366) | Add a helper shutdown routine in HadoopExecutor to ensure clean shutdown | Minor | . | Shashikant Banerjee | Shashikant Banerjee |
| [YARN-7905](https://issues.apache.org/jira/browse/YARN-7905) | Parent directory permission incorrect during public localization | Critical | . | Bibin A Chundatt | Bilwa S T |
| [HDFS-13376](https://issues.apache.org/jira/browse/HDFS-13376) | Specify minimum GCC version to avoid TLS support error in Build of hadoop-hdfs-native-client | Major | build, documentation, native | LiXin Ge | LiXin Ge |
| [HADOOP-15328](https://issues.apache.org/jira/browse/HADOOP-15328) | Fix the typo in HttpAuthentication.md | Minor | common | fang zhenyi | fang zhenyi |
| [HADOOP-15374](https://issues.apache.org/jira/browse/HADOOP-15374) | Add links of the new features of 3.1.0 to the top page | Major | documentation | Takanobu Asanuma | Takanobu Asanuma |
| [YARN-7804](https://issues.apache.org/jira/browse/YARN-7804) | Refresh action on Grid view page should not be redirected to graph view | Major | yarn-ui-v2 | Yesha Vora | Gergely Novák |
@ -390,6 +431,7 @@
| [HDFS-13339](https://issues.apache.org/jira/browse/HDFS-13339) | Volume reference can't be released and may lead to deadlock when DataXceiver does a check volume | Critical | datanode | liaoyuxiangqin | Zsolt Venczel |
| [YARN-8390](https://issues.apache.org/jira/browse/YARN-8390) | Fix API incompatible changes in FairScheduler's AllocationFileLoaderService | Major | fairscheduler | Gergo Repas | Gergo Repas |
| [YARN-8382](https://issues.apache.org/jira/browse/YARN-8382) | cgroup file leak in NM | Major | nodemanager | Hu Ziqian | Hu Ziqian |
| [YARN-8388](https://issues.apache.org/jira/browse/YARN-8388) | TestCGroupElasticMemoryController.testNormalExit() hangs on Linux | Major | nodemanager | Haibo Chen | Miklos Szegedi |
| [YARN-8365](https://issues.apache.org/jira/browse/YARN-8365) | Revisit the record type used by Registry DNS for upstream resolution | Major | yarn-native-services | Shane Kumpf | Shane Kumpf |
| [HDFS-13545](https://issues.apache.org/jira/browse/HDFS-13545) | "guarded" is misspelled as "gaurded" in FSPermissionChecker.java | Trivial | documentation | Jianchao Jia | Jianchao Jia |
| [YARN-8396](https://issues.apache.org/jira/browse/YARN-8396) | Click on an individual container continuously spins and doesn't load the page | Blocker | . | Charan Hebri | Sunil Govindan |
@ -433,6 +475,7 @@
| [YARN-8457](https://issues.apache.org/jira/browse/YARN-8457) | Compilation is broken with -Pyarn-ui | Major | webapp | Sunil Govindan | Sunil Govindan |
| [HADOOP-15550](https://issues.apache.org/jira/browse/HADOOP-15550) | Avoid static initialization of ObjectMappers | Minor | performance | Todd Lipcon | Todd Lipcon |
| [YARN-8438](https://issues.apache.org/jira/browse/YARN-8438) | TestContainer.testKillOnNew flaky on trunk | Major | nodemanager | Szilard Nemeth | Szilard Nemeth |
| [YARN-8108](https://issues.apache.org/jira/browse/YARN-8108) | RM metrics rest API throws GSSException in kerberized environment | Blocker | . | Kshitij Badani | Sunil Govindan |
| [YARN-8464](https://issues.apache.org/jira/browse/YARN-8464) | Async scheduling thread could be interrupted when there are no NodeManagers in cluster | Blocker | capacity scheduler | Charan Hebri | Sunil Govindan |
| [YARN-8423](https://issues.apache.org/jira/browse/YARN-8423) | GPU does not get released even though the application gets killed. | Critical | yarn | Sumana Sathish | Sunil Govindan |
| [YARN-8401](https://issues.apache.org/jira/browse/YARN-8401) | [UI2] new ui is not accessible with out internet connection | Blocker | . | Bibin A Chundatt | Bibin A Chundatt |
@ -478,7 +521,7 @@
| [HADOOP-15614](https://issues.apache.org/jira/browse/HADOOP-15614) | TestGroupsCaching.testExceptionOnBackgroundRefreshHandled reliably fails | Major | . | Kihwal Lee | Weiwei Yang |
| [YARN-8436](https://issues.apache.org/jira/browse/YARN-8436) | FSParentQueue: Comparison method violates its general contract | Minor | fairscheduler | Wilfred Spiegelenburg | Wilfred Spiegelenburg |
| [MAPREDUCE-7118](https://issues.apache.org/jira/browse/MAPREDUCE-7118) | Distributed cache conflicts breaks backwards compatability | Blocker | mrv2 | Jason Lowe | Jason Lowe |
| [YARN-8528](https://issues.apache.org/jira/browse/YARN-8528) | Final states in ContainerAllocation might be modified externally causing unexpected allocation results | Major | capacity scheduler | Xintong Song | Xintong Song |
| [YARN-8528](https://issues.apache.org/jira/browse/YARN-8528) | Final states in ContainerAllocation might be modified externally causing unexpected allocation results | Major | capacity scheduler | Tony Xintong Song | Tony Xintong Song |
| [YARN-6964](https://issues.apache.org/jira/browse/YARN-6964) | Fair scheduler misuses Resources operations | Major | fairscheduler | Daniel Templeton | Szilard Nemeth |
| [YARN-8360](https://issues.apache.org/jira/browse/YARN-8360) | Yarn service conflict between restart policy and NM configuration | Critical | yarn | Chandni Singh | Suma Shivaprasad |
| [YARN-8380](https://issues.apache.org/jira/browse/YARN-8380) | Support bind propagation options for mounts in docker runtime | Major | . | Billie Rinaldi | Billie Rinaldi |
@ -570,6 +613,127 @@
| [HADOOP-15698](https://issues.apache.org/jira/browse/HADOOP-15698) | KMS log4j is not initialized properly at startup | Major | kms | Kitti Nanasi | Kitti Nanasi |
| [HADOOP-15706](https://issues.apache.org/jira/browse/HADOOP-15706) | Typo in compatibility doc: SHOUD -\> SHOULD | Trivial | . | Daniel Templeton | Laszlo Kollar |
| [HDFS-13027](https://issues.apache.org/jira/browse/HDFS-13027) | Handle possible NPEs due to deleted blocks in race condition | Major | namenode | Vinayakumar B | Vinayakumar B |
| [YARN-8535](https://issues.apache.org/jira/browse/YARN-8535) | Fix DistributedShell unit tests | Major | distributed-shell, timelineservice | Eric Yang | Abhishek Modi |
| [HDFS-13867](https://issues.apache.org/jira/browse/HDFS-13867) | RBF: Add validation for max arguments for Router admin ls, clrQuota, setQuota, rm and nameservice commands | Major | . | Ayush Saxena | Ayush Saxena |
| [HDFS-13774](https://issues.apache.org/jira/browse/HDFS-13774) | EC: "hdfs ec -getPolicy" is not retrieving policy details when the special REPLICATION policy set on the directory | Minor | erasure-coding | Souryakanta Dwivedy | Ayush Saxena |
| [HADOOP-10219](https://issues.apache.org/jira/browse/HADOOP-10219) | ipc.Client.setupIOstreams() needs to check for ClientCache.stopClient requested shutdowns | Major | ipc | Steve Loughran | Kihwal Lee |
| [HDFS-13815](https://issues.apache.org/jira/browse/HDFS-13815) | RBF: Add check to order command | Major | federation | Soumyapn | Ranith Sardar |
| [HADOOP-15696](https://issues.apache.org/jira/browse/HADOOP-15696) | KMS performance regression due to too many open file descriptors after Jetty migration | Blocker | kms | Wei-Chiu Chuang | Wei-Chiu Chuang |
| [MAPREDUCE-7131](https://issues.apache.org/jira/browse/MAPREDUCE-7131) | Job History Server has race condition where it moves files from intermediate to finished but thinks file is in intermediate | Major | . | Anthony Hsu | Anthony Hsu |
| [HDFS-13836](https://issues.apache.org/jira/browse/HDFS-13836) | RBF: Handle mount table znode with null value | Major | federation, hdfs | yanghuafeng | yanghuafeng |
| [YARN-8751](https://issues.apache.org/jira/browse/YARN-8751) | Container-executor permission check errors cause the NM to be marked unhealthy | Critical | . | Shane Kumpf | Craig Condit |
| [HDFS-13862](https://issues.apache.org/jira/browse/HDFS-13862) | RBF: Router logs are not capturing few of the dfsrouteradmin commands | Major | . | Soumyapn | Ayush Saxena |
| [HDFS-12716](https://issues.apache.org/jira/browse/HDFS-12716) | 'dfs.datanode.failed.volumes.tolerated' to support minimum number of volumes to be available | Major | datanode | usharani | Ranith Sardar |
| [HDFS-13895](https://issues.apache.org/jira/browse/HDFS-13895) | EC: Fix Intermittent Failure in TestDFSStripedOutputStreamWithFailureWithRandomECPolicy | Major | erasure-coding | Ayush Saxena | Ayush Saxena |
| [YARN-8709](https://issues.apache.org/jira/browse/YARN-8709) | CS preemption monitor always fails since one under-served queue was deleted | Major | capacityscheduler, scheduler preemption | Tao Yang | Tao Yang |
| [HDFS-13051](https://issues.apache.org/jira/browse/HDFS-13051) | Fix dead lock during async editlog rolling if edit queue is full | Major | namenode | zhangwei | Daryn Sharp |
| [MAPREDUCE-7135](https://issues.apache.org/jira/browse/MAPREDUCE-7135) | TestTaskAttemptContainerRequest should reset UserGroupInformation | Minor | test | Oleksandr Shevchenko | Oleksandr Shevchenko |
| [MAPREDUCE-7136](https://issues.apache.org/jira/browse/MAPREDUCE-7136) | TestMRAppMetrics should shutdown DefaultMetricsSystem after completion | Minor | test | Oleksandr Shevchenko | Oleksandr Shevchenko |
| [HDFS-13846](https://issues.apache.org/jira/browse/HDFS-13846) | Safe blocks counter is not decremented correctly if the block is striped | Major | hdfs | Kitti Nanasi | Kitti Nanasi |
| [YARN-8768](https://issues.apache.org/jira/browse/YARN-8768) | Javadoc error in node attributes | Major | . | Sunil Govindan | Sunil Govindan |
| [HADOOP-12760](https://issues.apache.org/jira/browse/HADOOP-12760) | sun.misc.Cleaner has moved to a new location in OpenJDK 9 | Major | . | Chris Hegarty | Akira Ajisaka |
| [YARN-8630](https://issues.apache.org/jira/browse/YARN-8630) | ATSv2 REST APIs should honor filter-entity-list-by-user in non-secure cluster when ACls are enabled | Major | . | Rohith Sharma K S | Rohith Sharma K S |
| [YARN-8729](https://issues.apache.org/jira/browse/YARN-8729) | Node status updater thread could be lost after it is restarted | Critical | nodemanager | Tao Yang | Tao Yang |
| [HDFS-13914](https://issues.apache.org/jira/browse/HDFS-13914) | Fix DN UI logs link broken when https is enabled after HDFS-13902 | Minor | datanode | Jianfei Jiang | Jianfei Jiang |
| [MAPREDUCE-7133](https://issues.apache.org/jira/browse/MAPREDUCE-7133) | History Server task attempts REST API returns invalid data | Major | jobhistoryserver | Oleksandr Shevchenko | Oleksandr Shevchenko |
| [HDFS-13838](https://issues.apache.org/jira/browse/HDFS-13838) | WebHdfsFileSystem.getFileStatus() won't return correct "snapshot enabled" status | Major | hdfs, webhdfs | Siyao Meng | Siyao Meng |
| [HADOOP-15733](https://issues.apache.org/jira/browse/HADOOP-15733) | Correct the log when Invalid emptier Interval configured | Major | trash | Harshakiran Reddy | Ayush Saxena |
| [YARN-8772](https://issues.apache.org/jira/browse/YARN-8772) | Annotation javax.annotation.Generated has moved | Minor | api | Andrew Purtell | Andrew Purtell |
| [YARN-8720](https://issues.apache.org/jira/browse/YARN-8720) | CapacityScheduler does not enforce max resource allocation check at queue level | Major | capacity scheduler, capacityscheduler, resourcemanager | Tarun Parimi | Tarun Parimi |
| [YARN-8748](https://issues.apache.org/jira/browse/YARN-8748) | Javadoc warnings within the nodemanager package | Trivial | . | Shane Kumpf | Craig Condit |
| [YARN-8782](https://issues.apache.org/jira/browse/YARN-8782) | Fix exception message in Resource.throwExceptionWhenArrayOutOfBound | Minor | . | Szilard Nemeth | Gergely Pollak |
| [HDFS-13919](https://issues.apache.org/jira/browse/HDFS-13919) | Documentation: Improper formatting in Disk Balancer for Settings | Minor | . | Ayush Saxena | Ayush Saxena |
| [HDFS-13844](https://issues.apache.org/jira/browse/HDFS-13844) | Fix the fmt\_bytes function in the dfs-dust.js | Minor | hdfs, ui | yanghuafeng | yanghuafeng |
| [HADOOP-15220](https://issues.apache.org/jira/browse/HADOOP-15220) | Über-jira: S3a phase V: Hadoop 3.2 features | Major | . | Steve Loughran | Steve Loughran |
| [YARN-8726](https://issues.apache.org/jira/browse/YARN-8726) | [UI2] YARN UI2 is not accessible when config.env file failed to load | Critical | yarn-ui-v2 | Akhil PB | Akhil PB |
| [YARN-8652](https://issues.apache.org/jira/browse/YARN-8652) | [UI2] YARN UI2 breaks if getUserInfo REST API is not available in older versions. | Critical | . | Akhil PB | Akhil PB |
| [YARN-8787](https://issues.apache.org/jira/browse/YARN-8787) | Fix broken list items in PlacementConstraints documentation | Minor | documentation | Masahiro Tanaka | Masahiro Tanaka |
| [HDFS-11719](https://issues.apache.org/jira/browse/HDFS-11719) | Arrays.fill() wrong index in BlockSender.readChecksum() exception handling | Major | datanode | Tao Zhang | Tao Zhang |
| [YARN-8648](https://issues.apache.org/jira/browse/YARN-8648) | Container cgroups are leaked when using docker | Major | . | Jim Brennan | Jim Brennan |
| [HADOOP-15755](https://issues.apache.org/jira/browse/HADOOP-15755) | StringUtils#createStartupShutdownMessage throws NPE when args is null | Major | . | Lokesh Jain | Dinesh Chitlangia |
| [MAPREDUCE-3801](https://issues.apache.org/jira/browse/MAPREDUCE-3801) | org.apache.hadoop.mapreduce.v2.app.TestRuntimeEstimators.testExponentialEstimator fails intermittently | Major | mrv2 | Robert Joseph Evans | Jason Lowe |
| [MAPREDUCE-7137](https://issues.apache.org/jira/browse/MAPREDUCE-7137) | MRAppBenchmark.benchmark1() fails with NullPointerException | Minor | test | Oleksandr Shevchenko | Oleksandr Shevchenko |
| [MAPREDUCE-7138](https://issues.apache.org/jira/browse/MAPREDUCE-7138) | ThrottledContainerAllocator in MRAppBenchmark should implement RMHeartbeatHandler | Minor | test | Oleksandr Shevchenko | Oleksandr Shevchenko |
| [HDFS-13886](https://issues.apache.org/jira/browse/HDFS-13886) | HttpFSFileSystem.getFileStatus() doesn't return "snapshot enabled" bit | Major | httpfs | Siyao Meng | Siyao Meng |
| [HDFS-13833](https://issues.apache.org/jira/browse/HDFS-13833) | Improve BlockPlacementPolicyDefault's consider load logic | Major | . | Henrique Barros | Shweta |
| [HDFS-13868](https://issues.apache.org/jira/browse/HDFS-13868) | WebHDFS: GETSNAPSHOTDIFF API NPE when param "snapshotname" is given but "oldsnapshotname" is not. | Major | hdfs, webhdfs | Siyao Meng | Pranay Singh |
| [YARN-8771](https://issues.apache.org/jira/browse/YARN-8771) | CapacityScheduler fails to unreserve when cluster resource contains empty resource type | Critical | capacityscheduler | Tao Yang | Tao Yang |
| [HDFS-13908](https://issues.apache.org/jira/browse/HDFS-13908) | TestDataNodeMultipleRegistrations is flaky | Major | . | Íñigo Goiri | Ayush Saxena |
| [HADOOP-15684](https://issues.apache.org/jira/browse/HADOOP-15684) | triggerActiveLogRoll stuck on dead name node, when ConnectTimeoutException happens. | Critical | ha | Rong Tang | Rong Tang |
| [HADOOP-15772](https://issues.apache.org/jira/browse/HADOOP-15772) | Remove the 'Path ... should be specified as a URI' warnings on startup | Major | conf | Arpit Agarwal | Ayush Saxena |
| [YARN-8784](https://issues.apache.org/jira/browse/YARN-8784) | DockerLinuxContainerRuntime prevents access to distributed cache entries on a full disk | Major | nodemanager | Jason Lowe | Eric Badger |
| [HDFS-13892](https://issues.apache.org/jira/browse/HDFS-13892) | Disk Balancer : Make execute command documentation better. | Major | diskbalancer, documentation | Harshakiran Reddy | Ranith Sardar |
| [HADOOP-15736](https://issues.apache.org/jira/browse/HADOOP-15736) | Trash : Negative Value For Deletion Interval Leads To Abnormal Behaviour. | Major | . | Ayush Saxena | Ayush Saxena |
| [YARN-8628](https://issues.apache.org/jira/browse/YARN-8628) | [UI2] Few duplicated or inconsistent information displayed in UI2 | Major | yarn-ui-v2 | Akhil PB | Akhil PB |
| [YARN-8742](https://issues.apache.org/jira/browse/YARN-8742) | [UI2] Container logs on Application / Service pages on UI2 are not available many case, improve error messages in such cases. | Major | yarn-ui-v2 | Akhil PB | Akhil PB |
| [MAPREDUCE-7125](https://issues.apache.org/jira/browse/MAPREDUCE-7125) | JobResourceUploader creates LocalFileSystem when it's not necessary | Major | job submission | Peter Cseh | Peter Cseh |
| [YARN-6338](https://issues.apache.org/jira/browse/YARN-6338) | Typos in Docker docs: contains =\> containers | Minor | nodemanager | Daniel Templeton | Zoltan Siegl |
| [YARN-8815](https://issues.apache.org/jira/browse/YARN-8815) | RM fails to recover finished unmanaged AM | Critical | . | Rakesh Shah | Bibin A Chundatt |
| [YARN-8752](https://issues.apache.org/jira/browse/YARN-8752) | yarn-registry.md has wrong word ong-lived, it should be long-lived | Trivial | documentation | leiqiang | leiqiang |
| [YARN-8745](https://issues.apache.org/jira/browse/YARN-8745) | Misplaced the TestRMWebServicesFairScheduler.java file. | Major | fairscheduler, test | Y. SREENIVASULU REDDY | Y. SREENIVASULU REDDY |
| [HADOOP-15693](https://issues.apache.org/jira/browse/HADOOP-15693) | Typo in Secure Mode documentation | Trivial | documentation | Thomas Levine | Thomas Levine |
| [YARN-8824](https://issues.apache.org/jira/browse/YARN-8824) | App Nodelabel missed after RM restart for finished apps | Major | . | Bibin A Chundatt | Bibin A Chundatt |
| [HDFS-13840](https://issues.apache.org/jira/browse/HDFS-13840) | RBW Blocks which are having less GS should be added to Corrupt | Minor | . | Brahma Reddy Battula | Brahma Reddy Battula |
| [YARN-8804](https://issues.apache.org/jira/browse/YARN-8804) | resourceLimits may be wrongly calculated when leaf-queue is blocked in cluster with 3+ level queues | Critical | capacityscheduler | Tao Yang | Tao Yang |
| [HDFS-13932](https://issues.apache.org/jira/browse/HDFS-13932) | [JDK 11] Casts to BlockStoragePolicy[] in unit tests raise ClassCastExceptions | Minor | test | Andrew Purtell | Andrew Purtell |
| [HDFS-13938](https://issues.apache.org/jira/browse/HDFS-13938) | Add a missing "break" in BaseTestHttpFSWith | Minor | httpfs, test | Siyao Meng | Siyao Meng |
| [YARN-8829](https://issues.apache.org/jira/browse/YARN-8829) | Cluster metrics can fail with IndexOutOfBoundsException | Minor | router | Akshay Agarwal | Akshay Agarwal |
| [YARN-8774](https://issues.apache.org/jira/browse/YARN-8774) | Memory leak when CapacityScheduler allocates from reserved container with non-default label | Critical | capacityscheduler | Tao Yang | Tao Yang |
| [HADOOP-15814](https://issues.apache.org/jira/browse/HADOOP-15814) | Maven 3.3.3 unable to parse pom file | Major | . | Wei-Chiu Chuang | Wei-Chiu Chuang |
| [YARN-8840](https://issues.apache.org/jira/browse/YARN-8840) | Add missing cleanupSSLConfig() call for TestTimelineClient test | Minor | test, timelineclient | Aki Tanaka | Aki Tanaka |
| [HADOOP-15817](https://issues.apache.org/jira/browse/HADOOP-15817) | Reuse Object Mapper in KMSJSONReader | Major | kms | Jonathan Eagles | Jonathan Eagles |
| [YARN-8844](https://issues.apache.org/jira/browse/YARN-8844) | TestNMProxy unit test is failing | Major | yarn | Eric Yang | Eric Yang |
| [HDFS-13957](https://issues.apache.org/jira/browse/HDFS-13957) | Fix incorrect option used in description of InMemoryAliasMap | Minor | documentation | Yiqun Lin | Yiqun Lin |
| [YARN-7957](https://issues.apache.org/jira/browse/YARN-7957) | [UI2] YARN service delete option disappears after stopping application | Critical | yarn-ui-v2 | Yesha Vora | Akhil PB |
| [YARN-8797](https://issues.apache.org/jira/browse/YARN-8797) | [UI2] Improve error pages in new YARN UI | Major | yarn-ui-v2 | Akhil PB | Akhil PB |
| [HADOOP-15791](https://issues.apache.org/jira/browse/HADOOP-15791) | Remove Ozone related sources from the 3.2 branch | Major | . | Elek, Marton | Elek, Marton |
| [HADOOP-15820](https://issues.apache.org/jira/browse/HADOOP-15820) | ZStandardDecompressor native code sets an integer field as a long | Blocker | . | Jason Lowe | Jason Lowe |
| [YARN-8819](https://issues.apache.org/jira/browse/YARN-8819) | Fix findbugs warnings in YarnServiceUtils | Minor | . | Akira Ajisaka | Vidura Bhathiya Mudalige |
| [HDFS-13964](https://issues.apache.org/jira/browse/HDFS-13964) | RBF: TestRouterWebHDFSContractAppend fails with No Active Namenode under nameservice | Major | . | Ayush Saxena | Ayush Saxena |
| [YARN-7825](https://issues.apache.org/jira/browse/YARN-7825) | [UI2] Maintain constant horizontal application info bar for Application Attempt page | Major | yarn-ui-v2 | Yesha Vora | Akhil PB |
| [YARN-8843](https://issues.apache.org/jira/browse/YARN-8843) | updateNodeResource does not support units for memory | Minor | . | Íñigo Goiri | Manikandan R |
| [HDFS-13768](https://issues.apache.org/jira/browse/HDFS-13768) | Adding replicas to volume map makes DataNode start slowly | Major | . | Yiqun Lin | Surendra Singh Lilhore |
| [HDFS-13962](https://issues.apache.org/jira/browse/HDFS-13962) | Add null check for add-replica pool to avoid lock acquiring | Major | . | Yiqun Lin | Surendra Singh Lilhore |
| [HADOOP-15818](https://issues.apache.org/jira/browse/HADOOP-15818) | Fix deprecated maven-surefire-plugin configuration in hadoop-kms module | Minor | kms | Akira Ajisaka | Vidura Bhathiya Mudalige |
| [YARN-8853](https://issues.apache.org/jira/browse/YARN-8853) | [UI2] Application Attempts tab is not shown correctly when there are no attempts | Major | . | Charan Hebri | Akhil PB |
| [YARN-8845](https://issues.apache.org/jira/browse/YARN-8845) | hadoop.registry.rm.enabled is not used | Major | . | Íñigo Goiri | Íñigo Goiri |
| [HDFS-11396](https://issues.apache.org/jira/browse/HDFS-11396) | TestNameNodeMetadataConsistency#testGenerationStampInFuture timed out | Minor | namenode, test | John Zhuge | Ayush Saxena |
| [HDFS-13926](https://issues.apache.org/jira/browse/HDFS-13926) | ThreadLocal aggregations for FileSystem.Statistics are incorrect with striped reads | Major | erasure-coding | Xiao Chen | Hrishikesh Gadre |
| [HDFS-12459](https://issues.apache.org/jira/browse/HDFS-12459) | Fix revert: Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API | Major | webhdfs | Weiwei Yang | Weiwei Yang |
| [YARN-8666](https://issues.apache.org/jira/browse/YARN-8666) | [UI2] Remove application tab from YARN Queue Page | Major | yarn-ui-v2 | Yesha Vora | Yesha Vora |
| [YARN-8753](https://issues.apache.org/jira/browse/YARN-8753) | [UI2] Lost nodes representation missing from Nodemanagers Chart | Major | yarn-ui-v2 | Yesha Vora | Yesha Vora |
| [YARN-8861](https://issues.apache.org/jira/browse/YARN-8861) | executorLock is misleading in ContainerLaunch | Trivial | yarn | Chandni Singh | Chandni Singh |
| [HDFS-13945](https://issues.apache.org/jira/browse/HDFS-13945) | TestDataNodeVolumeFailure is Flaky | Major | . | Ayush Saxena | Ayush Saxena |
| [HDFS-13802](https://issues.apache.org/jira/browse/HDFS-13802) | RBF: Remove FSCK from Router Web UI | Major | . | Fei Hui | Fei Hui |
| [YARN-8830](https://issues.apache.org/jira/browse/YARN-8830) | SLS tool fix node addition | Critical | . | Bibin A Chundatt | Bibin A Chundatt |
| [YARN-8869](https://issues.apache.org/jira/browse/YARN-8869) | YARN Service Client might not work correctly with RM REST API for Kerberos authentication | Blocker | . | Eric Yang | Eric Yang |
| [HADOOP-15857](https://issues.apache.org/jira/browse/HADOOP-15857) | Remove ozonefs class name definition from core-default.xml | Blocker | fs | Elek, Marton | Elek, Marton |
| [YARN-8879](https://issues.apache.org/jira/browse/YARN-8879) | Kerberos principal is needed when submitting a submarine job | Critical | . | Zac Zhou | Zac Zhou |
| [MAPREDUCE-7132](https://issues.apache.org/jira/browse/MAPREDUCE-7132) | JobSplitWriter prints unnecessary warnings if EC(RS10,4) is used | Major | client, mrv2 | Peter Bacsko | Peter Bacsko |
| [YARN-8892](https://issues.apache.org/jira/browse/YARN-8892) | YARN UI2 doc changes to update security status (verified under security environment) | Blocker | . | Sunil Govindan | Sunil Govindan |
| [YARN-8810](https://issues.apache.org/jira/browse/YARN-8810) | Yarn Service: discrepancy between hashcode and equals of ConfigFile | Minor | . | Chandni Singh | Chandni Singh |
| [HADOOP-15802](https://issues.apache.org/jira/browse/HADOOP-15802) | start-build-env.sh creates an invalid /etc/sudoers.d/hadoop-build-${USER\_ID} file entry | Minor | build | Jon Boone | Jon Boone |
| [HDFS-13662](https://issues.apache.org/jira/browse/HDFS-13662) | TestBlockReaderLocal#testStatisticsForErasureCodingRead is flaky | Major | erasure-coding, test | Wei-Chiu Chuang | Xiao Chen |
| [HADOOP-15861](https://issues.apache.org/jira/browse/HADOOP-15861) | Move DelegationTokenIssuer to the right path | Blocker | security | Wei-Chiu Chuang | Wei-Chiu Chuang |
| [YARN-8759](https://issues.apache.org/jira/browse/YARN-8759) | Copy of resource-types.xml is not deleted if test fails, causes other test failures | Major | yarn | Antal Bálint Steinbach | Antal Bálint Steinbach |
| [HADOOP-15859](https://issues.apache.org/jira/browse/HADOOP-15859) | ZStandardDecompressor.c mistakes a class for an instance | Blocker | . | Ben Lau | Jason Lowe |
| [HDFS-14000](https://issues.apache.org/jira/browse/HDFS-14000) | RBF: Documentation should reflect right scripts for v3.0 and above | Major | . | CR Hota | CR Hota |
| [HDFS-14005](https://issues.apache.org/jira/browse/HDFS-14005) | RBF: Web UI update to bootstrap-3.3.7 | Major | . | Íñigo Goiri | Íñigo Goiri |
| [HDFS-14002](https://issues.apache.org/jira/browse/HDFS-14002) | TestLayoutVersion#testNameNodeFeatureMinimumCompatibleLayoutVersions fails | Major | test | Takanobu Asanuma | Takanobu Asanuma |
| [HDFS-14007](https://issues.apache.org/jira/browse/HDFS-14007) | Incompatible layout when generating FSImage | Major | . | Íñigo Goiri | Íñigo Goiri |
| [HADOOP-15850](https://issues.apache.org/jira/browse/HADOOP-15850) | CopyCommitter#concatFileChunks should check that the blocks per chunk is not 0 | Critical | tools/distcp | Ted Yu | Ted Yu |
| [HADOOP-15866](https://issues.apache.org/jira/browse/HADOOP-15866) | Renamed HADOOP\_SECURITY\_GROUP\_SHELL\_COMMAND\_TIMEOUT keys break compatibility | Blocker | . | Wei-Chiu Chuang | Wei-Chiu Chuang |
| [HADOOP-15822](https://issues.apache.org/jira/browse/HADOOP-15822) | zstd compressor can fail with a small output buffer | Major | . | Jason Lowe | Jason Lowe |
| [YARN-8938](https://issues.apache.org/jira/browse/YARN-8938) | Add service upgrade cancel and express examples to the service upgrade doc | Minor | . | Chandni Singh | Chandni Singh |
| [HADOOP-14445](https://issues.apache.org/jira/browse/HADOOP-14445) | Use DelegationTokenIssuer to create KMS delegation tokens that can authenticate to all KMS instances | Major | kms | Wei-Chiu Chuang | Xiao Chen |
| [YARN-8950](https://issues.apache.org/jira/browse/YARN-8950) | Compilation fails with dependency convergence error for hbase.profile=2.0 | Blocker | . | Rohith Sharma K S | Rohith Sharma K S |
| [YARN-8858](https://issues.apache.org/jira/browse/YARN-8858) | CapacityScheduler should respect maximum node resource when per-queue maximum-allocation is being used. | Major | . | Sumana Sathish | Wangda Tan |
| [YARN-8990](https://issues.apache.org/jira/browse/YARN-8990) | Fix fair scheduler race condition in app submit and queue cleanup | Blocker | fairscheduler | Wilfred Spiegelenburg | Wilfred Spiegelenburg |
| [HADOOP-15973](https://issues.apache.org/jira/browse/HADOOP-15973) | Configuration: Included properties are not cached if resource is a stream | Critical | . | Eric Payne | Eric Payne |
| [MAPREDUCE-7174](https://issues.apache.org/jira/browse/MAPREDUCE-7174) | MapReduce example wordmedian should handle generic options | Major | . | Fei Hui | Fei Hui |
| [HADOOP-15997](https://issues.apache.org/jira/browse/HADOOP-15997) | KMS client uses wrong UGI after HADOOP-14445 | Blocker | kms | Wei-Chiu Chuang | Wei-Chiu Chuang |
| [HADOOP-15992](https://issues.apache.org/jira/browse/HADOOP-15992) | JSON License is included in the transitive dependency of aliyun-sdk-oss 3.0.0 | Blocker | . | Akira Ajisaka | Akira Ajisaka |
| [HADOOP-16031](https://issues.apache.org/jira/browse/HADOOP-16031) | TestSecureLogins#testValidKerberosName fails | Major | security | Akira Ajisaka | Akira Ajisaka |
### TESTS:
@ -660,38 +824,54 @@
| [HDFS-12146](https://issues.apache.org/jira/browse/HDFS-12146) | [SPS] : Fix TestStoragePolicySatisfierWithStripedFile#testSPSWhenFileHasLowRedundancyBlocks | Major | namenode | Surendra Singh Lilhore | Surendra Singh Lilhore |
| [HDFS-12141](https://issues.apache.org/jira/browse/HDFS-12141) | [SPS]: Fix checkstyle warnings | Major | datanode, namenode | Rakesh R | Rakesh R |
| [HDFS-12152](https://issues.apache.org/jira/browse/HDFS-12152) | [SPS]: Re-arrange StoragePolicySatisfyWorker stopping sequence to improve thread cleanup time | Minor | datanode, namenode | Rakesh R | Rakesh R |
| [HDFS-12237](https://issues.apache.org/jira/browse/HDFS-12237) | libhdfs++: PROTOC\_IS\_COMPATIBLE check fails if protobuf library is built from source | Major | hdfs-client | Anatoli Shein | Anatoli Shein |
| [HDFS-12214](https://issues.apache.org/jira/browse/HDFS-12214) | [SPS]: Fix review comments of StoragePolicySatisfier feature | Major | datanode, namenode | Rakesh R | Rakesh R |
| [HDFS-12225](https://issues.apache.org/jira/browse/HDFS-12225) | [SPS]: Optimize extended attributes for tracking SPS movements | Major | datanode, namenode | Uma Maheswara Rao G | Surendra Singh Lilhore |
| [HDFS-12291](https://issues.apache.org/jira/browse/HDFS-12291) | [SPS]: Provide a mechanism to recursively iterate and satisfy storage policy of all the files under the given dir | Major | datanode, namenode | Rakesh R | Surendra Singh Lilhore |
| [YARN-6855](https://issues.apache.org/jira/browse/YARN-6855) | CLI Proto Modifications to support Node Attributes | Major | api, capacityscheduler, client | Naganarasimha G R | Naganarasimha G R |
| [HDFS-12570](https://issues.apache.org/jira/browse/HDFS-12570) | [SPS]: Refactor Co-ordinator datanode logic to track the block storage movements | Major | datanode, namenode | Rakesh R | Rakesh R |
| [HDFS-12556](https://issues.apache.org/jira/browse/HDFS-12556) | [SPS] : Block movement analysis should be done in read lock. | Major | datanode, namenode | Surendra Singh Lilhore | Surendra Singh Lilhore |
| [HDFS-12427](https://issues.apache.org/jira/browse/HDFS-12427) | libhdfs++: Prevent Requests from holding dangling pointer to RpcEngine | Critical | hdfs-client | James Clampffer | James Clampffer |
| [HDFS-12310](https://issues.apache.org/jira/browse/HDFS-12310) | [SPS]: Provide an option to track the status of in progress requests | Major | datanode, namenode | Uma Maheswara Rao G | Surendra Singh Lilhore |
| [HDFS-12790](https://issues.apache.org/jira/browse/HDFS-12790) | [SPS]: Rebasing HDFS-10285 branch after HDFS-10467, HDFS-12599 and HDFS-11968 commits | Major | datanode, namenode | Rakesh R | Rakesh R |
| [HDFS-12106](https://issues.apache.org/jira/browse/HDFS-12106) | [SPS]: Improve storage policy satisfier configurations | Major | namenode | Surendra Singh Lilhore | Surendra Singh Lilhore |
| [HDFS-12955](https://issues.apache.org/jira/browse/HDFS-12955) | [SPS]: Move SPS classes to a separate package | Trivial | nn | Uma Maheswara Rao G | Rakesh R |
| [HDFS-12982](https://issues.apache.org/jira/browse/HDFS-12982) | [SPS]: Reduce the locking and cleanup the Namesystem access | Major | datanode, namenode | Rakesh R | Rakesh R |
| [HDFS-12911](https://issues.apache.org/jira/browse/HDFS-12911) | [SPS]: Modularize the SPS code and expose necessary interfaces for external/internal implementations. | Major | datanode, namenode | Uma Maheswara Rao G | Uma Maheswara Rao G |
| [YARN-6856](https://issues.apache.org/jira/browse/YARN-6856) | Support CLI for Node Attributes Mapping | Major | api, capacityscheduler, client | Naganarasimha G R | Naganarasimha G R |
| [HDFS-13025](https://issues.apache.org/jira/browse/HDFS-13025) | [SPS]: Implement a mechanism to scan the files for external SPS | Major | namenode | Uma Maheswara Rao G | Uma Maheswara Rao G |
| [HDFS-13033](https://issues.apache.org/jira/browse/HDFS-13033) | [SPS]: Implement a mechanism to do file block movements for external SPS | Major | . | Rakesh R | Rakesh R |
| [HDFS-13057](https://issues.apache.org/jira/browse/HDFS-13057) | [SPS]: Revisit configurations to make SPS service modes internal/external/none | Blocker | . | Rakesh R | Rakesh R |
| [HDFS-13075](https://issues.apache.org/jira/browse/HDFS-13075) | [SPS]: Provide External Context implementation. | Major | . | Uma Maheswara Rao G | Uma Maheswara Rao G |
| [HDFS-13050](https://issues.apache.org/jira/browse/HDFS-13050) | [SPS] : Create start/stop script to start external SPS process. | Blocker | namenode | Surendra Singh Lilhore | Surendra Singh Lilhore |
| [HDFS-13077](https://issues.apache.org/jira/browse/HDFS-13077) | [SPS]: Fix review comments of external storage policy satisfier | Major | . | Rakesh R | Rakesh R |
| [YARN-7842](https://issues.apache.org/jira/browse/YARN-7842) | PB changes to carry node-attributes in NM heartbeat | Major | . | Weiwei Yang | Weiwei Yang |
| [YARN-7840](https://issues.apache.org/jira/browse/YARN-7840) | Update PB for prefix support of node attributes | Blocker | . | Weiwei Yang | Naganarasimha G R |
| [YARN-7757](https://issues.apache.org/jira/browse/YARN-7757) | Refactor NodeLabelsProvider to be more generic and reusable for node attributes providers | Blocker | nodemanager | Weiwei Yang | Weiwei Yang |
| [HDFS-13097](https://issues.apache.org/jira/browse/HDFS-13097) | [SPS]: Fix the branch review comments(Part1) | Major | namenode | Surendra Singh Lilhore | Surendra Singh Lilhore |
| [HDFS-13110](https://issues.apache.org/jira/browse/HDFS-13110) | [SPS]: Reduce the number of APIs in NamenodeProtocol used by external satisfier | Major | . | Rakesh R | Rakesh R |
| [YARN-6858](https://issues.apache.org/jira/browse/YARN-6858) | Attribute Manager to store and provide node attributes in RM | Major | api, capacityscheduler, client | Naganarasimha G R | Naganarasimha G R |
| [YARN-7856](https://issues.apache.org/jira/browse/YARN-7856) | Validate Node Attributes from NM | Major | nodemanager, RM | Weiwei Yang | Weiwei Yang |
| [YARN-7965](https://issues.apache.org/jira/browse/YARN-7965) | NodeAttributeManager add/get API is not working properly | Critical | resourcemanager | Weiwei Yang | Weiwei Yang |
| [HDFS-13166](https://issues.apache.org/jira/browse/HDFS-13166) | [SPS]: Implement caching mechanism to keep LIVE datanodes to minimize costly getLiveDatanodeStorageReport() calls | Major | . | Rakesh R | Rakesh R |
| [YARN-7871](https://issues.apache.org/jira/browse/YARN-7871) | Node attributes reporting from NM to RM | Major | nodemanager | Weiwei Yang | Weiwei Yang |
| [HADOOP-15262](https://issues.apache.org/jira/browse/HADOOP-15262) | AliyunOSS: move files under a directory in parallel when rename a directory | Major | fs/oss | wujinhu | wujinhu |
| [YARN-8002](https://issues.apache.org/jira/browse/YARN-8002) | Support NOT\_SELF and ALL namespace types for allocation tag | Major | resourcemanager | Weiwei Yang | Weiwei Yang |
| [HDFS-13307](https://issues.apache.org/jira/browse/HDFS-13307) | RBF: Improve the use of setQuota command | Major | . | liuhongtong | liuhongtong |
| [YARN-7988](https://issues.apache.org/jira/browse/YARN-7988) | Refactor FSNodeLabelStore code for Node Attributes store support | Major | . | Bibin A Chundatt | Bibin A Chundatt |
| [YARN-7497](https://issues.apache.org/jira/browse/YARN-7497) | Add file system based scheduler configuration store | Major | yarn | Jiandan Yang | Jiandan Yang |
| [HDFS-13289](https://issues.apache.org/jira/browse/HDFS-13289) | RBF: TestConnectionManager#testCleanup() test case need correction | Minor | . | Dibyendu Karmakar | Dibyendu Karmakar |
| [YARN-8094](https://issues.apache.org/jira/browse/YARN-8094) | Support configuration based Node Attribute provider | Major | nodemanager | Weiwei Yang | Weiwei Yang |
| [YARN-8092](https://issues.apache.org/jira/browse/YARN-8092) | Expose Node Attributes info via RM nodes REST API | Major | resourcemanager | Weiwei Yang | Weiwei Yang |
| [YARN-8033](https://issues.apache.org/jira/browse/YARN-8033) | CLI Integration with NodeAttributesManagerImpl | Major | . | Naganarasimha G R | Naganarasimha G R |
| [HADOOP-14758](https://issues.apache.org/jira/browse/HADOOP-14758) | S3GuardTool.prune to handle UnsupportedOperationException | Trivial | fs/s3 | Steve Loughran | Gabor Bota |
| [HDFS-13364](https://issues.apache.org/jira/browse/HDFS-13364) | RBF: Support NamenodeProtocol in the Router | Major | . | Íñigo Goiri | Íñigo Goiri |
| [YARN-8013](https://issues.apache.org/jira/browse/YARN-8013) | Support application tags when defining application namespaces for placement constraints | Major | . | Weiwei Yang | Weiwei Yang |
| [YARN-7946](https://issues.apache.org/jira/browse/YARN-7946) | Update TimelineServerV2 doc as per YARN-7919 | Major | . | Rohith Sharma K S | Haibo Chen |
| [YARN-8117](https://issues.apache.org/jira/browse/YARN-8117) | Fix TestRMWebServicesNodes test failure | Major | . | Bibin A Chundatt | Bibin A Chundatt |
| [YARN-6936](https://issues.apache.org/jira/browse/YARN-6936) | [Atsv2] Retrospect storing entities into sub application table from client perspective | Major | . | Rohith Sharma K S | Rohith Sharma K S |
| [HDFS-13353](https://issues.apache.org/jira/browse/HDFS-13353) | RBF: TestRouterWebHDFSContractCreate failed | Major | test | Takanobu Asanuma | Takanobu Asanuma |
| [HADOOP-15367](https://issues.apache.org/jira/browse/HADOOP-15367) | Update the initialization code in the docker hadoop-runner baseimage | Major | . | Elek, Marton | Elek, Marton |
| [YARN-7875](https://issues.apache.org/jira/browse/YARN-7875) | Node Attribute store for storing and recovering attributes | Major | . | Bibin A Chundatt | Bibin A Chundatt |
| [HADOOP-14759](https://issues.apache.org/jira/browse/HADOOP-14759) | S3GuardTool prune to prune specific bucket entries | Minor | fs/s3 | Steve Loughran | Gabor Bota |
| [YARN-8107](https://issues.apache.org/jira/browse/YARN-8107) | Give an informative message when incorrect format is used in ATSv2 filter attributes | Major | ATSv2 | Charan Hebri | Rohith Sharma K S |
| [YARN-8048](https://issues.apache.org/jira/browse/YARN-8048) | Support auto-spawning of admin configured services during bootstrap of rm/apiserver | Major | . | Rohith Sharma K S | Rohith Sharma K S |
@ -699,6 +879,7 @@
| [YARN-7574](https://issues.apache.org/jira/browse/YARN-7574) | Add support for Node Labels on Auto Created Leaf Queue Template | Major | capacity scheduler | Suma Shivaprasad | Suma Shivaprasad |
| [HDFS-13380](https://issues.apache.org/jira/browse/HDFS-13380) | RBF: mv/rm fail after the directory exceeded the quota limit | Major | . | Weiwei Wu | Yiqun Lin |
| [YARN-7667](https://issues.apache.org/jira/browse/YARN-7667) | Docker Stop grace period should be configurable | Major | yarn | Eric Badger | Eric Badger |
| [YARN-8100](https://issues.apache.org/jira/browse/YARN-8100) | Support API interface to query cluster attributes and attribute to nodes | Major | . | Bibin A Chundatt | Bibin A Chundatt |
| [HDFS-13410](https://issues.apache.org/jira/browse/HDFS-13410) | RBF: Support federation with no subclusters | Minor | . | Íñigo Goiri | Íñigo Goiri |
| [HDFS-13384](https://issues.apache.org/jira/browse/HDFS-13384) | RBF: Improve timeout RPC call mechanism | Minor | . | Íñigo Goiri | Íñigo Goiri |
| [HADOOP-15376](https://issues.apache.org/jira/browse/HADOOP-15376) | Remove double semi colons on imports that make Clover fall over. | Minor | . | Ewan Higgs | Ewan Higgs |
@ -723,6 +904,7 @@
| [HDFS-13466](https://issues.apache.org/jira/browse/HDFS-13466) | RBF: Add more router-related information to the UI | Minor | . | Wei Yan | Wei Yan |
| [YARN-5888](https://issues.apache.org/jira/browse/YARN-5888) | [UI2] Improve unit tests for new YARN UI | Minor | yarn-ui-v2 | Akhil PB | Akhil PB |
| [HDFS-13453](https://issues.apache.org/jira/browse/HDFS-13453) | RBF: getMountPointDates should fetch latest subdir time/date when parent dir is not present but /parent/child dirs are present in mount table | Major | . | Dibyendu Karmakar | Dibyendu Karmakar |
| [YARN-8104](https://issues.apache.org/jira/browse/YARN-8104) | Add API to fetch node to attribute mapping | Major | . | Bibin A Chundatt | Bibin A Chundatt |
| [YARN-8186](https://issues.apache.org/jira/browse/YARN-8186) | [Router] Federation: routing getAppState REST invocations transparently to multiple RMs | Major | . | Giovanni Matteo Fumarola | Giovanni Matteo Fumarola |
| [YARN-8137](https://issues.apache.org/jira/browse/YARN-8137) | Parallelize node addition in SLS | Major | . | Abhishek Modi | Abhishek Modi |
| [YARN-8111](https://issues.apache.org/jira/browse/YARN-8111) | Simplify PlacementConstraints API by removing allocationTagToIntraApp | Minor | resourcemanager | Weiwei Yang | Weiwei Yang |
@ -751,6 +933,7 @@
| [YARN-8212](https://issues.apache.org/jira/browse/YARN-8212) | Pending backlog for async allocation threads should be configurable | Major | . | Weiwei Yang | Tao Yang |
| [YARN-2674](https://issues.apache.org/jira/browse/YARN-2674) | Distributed shell AM may re-launch containers if RM work preserving restart happens | Major | applications, resourcemanager | Chun Chen | Shane Kumpf |
| [HDFS-13488](https://issues.apache.org/jira/browse/HDFS-13488) | RBF: Reject requests when a Router is overloaded | Major | . | Íñigo Goiri | Íñigo Goiri |
| [HDFS-11807](https://issues.apache.org/jira/browse/HDFS-11807) | libhdfs++: Get minidfscluster tests running under valgrind | Major | hdfs-client | James Clampffer | Anatoli Shein |
| [YARN-8113](https://issues.apache.org/jira/browse/YARN-8113) | Update placement constraints doc with application namespaces and inter-app constraints | Major | documentation | Weiwei Yang | Weiwei Yang |
| [YARN-8194](https://issues.apache.org/jira/browse/YARN-8194) | Exception when reinitializing a container using LinuxContainerExecutor | Blocker | . | Chandni Singh | Chandni Singh |
| [YARN-8151](https://issues.apache.org/jira/browse/YARN-8151) | Yarn RM Epoch should wrap around | Major | . | Young Chen | Young Chen |
@ -767,6 +950,7 @@
| [HDFS-13346](https://issues.apache.org/jira/browse/HDFS-13346) | RBF: Fix synchronization of router quota and nameservice quota | Major | . | liuhongtong | Yiqun Lin |
| [YARN-8243](https://issues.apache.org/jira/browse/YARN-8243) | Flex down should remove instance with largest component instance ID first | Critical | yarn-native-services | Gour Saha | Gour Saha |
| [YARN-7654](https://issues.apache.org/jira/browse/YARN-7654) | Support ENTRY\_POINT for docker container | Blocker | yarn | Eric Yang | Eric Yang |
| [YARN-7892](https://issues.apache.org/jira/browse/YARN-7892) | Revisit NodeAttribute class structure | Major | resourcemanager | Naganarasimha G R | Naganarasimha G R |
| [YARN-8247](https://issues.apache.org/jira/browse/YARN-8247) | Incorrect HTTP status code returned by ATSv2 for non-whitelisted users | Critical | ATSv2 | Charan Hebri | Rohith Sharma K S |
| [YARN-8130](https://issues.apache.org/jira/browse/YARN-8130) | Race condition when container events are published for KILLED applications | Major | ATSv2 | Charan Hebri | Rohith Sharma K S |
| [YARN-8081](https://issues.apache.org/jira/browse/YARN-8081) | Yarn Service Upgrade: Add support to upgrade a component | Major | . | Chandni Singh | Chandni Singh |
@ -780,6 +964,7 @@
| [YARN-7530](https://issues.apache.org/jira/browse/YARN-7530) | hadoop-yarn-services-api should be part of hadoop-yarn-services | Blocker | yarn-native-services | Eric Yang | Chandni Singh |
| [YARN-6919](https://issues.apache.org/jira/browse/YARN-6919) | Add default volume mount list | Major | yarn | Eric Badger | Eric Badger |
| [HADOOP-15494](https://issues.apache.org/jira/browse/HADOOP-15494) | TestRawLocalFileSystemContract fails on Windows | Minor | test | Anbang Hu | Anbang Hu |
| [YARN-8351](https://issues.apache.org/jira/browse/YARN-8351) | Node attribute manager logs are flooding RM logs | Major | . | Weiwei Yang | Weiwei Yang |
| [HADOOP-15498](https://issues.apache.org/jira/browse/HADOOP-15498) | TestHadoopArchiveLogs (#testGenerateScript, #testPrepareWorkingDir) fails on Windows | Minor | . | Anbang Hu | Anbang Hu |
| [HADOOP-15497](https://issues.apache.org/jira/browse/HADOOP-15497) | TestTrash should use proper test path to avoid failing on Windows | Minor | . | Anbang Hu | Anbang Hu |
| [YARN-8329](https://issues.apache.org/jira/browse/YARN-8329) | Docker client configuration can still be set incorrectly | Major | . | Shane Kumpf | Shane Kumpf |
@ -809,6 +994,8 @@
| [HADOOP-15458](https://issues.apache.org/jira/browse/HADOOP-15458) | TestLocalFileSystem#testFSOutputStreamBuilder fails on Windows | Minor | test | Xiao Liang | Xiao Liang |
| [HADOOP-15416](https://issues.apache.org/jira/browse/HADOOP-15416) | s3guard diff assert failure if source path not found | Minor | fs/s3 | Steve Loughran | Gabor Bota |
| [HADOOP-15423](https://issues.apache.org/jira/browse/HADOOP-15423) | Merge fileCache and dirCache into one single cache in LocalMetadataStore | Minor | . | Gabor Bota | Gabor Bota |
| [YARN-8103](https://issues.apache.org/jira/browse/YARN-8103) | Add CLI interface to query node attributes | Major | . | Bibin A Chundatt | Bibin A Chundatt |
| [YARN-8481](https://issues.apache.org/jira/browse/YARN-8481) | AMRMProxyPolicies should accept heartbeat response from new/unknown subclusters | Minor | amrmproxy, federation | Botong Huang | Botong Huang |
| [YARN-8465](https://issues.apache.org/jira/browse/YARN-8465) | Dshell docker container gets marked as lost after NM restart | Major | yarn-native-services | Yesha Vora | Shane Kumpf |
| [YARN-8485](https://issues.apache.org/jira/browse/YARN-8485) | Priviledged container app launch is failing intermittently | Major | yarn-native-services | Yesha Vora | Eric Yang |
| [HDFS-13381](https://issues.apache.org/jira/browse/HDFS-13381) | [SPS]: Use DFSUtilClient#makePathFromFileId() to prepare satisfier file path | Major | . | Rakesh R | Rakesh R |
@ -818,8 +1005,8 @@
| [HDFS-13710](https://issues.apache.org/jira/browse/HDFS-13710) | RBF: setQuota and getQuotaUsage should check the dfs.federation.router.quota.enable | Major | federation, hdfs | yanghuafeng | yanghuafeng |
| [YARN-7899](https://issues.apache.org/jira/browse/YARN-7899) | [AMRMProxy] Stateful FederationInterceptor for pending requests | Major | . | Botong Huang | Botong Huang |
| [HDFS-13726](https://issues.apache.org/jira/browse/HDFS-13726) | RBF: Fix RBF configuration links | Minor | documentation | Takanobu Asanuma | Takanobu Asanuma |
| [HADOOP-15349](https://issues.apache.org/jira/browse/HADOOP-15349) | S3Guard DDB retryBackoff to be more informative on limits exceeded | Major | fs/s3 | Steve Loughran | Gabor Bota |
| [HDFS-13475](https://issues.apache.org/jira/browse/HDFS-13475) | RBF: Admin cannot enforce Router enter SafeMode | Major | . | Wei Yan | Chao Sun |
| [YARN-8299](https://issues.apache.org/jira/browse/YARN-8299) | Yarn Service Upgrade: Add GET APIs that returns instances matching query params | Major | . | Chandni Singh | Chandni Singh |
| [HDFS-13733](https://issues.apache.org/jira/browse/HDFS-13733) | RBF: Add Web UI configurations and descriptions to RBF document | Minor | documentation | Takanobu Asanuma | Takanobu Asanuma |
| [YARN-6995](https://issues.apache.org/jira/browse/YARN-6995) | Improve use of ResourceNotFoundException in resource types code | Minor | nodemanager, resourcemanager | Daniel Templeton | Szilard Nemeth |
| [HDFS-13743](https://issues.apache.org/jira/browse/HDFS-13743) | RBF: Router throws NullPointerException due to the invalid initialization of MountTableResolver | Major | . | Takanobu Asanuma | Takanobu Asanuma |
@ -827,39 +1014,132 @@
| [HDFS-13076](https://issues.apache.org/jira/browse/HDFS-13076) | [SPS]: Cleanup work for HDFS-10285 | Major | . | Rakesh R | Rakesh R |
| [HDFS-13583](https://issues.apache.org/jira/browse/HDFS-13583) | RBF: Router admin clrQuota is not synchronized with nameservice | Major | . | Dibyendu Karmakar | Dibyendu Karmakar |
| [YARN-8546](https://issues.apache.org/jira/browse/YARN-8546) | Resource leak caused by a reserved container being released more than once under async scheduling | Major | capacity scheduler | Weiwei Yang | Tao Yang |
| [HADOOP-15636](https://issues.apache.org/jira/browse/HADOOP-15636) | Add ITestDynamoDBMetadataStore | Minor | fs/s3, test | Sean Mackrory | Gabor Bota |
| [YARN-8574](https://issues.apache.org/jira/browse/YARN-8574) | Allow dot in attribute values | Minor | . | Bibin A Chundatt | Bibin A Chundatt |
| [YARN-8175](https://issues.apache.org/jira/browse/YARN-8175) | Add support for Node Labels in SLS | Major | . | Abhishek Modi | Abhishek Modi |
| [HDFS-13084](https://issues.apache.org/jira/browse/HDFS-13084) | [SPS]: Fix the branch review comments | Major | . | Uma Maheswara Rao G | Rakesh R |
| [YARN-8263](https://issues.apache.org/jira/browse/YARN-8263) | DockerClient still touches hadoop.tmp.dir | Minor | . | Jason Lowe | Craig Condit |
| [YARN-7159](https://issues.apache.org/jira/browse/YARN-7159) | Normalize unit of resource objects in RM to avoid unit conversion in critical path | Critical | nodemanager, resourcemanager | Wangda Tan | Manikandan R |
| [HADOOP-15626](https://issues.apache.org/jira/browse/HADOOP-15626) | FileContextMainOperationsBaseTest.testBuilderCreateAppendExistingFile fails on filesystems without append. | Blocker | fs/s3, test | Steve Loughran | Steve Loughran |
| [YARN-8287](https://issues.apache.org/jira/browse/YARN-8287) | Update documentation and yarn-default related to the Docker runtime | Minor | . | Shane Kumpf | Craig Condit |
| [YARN-8624](https://issues.apache.org/jira/browse/YARN-8624) | Cleanup ENTRYPOINT documentation | Minor | . | Craig Condit | Craig Condit |
| [YARN-7089](https://issues.apache.org/jira/browse/YARN-7089) | Mark the log-aggregation-controller APIs as public | Major | . | Xuan Gong | Zian Chen |
| [YARN-8626](https://issues.apache.org/jira/browse/YARN-8626) | Create HomePolicyManager that sends all the requests to the home subcluster | Minor | . | Giovanni Matteo Fumarola | Íñigo Goiri |
| [HADOOP-15400](https://issues.apache.org/jira/browse/HADOOP-15400) | Improve S3Guard documentation on Authoritative Mode implementation | Minor | fs/s3 | Aaron Fabbri | Gabor Bota |
| [HADOOP-15546](https://issues.apache.org/jira/browse/HADOOP-15546) | ABFS: tune imports & javadocs; stabilise tests | Major | fs/azure | Steve Loughran | Thomas Marquardt |
| [HADOOP-15583](https://issues.apache.org/jira/browse/HADOOP-15583) | Stabilize S3A Assumed Role support | Blocker | fs/s3 | Steve Loughran | Steve Loughran |
| [YARN-8136](https://issues.apache.org/jira/browse/YARN-8136) | Add version attribute to site doc examples and quickstart | Major | site | Gour Saha | Eric Yang |
| [YARN-8588](https://issues.apache.org/jira/browse/YARN-8588) | Logging improvements for better debuggability | Major | . | Suma Shivaprasad | Suma Shivaprasad |
| [YARN-8520](https://issues.apache.org/jira/browse/YARN-8520) | Document best practice for user management | Major | documentation, yarn | Eric Yang | Eric Yang |
| [HADOOP-15659](https://issues.apache.org/jira/browse/HADOOP-15659) | ABFS: Code changes for bug fix and new tests | Major | fs/azure | Da Zhou | Da Zhou |
| [HDFS-13808](https://issues.apache.org/jira/browse/HDFS-13808) | [SPS]: Remove unwanted FSNamesystem #isFileOpenedForWrite() and #getFileInfo() function | Minor | . | Rakesh R | Rakesh R |
| [HADOOP-15576](https://issues.apache.org/jira/browse/HADOOP-15576) | S3A Multipart Uploader to work with S3Guard and encryption | Blocker | fs/s3 | Steve Loughran | Ewan Higgs |
| [YARN-8561](https://issues.apache.org/jira/browse/YARN-8561) | [Submarine] Initial implementation: Training job submission and job history retrieval | Major | . | Wangda Tan | Wangda Tan |
| [HADOOP-15645](https://issues.apache.org/jira/browse/HADOOP-15645) | ITestS3GuardToolLocal.testDiffCommand fails if bucket has per-bucket binding to DDB | Blocker | fs/s3 | Steve Loughran | Steve Loughran |
| [YARN-7417](https://issues.apache.org/jira/browse/YARN-7417) | re-factory IndexedFileAggregatedLogsBlock and TFileAggregatedLogsBlock to remove duplicate codes | Major | . | Xuan Gong | Zian Chen |
| [YARN-8160](https://issues.apache.org/jira/browse/YARN-8160) | Yarn Service Upgrade: Support upgrade of service that use docker containers | Major | . | Chandni Singh | Chandni Singh |
| [HADOOP-15552](https://issues.apache.org/jira/browse/HADOOP-15552) | Move logging APIs over to slf4j in hadoop-tools - Part2 | Major | . | Giovanni Matteo Fumarola | Ian Pickering |
| [HADOOP-15669](https://issues.apache.org/jira/browse/HADOOP-15669) | ABFS: Improve HTTPS Performance | Major | fs/azure | Thomas Marquardt | Vishwajeet Dusane |
| [HADOOP-15642](https://issues.apache.org/jira/browse/HADOOP-15642) | Update aws-sdk version to 1.11.375 | Blocker | build, fs/s3 | Steve Loughran | Steve Loughran |
| [HADOOP-14154](https://issues.apache.org/jira/browse/HADOOP-14154) | Persist isAuthoritative bit in DynamoDBMetaStore (authoritative mode support) | Minor | fs/s3 | Rajesh Balamohan | Gabor Bota |
| [HADOOP-14624](https://issues.apache.org/jira/browse/HADOOP-14624) | Add GenericTestUtils.DelayAnswer that accept slf4j logger API | Major | . | Wenxin He | Wenxin He |
| [HADOOP-15660](https://issues.apache.org/jira/browse/HADOOP-15660) | ABFS: Add support for OAuth | Major | . | Da Zhou | Da Zhou |
| [HDFS-13750](https://issues.apache.org/jira/browse/HDFS-13750) | RBF: Router ID in RouterRpcClient is always null | Major | . | Takanobu Asanuma | Takanobu Asanuma |
| [YARN-8129](https://issues.apache.org/jira/browse/YARN-8129) | Improve error message for invalid value in fields attribute | Minor | ATSv2 | Charan Hebri | Abhishek Modi |
| [YARN-7494](https://issues.apache.org/jira/browse/YARN-7494) | Add muti-node lookup mechanism and pluggable nodes sorting policies to optimize placement decision | Major | capacity scheduler | Sunil Govindan | Sunil Govindan |
| [YARN-8298](https://issues.apache.org/jira/browse/YARN-8298) | Yarn Service Upgrade: Support express upgrade of a service | Major | . | Chandni Singh | Chandni Singh |
| [YARN-8581](https://issues.apache.org/jira/browse/YARN-8581) | [AMRMProxy] Add sub-cluster timeout in LocalityMulticastAMRMProxyPolicy | Major | amrmproxy, federation | Botong Huang | Botong Huang |
| [YARN-8673](https://issues.apache.org/jira/browse/YARN-8673) | [AMRMProxy] More robust responseId resync after an YarnRM master slave switch | Major | amrmproxy | Botong Huang | Botong Huang |
| [HADOOP-15661](https://issues.apache.org/jira/browse/HADOOP-15661) | ABFS: Add support for ACL | Major | . | Da Zhou | Da Zhou |
| [YARN-8015](https://issues.apache.org/jira/browse/YARN-8015) | Support all types of placement constraint support for Capacity Scheduler | Critical | capacity scheduler | Weiwei Yang | Weiwei Yang |
| [HADOOP-15688](https://issues.apache.org/jira/browse/HADOOP-15688) | ABFS: InputStream wrapped in FSDataInputStream twice | Major | . | Sean Mackrory | Sean Mackrory |
| [HADOOP-15682](https://issues.apache.org/jira/browse/HADOOP-15682) | ABFS: Add support for StreamCapabilities. Fix javadoc and checkstyle | Major | fs/azure | Thomas Marquardt | Thomas Marquardt |
| [HDFS-13848](https://issues.apache.org/jira/browse/HDFS-13848) | Refactor NameNode failover proxy providers | Major | ha, hdfs-client | Konstantin Shvachko | Konstantin Shvachko |
| [YARN-7863](https://issues.apache.org/jira/browse/YARN-7863) | Modify placement constraints to support node attributes | Major | . | Sunil Govindan | Sunil Govindan |
| [YARN-8705](https://issues.apache.org/jira/browse/YARN-8705) | Refactor the UAM heartbeat thread in preparation for YARN-8696 | Major | . | Botong Huang | Botong Huang |
| [HADOOP-15699](https://issues.apache.org/jira/browse/HADOOP-15699) | Fix some of testContainerManager failures in Windows | Major | . | Botong Huang | Botong Huang |
| [YARN-8721](https://issues.apache.org/jira/browse/YARN-8721) | Relax NE node-attribute check when attribute doesn't exist on a node | Major | . | Sunil Govindan | Sunil Govindan |
| [YARN-8697](https://issues.apache.org/jira/browse/YARN-8697) | LocalityMulticastAMRMProxyPolicy should fallback to random sub-cluster when cannot resolve resource | Major | . | Botong Huang | Botong Huang |
| [HADOOP-15692](https://issues.apache.org/jira/browse/HADOOP-15692) | ABFS: extensible support for custom oauth | Major | fs/azure | Thomas Marquardt | Da Zhou |
| [HDFS-13634](https://issues.apache.org/jira/browse/HDFS-13634) | RBF: Configurable value in xml for async connection request queue size. | Major | federation | CR Hota | CR Hota |
| [YARN-8642](https://issues.apache.org/jira/browse/YARN-8642) | Add support for tmpfs mounts with the Docker runtime | Major | . | Shane Kumpf | Craig Condit |
| [HADOOP-15667](https://issues.apache.org/jira/browse/HADOOP-15667) | FileSystemMultipartUploader should verify that UploadHandle has non-0 length | Major | fs/s3 | Ewan Higgs | Ewan Higgs |
| [HADOOP-15663](https://issues.apache.org/jira/browse/HADOOP-15663) | ABFS: Simplify configuration | Major | fs/azure | Thomas Marquardt | Da Zhou |
| [YARN-7865](https://issues.apache.org/jira/browse/YARN-7865) | Node attributes documentation | Major | documentation | Weiwei Yang | Naganarasimha G R |
| [HADOOP-15664](https://issues.apache.org/jira/browse/HADOOP-15664) | ABFS: Reduce test run time via parallelization and grouping | Major | fs/azure | Thomas Marquardt | Da Zhou |
| [YARN-8739](https://issues.apache.org/jira/browse/YARN-8739) | Fix jenkins issues for Node Attributes branch | Major | . | Sunil Govindan | Sunil Govindan |
| [YARN-8740](https://issues.apache.org/jira/browse/YARN-8740) | Clear node attribute path after each test run | Major | . | Bibin A Chundatt | Bibin A Chundatt |
| [HADOOP-15703](https://issues.apache.org/jira/browse/HADOOP-15703) | ABFS - Implement client-side throttling | Major | . | Sneha Varma | Thomas Marquardt |
| [HDFS-13695](https://issues.apache.org/jira/browse/HDFS-13695) | Move logging to slf4j in HDFS package | Major | . | Giovanni Matteo Fumarola | Ian Pickering |
| [HADOOP-15728](https://issues.apache.org/jira/browse/HADOOP-15728) | ABFS: Add backward compatibility to handle Unsupported Operation for storage account with no namespace feature | Major | . | Da Zhou | Da Zhou |
| [HADOOP-15709](https://issues.apache.org/jira/browse/HADOOP-15709) | Move S3Guard LocalMetadataStore constants to org.apache.hadoop.fs.s3a.Constants | Minor | . | Gabor Bota | Gabor Bota |
| [YARN-8699](https://issues.apache.org/jira/browse/YARN-8699) | Add Yarnclient#yarnclusterMetrics API implementation in router | Major | . | Bibin A Chundatt | Bibin A Chundatt |
| [HADOOP-15731](https://issues.apache.org/jira/browse/HADOOP-15731) | TestDistributedShell fails on Windows | Major | . | Botong Huang | Botong Huang |
| [YARN-8698](https://issues.apache.org/jira/browse/YARN-8698) | [Submarine] Failed to reset Hadoop home environment when submitting a submarine job | Major | . | Zac Zhou | Zac Zhou |
| [HDFS-13237](https://issues.apache.org/jira/browse/HDFS-13237) | [Documentation] RBF: Mount points across multiple subclusters | Minor | . | Íñigo Goiri | Íñigo Goiri |
| [HADOOP-15745](https://issues.apache.org/jira/browse/HADOOP-15745) | Add ABFS configuration to ConfigRedactor | Major | . | Sean Mackrory | Sean Mackrory |
| [HADOOP-15740](https://issues.apache.org/jira/browse/HADOOP-15740) | ABFS: Check variable names during initialization of AbfsClientThrottlingIntercept | Minor | . | Sneha Varma | Sneha Varma |
| [HADOOP-15694](https://issues.apache.org/jira/browse/HADOOP-15694) | ABFS: Allow OAuth credentials to not be tied to accounts | Major | . | Sean Mackrory | Sean Mackrory |
| [HADOOP-14734](https://issues.apache.org/jira/browse/HADOOP-14734) | add option to tag DDB table(s) created | Minor | fs/s3 | Steve Loughran | Gabor Bota |
| [HADOOP-15426](https://issues.apache.org/jira/browse/HADOOP-15426) | Make S3guard client resilient to DDB throttle events and network failures | Blocker | fs/s3 | Steve Loughran | Steve Loughran |
| [YARN-8706](https://issues.apache.org/jira/browse/YARN-8706) | DelayedProcessKiller is executed for Docker containers even though docker stop sends a KILL signal after the specified grace period | Major | . | Chandni Singh | Chandni Singh |
| [HADOOP-15753](https://issues.apache.org/jira/browse/HADOOP-15753) | ABFS: support path "abfs://mycluster/file/path" | Major | . | Da Zhou | Da Zhou |
| [HADOOP-15757](https://issues.apache.org/jira/browse/HADOOP-15757) | ABFS: remove dependency on common-codec Base64 | Major | . | Da Zhou | Da Zhou |
| [YARN-8715](https://issues.apache.org/jira/browse/YARN-8715) | Make allocation tags in the placement spec optional for node-attributes | Major | . | Weiwei Yang | Weiwei Yang |
| [HADOOP-15754](https://issues.apache.org/jira/browse/HADOOP-15754) | s3guard: testDynamoTableTagging should clear existing config | Minor | fs/s3 | Aaron Fabbri | Gabor Bota |
| [HADOOP-15741](https://issues.apache.org/jira/browse/HADOOP-15741) | [JDK10] Upgrade Maven Javadoc Plugin from 3.0.0-M1 to 3.0.1 | Minor | build | Akira Ajisaka | Takanobu Asanuma |
| [HADOOP-15304](https://issues.apache.org/jira/browse/HADOOP-15304) | [JDK10] Migrate from com.sun.tools.doclets to the replacement | Major | build | Akira Ajisaka | Akira Ajisaka |
| [HADOOP-15759](https://issues.apache.org/jira/browse/HADOOP-15759) | AliyunOSS: update oss-sdk version to 3.0.0 | Major | fs/oss | wujinhu | wujinhu |
| [YARN-8757](https://issues.apache.org/jira/browse/YARN-8757) | [Submarine] Add Tensorboard component when --tensorboard is specified | Critical | . | Wangda Tan | Wangda Tan |
| [YARN-8791](https://issues.apache.org/jira/browse/YARN-8791) | When STOPSIGNAL is not present then docker inspect returns an extra line feed | Major | . | Chandni Singh | Chandni Singh |
| [HADOOP-15773](https://issues.apache.org/jira/browse/HADOOP-15773) | ABFS: Fix issues raised by Yetus | Major | fs/azure | Sean Mackrory | Sean Mackrory |
| [HADOOP-15748](https://issues.apache.org/jira/browse/HADOOP-15748) | S3 listing inconsistency can raise NPE in globber | Major | fs | Steve Loughran | Steve Loughran |
| [HADOOP-15756](https://issues.apache.org/jira/browse/HADOOP-15756) | [JDK10] Migrate from sun.net.util.IPAddressUtil to the replacement | Major | net, util | Akira Ajisaka | Akira Ajisaka |
| [YARN-8801](https://issues.apache.org/jira/browse/YARN-8801) | java doc comments in docker-util.h is confusing | Minor | . | Zian Chen | Zian Chen |
| [YARN-8658](https://issues.apache.org/jira/browse/YARN-8658) | [AMRMProxy] Metrics for AMRMClientRelayer inside FederationInterceptor | Major | . | Botong Huang | Young Chen |
| [YARN-8769](https://issues.apache.org/jira/browse/YARN-8769) | [Submarine] Allow user to specify customized quicklink(s) when submit Submarine job | Critical | . | Wangda Tan | Wangda Tan |
| [HDFS-13937](https://issues.apache.org/jira/browse/HDFS-13937) | Multipart Uploader APIs to be marked as private/unstable in 3.2.0 | Blocker | fs | Steve Loughran | Steve Loughran |
| [HDFS-13876](https://issues.apache.org/jira/browse/HDFS-13876) | HttpFS: Implement ALLOWSNAPSHOT, DISALLOWSNAPSHOT | Major | httpfs | Siyao Meng | Siyao Meng |
| [YARN-8696](https://issues.apache.org/jira/browse/YARN-8696) | [AMRMProxy] FederationInterceptor upgrade: home sub-cluster heartbeat async | Major | nodemanager | Botong Huang | Botong Huang |
| [YARN-8817](https://issues.apache.org/jira/browse/YARN-8817) | [Submarine] In cases when user doesn't ask HDFS path while submitting job but framework requires user to set HDFS related environments | Critical | submarine | Wangda Tan | Wangda Tan |
| [HADOOP-15783](https://issues.apache.org/jira/browse/HADOOP-15783) | [JDK10] TestSFTPFileSystem.testGetModifyTime fails | Major | . | Akira Ajisaka | Akira Ajisaka |
| [HADOOP-15671](https://issues.apache.org/jira/browse/HADOOP-15671) | AliyunOSS: Support Assume Roles in AliyunOSS | Major | fs/oss | wujinhu | wujinhu |
| [YARN-8623](https://issues.apache.org/jira/browse/YARN-8623) | Update Docker examples to use image which exists | Minor | . | Craig Condit | Craig Condit |
| [HADOOP-15723](https://issues.apache.org/jira/browse/HADOOP-15723) | ABFS: Ranger Support | Major | fs/azure | Thomas Marquardt | Yuan Gao |
| [HADOOP-15739](https://issues.apache.org/jira/browse/HADOOP-15739) | ABFS: remove unused maven dependencies and add used undeclared dependencies | Major | build, fs/azure | Da Zhou | Da Zhou |
| [HDFS-13790](https://issues.apache.org/jira/browse/HDFS-13790) | RBF: Move ClientProtocol APIs to its own module | Major | . | Íñigo Goiri | Chao Sun |
| [YARN-6456](https://issues.apache.org/jira/browse/YARN-6456) | Allow administrators to control available container runtimes and set defaults for all containers | Major | nodemanager | Miklos Szegedi | Craig Condit |
| [YARN-8270](https://issues.apache.org/jira/browse/YARN-8270) | Adding JMX Metrics for Timeline Collector and Reader | Major | ATSv2, timelineserver | Sushil Ks | Sushil Ks |
| [HADOOP-15764](https://issues.apache.org/jira/browse/HADOOP-15764) | [JDK10] Migrate from sun.net.dns.ResolverConfiguration to the replacement | Major | net, util | Akira Ajisaka | Akira Ajisaka |
| [YARN-8800](https://issues.apache.org/jira/browse/YARN-8800) | Updated documentation of Submarine with latest examples. | Critical | . | Wangda Tan | Wangda Tan |
| [YARN-8760](https://issues.apache.org/jira/browse/YARN-8760) | [AMRMProxy] Fix concurrent re-register due to YarnRM failover in AMRMClientRelayer | Major | . | Botong Huang | Botong Huang |
| [HDFS-13943](https://issues.apache.org/jira/browse/HDFS-13943) | [JDK10] Fix javadoc errors in hadoop-hdfs-client module | Major | documentation | Akira Ajisaka | Akira Ajisaka |
| [HADOOP-15793](https://issues.apache.org/jira/browse/HADOOP-15793) | ABFS: Skip unsupported test cases when non namespace enabled in ITestAzureBlobFileSystemAuthorization | Minor | fs/azure, test | Yuan Gao | Yuan Gao |
| [HADOOP-15801](https://issues.apache.org/jira/browse/HADOOP-15801) | ABFS: Fixing skipUserGroupMetadata in AzureBlobFileSystemStore | Major | fs/azure | Da Zhou | Da Zhou |
| [YARN-8785](https://issues.apache.org/jira/browse/YARN-8785) | Improve the error message when a bind mount is not whitelisted | Major | . | Simon Prewo | Simon Prewo |
| [HADOOP-15792](https://issues.apache.org/jira/browse/HADOOP-15792) | typo in AzureBlobFileSystem.getIsNamespaceEnabeld | Minor | fs/azure | Steve Loughran | Abhishek Modi |
| [HADOOP-15795](https://issues.apache.org/jira/browse/HADOOP-15795) | Make HTTPS the default protocol for ABFS | Major | fs/azure | Da Zhou | Da Zhou |
| [HDFS-13877](https://issues.apache.org/jira/browse/HDFS-13877) | HttpFS: Implement GETSNAPSHOTDIFF | Major | httpfs | Siyao Meng | Siyao Meng |
| [HADOOP-15809](https://issues.apache.org/jira/browse/HADOOP-15809) | ABFS: better exception handling when making getAccessToken call | Major | fs/azure | Da Zhou | Da Zhou |
| [HDFS-13936](https://issues.apache.org/jira/browse/HDFS-13936) | multipart upload to HDFS to support 0 byte upload | Major | fs, hdfs | Steve Loughran | Ewan Higgs |
| [YARN-8852](https://issues.apache.org/jira/browse/YARN-8852) | [Submarine] Add documentation for submarine installation details | Major | documentation, submarine | Zac Zhou | Zac Zhou |
| [YARN-7644](https://issues.apache.org/jira/browse/YARN-7644) | NM gets backed up deleting docker containers | Major | nodemanager | Eric Badger | Chandni Singh |
| [HDFS-13878](https://issues.apache.org/jira/browse/HDFS-13878) | HttpFS: Implement GETSNAPSHOTTABLEDIRECTORYLIST | Major | httpfs | Siyao Meng | Siyao Meng |
| [YARN-5742](https://issues.apache.org/jira/browse/YARN-5742) | Serve aggregated logs of historical apps from timeline service | Critical | timelineserver | Varun Saxena | Rohith Sharma K S |
| [HADOOP-15837](https://issues.apache.org/jira/browse/HADOOP-15837) | DynamoDB table Update can fail S3A FS init | Major | fs/s3 | Steve Loughran | Steve Loughran |
| [HADOOP-15848](https://issues.apache.org/jira/browse/HADOOP-15848) | ITestS3AContractMultipartUploader#testMultipartUploadEmptyPart test error | Major | fs/s3 | Gabor Bota | Ewan Higgs |
| [YARN-8875](https://issues.apache.org/jira/browse/YARN-8875) | [Submarine] Add documentation for submarine installation script details | Critical | . | Xun Liu | Xun Liu |
| [YARN-8798](https://issues.apache.org/jira/browse/YARN-8798) | [Submarine] Job should not be submitted if "--input\_path" option is missing | Critical | . | Zhankun Tang | Zhankun Tang |
| [HADOOP-15839](https://issues.apache.org/jira/browse/HADOOP-15839) | Review + update cloud store sensitive keys in hadoop.security.sensitive-config-keys | Minor | conf | Steve Loughran | Steve Loughran |
| [YARN-8687](https://issues.apache.org/jira/browse/YARN-8687) | YARN service example is out-dated | Major | yarn-native-services | Eric Yang | Suma Shivaprasad |
| [HADOOP-15607](https://issues.apache.org/jira/browse/HADOOP-15607) | AliyunOSS: fix duplicated partNumber issue in AliyunOSSBlockOutputStream | Critical | . | wujinhu | wujinhu |
| [HDFS-14033](https://issues.apache.org/jira/browse/HDFS-14033) | [libhdfs++] Disable libhdfs++ build on systems that do not support thread\_local | Major | . | Anatoli Shein | Anatoli Shein |
| [HDFS-12026](https://issues.apache.org/jira/browse/HDFS-12026) | libhdfs++: Fix compilation errors and warnings when compiling with Clang | Blocker | hdfs-client | Anatoli Shein | Anatoli Shein |
| [YARN-8299](https://issues.apache.org/jira/browse/YARN-8299) | Yarn Service Upgrade: Add GET APIs that returns instances matching query params | Critical | . | Chandni Singh | Chandni Singh |
| [YARN-8160](https://issues.apache.org/jira/browse/YARN-8160) | Yarn Service Upgrade: Support upgrade of service that use docker containers | Major | . | Chandni Singh | Chandni Singh |
| [YARN-8298](https://issues.apache.org/jira/browse/YARN-8298) | Yarn Service Upgrade: Support express upgrade of a service | Major | . | Chandni Singh | Chandni Singh |
| [YARN-8665](https://issues.apache.org/jira/browse/YARN-8665) | Yarn Service Upgrade: Support cancelling upgrade | Major | . | Chandni Singh | Chandni Singh |
| [HDFS-13547](https://issues.apache.org/jira/browse/HDFS-13547) | Add ingress port based sasl resolver | Major | security | Chen Liang | Chen Liang |
| [YARN-9125](https://issues.apache.org/jira/browse/YARN-9125) | Carriage Return character in launch command cause node manager to become unhealthy | Major | . | Eric Yang | Billie Rinaldi |
| [YARN-9090](https://issues.apache.org/jira/browse/YARN-9090) | [Submarine] Update submarine installation script document | Blocker | . | Xun Liu | Xun Liu |
| [YARN-8822](https://issues.apache.org/jira/browse/YARN-8822) | Nvidia-docker v2 support for YARN GPU feature | Critical | . | Zhankun Tang | Charo Zhang |
### OTHER:
@ -867,9 +1147,12 @@
| JIRA | Summary | Priority | Component | Reporter | Contributor |
|:---- |:---- | :--- |:---- |:---- |:---- |
| [YARN-8016](https://issues.apache.org/jira/browse/YARN-8016) | Refine PlacementRule interface and add a app-name queue mapping rule as an example | Major | . | Zian Chen | Zian Chen |
| [HDFS-13338](https://issues.apache.org/jira/browse/HDFS-13338) | Update BUILDING.txt for building native libraries | Critical | build, documentation, native | Takanobu Asanuma | Takanobu Asanuma |
| [YARN-8091](https://issues.apache.org/jira/browse/YARN-8091) | Revisit checkUserAccessToQueue RM REST API | Critical | . | Wangda Tan | Wangda Tan |
| [HDFS-13403](https://issues.apache.org/jira/browse/HDFS-13403) | libhdfs++: Use hdfs::IoService object rather than asio::io\_service | Critical | . | James Clampffer | James Clampffer |
| [YARN-8274](https://issues.apache.org/jira/browse/YARN-8274) | Docker command error during container relaunch | Critical | . | Billie Rinaldi | Jason Lowe |
| [YARN-8080](https://issues.apache.org/jira/browse/YARN-8080) | YARN native service should support component restart policy | Critical | . | Wangda Tan | Suma Shivaprasad |
| [HDFS-13534](https://issues.apache.org/jira/browse/HDFS-13534) | libhdfs++: Fix GCC7 build | Major | . | James Clampffer | James Clampffer |
| [HADOOP-15482](https://issues.apache.org/jira/browse/HADOOP-15482) | Upgrade jackson-databind to version 2.9.5 | Major | . | Lokesh Jain | Lokesh Jain |
| [YARN-7668](https://issues.apache.org/jira/browse/YARN-7668) | Remove unused variables from ContainerLocalizer | Trivial | . | Ray Chiang | Dedunu Dhananjaya |
| [YARN-8412](https://issues.apache.org/jira/browse/YARN-8412) | Move ResourceRequest.clone logic everywhere into a proper API | Minor | . | Botong Huang | Botong Huang |
@ -878,4 +1161,10 @@
| [YARN-8545](https://issues.apache.org/jira/browse/YARN-8545) | YARN native service should return container if launch failed | Critical | . | Wangda Tan | Chandni Singh |
| [HDFS-11610](https://issues.apache.org/jira/browse/HDFS-11610) | sun.net.spi.nameservice.NameService has moved to a new location | Major | . | Akira Ajisaka | Akira Ajisaka |
| [HDFS-13788](https://issues.apache.org/jira/browse/HDFS-13788) | Update EC documentation about rack fault tolerance | Major | documentation, erasure-coding | Xiao Chen | Kitti Nanasi |
| [HADOOP-15816](https://issues.apache.org/jira/browse/HADOOP-15816) | Upgrade Apache Zookeeper version due to security concerns | Major | . | Boris Vulikh | Akira Ajisaka |
| [HADOOP-15882](https://issues.apache.org/jira/browse/HADOOP-15882) | Upgrade maven-shade-plugin from 2.4.3 to 3.2.0 | Major | . | Takanobu Asanuma | Takanobu Asanuma |
| [HADOOP-15815](https://issues.apache.org/jira/browse/HADOOP-15815) | Upgrade Eclipse Jetty version to 9.3.24 | Major | . | Boris Vulikh | Boris Vulikh |
| [YARN-8488](https://issues.apache.org/jira/browse/YARN-8488) | YARN service/components/instances should have SUCCEEDED/FAILED states | Major | yarn-native-services | Wangda Tan | Suma Shivaprasad |
| [HADOOP-16025](https://issues.apache.org/jira/browse/HADOOP-16025) | Update the year to 2019 | Major | build | Ayush Saxena | Ayush Saxena |

100
hadoop-common-project/hadoop-common/src/site/markdown/release/3.2.0/RELEASENOTES.3.2.0.md
View File

@ -42,16 +42,16 @@ It is very important that solutions for any patches remain at the VS 2010-level.
---
* [YARN-6257](https://issues.apache.org/jira/browse/YARN-6257) | *Minor* | **CapacityScheduler REST API produces incorrect JSON - JSON object operationsInfo contains deplicate key**
* [YARN-6257](https://issues.apache.org/jira/browse/YARN-6257) | *Minor* | **CapacityScheduler REST API produces incorrect JSON - JSON object operationsInfo contains duplicate keys**
**WARNING: No release note provided for this change.**
The response format of resource manager's REST API endpoint "ws/v1/cluster/scheduler" is optimized, where the "operationInfos" field is converted from a map to list. This change makes the content to be more JSON parser friendly.
---
* [HADOOP-15146](https://issues.apache.org/jira/browse/HADOOP-15146) | *Minor* | **Remove DataOutputByteBuffer**
**WARNING: No release note provided for this change.**
This jira helped to remove the private and unused class DataOutputByteBuffer.
---
@ -79,7 +79,7 @@ WASB: Bug fix to support non-sequential page blob reads. Required for HBASE rep
New command is added to dfsadmin.
hdfs dfsadmin [-upgrade [query \| finalize]
1. -upgrade query gives the upgradeStatus
1. -upgrade query gives the upgradeStatus
2. -upgrade finalize is equivalent to -finalizeUpgrade.
@ -87,7 +87,7 @@ hdfs dfsadmin [-upgrade [query \| finalize]
* [YARN-8191](https://issues.apache.org/jira/browse/YARN-8191) | *Major* | **Fair scheduler: queue deletion without RM restart**
**WARNING: No release note provided for this change.**
To support Queue deletion with RM restart feature, AllocationFileLoaderService constructor signature was changed. YARN-8390 corrected this and made as a compatible change.
---
@ -132,3 +132,93 @@ FUSE lib now recognize the change of the Kerberos ticket cache path if it was ch
* [HADOOP-15638](https://issues.apache.org/jira/browse/HADOOP-15638) | *Major* | **KMS Accept Queue Size default changed from 500 to 128 in Hadoop 3.x**
Restore the KMS accept queue size to 500 in Hadoop 3.x, making it the same as in Hadoop 2.x.
---
* [HADOOP-15669](https://issues.apache.org/jira/browse/HADOOP-15669) | *Major* | **ABFS: Improve HTTPS Performance**
ABFS: Improved HTTPS performance
---
* [HADOOP-15660](https://issues.apache.org/jira/browse/HADOOP-15660) | *Major* | **ABFS: Add support for OAuth**
ABFS: Support for OAuth
---
* [HDFS-13806](https://issues.apache.org/jira/browse/HDFS-13806) | *Minor* | **EC: No error message for unsetting EC policy of the directory inherits the erasure coding policy from an ancestor directory**
After this change, attempt to unsetErasureCodingPolicy() on a directory without EC policy explicitly set on it, will get NoECPolicySetException.
---
* [HADOOP-14833](https://issues.apache.org/jira/browse/HADOOP-14833) | *Major* | **Remove s3a user:secret authentication**
The S3A connector no longer supports username and secrets in URLs of the form \`s3a://key:secret@bucket/\`. It is near-impossible to stop those secrets being logged —which is why a warning has been printed since Hadoop 2.8 whenever such a URL was used.
Fix: use a more secure mechanism to pass down the secrets.
---
* [YARN-3409](https://issues.apache.org/jira/browse/YARN-3409) | *Major* | **Support Node Attribute functionality**
With this feature task, Node Attributes is supported in YARN which will help user's to effectively use resources and assign to applications based on characteristics of each node's in the cluster.
---
* [HADOOP-15684](https://issues.apache.org/jira/browse/HADOOP-15684) | *Critical* | **triggerActiveLogRoll stuck on dead name node, when ConnectTimeoutException happens.**
When a namenode A sends request RollEditLog to a remote NN, either the remote NN is standby or IO Exception happens, A should continue to try next NN, instead of getting stuck on the problematic one. This Patch is based on trunk.
---
* [HDFS-10285](https://issues.apache.org/jira/browse/HDFS-10285) | *Major* | **Storage Policy Satisfier in HDFS**
StoragePolicySatisfier(SPS) allows users to track and satisfy the storage policy requirement of a given file/directory in HDFS. User can specify a file/directory path by invoking “hdfs storagepolicies -satisfyStoragePolicy -path \<path\>” command or via HdfsAdmin#satisfyStoragePolicy(path) API. For the blocks which has storage policy mismatches, it moves the replicas to a different storage type in order to fulfill the storage policy requirement. Since API calls goes to NN for tracking the invoked satisfier path(iNodes), administrator need to enable dfs.storage.policy.satisfier.mode’ config at NN to allow these operations. It can be enabled by setting ‘dfs.storage.policy.satisfier.mode’ to ‘external’ in hdfs-site.xml. The configs can be disabled dynamically without restarting Namenode. SPS should be started outside Namenode using "hdfs --daemon start sps". If administrator is looking to run Mover tool explicitly, then he/she should make sure to disable SPS first and then run Mover. See the "Storage Policy Satisfier (SPS)" section in the Archival Storage guide for detailed usage.
---
* [HADOOP-15407](https://issues.apache.org/jira/browse/HADOOP-15407) | *Blocker* | **Support Windows Azure Storage - Blob file system in Hadoop**
The abfs connector in the hadoop-azure module supports Microsoft Azure Datalake (Gen 2), which at the time of writing (September 2018) was in preview, soon to go GA. As with all cloud connectors, corner-cases will inevitably surface. If you encounter one, please file a bug report.
---
* [HADOOP-14445](https://issues.apache.org/jira/browse/HADOOP-14445) | *Major* | **Use DelegationTokenIssuer to create KMS delegation tokens that can authenticate to all KMS instances**
<!-- markdown -->
This patch improves the KMS delegation token issuing and authentication logic, to enable tokens to authenticate with a set of KMS servers. The change is backport compatible, in that it keeps the existing authentication logic as a fall back.
Historically, KMS delegation tokens have ip:port as service, making KMS clients only able to use the token to authenticate with the KMS server specified as ip:port, even though the token is shared among all KMS servers at server-side. After this patch, newly created tokens will have the KMS URL as service.
A `DelegationTokenIssuer` interface is introduced for token creation.
---
* [HDFS-14053](https://issues.apache.org/jira/browse/HDFS-14053) | *Major* | **Provide ability for NN to re-replicate based on topology changes.**
A new option (-replicate) is added to fsck command to re-trigger the replication for mis-replicated blocks. This option should be used instead of previous workaround of increasing and then decreasing replication factor (using hadoop fs -setrep command).
---
* [HADOOP-15996](https://issues.apache.org/jira/browse/HADOOP-15996) | *Major* | **Plugin interface to support more complex usernames in Hadoop**
This patch enables "Hadoop" and "MIT" as options for "hadoop.security.auth\_to\_local.mechanism" and defaults to 'hadoop'. This should be backward compatible with pre-HADOOP-12751.
This is basically HADOOP-12751 plus configurable + extended tests.