Commit Graph

25081 Commits

Author SHA1 Message Date
Viraj Jasani
c488abbc79
HDFS-16075. Use empty array constants present in StorageType and DatanodeInfo to avoid creating redundant objects (#3115)
Reviewed-by: Hui Fei <ferhui@apache.org>
2021-06-21 10:25:12 +09:00
litao
bdc9c8809e
HDFS-16079. Improve the block state change log (#3120). Contributed by tomscut.
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2021-06-20 17:23:11 +05:30
Viraj Jasani
d9eccb1eb3
HDFS-16080. RBF: Invoking method in all locations should break the loop after successful result (#3121). Contributed by Viraj Jasani.
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2021-06-20 17:19:05 +05:30
Ahmed Hussein
c255feb5d7
HDFS-16061. DFTestUtil.waitReplication can produce false positives (#3095). Contributed by Ahmed Hussein.
Reviewed-by: Jim Brennan <jbrennan@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2021-06-20 10:22:47 +05:30
litao
c491f81a30
HDFS-16078. Remove unused parameters for DatanodeManager.handleLifeline() (#3119). Contributed by tomscut.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2021-06-20 10:20:52 +05:30
LeonGao
643dfd60e4
HDFS-15842. HDFS mover to emit metrics. (#2738) 2021-06-19 15:39:46 -07:00
Gautham B A
51991c4907
HADOOP-17724. Add Dockerfile for Debian 10 (#3038) 2021-06-17 10:14:44 -07:00
Wei-Chiu Chuang
1ad674e548
HDFS-16074. Remove an expensive debug string concatenation (#3107)
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: Hui Fei <ferhui@apache.org>
Reviewed-by: Viraj Jasani <vjasani@apache.org>
Reviewed-by: litao <tomleescut@gmail.com>
2021-06-16 20:37:56 -07:00
hust_hhb
56d249759f
HDFS-13671. Namenode deletes large dir slowly caused by FoldedTreeSet#removeAndGet (#3065) 2021-06-17 11:08:29 +08:00
Takanobu Asanuma
9e7c7ad129
HADOOP-17760. Delete hadoop.ssl.enabled and dfs.https.enable from docs and core-default.xml (#3099)
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
2021-06-17 09:58:47 +09:00
Szilard Nemeth
6562391737 YARN-10813. Set default capacity of root for node labels. Contributed by Andras Gyori 2021-06-16 18:55:09 +02:00
Szilard Nemeth
428478bbe2 YARN-10801. Fix Auto Queue template to properly set all configuration properties. Contributed by Andras Gyori 2021-06-16 18:26:58 +02:00
Ayush Saxena
2b304ad645 HDFS-16073. Remove redundant RPC requests for getFileLinkInfo in ClientNamenodeProtocolTranslatorPB. Contributed by lei w. 2021-06-16 15:53:12 +05:30
Jim Brennan
9a6a11c452 YARN-10767. Yarn Logs Command retrying on Standby RM for 30 times. Contributed by D M Murali Krishna Reddy. 2021-06-15 18:58:42 +00:00
Wei-Chiu Chuang
a77bf7cf07 Make upstream aware of 3.3.1 release 2021-06-15 15:43:36 +00:00
Takanobu Asanuma
839fcf7682
HDFS-16068. WebHdfsFileSystem has a possible connection leak in connection with HttpFS (#3104) 2021-06-15 13:20:09 +05:30
Viraj Jasani
c1bf3cb0da
HDFS-16016. BPServiceActor to provide new thread to handle IBR (#2998)
Contributed by Viraj Jasani
2021-06-14 14:23:50 -07:00
Szilard Nemeth
e31d06032b YARN-10802. Change Capacity Scheduler minimum-user-limit-percent to accept decimal values. Contributed by Benjamin Teke 2021-06-14 22:33:04 +02:00
Siyao Meng
ebee2aed00
HDFS-16055. Quota is not preserved in snapshot INode (#3078) 2021-06-14 10:48:22 -07:00
Takanobu Asanuma
ae9630f580
HADOOP-17756. Increase precommit job timeout from 20 hours to 24 hours. (#3090)
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
Reviewed-by: Viraj Jasani <vjasani@apache.org>
2021-06-14 10:17:39 +09:00
snehavarma
35e4c31fff
HADOOP-17714 ABFS: testBlobBackCompatibility, testRandomRead & WasbAbfsCompatibility tests fail when triggered with default configs (#3035) 2021-06-13 23:52:29 +05:30
Anoop Sam John
5970c632d4
HADOOP-17645 Fix test failures in org.apache.hadoop.fs.azure.ITestOutputStreamSemantics. (#2926) 2021-06-13 23:07:10 +05:30
Petre Bogdan Stolojan
de9ca9f155
HADOOP-17547 Magic committer to downgrade abort in cleanup if list uploads fails with access denied (#3051)
Contributed by Bogdan Stolojan
2021-06-12 17:45:12 +01:00
Szilard Nemeth
f0bdc422aa YARN-10816. Avoid doing delegation token ops when yarn.timeline-service.http-authentication.type=simple. Contributed by Tarun Parimi 2021-06-12 15:18:41 +02:00
Szilard Nemeth
7003997e36 YARN-10789. RM HA startup can fail due to race conditions in ZKConfigurationStore. Contributed by Tarun Parimi 2021-06-12 14:49:52 +02:00
Anoop Sam John
2cf952baf4
HADOOP-17643 WASB : Make metadata checks case insensitive (#2972) 2021-06-12 15:25:03 +05:30
Ahmed Hussein
352949d070
HDFS-15671. testBalancerRPCDelayQpsDefault fails intermittently (#3097) 2021-06-11 11:00:42 -05:00
litao
51ebf7b2a0
HDFS-16057. Make sure the order for location in ENTERING_MAINTENANCE state (#3084)
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
2021-06-11 17:30:55 +09:00
Mingliang Liu
6e5692e7e2
Revert "HDFS-16033 Fix issue of the StatisticsDataReferenceCleaner cleanUp (#3042)"
This reverts commit 4a26a61ecd.
2021-06-11 00:34:24 -07:00
Viraj Jasani
4ef27a596f
HADOOP-17753. Keep restrict-imports-enforcer-rule for Guava Lists in top level hadoop-main pom (#3087) 2021-06-11 12:15:52 +09:00
Akira Ajisaka
9a17f97133
YARN-10803. [JDK 11] TestRMFailoverProxyProvider and TestNoHaRMFailoverProxyProvider fails by ClassCastException. (#3068)
Reviewed-by: Takanobu Asanuma <tasanuma@apache.org>
2021-06-10 14:29:47 +09:00
Takanobu Asanuma
b3c7254563
HADOOP-17750. Fix asf license errors in newly added files by HADOOP-17727. (#3083)
Reviewed-by: Viraj Jasani <vjasani@apache.org>
Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
Reviewed-by: Gautham B A <gautham.bangalore@gmail.com>
2021-06-09 15:32:07 +09:00
Viraj Jasani
81d7069316
YARN-10805. Replace Guava Lists usage by Hadoop's own Lists in hadoop-yarn-project (#3075) 2021-06-09 15:15:47 +09:00
Viraj Jasani
eefa664fea
HDFS-16054. Replace Guava Lists usage by Hadoop's own Lists in hadoop-hdfs-project (#3073) 2021-06-09 15:12:48 +09:00
snehavarma
4c039fafeb
HADOOP-17715 ABFS: Append blob tests with non HNS accounts fail (#3028) 2021-06-09 10:54:10 +05:30
Viraj Jasani
ca27744770
YARN-10809. Missing dependency causing NoClassDefFoundError in TestHBaseTimelineStorageUtils (#3081)
Reviewed-by: Takanobu Asanuma <tasanuma@apache.org>
2021-06-08 19:37:23 -07:00
Ayush Saxena
a1a3184171
HDFS-15916. Addendum. DistCp: Backward compatibility: Distcp fails from Hadoop 3 to Hadoop 2 for snapshotdiff. (#3056) 2021-06-08 19:34:45 -07:00
Viraj Jasani
00d372b663
HADOOP-17725. Improve error message for token providers in ABFS (#3041)
Contributed by Viraj Jasani.
2021-06-08 22:03:03 +01:00
Steve Loughran
762a83e044
HADOOP-17631. Configuration ${env.VAR:-FALLBACK} to eval FALLBACK when restrictSystemProps=true (#2977)
Contributed by Steve Loughran.
2021-06-08 21:56:40 +01:00
Ahmed Hussein
a2a0283c7b
HDFS-16042. DatanodeAdminMonitor scan should be delay based (#3058) 2021-06-08 11:09:31 -05:00
Prabhu Josephraj
9445abb500 YARN-10792. Set Completed AppAttempt LogsLink to Log Server URL. Contributed by Abhinaba Sarkar 2021-06-08 20:37:40 +05:30
zhuqi-lucas
ec16b1d3b9 YARN-10807. Parents node labels are incorrectly added to child queues in weight mode. Contributed by Benjamin Teke. 2021-06-08 21:03:43 +08:00
litao
c748fce17a
HDFS-16048. RBF: Print network topology on the router web (#3062)
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: Hemanth Boyina <hemanthboyina@apache.org>
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
2021-06-08 16:14:06 +09:00
Gautham B A
1c0b2edde9
HADOOP-17727. Modularize docker images (#3043)
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
2021-06-07 20:11:08 -07:00
Akira Ajisaka
57a3613e5d
HDFS-16050. Some dynamometer tests fail. (#3079)
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2021-06-07 14:37:30 +09:00
Viraj Jasani
f4b24c68e7
HADOOP-17743. Replace Guava Lists usage by Hadoop's own Lists in hadoop-common, hadoop-tools and cloud-storage projects (#3072) 2021-06-07 13:24:09 +09:00
Viraj Jasani
207c92753f
MAPREDUCE-7350. Replace Guava Lists usage by Hadoop's own Lists in hadoop-mapreduce-project (#3074) 2021-06-07 11:51:29 +09:00
Hideyuki Furue
9c7b8cf54e
Fix container-executor
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-06-06 07:05:29 +09:00
July
4a26a61ecd
HDFS-16033 Fix issue of the StatisticsDataReferenceCleaner cleanUp (#3042)
Contributed by kaifeiYi (yikf).

Signed-off-by: Mingliang Liu <liuml07@apache.org>
Reviewed-by: Steve Loughran <stevel@apache.org>
2021-06-04 13:36:09 -07:00
Szilard Nemeth
200eec8f2e YARN-10796. Capacity Scheduler: dynamic queue cannot scale out properly if its capacity is 0%. Contributed by Peter Bacsko 2021-06-03 13:27:36 +02:00