big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
24,746 Commits 2 Branches 2 Tags 264 MiB
7b51e6824d
Commit Graph

24746 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Wei-Chiu Chuang
0df4743284
HADOOP-17703. checkcompatibility.py errors out when specifying annotations. (#3017) 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 110cda3de6)
 2021-05-18 11:36:13 +08:00
Wei-Chiu Chuang
4ffe5eb1dd
HADOOP-17669. Backport HADOOP-17079, HADOOP-17505 to branch-3.3 (#2959) 
* HADOOP-17079. Optimize UGI#getGroups by adding UGI#getGroupsSet.

Co-authored-by: Wei-Chiu Chuang <weichiu@apache.org>
Change-Id: I0f31409923ece24a82dfba4c4610d8a38c52d9fb

* HADOOP-17505. public interface GroupMappingServiceProvider needs default impl for getGroupsSet() (#2661). Contributed by Vinayakumar B.

(cherry picked from commit c4c0683dff)

Co-authored-by: Xiaoyu Yao <xyao@apache.org>
Co-authored-by: Vinayakumar B <vinayakumarb@apache.org>
 2021-05-17 18:57:46 -07:00
lujiefsi
137e20cc9b
YARN-10555. Missing access check before getAppAttempts (#2608) 
Co-authored-by: lujie <lujie@foxmail.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit d92a25b790)
 2021-05-17 13:53:27 +09:00
litao
2f56e2b54d
HDFS-15873. Add namenode address in logs for block report (#2743) 
(cherry picked from commit 6a922f9896)
 2021-05-17 12:29:11 +08:00
Xiaoyu Yao
3f9c9ccf46
HADOOP-17284. Support BCFKS keystores for Hadoop Credential Provider.… (#3010) 
* HADOOP-17284. Support BCFKS keystores for Hadoop Credential Provider. (#2334)

(cherry picked from commit 4c5ad57818)
 2021-05-13 16:57:58 -07:00
Wei-Chiu Chuang
1b8eb2654a
HADOOP-16822. Provide source artifacts for hadoop-client-api. Contributed by Karel Kolman. 
(cherry picked from commit 2c4ab72a60)
 2021-05-13 20:30:40 +08:00
Aryan Gupta
7142dfec04
HADOOP-17302. Upgrade to jQuery 3.5.1 in hadoop-sls. (#2379) 
Co-authored-by: Aryan Gupta
(cherry picked from commit d60d5fe43d)
 2021-05-13 12:16:24 +08:00
Aryan Gupta
5f147112cf
HADOOP-17286. Updated jQuery in yarn-common. (#2340) 
(cherry picked from commit bbbfa7d415)
 2021-05-13 12:16:20 +08:00
Aryan Gupta
28079e9c30
HADOOP-17283. Hadoop - Upgrade to jQuery 3.5.1 (#2330) 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
(cherry picked from commit 486ddb73f9)
 2021-05-13 12:16:17 +08:00
Ayush Saxena
41b39ff292
HDFS-15245. Improve JournalNode web UI. Contributed by Jianfei Jiang. 
(cherry picked from commit 960c9ebaea)

Conflicts:
    hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/qjournal/server/JournalNode.java
 2021-05-13 12:14:09 +08:00
bilaharith
1c6c3920b6
HADOOP-17444. ADLS Gen1: Update adls SDK to 2.3.9 (#2842) 
Contributed by bilaharith
 2021-05-12 18:15:27 +01:00
dependabot[bot]
b2897fdd66
HADOOP-17683. Update commons-io to 2.8.0 (#2974) 
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Akira Ajisaka <aajisaka@apache.org>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 29105ffb63)
 2021-05-12 10:58:39 +09:00
Peter Bacsko
051a5068dd YARN-9615. Add dispatcher metrics to RM. Contributed by Jonathan Hung and Qi Zhu. 2021-05-11 19:23:45 +02:00
Viraj Jasani
3176ab028b
HDFS-16007. Deserialization of ReplicaState should avoid throwing ArrayIndexOutOfBoundsException (#2982) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit b944084b32)
 2021-05-11 12:39:42 +09:00
Borislav Iordanov
f5ef78d46f HADOOP-17665 Ignore missing keystore configuration in reloading mechanism 2021-05-10 15:41:56 -05:00
Ayush Saxena
43db94c7a3 HDFS-16003. ProcessReport print invalidatedBlocks should judge debug level at first. Contributed by lei w. 2021-05-10 17:11:09 +05:30
sumangala-patki
b20bc668d5
HADOOP-17548. ABFS: Toggle Store Mkdirs request overwrite parameter (#2729) (#2781) 
Contributed by Sumangala Patki.

(cherry picked from commit fe633d4739)
 2021-05-10 11:50:01 +05:30
Ayush Saxena
2a95ba29bb HADOOP-17644. Add back the exceptions removed by HADOOP-17432 for compatibility. Contributed by Quan Li. 2021-05-09 04:00:30 +05:30
Akira Ajisaka
1326cad66a
HDFS-16001. TestOfflineEditsViewer.testStored() fails reading negative value of FSEditsLogOpCodes (branch-3.3) (#2989). Contributed by Akira Ajisaka. 2021-05-08 10:48:56 +08:00
Szilard Nemeth
3d715c2e4c
YARN-9333. TestFairSchedulerPreemption.testRelaxLocalityPreemptionWithNoLessAMInRemainingNodes fails intermittently. Contributed by Peter Bacsko 
(cherry picked from commit eacbe07b56)
 2021-05-07 14:32:17 +09:00
Szilard Nemeth
3303aa5947
YARN-10515. Fix flaky test TestCapacitySchedulerAutoQueueCreation.testDynamicAutoQueueCreationWithTags. Contributed by Peter Bacsko 
(cherry picked from commit 8620984b8d)
 2021-05-07 14:26:58 +09:00
Mike
0f12f3e125
HADOOP-17036. TestFTPFileSystem failing as ftp server dir already exists. 
Contributed by Mikhail Pryakhin.

(cherry picked from commit 017d24e970)
 2021-05-07 14:20:29 +09:00
Takanobu Asanuma
df50e210dd
HADOOP-17375. Fix the error of TestDynamometerInfra. (#2471) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 207210263a)
 2021-05-07 14:09:03 +09:00
hchaverr
cedebf1c27 HADOOP-17680. Allow ProtobufRpcEngine to be extensible (#2905) Contributed by Hector Chaverri. 
(cherry picked from commit f40e3eb059)
 2021-05-06 17:50:32 -07:00
Akira Ajisaka
217655269a YARN-10756. Remove additional junit 4.11 dependency from javadoc. (#2960) 
Reviewed-by: Viraj Jasani <vjasani@apache.org>
Reviewed-by: Bilwa S T <stbilwa@gmail.com>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
(cherry picked from commit 7cb7b8adc1)
 2021-05-06 23:47:24 +09:00
Viraj Jasani
be34c1222a HADOOP-11616. Remove workaround for Curator's ChildReaper requiring Guava 15+ (#2973) 
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit b93e448f9a)
 2021-05-06 04:53:06 +09:00
Takanobu Asanuma
65bf544118 HADOOP-16954. Add -S option in "Count" command to show only Snapshot Counts. Contributed by hemanthboyina. 
(cherry picked from commit b89d875f7b)
 2021-05-04 17:44:34 +01:00
Steve Loughran
e944cc0338
HADOOP-16742. NullPointerException in S3A MultiObjectDeleteSupport 
Contributed by Tor Arvid Lund.

Change-Id: Iadfe9b2f355cf373031075bfbe681705a2c65bdc
 2021-05-04 16:15:13 +01:00
Istvan Fajth
00ba886a22 HADOOP-17675. LdapGroupsMapping$LdapSslSocketFactory ClassNotFoundException (#2965) 
(cherry picked from commit 0d78d73973)
 2021-05-04 11:36:15 +01:00
Karthik Palanisamy
00c3f3c42f HDFS-15865. Interrupt DataStreamer thread if no ack (#2728) 
(cherry picked from commit bd3da73a0f)
 2021-05-04 11:11:36 +01:00
kishendas
98aa4fc32c
HADOOP-17657: implement StreamCapabilities in SequenceFile.Writer and fall back to flush, if hflush is not supported (#2949) 
Co-authored-by: Kishen Das <kishen@cloudera.com>
Reviewed-by: Steve Loughran <stevel@apache.org>
(cherry picked from commit e571025f5b)
 2021-05-04 16:35:00 +08:00
Viraj Jasani
658bb49e66
HADOOP-17676. Restrict imports from org.apache.curator.shaded (#2969) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 62bcc79a2c)
 2021-05-04 01:11:30 +09:00
Akira Ajisaka
25244c4eec
HDFS-15952. TestRouterRpcMultiDestination#testProxyGetTransactionID and testProxyVersionRequest are flaky. (#2970) 
(cherry picked from commit 7c561e2a02)
 2021-05-04 00:26:34 +09:00
lfengnan
43fac739bb
HDFS-15810. RBF: RBFMetrics's TotalCapacity out of bounds (#2910) 
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 6e525ab81c)
 2021-05-02 19:19:55 +09:00
Wei-Chiu Chuang
670205c541
HADOOP-17653. Do not use guava's Files.createTempDir(). (#2945) 
Reviewed-by: Steve Loughran <stevel@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit f1e1809029)
 2021-05-02 11:12:37 +09:00
Hui Fei
9aa6106689 HDFS-15652. Make block size from NNThroughputBenchmark configurable (#2416) 
(cherry picked from commit 8a6d5b9151)
 2021-04-29 12:10:04 -07:00
Eric Badger
da2ebfa8a9 YARN-10707. Support custom resources in ResourceUtilization, and update Node GPU Utilization to use. Contributed by Qi Zhu 
(cherry picked from commit 803ac4b1a0)
 2021-04-29 17:52:03 +00:00
litao
0e9042fed5 HDFS-15991. Add location into datanode info for NameNodeMXBean (#2933) 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
(cherry picked from commit b968fa0957)
 2021-04-28 18:52:05 +09:00
Hui Fei
842e05ba53 HDFS-15698. Fix the typo of dfshealth.html after HDFS-15358 (#2495) 
(cherry picked from commit 68442b48c8)
 2021-04-28 18:51:50 +09:00
Ayush Saxena
f809b863fe HDFS-15358. RBF: Unify router datanode UI with namenode datanode UI. Contributed by Ayush Saxena. 
(cherry picked from commit 6e416a83d1)
 2021-04-28 18:51:43 +09:00
Ayush Saxena
3c7433f2a1 HDFS-15247. RBF: Provide Non DFS Used per DataNode in DataNode UI. Contributed by Lisheng Sun. 
(cherry picked from commit 8d49229c37)
 2021-04-28 18:51:37 +09:00
Ayush Saxena
187b4a63a4 HDFS-15378. TestReconstructStripedFile#testErasureCodingWorkerXmitsWeight is failing on trunk. Contributed by hemanthboyina. 
(cherry picked from commit 8db38c98a6)
 2021-04-27 23:18:12 +09:00
litao
f45365f201
HDFS-15975. Use LongAdder instead of AtomicLong for branch-3.3 (#2940) 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2021-04-27 22:39:25 +09:00
bilaharith
6649e5888b
HADOOP-17536. ABFS: Supporting customer provided encryption key (#2707) 
Contributed by bilahari t h

Change-Id: I86216e755b81e9d14f5e87844d9fd58e8940560c
 2021-04-27 13:16:33 +01:00
Wei-Chiu Chuang
99e4e9bad6
HADOOP-17661. mvn versions:set fails to parse pom.xml. (#2956) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>

Change-Id: I7de255cb9a167e3a5e25ae339b050e82cb4b7c30
(cherry picked from commit 64264d726d)
 2021-04-27 10:43:18 +09:00
Wei-Chiu Chuang
df29aaaf80
YARN-10752. Shaded guava not found when compiling with profile hbase2.0. (#2957) 
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>

Change-Id: Ic9943d18d4086b075ceed7924dcfa566653a16cb
 2021-04-27 00:41:42 +09:00
Stephen O'Donnell
f6efb58b07 HDFS-15621. Datanode DirectoryScanner uses excessive memory (#2849). Contributed by Stephen O'Donnell 
(cherry picked from commit 605ed85c29)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsDatasetImpl.java
 2021-04-26 11:23:56 +01:00
Wei-Chiu Chuang
1425512032 Revert "HADOOP-17661. mvn versions:set fails to parse pom.xml. (#2952)" 
This reverts commit d525e1e33d.
 2021-04-26 16:42:44 +08:00
Wei-Chiu Chuang
d525e1e33d HADOOP-17661. mvn versions:set fails to parse pom.xml. (#2952) 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
Reviewed-by: Hui Fei <ferhui@apache.org>
(cherry picked from commit 66dda86fec)
 2021-04-26 14:07:35 +08:00
Takanobu Asanuma
a5f038b3f6 HDFS-15967. Improve the log for Short Circuit Local Reads. Contributed by Bhavik Patel. 
(cherry picked from commit 01bad0e92a)
 2021-04-26 13:31:51 +09:00