big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
25,207 Commits 2 Branches 2 Tags 264 MiB
dd6b2cebed
Commit Graph

25207 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Mehakmeet Singh
6085f09db5
HADOOP-17471. ABFS to collect IOStatistics (#2731) 
The ABFS Filesystem and its input and output streams now implement
the IOStatisticSource interface and provide IOStatistics on
their interactions with Azure Storage.

This includes the min/max/mean durations of all REST API calls.

Contributed by Mehakmeet Singh <mehakmeet.singh@cloudera.com>
 2021-04-23 10:28:31 +01:00
zhuxiangyi
dff95c5eca
HDFS-15974. RBF: Unable to display the datanode UI of the router (#2915) 
Co-authored-by: zhuxiangyi <zhuxiangyi@360.cn>
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2021-04-23 10:19:11 +09:00
Vinayakumar B
c4c0683dff
HADOOP-17505. public interface GroupMappingServiceProvider needs default impl for getGroupsSet() (#2661). Contributed by Vinayakumar B. 2021-04-22 01:02:03 +05:30
Steve Loughran
5221322b96
HADOOP-17535. ABFS: ITestAzureBlobFileSystemCheckAccess test failure if no oauth key. (#2920) 
Contributed by Steve Loughran.
 2021-04-21 16:06:06 +01:00
Viraj Jasani
acb49f2fc3
HDFS-15989. Split TestBalancer and De-flake testMaxIterationTime() (#2923) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2021-04-21 19:45:49 +09:00
Eric Badger
6cb90005a7 YARN-10723. Change CS nodes page in UI to support custom resource. Contributed by Qi Zhu 2021-04-20 17:34:49 +00:00
Steve Loughran
2dd1e04010
HADOOP-17641. ITestWasbUriAndConfiguration failing. (#2937) 
This moves the mock account name --which is required to never exist-- from
"mockAccount"  to an account name containing a static UUID.

Contributed by Steve Loughran.
 2021-04-20 15:32:01 +01:00
litao
85a3532849
HDFS-15970. Print network topology on the web (#2896) 2021-04-19 20:42:30 +09:00
Masatake Iwasaki
e66d7d91be
HADOOP-17635. Update the ubuntu version in the build instruction. (#2931) 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
 2021-04-19 16:44:16 +09:00
Viraj Jasani
9a261b0a16
HADOOP-17642. Remove appender EventCounter to avoid instantiation (#2922) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-04-17 20:47:44 +09:00
litao
af0448d37b
HDFS-15975. Use LongAdder instead of AtomicLong (#2907) 2021-04-17 16:49:09 +09:00
Eric Badger
aed13f0f42 Revert "HDFS-15971. Make mkstemp cross platform (#2898)" 
This reverts commit b088f46b68.
 2021-04-16 17:15:33 +00:00
zhangshuyan0
14816be0b1
HDFS-15963. Unreleased volume references cause an infinite loop. (#2889) Contributed by Shuyan Zhang. 
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2021-04-17 00:08:31 +08:00
Viraj Jasani
f182798695
HADOOP-17633. Bump json-smart to 2.4.2 and nimbus-jose-jwt to 9.8 due to CVEs (#2895). Contributed by Viraj Jasani. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2021-04-16 12:36:01 +05:30
Akira Ajisaka
f0241ec216
HDFS-15977. Call explicit_bzero only if it is available. (#2914) 
Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
 2021-04-16 13:26:45 +09:00
lfengnan
17be99f9f9
HDFS-15423 RBF: WebHDFS create shouldn't choose DN from all sub-clusters (#2903) 2021-04-15 16:11:47 -07:00
Vivek Ratnavel Subramanian
c821008836
HDFS-15850. Superuser actions should be reported to external enforcers (#2784) 2021-04-15 14:45:51 -07:00
Masatake Iwasaki
2717203f85
HADOOP-17569. Building native code fails on Fedora 33. (#2886) 2021-04-15 21:33:18 +09:00
Viraj Jasani
9179638017
HADOOP-17524. Remove EventCounter and Log counters from JVM Metrics (#2909) 
Reviewed-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-04-15 18:04:46 +09:00
Miklos Gergely
ba3bc53f4e
YARN-10736. Fix GetApplicationsRequest JavaDoc. Contributed by Miklos Gergely. (#2906) 2021-04-14 17:16:30 +02:00
Ayush Saxena
7d6f0ca0bd Revert "HDFS-15884. RBF: Remove unused method getCreateLocation in RouterRpcServer (#2754). Contributed by tomscut." 
This reverts commit e565b05c80.

The removed method needs to be used in HDFS-15423.
 2021-04-13 23:28:42 +05:30
Gautham B A
b088f46b68
HDFS-15971. Make mkstemp cross platform (#2898) 2021-04-13 08:54:45 -07:00
Akira Ajisaka
156ecc89be
HADOOP-17630. [JDK 15] TestPrintableString fails due to Unicode 13.0 support. (#2890) 
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
 2021-04-13 17:08:49 +09:00
Inigo Goiri
82462739f8
Revert "HDFS-15423 RBF: WebHDFS create shouldn't choose DN from all sub-clusters (#2605)" (#2900) 
This reverts commit cb3ed32fe0.
 2021-04-12 20:44:31 -07:00
billierinaldi
c1fde4fe94
HADOOP-16948. Support infinite lease dirs. (#1925) 
* HADOOP-16948. Support single writer dirs.

* HADOOP-16948. Fix findbugs and checkstyle problems.

* HADOOP-16948. Fix remaining checkstyle problems.

* HADOOP-16948. Add DurationInfo, retry policy for acquiring lease, and javadocs

* HADOOP-16948. Convert ABFS client to use an executor for lease ops

* HADOOP-16948. Fix ABFS lease test for non-HNS

* HADOOP-16948. Fix checkstyle and javadoc

* HADOOP-16948. Address review comments

* HADOOP-16948. Use daemon threads for ABFS lease ops

* HADOOP-16948. Make lease duration configurable

* HADOOP-16948. Add error messages to test assertions

* HADOOP-16948. Remove extra isSingleWriterKey call

* HADOOP-16948. Use only infinite lease duration due to cost of renewal ops

* HADOOP-16948. Remove acquire/renew/release lease methods

* HADOOP-16948. Rename single writer dirs to infinite lease dirs

* HADOOP-16948. Fix checkstyle

* HADOOP-16948. Wait for acquire lease future

* HADOOP-16948. Add unit test for acquire lease failure
 2021-04-12 19:47:59 -04:00
lfengnan
cb3ed32fe0
HDFS-15423 RBF: WebHDFS create shouldn't choose DN from all sub-clusters (#2605) 2021-04-12 12:42:33 -07:00
Akira Ajisaka
2bd810a507
HADOOP-17608. Fix TestKMS failure (#2880) 
Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
 2021-04-12 12:53:58 +09:00
dependabot[bot]
6a9a5ae5fd
Bump y18n (#2843) 
Bumps [y18n](https://github.com/yargs/y18n) from 3.2.1 to 3.2.2.
- [Release notes](https://github.com/yargs/y18n/releases)
- [Changelog](https://github.com/yargs/y18n/blob/master/CHANGELOG.md)
- [Commits](https://github.com/yargs/y18n/commits)

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-04-11 01:30:49 +09:00
Gautham B A
3148791da4
HDFS-15962. Make strcasecmp cross platform (#2883) 2021-04-09 09:01:33 -07:00
Akira Ajisaka
1448756505
MAPREDUCE-7334. TestJobEndNotifier fails. (#2877) 2021-04-08 23:53:51 -07:00
sumangala-patki
6f640abbaf
HADOOP-17576. ABFS: Disable throttling update for auth failures (#2761) 
Contributed by Sumangala Patki
 2021-04-09 09:31:23 +05:30
lichaojacobs
663ca14a76
MAPREDUCE-7329: HadoopPipes task may fail when linux kernel version change from 3.x to 4.x (#2775) 2021-04-09 11:58:53 +09:00
Viraj Jasani
56bd968fb4
HDFS-15940. Fix TestBlockRecovery2#testRaceBetweenReplicaRecoveryAndFinalizeBlock (ADDENDUM) (#2874) 2021-04-09 11:10:18 +09:00
Eric Badger
213d3deb26 YARN-10503. Support queue capacity in terms of absolute resources with custom 
resourceType. Contributed by Qi Zhu.
 2021-04-09 00:34:15 +00:00
Gautham B A
bf66116407
HDFS-15955. Make explicit_bzero cross platform (#2875) 2021-04-08 09:44:47 -07:00
Ayush Saxena
c6539e3289
HDFS-15916. DistCp: Backward compatibility: Distcp fails from Hadoop 3 to Hadoop 2 for snapshotdiff. (#2863). Contributed by Ayush Saxena. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2021-04-08 20:49:08 +05:30
Stephen O'Donnell
4c567fcff7
HDFS-15937. Reduce memory used during datanode layout upgrade. Contributed by Stephen O'Donnell (#2838) 2021-04-08 11:59:02 +01:00
Peter Bacsko
ca9aa91d10 YARN-10564. Support Auto Queue Creation template configurations. Contributed by Andras Gyori. 2021-04-08 12:42:48 +02:00
Sungpeo Kook
46a5979805
MAPREDUCE-7270. TestHistoryViewerPrinter could be failed when the locale isn't English. (#1942) 
Contributed by Sungpeo Kook.
Signed-off-by: Mingliang Liu <liuml07@apache.org>
 2021-04-07 23:58:10 -07:00
Wei-Chiu Chuang
6040e86e99
HADOOP-17625. Update to Jetty 9.4.39. (#2870) 
Reviewed-by: cxorm <lianp964@gmail.com>
 2021-04-07 23:25:11 -07:00
Brahma Reddy Battula
ae88174c29 HADOOP-17617. Incorrect representation of RESPONSE for Get Key Version in KMS index.md.vm file. Contributed by Ravuri Sushma sree 2021-04-07 23:49:17 +05:30
Gautham B A
02b9506c5a
HDFS-15948. Fix test4tests for libhdfspp (#2873) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-04-07 19:06:25 +09:00
Szilard Nemeth
9cd69c20c4 YARN-10714. Remove dangling dynamic queues on reinitialization. Contributed by Andras Gyori 2021-04-07 11:52:21 +02:00
Wei-Chiu Chuang
dac60b8282
HADOOP-17621. hadoop-auth to remove jetty-server dependency. (#2865) 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
 2021-04-06 21:55:01 -07:00
Wei-Chiu Chuang
f608cda729
HADOOP-17624. Remove any rocksdb exclusion code. (#2867) 
Reviewed-by: Márton Elek <elek@apache.org>
Reviewed-by: Viraj Jasani <vjasani@apache.org>
 2021-04-06 21:52:23 -07:00
He Xiaoqiao
3a89471c36
HADOOP-17613. Log not flushed fully when daemon shutdown. Contributed by Renukaprasad C. 2021-04-07 12:18:23 +08:00
Gautham B A
42ddb5c6fe
HDFS-15909. Make fnmatch cross platform (#2792) 2021-04-06 09:23:21 -07:00
He Xiaoqiao
459df41997
HDFS-15942. Increase Quota initialization threads. Contributed by Stephen O'Donnell. 2021-04-06 21:22:48 +08:00
Boyina, Hemanth Kumar
e86050fae5 HADOOP-17588. CryptoInputStream#close() should be syncronized. Contributed by RenukaPrasad C 2021-04-06 18:06:10 +05:30
Gautham B A
36014b8282
HDFS-15949. Fix integer overflow (#2857) 
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-04-06 18:22:54 +09:00