big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
24,886 Commits 2 Branches 2 Tags 264 MiB
b2e54762a4
Commit Graph

9223 Commits

Author SHA1 Message Date
Wei-Chiu Chuang
b2e54762a4
HDFS-15624. fix the function of setting quota by storage type (#2377) (#2955) 
1. puts NVDIMM to the end of storage type enum to make sure compatibility.
2. adds check to make sure the software layout version is satisfied

Co-authored-by: su xu <kevinbrandon@163.com>
Co-authored-by: huangtianhua <huangtianhua223@gmail.com>
Co-authored-by: YaYun-Wang <34060507+YaYun-Wang@users.noreply.github.com>

Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Vinayakumar B <vinayakumarb@apache.org>

Change-Id: I3c58beef50730827a09b3c968e9ad637baa57d44
 2021-04-28 23:54:39 -07:00
Wei-Chiu Chuang
f41a368c14
HADOOP-11245. Update NFS gateway to use Netty4 (#2832) 
Reviewed-by: Tsz-Wo Nicholas Sze <szetszwo@apache.org>
 2021-04-28 23:43:11 -07:00
bshashikant
ef13f8ad6b
HDFS-15961. standby namenode failed to start ordered snapshot deletion is enabled while having snapshottable directories (#2881) 2021-04-27 09:15:06 +05:30
Stephen O'Donnell
605ed85c29
HDFS-15621. Datanode DirectoryScanner uses excessive memory (#2849). Contributed by Stephen O'Donnell 2021-04-26 11:00:23 +01:00
litao
b968fa0957
HDFS-15991. Add location into datanode info for NameNodeMXBean (#2933) 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2021-04-26 18:38:43 +09:00
Takanobu Asanuma
01bad0e92a HDFS-15967. Improve the log for Short Circuit Local Reads. Contributed by Bhavik Patel. 2021-04-26 13:29:28 +09:00
Brahma Reddy Battula
2621d3f15b HDFS-15566. NN restart fails after RollingUpgrade from 3.1.3/3.2.1 to 3.3.0. Contributed by Brahma Reddy Battula. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2021-04-26 11:39:50 +08:00
Wei-Chiu Chuang
90c6caf650 Revert "HDFS-15624. fix the function of setting quota by storage type (#2377)" 
This reverts commit 394b9f7a5c.

Ref: HDFS-15995.
Had to revert this commit, so we can commit HDFS-15566 (a critical bug preventing rolling upgrade to Hadoop 3.3)
Will re-work this fix again later.
 2021-04-26 11:27:15 +08:00
jianghuazhu
c486dc396c
HDFS-15978. Solve DatanodeManager#getBlockRecoveryCommand() printing IOException. (#2913) Contributed by JiangHua Zhu. 2021-04-25 14:31:51 +08:00
zhuxiangyi
dff95c5eca
HDFS-15974. RBF: Unable to display the datanode UI of the router (#2915) 
Co-authored-by: zhuxiangyi <zhuxiangyi@360.cn>
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2021-04-23 10:19:11 +09:00
Viraj Jasani
acb49f2fc3
HDFS-15989. Split TestBalancer and De-flake testMaxIterationTime() (#2923) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2021-04-21 19:45:49 +09:00
litao
85a3532849
HDFS-15970. Print network topology on the web (#2896) 2021-04-19 20:42:30 +09:00
litao
af0448d37b
HDFS-15975. Use LongAdder instead of AtomicLong (#2907) 2021-04-17 16:49:09 +09:00
Eric Badger
aed13f0f42 Revert "HDFS-15971. Make mkstemp cross platform (#2898)" 
This reverts commit b088f46b68.
 2021-04-16 17:15:33 +00:00
zhangshuyan0
14816be0b1
HDFS-15963. Unreleased volume references cause an infinite loop. (#2889) Contributed by Shuyan Zhang. 
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2021-04-17 00:08:31 +08:00
Akira Ajisaka
f0241ec216
HDFS-15977. Call explicit_bzero only if it is available. (#2914) 
Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
 2021-04-16 13:26:45 +09:00
lfengnan
17be99f9f9
HDFS-15423 RBF: WebHDFS create shouldn't choose DN from all sub-clusters (#2903) 2021-04-15 16:11:47 -07:00
Vivek Ratnavel Subramanian
c821008836
HDFS-15850. Superuser actions should be reported to external enforcers (#2784) 2021-04-15 14:45:51 -07:00
Masatake Iwasaki
2717203f85
HADOOP-17569. Building native code fails on Fedora 33. (#2886) 2021-04-15 21:33:18 +09:00
Ayush Saxena
7d6f0ca0bd Revert "HDFS-15884. RBF: Remove unused method getCreateLocation in RouterRpcServer (#2754). Contributed by tomscut." 
This reverts commit e565b05c80.

The removed method needs to be used in HDFS-15423.
 2021-04-13 23:28:42 +05:30
Gautham B A
b088f46b68
HDFS-15971. Make mkstemp cross platform (#2898) 2021-04-13 08:54:45 -07:00
Inigo Goiri
82462739f8
Revert "HDFS-15423 RBF: WebHDFS create shouldn't choose DN from all sub-clusters (#2605)" (#2900) 
This reverts commit cb3ed32fe0.
 2021-04-12 20:44:31 -07:00
lfengnan
cb3ed32fe0
HDFS-15423 RBF: WebHDFS create shouldn't choose DN from all sub-clusters (#2605) 2021-04-12 12:42:33 -07:00
Gautham B A
3148791da4
HDFS-15962. Make strcasecmp cross platform (#2883) 2021-04-09 09:01:33 -07:00
Viraj Jasani
56bd968fb4
HDFS-15940. Fix TestBlockRecovery2#testRaceBetweenReplicaRecoveryAndFinalizeBlock (ADDENDUM) (#2874) 2021-04-09 11:10:18 +09:00
Gautham B A
bf66116407
HDFS-15955. Make explicit_bzero cross platform (#2875) 2021-04-08 09:44:47 -07:00
Ayush Saxena
c6539e3289
HDFS-15916. DistCp: Backward compatibility: Distcp fails from Hadoop 3 to Hadoop 2 for snapshotdiff. (#2863). Contributed by Ayush Saxena. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2021-04-08 20:49:08 +05:30
Stephen O'Donnell
4c567fcff7
HDFS-15937. Reduce memory used during datanode layout upgrade. Contributed by Stephen O'Donnell (#2838) 2021-04-08 11:59:02 +01:00
Gautham B A
42ddb5c6fe
HDFS-15909. Make fnmatch cross platform (#2792) 2021-04-06 09:23:21 -07:00
He Xiaoqiao
459df41997
HDFS-15942. Increase Quota initialization threads. Contributed by Stephen O'Donnell. 2021-04-06 21:22:48 +08:00
Gautham B A
36014b8282
HDFS-15949. Fix integer overflow (#2857) 
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-04-06 18:22:54 +09:00
Viraj Jasani
3f2682b92b
HADOOP-17622. Avoid usage of deprecated IOUtils#cleanup API. (#2862) 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2021-04-06 13:39:10 +09:00
Viraj Jasani
de05cefbae
HDFS-15940. Fixing and refactoring tests specific to Block recovery.(#2844). Contributed by Viraj Jasani 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2021-04-06 00:19:21 +05:30
Gautham B A
6e1df1c048
HDFS-15947. Replace deprecated protobuf APIs (#2856) 2021-04-05 09:52:48 -07:00
litao
9e2d5d6d05
HDFS-15951. Remove unused parameters in NameNodeProxiesClient (#2859) 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2021-04-05 17:47:10 +09:00
Gautham B A
ca5e13fd74
HDFS-15950. Remove unused hdfs.proto import (#2858) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-04-05 11:42:05 +09:00
zhuqi
96e410a127
HDFS-15930: Fix some @param errors in DirectoryScanner. (#2829). Contributed by Qi Zhu. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2021-04-04 03:11:02 +05:30
litao
3cb76447f5
HDFS-15946. Fix java doc in FSPermissionChecker (#2855). Contributed by tomscut. 
Signed-off-by: Mingliang Liu <liuml07@apache.org>
 2021-04-02 10:37:04 -07:00
Gautham B A
70536ba1f9
HDFS-15944. Prevent truncation by snprintf (#2853) 2021-04-02 09:37:33 -07:00
Brahma Reddy Battula
ed74479ea5 HDFS-15222. Correct the "hdfs fsck -list-corruptfileblocks" command output. Contributed by Ravuri Sushma sree. 2021-04-02 09:47:20 +05:30
litao
7c83f140dc
HDFS-15938. Fix java doc in FSEditLog (#2837). Contributed by tomscut. 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
Signed-off-by: Mingliang Liu <liuml07@apache.org>
 2021-04-01 19:28:17 -07:00
litao
4bd04126d6
HDFS-15892. Add metric for editPendingQ in FSEditLogAsync (#2770) 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2021-04-02 10:33:40 +09:00
Viraj Jasani
4f28738010
HDFS-15931 : Fix non-static inner classes for better memory management (#2830). Contributed by Viraj Jasani 
Signed-off-by: Mingliang Liu <liuml07@apache.org>
 2021-04-01 16:34:31 -07:00
Brahma Reddy Battula
0665ce9930 HDFS-15494. TestReplicaCachingGetSpaceUsed#testReplicaCachingGetSpaceUsedByRBWReplica Fails on Windows. Contributed by Ravuri Sushma sree. 2021-04-01 09:19:39 +05:30
He Xiaoqiao
5eca748970
HDFS-15921. Improve the log for the Storage Policy Operations. Contributed by Bhavik Patel. 2021-04-01 11:32:03 +08:00
Borislav Iordanov
2c482fbacf HADOOP-16524. Automatic keystore reloading for HttpServer2 
Reapply of issue reverted first because it caused yarn failures and
then again because the commit message was incorrectly formatted
(and yet again because of commit message format).

Signed-off-by: stack <stack@apache.org>
 2021-03-31 10:46:35 -07:00
stack
22961a615d Revert "HADOOP-16524. Automatic keystore reloading for HttpServer2" 
This reverts commit a2975d2153.
 2021-03-31 10:43:09 -07:00
stack
a2975d2153 HADOOP-16524. Automatic keystore reloading for HttpServer2 
Reapply of issue reverted first because it caused yarn failures and
then again because the commit message was incorrectly formatted.
 2021-03-31 10:40:20 -07:00
stack
5183aaeda2 Revert "Hadoop 16524 - resubmission following some unit test fixes (#2693)" 
Revert to fix the summary message.

This reverts commit 9509bebf7f.
 2021-03-31 10:39:55 -07:00
Borislav Iordanov
9509bebf7f
Hadoop 16524 - resubmission following some unit test fixes (#2693) 
Signed-off-by: stack <stack@apache.org>
 2021-03-31 10:07:42 -07:00