Commit Graph

25631 Commits

Author SHA1 Message Date
Szilard Nemeth
d75f2db2f9 YARN-11014. YARN incorrectly validates maximum capacity resources on the validation API. Contributed by Benjamin Teke 2021-12-07 08:42:57 +01:00
Ashutosh Gupta
5a950b8900
YARN-9063. ATS 1.5 fails to start if RollingLevelDb files are corrupt or missing (#3728)
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-12-06 19:45:44 +09:00
GuoPhilipse
1509ea5de2
HDFS-16338. Correct fsimage error configuration message (#3684). Contributed by guophilipse.
Reviewed-by: cxorm <lianp964@gmail.com>
Reviewed-by: tomscut <litao@bigo.sg>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2021-12-06 14:00:50 +05:30
Takanobu Asanuma
e8e69de106 HDFS-16293. Client sleeps and holds 'dataQueue' when DataNodes are congested. Contributed by Yuanxin Zhu. 2021-12-06 10:44:36 +09:00
litao
c2afb6a00b
HDFS-16370. Fix assert message for BlockInfo (#3747). Contributed by tomscut.
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2021-12-04 22:54:44 +05:30
GuoPhilipse
2f55a6cced
HDFS-16357. Fix log format in DFSUtilClient (#3729). Contributed by guophilipse.
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2021-12-04 18:35:05 +05:30
Ayush Saxena
cab7086fbc
HDFS-16369. RBF: Fix the retry logic of RouterRpcServer#invokeAtAvailableNs. (#3745). Contributed by Ayush Saxena.
Reviewed-by: litao <tomleescut@gmail.com>
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
2021-12-04 10:54:29 +05:30
huhaiyang
c0f405a46b
HDFS-16314. Support to make dfs.namenode.block-placement-policy.exclude-slow-nodes.enabled reconfigurable (#3664)
Reviewed-by: Fei Hui <feihui.ustc@gmail.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-12-04 02:19:36 +09:00
Andras Gyori
47ea0d734f
HADOOP-18021. Provide a public wrapper of Configuration#substituteVars (#3710)
Contributed by Andras Gyori
2021-12-03 16:44:58 +00:00
bitterfox
dd6b987c93
HDFS-16332. Handle invalid token exception in sasl handshake (#3677)
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-12-03 23:30:13 +09:00
wangzhaohui
0cb6c28d19
HDFS-16364. Remove unnecessary brackets in NameNodeRpcServer#L453 (#3742) 2021-12-03 16:51:04 +05:30
litao
52ec65fd10
HDFS-16331. Make dfs.blockreport.intervalMsec reconfigurable (#3676)
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2021-12-03 14:12:05 +09:00
Viraj Jasani
0c62a514f7
HDFS-16358. HttpFS implementation for getSnapshotDiffReportListing (#3730) 2021-12-03 13:25:32 +09:00
litao
196935a8d2
HDFS-16359. RBF: RouterRpcServer#invokeAtAvailableNs does not take effect when retrying (#3731). Contributed by tomscut.
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2021-12-02 19:10:35 +05:30
Viraj Jasani
b34dcb5b3a
HADOOP-18025. Upgrade HBase version to 1.7.1 for hbase1 profile (#3722) 2021-12-02 10:40:30 +08:00
Desmond Sisson
df4197592f
HADOOP-18029: Update CompressionCodecFactory to handle uppercase file extensions (#3739)
Co-authored-by: Desmond Sisson <sissonde@amazon.com>
2021-12-01 15:36:54 -08:00
Viraj Jasani
4dea4a7b67
HADOOP-18027. Include static imports in the maven plugin rules (#3733)
Reviewed-by: Steve Loughran <stevel@apache.org>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2021-12-01 16:48:43 +09:00
litao
1ebdac2cd6
HDFS-16361. Fix log format for QueryCommand (#3732). Contributed by tomscut. 2021-12-01 12:48:21 +05:30
Sushmasree-28
7b840f2693
YARN-10975 EntityGroupFSTimelineStore#ActiveLogParser parses already processed files (#3735)
Contributed by  sravuri <sravuri@microsoft.com> and reviewed by Prabhu Joseph
2021-11-30 00:00:25 +05:30
GuoPhilipse
c65c87f211
HADOOP-18026. Fix default value of Magic committer (#3723)
Contributed by guophilipse
2021-11-29 15:50:30 +00:00
smarthan
932a78fe38
HADOOP-18023. Allow cp command to run with multi threads. (#3721) 2021-11-29 12:45:08 +00:00
litao
829af89dc4
HDFS-16344. Improve DirectoryScanner.Stats#toString (#3695)
Reviewed-by: Viraj Jasani <vjasani@apache.org>
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Reviewed-by: Hui Fei <ferhui@apache.org>
2021-11-29 17:48:10 +09:00
Viraj Jasani
215388beea
HADOOP-18022. Add restrict-imports-enforcer-rule for Guava Preconditions and remove remaining usages (#3712)
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2021-11-29 17:37:30 +09:00
litao
86705eda82
HDFS-16339. Show the threshold when mover threads quota is exceeded (#3689)
Reviewed-by: Viraj Jasani <vjasani@apache.org>
Reviewed-by: Hui Fei <ferhui@apache.org>
2021-11-26 19:52:14 +09:00
GuoPhilipse
c9d64bad37
YARN-11007. Correct words in YARN documents (#3680)
Reviewed-by: cxorm <lianp964@gmail.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-11-25 17:48:54 +09:00
Takanobu Asanuma
9c887e5b82
HADOOP-18014. CallerContext should not include some characters. (#3698)
Reviewed-by: Viraj Jasani <vjasani@apache.org>
Reviewed-by: Mingliang Liu <liuml07@apache.org>
Reviewed-by: Hui Fei <ferhui@apache.org>
2021-11-25 14:05:04 +09:00
Viraj Jasani
cdc13e91b6
HDFS-16350. Datanode start time should be set after RPC server starts successfully (#3711) 2021-11-25 10:23:04 +08:00
huhaiyang
99b161dec7
HADOOP-17995. Stale record should be remove when DataNodePeerMetrics#dumpSendPacketDownstreamAvgInfoAsJson (#3708) 2021-11-25 10:20:42 +08:00
Steve Loughran
98fe0d0fc3
HADOOP-17979. Add Interface EtagSource to allow FileStatus subclasses to provide etags (#3633)
Contributed by Steve Loughran
2021-11-24 17:33:12 +00:00
Viraj Jasani
e8566b3812
HDFS-16336. De-flake TestRollingUpgrade#testRollback (#3686)
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-11-24 22:54:11 +09:00
Istvan Fajth
ae3ba45db5
HADOOP-17975. Fallback to simple auth does not work for a secondary DistributedFileSystem instance. (#3579) 2021-11-24 10:44:57 +00:00
Ashutosh Gupta
08f3df3ea2
YARN-10991. Fix to ignore the grouping "[]" for resourcesStr in parseResourcesString method (#3592)
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-11-24 18:41:22 +09:00
Mehakmeet Singh
a35f7dec25
HADOOP-18016. Make certain methods LimitedPrivate in S3AUtils.java (#3685)
Contributed By: Mehakmeet Singh
2021-11-24 13:32:59 +05:30
Andrew Chung
d3f0b7eab7
YARN-10760. Number of allocated OPPORTUNISTIC containers can dip below 0 (#3642) 2021-11-23 13:21:51 -08:00
Andrew Chung
5b1b2c8ef6
YARN-11003. Make RMNode aware of all (OContainer inclusive) allocated resources (#3646) 2021-11-23 13:20:08 -08:00
Mukul Kumar Singh
faa4eeacb1
HDFS-16343. Add some debug logs when the dfsUsed are not used during Datanode startup. (#3694) 2021-11-23 13:36:14 +00:00
Viraj Jasani
c0bdba8fac
HADOOP-18017. unguava: remove Preconditions from hadoop-yarn-project modules (#3687) 2021-11-23 13:36:22 +09:00
Viraj Jasani
c7ec1897c4
HADOOP-18018. unguava: remove Preconditions from hadoop-tools modules (#3688) 2021-11-23 13:34:10 +09:00
litao
0ed817babb
HDFS-16337. Show start time of Datanode on Web (#3682)
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
2021-11-23 10:18:30 +08:00
smarthan
63018dc73f
HADOOP-17998. Allow get command to run with multi threads. (#3645) 2021-11-22 11:37:05 +00:00
Symious
c88640c4ad
HDFS-16320. Datanode retrieve slownode information from NameNode (#3654) 2021-11-22 18:04:32 +08:00
litao
20b78c8f97
HDFS-16335. Fix HDFSCommands.md (#3681) 2021-11-22 17:07:52 +09:00
Hui Fei
8e9ad2d7af
Revert "HADOOP-17995. Stale record should be remove when DataNodePeerMetrics#dumpSendPacketDownstreamAvgInfoAsJson (#3630)" (#3697) 2021-11-22 10:05:21 +08:00
GuoPhilipse
3d1ca6f6dc
HDFS-16334. Correct NameNode ACL description (#3678)
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-11-22 07:41:57 +09:00
litao
965e3874e9
HDFS-16326. Simplify the code for DiskBalancer (#3663)
Reviewed-by: Viraj Jasani <vjasani@apache.org>
2021-11-19 11:37:10 +09:00
GuoPhilipse
0c1eb0125b
HDFS-16328. Correct disk balancer desc (#3669)
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-11-19 01:02:01 +09:00
Viraj Jasani
38614d05e1
HDFS-16330. Fix incorrect placeholder for Exception logs in DiskBalancer (#3672)
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-11-19 00:55:39 +09:00
litao
dc751df63b
HDFS-16329. Fix log format for BlockManager (#3670)
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-11-19 00:42:01 +09:00
litao
5b05068fdc
HDFS-16310. RBF: Add client port to CallerContext for Router (#3635) 2021-11-18 13:46:35 +09:00
huhaiyang
91af256a5b
HADOOP-17995. Stale record should be remove when DataNodePeerMetrics#dumpSendPacketDownstreamAvgInfoAsJson (#3630) 2021-11-17 22:41:06 +08:00