big-data/hadoop
big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Projects Wiki Activity
24,188 Commits 2 Branches 2 Tags
Commit Graph

5909 Commits

Author SHA1 Message Date
Ayush Saxena
b5698e0c33 HDFS-15176. Enable GcTimePercentage Metric in NameNode's JvmMetrics. Contributed by Jinglun. 2020-02-24 00:07:18 +05:30
Ayush Saxena
9eb7a8bdf8 HDFS-15041. Make MAX_LOCK_HOLD_MS and full queue size configurable. Contributed by zhuqi. 2020-02-23 14:25:39 +05:30
Konstantin V Shvachko
ecbcb058b8 HDFS-14731. [FGL] Remove redundant locking on NameNode. Contributed by Konstantin V Shvachko. 2020-02-21 17:53:37 -08:00
Konstantin V Shvachko
6f84269bcd HDFS-15185. StartupProgress reports edits segments until the entire startup completes. Contributed by Konstantin V Shvachko. 2020-02-21 10:51:14 -08:00
Masatake Iwasaki
2338d25dc7
HDFS-15052. WebHDFS getTrashRoot leads to OOM due to FileSystem object creation. (#1758) 2020-02-21 11:56:07 +09:00
Inigo Goiri
ec7507162c HDFS-15165. In Du missed calling getAttributesProvider. Contributed by Bharat Viswanadham. 2020-02-19 11:33:58 -08:00
Ayush Saxena
ac4b556e2d HDFS-13739. Add option to disable rack local write preference. Contributed by Ayush Saxena. 2020-02-19 08:20:59 +05:30
Surendra Singh Lilhore
810783d443 HDFS-15135. EC : ArrayIndexOutOfBoundsException in BlockRecoveryWorker#RecoveryTaskStriped. Contributed by Ravuri Sushma sree. 2020-02-16 12:15:03 +05:30
Surendra Singh Lilhore
a98352ced1 HDFS-15086. Block scheduled counter never get decremet if the block got deleted before replication. Contributed by hemanthboyina. 2020-02-13 16:57:41 +05:30
Akira Ajisaka
0ddb5f0881
HDFS-13989. RBF: Add FSCK to the Router (#1832) 
Co-authored-by: Inigo Goiri <inigoiri@apache.org>
 2020-02-13 10:06:07 +09:00
Kihwal Lee
9b8a78d97b HDFS-14758. Make lease hard limit configurable and reduce the default. 
Contributed by hemanthboyina.
 2020-02-11 12:40:00 -06:00
Stephen O'Donnell
d7c136b9ed HDFS-15150. Introduce read write lock to Datanode. Contributed Stephen O'Donnell. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2020-02-11 08:00:15 -08:00
Ayush Saxena
6191d4b4a0 HDFS-15158. The number of failed volumes mismatch with volumeFailures of Datanode metrics. Contributed by Yang Yun. 2020-02-09 23:32:22 +05:30
Ayush Saxena
d23317b102 HDFS-15115. Namenode crash caused by NPE in BlockPlacementPolicyDefault when dynamically change logger to debug. Contributed by wangzhixiang 2020-02-08 10:33:57 +05:30
Vinayakumar B
7dac7e1d13
HADOOP-16596. [pb-upgrade] Use shaded protobuf classes from hadoop-thirdparty dependency (#1635). Contributed by Vinayakumar B. 2020-02-07 14:51:24 +05:30
Chen Liang
ce7b8b5634 HDFS-15148. dfs.namenode.send.qop.enabled should not apply to primary NN port. Contributed by Chen Liang. 2020-02-04 12:12:35 -08:00
Stephen O'Donnell
1e3a0b0d93 HDFS-7175. Client-side SocketTimeoutException during Fsck. Contributed by Stephen O'Donnell, Akira Ajisaka. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Co-authored-by: Akira Ajisaka <aajisaka@apache.org>
 2020-01-31 16:13:02 -08:00
Inigo Goiri
1839c467f6 HDFS-13179. TestLazyPersistReplicaRecovery#testDnRestartWithSavedReplicas fails intermittently. Contributed by Ahmed Hussein. 2020-01-28 10:10:35 -08:00
Ayush Saxena
87c198468b HDFS-14993. checkDiskError doesn't work during datanode startup. Contributed by Yang Yun. 2020-01-28 23:09:38 +05:30
Kihwal Lee
d10f77e3c9 HDFS-15119. Allow expiration of cached locations in DFSInputStream. 
Contributed by Ahmed Hussein.
 2020-01-24 09:15:27 -06:00
Ayush Saxena
92c58901d7 HDFS-15117. EC: Add getECTopologyResultForPolicies to DistributedFileSystem. Contributed by Ayush Saxena 2020-01-23 18:18:34 +05:30
Kihwal Lee
bd03053ea2 HDFS-14968. Add ability to log stale datanodes. Contributed by Ahmed Hussein. 2020-01-22 09:14:38 -06:00
Akira Ajisaka
1defe3a65a
HADOOP-16753. Refactor HAAdmin. Contributed by Xieming Li. 2020-01-21 10:58:32 +09:00
Chao Sun
d7c4f8ab21 HDFS-13616. Batch listing of multiple directories (#1725) 2020-01-15 17:22:47 -08:00
Ayush Saxena
cebce0a348 HDFS-14578. AvailableSpaceBlockPlacementPolicy always prefers local node. Contributed by Ayush Saxena. 2020-01-11 10:29:11 +05:30
Chen Liang
26a969ec73 HDFS-15099. [SBN Read] checkOperation(WRITE) should throw ObserverRetryOnActiveException for ObserverNode. Contributed by Chen Liang. 2020-01-10 17:06:59 -08:00
Ayush Saxena
b32757c616 HDFS-15107. dfs.client.server-defaults.validity.period.ms to support time units. Contributed by Ayush Saxena. 2020-01-10 08:14:56 +05:30
Surendra Singh Lilhore
bf45f3b80a HDFS-14957. INodeReference Space Consumed was not same in QuotaUsage and ContentSummary. Contributed by hemanthboyina. 2020-01-09 12:04:05 +05:30
Rakesh Radhakrishnan
7030722e5d HDFS-15080. Fix the issue in reading persistent memory cached data with an offset. Contributed by Feilong He. 2020-01-08 14:25:17 +05:30
Masatake Iwasaki
a43c177f1d HDFS-15072. HDFS MiniCluster fails to start when run in directory path with a %. (#1775) 2020-01-08 11:28:34 +09:00
Ayush Saxena
f8644fbe9f HDFS-15091. Cache Admin and Quota Commands Should Check SuperUser Before Taking Lock. Contributed by Ayush Saxena. 2020-01-04 19:02:59 +05:30
Masatake Iwasaki
037ec8cfb1 HDFS-15068. DataNode could meet deadlock if invoke refreshVolumes when register. Contributed by Aiphago. 
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
 2020-01-04 01:55:36 +09:00
Rakesh Radhakrishnan
d79cce20ab HDFS-14740. Recover data blocks from persistent memory read cache during datanode restarts. Contributed by Feilong He. 2020-01-02 11:44:00 +05:30
Takanobu Asanuma
074050ca59 HDFS-15063. HttpFS: getFileStatus doesn't return ecPolicy. Contributed by hemanthboyina. 2020-01-01 11:26:38 +09:00
Surendra Singh Lilhore
ee51eadda0 HDFS-15074. DataNode.DataTransfer thread should catch all the expception and log it. Contributed by hemanthboyina. 2019-12-29 11:15:54 +05:30
Takanobu Asanuma
dc32f583af HDFS-14934. [SBN Read] Standby NN throws many InterruptedExceptions when dfs.ha.tail-edits.period is 0. Contributed by Ayush Saxena. 2019-12-28 21:32:15 +09:00
Ayush Saxena
80f91d14ab HDFS-14997. Addendum: BPServiceActor processes commands from NameNode asynchronously. Contributed by Xiaoqiao He. 2019-12-27 09:12:44 +05:30
Ayush Saxena
300505c562 HDFS-15054. Delete Snapshot not updating new modification time. Contributed by hemanthboyina. 2019-12-25 12:42:50 +05:30
Inigo Goiri
52d7b745c6 HDFS-15062. Add LOG when sendIBRs failed. Contributed by Fei Hui. 2019-12-19 09:42:17 -08:00
Inigo Goiri
b86895485d HDFS-14997. BPServiceActor processes commands from NameNode asynchronously. Contributed by Xiaoqiao He. 2019-12-19 09:34:43 -08:00
Shashikant Banerjee
fdd96e46d1 HDFS-15012. NN fails to parse Edit logs after applying HDFS-13101. Contributed by Shashikant Banerjee. 2019-12-18 22:50:46 +05:30
Inigo Goiri
24080666e5 HDFS-14908. LeaseManager should check parent-child relationship when filter open files. Contributed by Jinglun. 2019-12-16 18:41:45 -08:00
Chen Liang
65c4660bcd HDFS-15036. Active NameNode should not silently fail the image transfer. Contributed by Chen Liang. 2019-12-12 10:22:05 -08:00
Stephen O'Donnell
c93cb6790e HDFS-14854. Create improved decommission monitor implementation. Contributed by Stephen O'Donnell. 
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-12-10 17:16:43 -08:00
Masatake Iwasaki
11cd5b6e39 HDFS-15028. Keep the capacity of volume and reduce a system call. Contributed by Yang Yun. 
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
 2019-12-07 08:46:45 +09:00
Wei-Chiu Chuang
ecd461f940 HDFS-14751. Synchronize on diffs in DirectoryScanner. Contributed by Lisheng Sun. 2019-12-06 15:10:40 -08:00
Wei-Chiu Chuang
313b76f8e9 HDFS-14476. lock too long when fix inconsistent blocks between disk and in-memory. Contributed by Sean Chow. 2019-12-06 15:10:40 -08:00
Ayush Saxena
6b2d6d4aaf HDFS-15009. FSCK -list-corruptfileblocks return Invalid Entries. Contributed by hemanthboyina. 2019-11-29 23:55:37 +05:30
Surendra Singh Lilhore
44f7b9159d HDFS-15013. Reduce NameNode overview tab response time. Contributed by HuangTao. 2019-11-29 00:42:31 +05:30
Surendra Singh Lilhore
0384687811 HDFS-15010. BlockPoolSlice#addReplicaThreadPool static pool should be initialized by static method. Contributed by Surendra Singh Lilhore. 2019-11-28 23:49:35 +05:30