Inigo Goiri
|
3f223bebfa
|
HDFS-14844. Make buffer of BlockReaderRemote#newBlockReader#BufferedOutputStream configurable. Contributed by Lisheng Sun.
|
2019-09-20 11:45:01 -07:00 |
|
Vinayakumar B
|
1654497f98
|
HADOOP-16557. [pb-upgrade] Upgrade protobuf.version to 3.7.1 (#1432)
HADOOP-16557. [pb-upgrade] Upgrade protobuf.version to 3.7.1. Contributed by Vinayakumar B.
|
2019-09-20 16:08:30 +05:30 |
|
Chen Liang
|
4ed0aefe9f
|
HDFS-14822. [SBN read] Revisit GlobalStateIdContext locking when getting server state id. Contributed by Chen Liang.
|
2019-09-18 16:47:46 -07:00 |
|
Wei-Chiu Chuang
|
5dd859a8a0
|
HDFS-14836. FileIoProvider should not increase FileIoErrors metric in datanode volume metric. Contributed by Aiphago.
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
|
2019-09-18 09:34:01 -07:00 |
|
Inigo Goiri
|
f580a87079
|
HDFS-14795. Add Throttler for writing block. Contributed by Lisheng Sun.
|
2019-09-17 14:55:34 -07:00 |
|
Ayush Saxena
|
e04b8a46c3
|
HDFS-14799. Do Not Call Map containsKey In Conjunction with get. Contributed by hemanthboyina.
|
2019-09-14 10:39:20 +05:30 |
|
Stephen O'Donnell
|
06ad6540b3
|
HDFS-14821. Make HDFS-14617 (fsimage sub-sections) off by default. Contributed by Stephen O'Donnell.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
|
2019-09-13 10:36:45 -07:00 |
|
Surendra Singh Lilhore
|
4852a90e4b
|
HDFS-14754. Erasure Coding : The number of Under-Replicated Blocks never reduced. Contributed by hemanthboyina.
|
2019-09-13 00:17:13 +05:30 |
|
Surendra Singh Lilhore
|
2ff2a7f612
|
HDFS-14798. Synchronize invalidateBlocks in DatanodeDescriptor. Contributed by hemanthboyina.
|
2019-09-12 19:43:18 +05:30 |
|
Surendra Singh Lilhore
|
d1c303a497
|
HDFS-14699. Erasure Coding: Storage not considered in live replica when replication streams hard limit reached to threshold. Contributed by Zhao Yi Ming.
|
2019-09-12 19:11:50 +05:30 |
|
Akira Ajisaka
|
68612a0410
|
HDFS-14840. Use Java Conccurent Instead of Synchronization in BlockPoolTokenSecretManager. Contributed by David Mollitor.
|
2019-09-12 12:41:57 +09:00 |
|
Surendra Singh Lilhore
|
d98c54816d
|
HDFS-13913. LazyPersistFileScrubber.run() should log meaningful warn message. Contributed by Daniel Green.
|
2019-09-06 18:12:42 +05:30 |
|
Surendra Singh Lilhore
|
fa7f03fc56
|
HDFS-14826. dfs.ha.zkfc.port property duplicated in hdfs-default.xml. Contributed by Renukaprasad C.
|
2019-09-06 17:55:30 +05:30 |
|
Inigo Goiri
|
72d8b92ba5
|
HDFS-12904. Add DataTransferThrottler to the Datanode transfers. Contributed by Lisheng Sun.
|
2019-09-05 11:44:02 -07:00 |
|
Xiaoyu Yao
|
3c117163a3
|
HDFS-14633. The StorageType quota and consume in QuotaFeature is not handled for rename. Contributed by Jinglun.
|
2019-09-03 09:29:58 -07:00 |
|
Ayush Saxena
|
91b01a1db7
|
HDFS-14807. SetTimes updates all negative values apart from -1. Contributed by Ayush Saxena.
|
2019-09-03 18:08:42 +05:30 |
|
Surendra Singh Lilhore
|
5ff76cb8bc
|
HDFS-14630. Configuration.getTimeDurationHelper() should not log time unit warning in info log. Contributed by hemanthboyina.
|
2019-09-03 12:37:09 +05:30 |
|
Stephen O'Donnell
|
915cbc91c0
|
HDFS-14706. Checksums are not checked if block meta file is less than 7 bytes. Contributed by Stephen O'Donnell.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
|
2019-09-02 09:47:04 -07:00 |
|
Surendra Singh Lilhore
|
96f7dc1992
|
HDFS-14796. Define LOG instead of BlockManager.LOG in ErasureCodingWork/ReplicationWork. Contributed by Fei Hui.
|
2019-08-30 14:31:05 +05:30 |
|
Ayush Saxena
|
e220dac15c
|
HDFS-12212. Options.Rename.To_TRASH is considered even when Options.Rename.NONE is specified. Contributed by Vinayakumar B.
|
2019-08-30 08:40:13 +05:30 |
|
leosunli
|
915271245b
|
Make DataNodePeerMetrics#minOutlierDetectionSamples configurable (#1314). Contributed by Lisheng Sun.
Signed-off-by: sunlisheng <sunlisheng@xiaomi.com>
|
2019-08-29 18:35:26 -07:00 |
|
Istvan Fajth
|
fcb7884bfc
|
HDFS-8178. QJM doesn't move aside stale inprogress edits files. Contributed by Istvan Fajth.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
|
2019-08-29 16:56:14 -07:00 |
|
Inigo Goiri
|
3b22fcd377
|
HDFS-14256. Review Logging of NameNode Class. Contributed by David Mollitor.
|
2019-08-29 14:21:09 -07:00 |
|
Inigo Goiri
|
ffca734c62
|
HDFS-14104. Review getImageTxIdToRetain. Contributed by David Mollitor.
|
2019-08-29 14:18:35 -07:00 |
|
He Xiaoqiao
|
f600fbb6c4
|
HDFS-11246. FSNameSystem#logAuditEvent should be called outside the read or write locks. Contributed by He Xiaoqiao, Kuhu Shukla.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Co-authored-by: Kuhu Shukla <kshukla@apache.org>
|
2019-08-29 10:10:52 -07:00 |
|
Surendra Singh Lilhore
|
29bd6f3fc3
|
HDFS-8631. WebHDFS : Support setQuota. Contributed by Chao Sun.
|
2019-08-28 23:58:23 +05:30 |
|
CR Hota
|
6e37d65b03
|
HDFS-14760. Log INFO mode if snapshot usage and actual usage differ. Contributed by CR Hota.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
|
2019-08-27 16:20:25 -07:00 |
|
He Xiaoqiao
|
dde9399b37
|
HDFS-14497. Addendum: Write lock held by metasave impact following RPC processing.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
|
2019-08-27 15:26:59 -07:00 |
|
Shashikant Banerjee
|
ce8eb1283a
|
Revert "HDDS-1610. applyTransaction failure should not be lost on restart. Contributed by Shashikant Banerjee."
This reverts commit 62445021d5 as it has unintended changes in DirectoryWithSnapshotFeature class..
|
2019-08-27 23:23:44 +05:30 |
|
Arpit Agarwal
|
07e3cf952e
|
HDFS-2470. NN should automatically set permissions on dfs.namenode.*.dir. Contributed by Siddharth Wagle.
|
2019-08-26 15:57:55 -07:00 |
|
Chen Liang
|
ebef99dcf4
|
HDFS-14674. [SBN read] Got an unexpected txid when tail editlog. Contributed by wangzhaohui.
|
2019-08-23 12:18:43 -07:00 |
|
Erik Krogen
|
d699022fce
|
HDFS-13977. Override shouldForceSync in QuorumOutputStream to allow for proper auto-sync behavior. Contributed by Erik Krogen.
|
2019-08-23 11:33:21 -07:00 |
|
Akira Ajisaka
|
bd7baea5a5
|
HDFS-14396. Failed to load image from FSImageFile when downgrade from 3.x to 2.x. Contributed by Fei Hui.
|
2019-08-23 15:28:27 +09:00 |
|
Akira Ajisaka
|
abc8fde4ca
|
HDFS-13596. NN restart fails after RollingUpgrade from 2.x to 3.x. Contributed by Fei Hui.
|
2019-08-23 14:44:34 +09:00 |
|
Stephen O'Donnell
|
b67812ea21
|
HDFS-14617. Improve fsimage load time by writing sub-sections to the fsimage index (#1028). Contributed by Stephen O'Donnell.
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
|
2019-08-22 17:09:57 -07:00 |
|
Stephen O'Donnell
|
93daf69f90
|
HDFS-14675. Increase Balancer Defaults Further. Contributed by Stephen O'Donnell.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Gabor Bota <gabota@apache.org>
|
2019-08-22 15:21:33 -07:00 |
|
Surendra Singh Lilhore
|
76790a1e67
|
HDFS-14358. Provide LiveNode and DeadNode filter in DataNode UI. Contributed by hemanthboyina.
|
2019-08-22 12:27:45 +05:30 |
|
Ayush Saxena
|
0f598aed13
|
HDFS-14276. [SBN read] Reduce tailing overhead. Contributed by Wei-Chiu Chuang.
|
2019-08-22 02:25:06 +05:30 |
|
Wei-Chiu Chuang
|
57f7370174
|
Revert "HDFS-14476. lock too long when fix inconsistent blocks between disk and in-memory. Contributed by Sean Chow."
This reverts commit b58a35f374 .
|
2019-08-21 08:59:49 -07:00 |
|
Surendra Singh Lilhore
|
3a145e2918
|
HDFS-14582. Failed to start DN with ArithmeticException when NULL checksum used. Contributed by Surendra Singh Lilhore.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
|
2019-08-20 15:55:26 -07:00 |
|
Sandeep Kumar
|
269b543367
|
HADOOP-16523. Minor spell mistake in comment : Line number 101 (#388)
|
2019-08-20 13:36:23 -07:00 |
|
Wei-Chiu Chuang
|
4cb22cd867
|
HDFS-14311. Multi-threading conflict at layoutVersion when loading block pool storage. Contributed by Yicong Cai.
|
2019-08-20 10:37:11 -07:00 |
|
Sunil G
|
bd92462321
|
HDFS-14729. Upgrade Bootstrap and jQuery versions used in HDFS UIs. Contributed by Vivek Ratnavel Subramanian. This closes #1297
|
2019-08-20 18:42:19 +05:30 |
|
Shashikant Banerjee
|
62445021d5
|
HDDS-1610. applyTransaction failure should not be lost on restart. Contributed by Shashikant Banerjee.
|
2019-08-20 14:37:01 +05:30 |
|
Wei-Chiu Chuang
|
360a96f342
|
HDFS-13709. Report bad block to NN when transfer block encounter EIO exception. Contributed by Chen Zhang.
|
2019-08-19 13:08:55 -07:00 |
|
Surendra Singh Lilhore
|
b8db5b9a98
|
HDFS-14687. Standby Namenode never come out of safemode when EC files are being written. Contributed by Surendra Singh Lilhore.
Reviewed-by: Siyao Meng <smeng@cloudera.com>
Reviewed-by: Wei-Chiu CHuang <weichiu@apache.org>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
|
2019-08-18 18:13:35 -07:00 |
|
Wei-Chiu Chuang
|
b58a35f374
|
HDFS-14476. lock too long when fix inconsistent blocks between disk and in-memory. Contributed by Sean Chow.
|
2019-08-18 17:46:00 -07:00 |
|
Wei-Chiu Chuang
|
d873ddd656
|
HDFS-14127. Add a description about the observer read configuration. Contributed by xiangheng.
|
2019-08-17 04:39:22 -07:00 |
|
Wei-Chiu Chuang
|
528378784f
|
HDFS-12012. Fix spelling mistakes in BPServiceActor.java. Contributed by chencan.
|
2019-08-17 04:37:05 -07:00 |
|
hunshenshi
|
a38b9e137e
|
HDFS-14456:HAState#prepareToEnterState neednt a lock (#770) Contributed by hunshenshi.
|
2019-08-16 14:53:06 -07:00 |
|
LeonGao91
|
9a1d8cfaf5
|
HDFS-14678. Allow triggerBlockReport to a specific namenode. (#1252). Contributed by Leon Gao.
|
2019-08-16 08:00:51 -07:00 |
|
Surendra Singh Lilhore
|
34681643e9
|
HDFS-14719. Correct the safemode threshold value in BlockManagerSafeMode. Contributed by hemanthboyina.
|
2019-08-15 12:12:18 +05:30 |
|
Shashikant Banerjee
|
0a85af959c
|
HDFS-13101. Yet another fsimage corruption related to snapshot. Contributed by Shashikant Banerjee.
|
2019-08-15 10:16:25 +05:30 |
|
Wei-Chiu Chuang
|
c13ec7ab66
|
HDFS-14491. More Clarity on Namenode UI Around Blocks and Replicas. Contributed by Siyao Meng.
|
2019-08-13 17:15:26 -07:00 |
|
Wei-Chiu Chuang
|
633b7c1cfe
|
HDFS-14625. Make DefaultAuditLogger class in FSnamesystem to Abstract. Contributed by hemanthboyina.
|
2019-08-13 16:50:49 -07:00 |
|
Masatake Iwasaki
|
da0006fe04
|
HDFS-14423. Percent (%) and plus (+) characters no longer work in WebHDFS.
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
|
2019-08-14 08:39:40 +09:00 |
|
Ayush Saxena
|
e9b6b81de4
|
HDFS-13505. Turn on HDFS ACLs by default. Contributed by Siyao Meng.
|
2019-08-13 19:17:10 +05:30 |
|
Siyao Meng
|
c92b49876a
|
HDFS-14148. HDFS OIV ReverseXML SnapshotSection parser throws exception when there are more than one snapshottable directory (#1274) Contributed by Siyao Meng.
|
2019-08-12 17:26:20 -07:00 |
|
Yiqun Lin
|
8a77a224c7
|
HDFS-13359. DataXceiver hung due to the lock in FsDatasetImpl#getBlockInputStream. Contributed by Yiqun Lin.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
|
2019-08-09 18:40:28 -07:00 |
|
Wei-Chiu Chuang
|
ce3c5a3e3b
|
HDFS-14623. In NameNode Web UI, for Head the file (first 32K) old data is showing. Contributed by hemanthboyina.
|
2019-08-09 15:41:37 -07:00 |
|
Wei-Chiu Chuang
|
865021b8c9
|
HDFS-14195. OIV: print out storage policy id in oiv Delimited output. Contributed by Wang, Xinglong.
|
2019-08-09 15:37:29 -07:00 |
|
Rakesh Radhakrishnan
|
f6fa865d6f
|
HDFS-14700. Clean up pmem cache before setting pmem cache capacity. Contributed by Feilong He.
|
2019-08-09 14:07:54 +05:30 |
|
Wei-Chiu Chuang
|
6ad9a11494
|
HDFS-14693. NameNode should log a warning when EditLog IPC logger's pending size exceeds limit. Contributed by Xudong Cao.
|
2019-08-08 13:50:30 -07:00 |
|
Stephen O'Donnell
|
b0799148cf
|
HDFS-14459. ClosedChannelException silently ignored in FsVolumeList.addBlockPool(). Contributed by Stephen O'Donnell.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
|
2019-08-08 13:36:39 -07:00 |
|
Ayush Saxena
|
11f750e6a7
|
HDFS-14616. Add the warn log when the volume available space isn't enough. Contributed by liying.
|
2019-08-08 00:49:58 +05:30 |
|
Ayush Saxena
|
de64253892
|
HDFS-14608. DataNode#DataTransfer should be named. Contributed by Inigo Goiri.
|
2019-08-08 00:37:56 +05:30 |
|
Erik Krogen
|
827dbb11e2
|
HDFS-14370. Add exponential backoff to the edit log tailer to avoid spinning on empty edit tail requests. Contributed by Erik Krogen.
|
2019-08-07 09:25:58 -07:00 |
|
Yiqun Lin
|
a5bb1e8ee8
|
HDFS-14313. Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du. Contributed by Lisheng Sun.
|
2019-08-07 10:18:11 +08:00 |
|
Lokesh Jain
|
954ff36360
|
HDFS-14692. Upload button should not encode complete url. Contributed by Lokesh Jain.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
|
2019-08-06 15:30:50 -07:00 |
|
Stephen O'Donnell
|
35e0a01d7b
|
HDFS-14557. JournalNode error: Can't scan a pre-transactional edit log. Contributed by Stephen O'Donnell.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
|
2019-08-06 07:57:46 -07:00 |
|
Ayush Saxena
|
11272159bb
|
HDFS-14679. Failed to add erasure code policies with example template. Contributed by Yuan Zhou.
|
2019-08-06 08:42:15 +05:30 |
|
Wei-Chiu Chuang
|
b964b81f85
|
HDFS-14685. DefaultAuditLogger doesn't print CallerContext. Contributed by xuzq.
|
2019-08-02 10:54:32 -07:00 |
|
Siyao Meng
|
99bf1dc9eb
|
HDFS-14683. WebHDFS: Add erasureCodingPolicy field to GETCONTENTSUMMARY response (#1189) Contributed by Siyao Meng.
|
2019-08-01 17:14:07 -07:00 |
|
Wei-Chiu Chuang
|
32607dbd98
|
HDFS-14631.The DirectoryScanner doesn't fix the wrongly placed replica. Contributed by Jinglun.
|
2019-08-01 15:48:36 -07:00 |
|
Wei-Chiu Chuang
|
99f88c30cb
|
HDFS-14419. Avoid repeated calls to the listOpenFiles function. Contributed by HuangTao.
|
2019-07-30 16:50:06 -07:00 |
|
Chao Sun
|
3ae775d740
|
HDFS-14034. Support getQuotaUsage API in WebHDFS. Contributed by Chao Sun.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
|
2019-07-30 16:01:17 -07:00 |
|
Erik Krogen
|
1f26cc8705
|
HDFS-13783. Add an option to the Balancer to make it run as a long-running service. Contributed by Chen Zhang.
|
2019-07-30 15:42:55 -07:00 |
|
Inigo Goiri
|
7849bdcf70
|
HDFS-14449. Expose total number of DT in JMX for Namenode. Contributed by Fengnan Li.
|
2019-07-30 13:45:27 -07:00 |
|
Wei-Chiu Chuang
|
8053085388
|
HDFS-14429. Block remain in COMMITTED but not COMPLETE caused by Decommission. Contributed by Yicong Cai.
|
2019-07-29 14:31:34 -07:00 |
|
Ayush Saxena
|
02bd02b5af
|
HDFS-14660. [SBN Read] ObserverNameNode should throw StandbyException for requests not from ObserverProxyProvider. Contributed by Chao Sun.
|
2019-07-28 08:41:42 +05:30 |
|
Ayush Saxena
|
62deab17a3
|
HDFS-14647. NPE during secure namenode startup. Contributed by Fengnan Li.
|
2019-07-25 06:51:07 +05:30 |
|
Ayush Saxena
|
377f95bbe8
|
HDFS-13693. Remove unnecessary search in INodeDirectory.addChild during image loading. Contributed by Lisheng Sun.
|
2019-07-23 08:37:55 +05:30 |
|
Chen Liang
|
5e6cc6fe8a
|
HDFS-12979. [SBN read] StandbyNode should upload FsImage to ObserverNode after checkpointing. Contributed by Chen Liang.
|
2019-07-17 14:37:16 -07:00 |
|
Ayush Saxena
|
af0665c7f5
|
HDFS-13647. Fix the description of storageType option for space quota. Contributed by Takanobu Asanuma.
|
2019-07-17 23:22:32 +05:30 |
|
Ayush Saxena
|
f77d54c243
|
HDFS-14642. processMisReplicatedBlocks does not return correct processed count. Contributed by Stephen O'Donnell.
|
2019-07-16 08:14:27 +05:30 |
|
Rakesh Radhakrishnan
|
e98adb00b7
|
HDFS-14458. Report pmem stats to namenode. Contributed by Feilong He.
|
2019-07-15 13:02:37 +05:30 |
|
Shashikant Banerjee
|
f9fab9f22a
|
HDFS-14499. Misleading REM_QUOTA value with snapshot and trash feature enabled for a directory. Contributed by Shashikant Banerjee.
|
2019-07-12 15:42:06 +05:30 |
|
Chen Liang
|
8fb5ca3f40
|
HDFS-14611. Move handshake secret field from Token to BlockAccessToken. Contributed by Chen Liang.
|
2019-07-11 13:23:25 -07:00 |
|
Inigo Goiri
|
5747f6cff5
|
HDFS-14624. When decommissioning a node, log remaining blocks to replicate periodically. Contributed by Stephen O'Donnell.
|
2019-07-11 08:55:44 -07:00 |
|
Inigo Goiri
|
eccc9a40de
|
HDFS-12703. Exceptions are fatal to decommissioning monitor. Contributed by He Xiaoqiao.
|
2019-07-10 11:11:52 -07:00 |
|
Yiqun Lin
|
993dc8726b
|
HDFS-14632. Reduce useless #getNumLiveDataNodes call in SafeModeMonitor. Contributed by He Xiaoqiao.
|
2019-07-10 10:53:34 +08:00 |
|
Erik Krogen
|
4632708148
|
HDFS-14547. Improve memory efficiency of quotas when storage type quotas are not set. Contributed by Jinglun.
|
2019-07-08 14:46:25 -07:00 |
|
Wei-Chiu Chuang
|
ec851e4db2
|
HDFS-12862. CacheDirective becomes invalid when NN restart or failover. Contributed by Wang XL.
|
2019-07-08 15:23:46 +08:00 |
|
leosunli
|
9c90729486
|
HDFS-13694. Making md5 computing being in parallel with image loading.
|
2019-07-05 10:17:37 -07:00 |
|
Masatake Iwasaki
|
1c254a8ec7
|
HDFS-14629. Property value Hard Coded in DNConf.java. Contributed by hemanthboyina.
|
2019-07-04 23:20:42 +09:00 |
|
Weiwei Yang
|
729cb3aefe
|
HDFS-12748. NameNode memory leak when accessing webhdfs GETHOMEDIRECTORY. Contributed by Weiwei Yang.
|
2019-07-04 09:37:24 +08:00 |
|
paulward24
|
d8bac50e12
|
HDFS-14610. HashMap is not thread safe. Field storageMap is typically synchronized by storageMap. However, in one place, field storageMap is not protected with synchronized. (#1015)
|
2019-07-01 13:54:49 -07:00 |
|
paulward24
|
d203045c30
|
HDFS-14618. Incorrect synchronization of ArrayList field (ArrayList is thread-unsafe). Contributed by Paul Ward.
|
2019-06-28 16:06:43 -07:00 |
|
Inigo Goiri
|
4a212242d9
|
HDFS-14599. HDFS-12487 breaks test TestDiskBalancer.testDiskBalancerWithFedClusterWithOneNameServiceEmpty. Contributed by He Xiaoqiao.
|
2019-06-27 10:00:05 -07:00 |
|
Anu Engineer
|
041e7a7dee
|
HDFS-14598. Findbugs warning caused by HDFS-12487.
Contributed by He Xiaoqiao.
|
2019-06-24 19:07:22 -07:00 |
|