big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
23,360 Commits 2 Branches 2 Tags 264 MiB
b98ac2a3af
Commit Graph

8615 Commits

Author SHA1 Message Date
Wei-Chiu Chuang
5dd859a8a0 HDFS-14836. FileIoProvider should not increase FileIoErrors metric in datanode volume metric. Contributed by Aiphago. 
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2019-09-18 09:34:01 -07:00
Sahil Takiar
3cf6e4272f HDFS-14846: libhdfs tests are failing on trunk due to jni usage bugs 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-09-17 15:51:39 -07:00
Inigo Goiri
f580a87079 HDFS-14795. Add Throttler for writing block. Contributed by Lisheng Sun. 2019-09-17 14:55:34 -07:00
Ayush Saxena
e04b8a46c3 HDFS-14799. Do Not Call Map containsKey In Conjunction with get. Contributed by hemanthboyina. 2019-09-14 10:39:20 +05:30
Stephen O'Donnell
06ad6540b3 HDFS-14821. Make HDFS-14617 (fsimage sub-sections) off by default. Contributed by Stephen O'Donnell. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-09-13 10:36:45 -07:00
Surendra Singh Lilhore
4852a90e4b HDFS-14754. Erasure Coding : The number of Under-Replicated Blocks never reduced. Contributed by hemanthboyina. 2019-09-13 00:17:13 +05:30
Surendra Singh Lilhore
2ff2a7f612 HDFS-14798. Synchronize invalidateBlocks in DatanodeDescriptor. Contributed by hemanthboyina. 2019-09-12 19:43:18 +05:30
Surendra Singh Lilhore
d1c303a497 HDFS-14699. Erasure Coding: Storage not considered in live replica when replication streams hard limit reached to threshold. Contributed by Zhao Yi Ming. 2019-09-12 19:11:50 +05:30
Akira Ajisaka
68612a0410
HDFS-14840. Use Java Conccurent Instead of Synchronization in BlockPoolTokenSecretManager. Contributed by David Mollitor. 2019-09-12 12:41:57 +09:00
Takanobu Asanuma
c255333e20 HDFS-14838. RBF: Display RPC (instead of HTTP) Port Number in RBF web UI. Contributed by Xieming Li 2019-09-11 16:54:08 +09:00
Takanobu Asanuma
524b553a5f
HDFS-14835. RBF: Secured Router should not run when it can't initialize DelegationTokenSecretManager. (#1414) 2019-09-11 10:32:07 +09:00
Erik Krogen
c92a3e94d8 HADOOP-15565. Add an inner FS cache to ViewFileSystem, separate from the global cache, to avoid file system leaks. Contributed by Jinglun. 2019-09-06 10:22:28 -07:00
Surendra Singh Lilhore
d98c54816d HDFS-13913. LazyPersistFileScrubber.run() should log meaningful warn message. Contributed by Daniel Green. 2019-09-06 18:12:42 +05:30
Surendra Singh Lilhore
fa7f03fc56 HDFS-14826. dfs.ha.zkfc.port property duplicated in hdfs-default.xml. Contributed by Renukaprasad C. 2019-09-06 17:55:30 +05:30
Inigo Goiri
494d75eb2b HDFS-14784. Add more methods to WebHdfsTestUtil to support tests outside of package. Contributed by Chen Zhang. 2019-09-05 21:15:17 -07:00
Inigo Goiri
72d8b92ba5 HDFS-12904. Add DataTransferThrottler to the Datanode transfers. Contributed by Lisheng Sun. 2019-09-05 11:44:02 -07:00
Ayush Saxena
2f70b52a5b HDFS-14812. RBF: MountTableRefresherService should load cache when refresh. Contributed by xuzq. 2019-09-05 08:29:56 +05:30
Surendra Singh Lilhore
05704754a0 HDFS-14777. RBF: Set ReadOnly is failing for mount Table but actually readonly succeed to set. Contributed by Ranith Sardar. 2019-09-04 15:09:09 +05:30
Xiaoyu Yao
3c117163a3
HDFS-14633. The StorageType quota and consume in QuotaFeature is not handled for rename. Contributed by Jinglun. 2019-09-03 09:29:58 -07:00
Ayush Saxena
91b01a1db7 HDFS-14807. SetTimes updates all negative values apart from -1. Contributed by Ayush Saxena. 2019-09-03 18:08:42 +05:30
Surendra Singh Lilhore
5ff76cb8bc HDFS-14630. Configuration.getTimeDurationHelper() should not log time unit warning in info log. Contributed by hemanthboyina. 2019-09-03 12:37:09 +05:30
Stephen O'Donnell
915cbc91c0 HDFS-14706. Checksums are not checked if block meta file is less than 7 bytes. Contributed by Stephen O'Donnell. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-09-02 09:47:04 -07:00
Wei-Chiu Chuang
d207aba026 Revert "HDFS-14706. Checksums are not checked if block meta file is less than 7 bytes. Contributed by Stephen O'Donnell." 
This reverts commit 7bebad61d9.
 2019-09-02 09:43:44 -07:00
Ayush Saxena
040f6e93bb HDFS-14654. RBF: TestRouterRpc#testNamenodeMetrics is flaky. Contributed by Chen Zhang. 2019-09-02 13:01:52 +05:30
Ayush Saxena
c3abfcefdd HDFS-13843. RBF: Add optional parameter -d for detailed listing of mount points. Contributed by Ayush Saxena. 2019-09-02 07:24:04 +05:30
Ayush Saxena
18d74fe41c HDFS-14711. RBF: RBFMetrics throws NullPointerException if stateStore disabled. Contributed by Chen Zhang. 2019-09-01 12:53:10 +05:30
Surendra Singh Lilhore
96f7dc1992 HDFS-14796. Define LOG instead of BlockManager.LOG in ErasureCodingWork/ReplicationWork. Contributed by Fei Hui. 2019-08-30 14:31:05 +05:30
Ayush Saxena
e220dac15c HDFS-12212. Options.Rename.To_TRASH is considered even when Options.Rename.NONE is specified. Contributed by Vinayakumar B. 2019-08-30 08:40:13 +05:30
leosunli
915271245b Make DataNodePeerMetrics#minOutlierDetectionSamples configurable (#1314). Contributed by Lisheng Sun. 
Signed-off-by: sunlisheng <sunlisheng@xiaomi.com>
 2019-08-29 18:35:26 -07:00
Stephen O'Donnell
7bebad61d9 HDFS-14706. Checksums are not checked if block meta file is less than 7 bytes. Contributed by Stephen O'Donnell. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-08-29 17:38:07 -07:00
Istvan Fajth
fcb7884bfc HDFS-8178. QJM doesn't move aside stale inprogress edits files. Contributed by Istvan Fajth. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-08-29 16:56:14 -07:00
Inigo Goiri
3b22fcd377 HDFS-14256. Review Logging of NameNode Class. Contributed by David Mollitor. 2019-08-29 14:21:09 -07:00
Inigo Goiri
ffca734c62 HDFS-14104. Review getImageTxIdToRetain. Contributed by David Mollitor. 2019-08-29 14:18:35 -07:00
He Xiaoqiao
f600fbb6c4 HDFS-11246. FSNameSystem#logAuditEvent should be called outside the read or write locks. Contributed by He Xiaoqiao, Kuhu Shukla. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Co-authored-by: Kuhu Shukla <kshukla@apache.org>
 2019-08-29 10:10:52 -07:00
Ayush Saxena
8e779a151e HDFS-14721. RBF: ProxyOpComplete is not accurate in FederationRPCPerformanceMonitor. Contributed by xuzq. 2019-08-29 20:08:38 +05:30
Surendra Singh Lilhore
29bd6f3fc3 HDFS-8631. WebHDFS : Support setQuota. Contributed by Chao Sun. 2019-08-28 23:58:23 +05:30
Inigo Goiri
48cb583906 HDFS-14710. RBF: Improve some RPC performance by using previous block. Contributed by xuzq. 2019-08-28 10:48:00 -07:00
CR Hota
6e37d65b03 HDFS-14760. Log INFO mode if snapshot usage and actual usage differ. Contributed by CR Hota. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-08-27 16:20:25 -07:00
He Xiaoqiao
dde9399b37 HDFS-14497. Addendum: Write lock held by metasave impact following RPC processing. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-08-27 15:26:59 -07:00
Shashikant Banerjee
ce8eb1283a Revert "HDDS-1610. applyTransaction failure should not be lost on restart. Contributed by Shashikant Banerjee." 
This reverts commit 62445021d5 as it has unintended changes in DirectoryWithSnapshotFeature class..
 2019-08-27 23:23:44 +05:30
Jonathan Hung
8ab7020e64 HDFS-14779. Fix logging error in TestEditLog#testMultiStreamsLoadEditWithConfMaxTxns 2019-08-27 10:30:31 -07:00
Takanobu Asanuma
b69ac575a1 HDFS-14772. RBF: hdfs-rbf-site.xml can't be loaded automatically. Contributed by Yuxuan Wang 2019-08-27 14:32:32 +09:00
Arpit Agarwal
07e3cf952e
HDFS-2470. NN should automatically set permissions on dfs.namenode.*.dir. Contributed by Siddharth Wagle. 2019-08-26 15:57:55 -07:00
Ayush Saxena
d2225c8ca8 HDFS-14722. RBF: GetMountPointStatus should return mountTable information when getFileInfoAll throw IOException. Contributed by xuzq. 2019-08-24 08:47:43 +05:30
Inigo Goiri
0b796754b9 HDFS-14766. RBF: MountTableStoreImpl#getMountTableEntries returns extra entry. Contributed by Chen Zhang. 2019-08-23 14:53:57 -07:00
Chen Liang
ebef99dcf4 HDFS-14674. [SBN read] Got an unexpected txid when tail editlog. Contributed by wangzhaohui. 2019-08-23 12:18:43 -07:00
Erik Krogen
d699022fce HDFS-13977. Override shouldForceSync in QuorumOutputStream to allow for proper auto-sync behavior. Contributed by Erik Krogen. 2019-08-23 11:33:21 -07:00
Wang Yuxuan
894e2300d6 HDFS-14761. RBF: MountTableResolver cannot invalidate cache correctly (#1334) 
HDFS-14761. RBF: MountTableResolver cannot invalidate cache correctly
 2019-08-23 10:46:39 -07:00
Ayush Saxena
c92de8209d HDFS-14747. RBF: IsFileClosed should be return false when the file is open in multiple destination. Contributed by xuzq. 2019-08-23 19:08:07 +05:30
Akira Ajisaka
bd7baea5a5
HDFS-14396. Failed to load image from FSImageFile when downgrade from 3.x to 2.x. Contributed by Fei Hui. 2019-08-23 15:28:27 +09:00
Akira Ajisaka
abc8fde4ca
HDFS-13596. NN restart fails after RollingUpgrade from 2.x to 3.x. Contributed by Fei Hui. 2019-08-23 14:44:34 +09:00
Stephen O'Donnell
b67812ea21 HDFS-14617. Improve fsimage load time by writing sub-sections to the fsimage index (#1028). Contributed by Stephen O'Donnell. 
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2019-08-22 17:09:57 -07:00
Stephen O'Donnell
93daf69f90 HDFS-14675. Increase Balancer Defaults Further. Contributed by Stephen O'Donnell. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Gabor Bota <gabota@apache.org>
 2019-08-22 15:21:33 -07:00
Inigo Goiri
e04dcfdc57 HDFS-14583. FileStatus#toString() will throw IllegalArgumentException. Contributed by xuzq. 2019-08-22 10:22:38 -07:00
Surendra Singh Lilhore
76790a1e67 HDFS-14358. Provide LiveNode and DeadNode filter in DataNode UI. Contributed by hemanthboyina. 2019-08-22 12:27:45 +05:30
Ayush Saxena
52c77bc160 HDFS-14741. RBF: RecoverLease should be return false when the file is open in multiple destination. Contributed by xuzq 2019-08-22 08:57:22 +05:30
Ayush Saxena
f9029c4070 HDFS-14744. RBF: Non secured routers should not log in error mode when UGI is default. Contributed by CR Hota. 2019-08-22 02:40:39 +05:30
Ayush Saxena
0f598aed13 HDFS-14276. [SBN read] Reduce tailing overhead. Contributed by Wei-Chiu Chuang. 2019-08-22 02:25:06 +05:30
Inigo Goiri
93595febaa HDFS-14756. RBF: getQuotaUsage may ignore some folders. Contributed by Chen Zhang. 2019-08-21 09:39:57 -07:00
Inigo Goiri
5eeb6da2d4 HDFS-14714. RBF: implement getReplicatedBlockStats interface. Contributed by Chen Zhang. 2019-08-21 09:38:17 -07:00
Wei-Chiu Chuang
57f7370174 Revert "HDFS-14476. lock too long when fix inconsistent blocks between disk and in-memory. Contributed by Sean Chow." 
This reverts commit b58a35f374.
 2019-08-21 08:59:49 -07:00
Anu Engineer
8aaf5e1a14 HDFS-14759. HDFS cat logs an info message. 
Contributed by Eric Badger.
 2019-08-20 20:24:19 -07:00
Surendra Singh Lilhore
3a145e2918 HDFS-14582. Failed to start DN with ArithmeticException when NULL checksum used. Contributed by Surendra Singh Lilhore. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-08-20 15:55:26 -07:00
Sandeep Kumar
269b543367 HADOOP-16523. Minor spell mistake in comment : Line number 101 (#388) 2019-08-20 13:36:23 -07:00
Wei-Chiu Chuang
4cb22cd867 HDFS-14311. Multi-threading conflict at layoutVersion when loading block pool storage. Contributed by Yicong Cai. 2019-08-20 10:37:11 -07:00
Wei-Chiu Chuang
aa6995fde2 HDFS-13201. Fix prompt message in testPolicyAndStateCantBeNull. Contributed by chencan. 2019-08-20 10:32:55 -07:00
Sunil G
bd92462321 HDFS-14729. Upgrade Bootstrap and jQuery versions used in HDFS UIs. Contributed by Vivek Ratnavel Subramanian. This closes #1297 2019-08-20 18:42:19 +05:30
Shashikant Banerjee
62445021d5 HDDS-1610. applyTransaction failure should not be lost on restart. Contributed by Shashikant Banerjee. 2019-08-20 14:37:01 +05:30
Wei-Chiu Chuang
360a96f342 HDFS-13709. Report bad block to NN when transfer block encounter EIO exception. Contributed by Chen Zhang. 2019-08-19 13:08:55 -07:00
HUAN-PING SU
abae6ff2a2 HDFS-14746. Trivial test code update after HDFS-14687. Contributed by kevin su. 
Reviewed-by: Surendra Singh Lilhore <surendralilhore@apache.org>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-08-19 11:14:51 -07:00
Surendra Singh Lilhore
b8db5b9a98 HDFS-14687. Standby Namenode never come out of safemode when EC files are being written. Contributed by Surendra Singh Lilhore. 
Reviewed-by: Siyao Meng <smeng@cloudera.com>
Reviewed-by: Wei-Chiu CHuang <weichiu@apache.org>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-08-18 18:13:35 -07:00
Wei-Chiu Chuang
b58a35f374 HDFS-14476. lock too long when fix inconsistent blocks between disk and in-memory. Contributed by Sean Chow. 2019-08-18 17:46:00 -07:00
Wei-Chiu Chuang
d873ddd656 HDFS-14127. Add a description about the observer read configuration. Contributed by xiangheng. 2019-08-17 04:39:22 -07:00
Wei-Chiu Chuang
528378784f HDFS-12012. Fix spelling mistakes in BPServiceActor.java. Contributed by chencan. 2019-08-17 04:37:05 -07:00
hunshenshi
a38b9e137e HDFS-14456:HAState#prepareToEnterState neednt a lock (#770) Contributed by hunshenshi. 2019-08-16 14:53:06 -07:00
LeonGao91
9a1d8cfaf5 HDFS-14678. Allow triggerBlockReport to a specific namenode. (#1252). Contributed by Leon Gao. 2019-08-16 08:00:51 -07:00
Surendra Singh Lilhore
34681643e9 HDFS-14719. Correct the safemode threshold value in BlockManagerSafeMode. Contributed by hemanthboyina. 2019-08-15 12:12:18 +05:30
Shashikant Banerjee
0a85af959c HDFS-13101. Yet another fsimage corruption related to snapshot. Contributed by Shashikant Banerjee. 2019-08-15 10:16:25 +05:30
Ayush Saxena
b06c2345ef HDFS-14713. RBF: RouterAdmin supports refreshRouterArgs command but not on display. Contributed by wangzhaohui. 2019-08-14 20:44:31 +05:30
Siyao Meng
3c0382f1b9 HDFS-14595. HDFS-11848 breaks API compatibility. Contributed by Siyao Meng. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
 2019-08-14 07:30:35 -07:00
Wei-Chiu Chuang
c13ec7ab66 HDFS-14491. More Clarity on Namenode UI Around Blocks and Replicas. Contributed by Siyao Meng. 2019-08-13 17:15:26 -07:00
Wei-Chiu Chuang
633b7c1cfe HDFS-14625. Make DefaultAuditLogger class in FSnamesystem to Abstract. Contributed by hemanthboyina. 2019-08-13 16:50:49 -07:00
Masatake Iwasaki
da0006fe04 HDFS-14423. Percent (%) and plus (+) characters no longer work in WebHDFS. 
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
 2019-08-14 08:39:40 +09:00
Siyao Meng
6ae8bc3a4a HDFS-14665. HttpFS: LISTSTATUS response is missing HDFS-specific fields (#1267) Contributed by Siyao Meng. 2019-08-13 16:27:57 -07:00
Ayush Saxena
e9b6b81de4 HDFS-13505. Turn on HDFS ACLs by default. Contributed by Siyao Meng. 2019-08-13 19:17:10 +05:30
Ayush Saxena
454420e4f2 HDFS-14708. TestLargeBlockReport#testBlockReportSucceedsWithLargerLengthLimit fails in trunk. Contributed by Lisheng Sun. 2019-08-13 08:27:20 +05:30
Siyao Meng
c92b49876a HDFS-14148. HDFS OIV ReverseXML SnapshotSection parser throws exception when there are more than one snapshottable directory (#1274) Contributed by Siyao Meng. 2019-08-12 17:26:20 -07:00
Yiqun Lin
8a77a224c7 HDFS-13359. DataXceiver hung due to the lock in FsDatasetImpl#getBlockInputStream. Contributed by Yiqun Lin. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-08-09 18:40:28 -07:00
Siyao Meng
e02ffed1b1 HDFS-12125. Document the missing EC removePolicy command (#1258) Contributed by Wenxin He, Siyao Meng. 
Co-authored-by: Wenxin He <wenxin.he@gmail.com>
 2019-08-09 18:00:22 -07:00
Wei-Chiu Chuang
ce3c5a3e3b HDFS-14623. In NameNode Web UI, for Head the file (first 32K) old data is showing. Contributed by hemanthboyina. 2019-08-09 15:41:37 -07:00
Wei-Chiu Chuang
865021b8c9 HDFS-14195. OIV: print out storage policy id in oiv Delimited output. Contributed by Wang, Xinglong. 2019-08-09 15:37:29 -07:00
Rakesh Radhakrishnan
f6fa865d6f HDFS-14700. Clean up pmem cache before setting pmem cache capacity. Contributed by Feilong He. 2019-08-09 14:07:54 +05:30
Wei-Chiu Chuang
6ad9a11494 HDFS-14693. NameNode should log a warning when EditLog IPC logger's pending size exceeds limit. Contributed by Xudong Cao. 2019-08-08 13:50:30 -07:00
Wei-Chiu Chuang
2265872c2d HDFS-14705. Remove unused configuration dfs.min.replication. Contributed by CR Hota. 2019-08-08 13:48:29 -07:00
Wei-Chiu Chuang
28a848412c HDFS-14701. Change Log Level to warn in SlotReleaser. Contributed by Lisheng Sun. 2019-08-08 13:46:31 -07:00
Wei-Chiu Chuang
23f91f68b8 HDFS-14662. Document the usage of the new Balancer "asService" parameter. Contributed by Chen Zhang. 2019-08-08 13:45:29 -07:00
Stephen O'Donnell
b0799148cf HDFS-14459. ClosedChannelException silently ignored in FsVolumeList.addBlockPool(). Contributed by Stephen O'Donnell. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-08-08 13:36:39 -07:00
Ayush Saxena
11f750e6a7 HDFS-14616. Add the warn log when the volume available space isn't enough. Contributed by liying. 2019-08-08 00:49:58 +05:30
Ayush Saxena
de64253892 HDFS-14608. DataNode#DataTransfer should be named. Contributed by Inigo Goiri. 2019-08-08 00:37:56 +05:30
Erik Krogen
827dbb11e2 HDFS-14370. Add exponential backoff to the edit log tailer to avoid spinning on empty edit tail requests. Contributed by Erik Krogen. 2019-08-07 09:25:58 -07:00
Yiqun Lin
a5bb1e8ee8 HDFS-14313. Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du. Contributed by Lisheng Sun. 2019-08-07 10:18:11 +08:00
Lokesh Jain
954ff36360 HDFS-14692. Upload button should not encode complete url. Contributed by Lokesh Jain. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-08-06 15:30:50 -07:00
Stephen O'Donnell
35e0a01d7b HDFS-14557. JournalNode error: Can't scan a pre-transactional edit log. Contributed by Stephen O'Donnell. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-08-06 07:57:46 -07:00
Ayush Saxena
11272159bb HDFS-14679. Failed to add erasure code policies with example template. Contributed by Yuan Zhou. 2019-08-06 08:42:15 +05:30
Ayush Saxena
8f40856f76 HDFS-14669. TestDirectoryScanner#testDirectoryScannerInFederatedCluster fails intermittently in trunk. Contributed by qiang Liu. 2019-08-03 12:15:41 +05:30
Ayush Saxena
e503db5f44 HDFS-12826. Document Saying the RPC port, But it's required IPC port in HDFS Federation Document. Contributed by usharani. 2019-08-03 12:05:46 +05:30
Erik Krogen
e7a0b8aa83 HDFS-14462 Ensure WebHDFS client throws the correct exception during writes. Contributed by Simbarashe Dzinamarira. 2019-08-02 11:48:31 -07:00
Wei-Chiu Chuang
b964b81f85 HDFS-14685. DefaultAuditLogger doesn't print CallerContext. Contributed by xuzq. 2019-08-02 10:54:32 -07:00
Wei-Chiu Chuang
c2d00c8450 HDFS-13131. Modifying testcase testEnableAndDisableErasureCodingPolicy. Contributed by chencan. 2019-08-01 19:11:43 -07:00
Siyao Meng
17e8cf501b HDFS-14686. HttpFS: HttpFSFileSystem#getErasureCodingPolicy always returns null (#1192) Contributed by Siyao Meng. 2019-08-01 17:15:22 -07:00
Siyao Meng
99bf1dc9eb HDFS-14683. WebHDFS: Add erasureCodingPolicy field to GETCONTENTSUMMARY response (#1189) Contributed by Siyao Meng. 2019-08-01 17:14:07 -07:00
Wei-Chiu Chuang
32607dbd98 HDFS-14631.The DirectoryScanner doesn't fix the wrongly placed replica. Contributed by Jinglun. 2019-08-01 15:48:36 -07:00
Ayush Saxena
89b102f916 HDFS-14661. RBF: updateMountTableEntry shouldn't update mountTableEntry if targetPath not exist. Contributed by xuzq. 2019-08-01 08:43:39 +05:30
Chao Sun
b008072044 HDFS-14681. RBF: TestDisableRouterQuota failed because port 8888 was occupied. 2019-07-31 10:24:48 -07:00
Wei-Chiu Chuang
99f88c30cb HDFS-14419. Avoid repeated calls to the listOpenFiles function. Contributed by HuangTao. 2019-07-30 16:50:06 -07:00
Chao Sun
3ae775d740 HDFS-14034. Support getQuotaUsage API in WebHDFS. Contributed by Chao Sun. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-07-30 16:01:17 -07:00
Erik Krogen
1f26cc8705 HDFS-13783. Add an option to the Balancer to make it run as a long-running service. Contributed by Chen Zhang. 2019-07-30 15:42:55 -07:00
Inigo Goiri
7849bdcf70 HDFS-14449. Expose total number of DT in JMX for Namenode. Contributed by Fengnan Li. 2019-07-30 13:45:27 -07:00
Ayush Saxena
8deced0073 HDFS-14677. TestDataNodeHotSwapVolumes#testAddVolumesConcurrently fails intermittently in trunk. Contributed by Chen Zhang. 2019-07-30 22:17:39 +05:30
Wei-Chiu Chuang
8053085388 HDFS-14429. Block remain in COMMITTED but not COMPLETE caused by Decommission. Contributed by Yicong Cai. 2019-07-29 14:31:34 -07:00
CR Hota
611718f59f HDFS-14670: RBF: Create secret manager instance using FederationUtil#newInstance. 2019-07-29 11:00:22 -07:00
Ayush Saxena
02bd02b5af HDFS-14660. [SBN Read] ObserverNameNode should throw StandbyException for requests not from ObserverProxyProvider. Contributed by Chao Sun. 2019-07-28 08:41:42 +05:30
hunshenshi
2fe450cb5e HDFS-14425. Native build fails on macos due to jlong in hdfs.c (#741) 2019-07-26 15:33:12 -07:00
HUAN-PING SU
ecc8acfd24 HDFS-14673. The console log is noisy when using DNSDomainNameResolver to resolve NameNode. 2019-07-26 11:08:46 -07:00
Ayush Saxena
ce99cc31e9 HDFS-14303. Addendum: check block directory logic not correct when there is only meta file, print no meaning warn log. Contributed by qiang Liu. 2019-07-26 08:31:00 +05:30
Masatake Iwasaki
6b8107ad97 HDFS-14135. TestWebHdfsTimeouts Fails intermittently in trunk. Contributed by Ayush Saxena. 
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
 2019-07-26 09:35:23 +09:00
Ayush Saxena
62deab17a3 HDFS-14647. NPE during secure namenode startup. Contributed by Fengnan Li. 2019-07-25 06:51:07 +05:30
Ayush Saxena
377f95bbe8 HDFS-13693. Remove unnecessary search in INodeDirectory.addChild during image loading. Contributed by Lisheng Sun. 2019-07-23 08:37:55 +05:30
Ayush Saxena
6282c02d7d HDFS-14577. RBF: FederationUtil#newInstance should allow constructor without context. Contributed by CR Hota. 2019-07-19 17:44:13 +05:30
Takanobu Asanuma
9792f58662 HDFS-14653. RBF: Correct the default value for dfs.federation.router.namenode.heartbeat.enable. Contributed by Ayush Saxena. 2019-07-18 10:16:20 +09:00
Chen Liang
5e6cc6fe8a HDFS-12979. [SBN read] StandbyNode should upload FsImage to ObserverNode after checkpointing. Contributed by Chen Liang. 2019-07-17 14:37:16 -07:00
Ayush Saxena
af0665c7f5 HDFS-13647. Fix the description of storageType option for space quota. Contributed by Takanobu Asanuma. 2019-07-17 23:22:32 +05:30
Ayush Saxena
f77d54c243 HDFS-14642. processMisReplicatedBlocks does not return correct processed count. Contributed by Stephen O'Donnell. 2019-07-16 08:14:27 +05:30
Ayush Saxena
64d4abf489 HDFS-14593. RBF: Implement deletion feature for expired records in State Store. Contributed by Takanobu Asanuma. 2019-07-15 22:38:00 +05:30
Rakesh Radhakrishnan
30a8f840f1 HDFS-14357. Update documentation for HDFS cache on SCM support. Contributed by Feilong He. 2019-07-15 13:18:23 +05:30
Rakesh Radhakrishnan
e98adb00b7 HDFS-14458. Report pmem stats to namenode. Contributed by Feilong He. 2019-07-15 13:02:37 +05:30
Shashikant Banerjee
f9fab9f22a HDFS-14499. Misleading REM_QUOTA value with snapshot and trash feature enabled for a directory. Contributed by Shashikant Banerjee. 2019-07-12 15:42:06 +05:30
Masatake Iwasaki
00dd843a1a
HDFS-14466. Add a regression test for HDFS-14323. 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2019-07-12 10:04:55 +09:00
Chen Liang
8fb5ca3f40 HDFS-14611. Move handshake secret field from Token to BlockAccessToken. Contributed by Chen Liang. 2019-07-11 13:23:25 -07:00
Inigo Goiri
5747f6cff5 HDFS-14624. When decommissioning a node, log remaining blocks to replicate periodically. Contributed by Stephen O'Donnell. 2019-07-11 08:55:44 -07:00
Inigo Goiri
eccc9a40de HDFS-12703. Exceptions are fatal to decommissioning monitor. Contributed by He Xiaoqiao. 2019-07-10 11:11:52 -07:00
Ajay Yadav
cdb20adfcc HDDS-1611. Evaluate ACL on volume bucket key and prefix to authorize access. Contributed by Ajay Kumar. (#973) 2019-07-10 11:03:58 -07:00
Yiqun Lin
993dc8726b HDFS-14632. Reduce useless #getNumLiveDataNodes call in SafeModeMonitor. Contributed by He Xiaoqiao. 2019-07-10 10:53:34 +08:00
Erik Krogen
4632708148 HDFS-14547. Improve memory efficiency of quotas when storage type quotas are not set. Contributed by Jinglun. 2019-07-08 14:46:25 -07:00
Wei-Chiu Chuang
ec851e4db2 HDFS-12862. CacheDirective becomes invalid when NN restart or failover. Contributed by Wang XL. 2019-07-08 15:23:46 +08:00
leosunli
9c90729486 HDFS-13694. Making md5 computing being in parallel with image loading. 2019-07-05 10:17:37 -07:00
Inigo Goiri
c3ca348b81 HDFS-14620. RBF: Fix 'not a super user' error when disabling a namespace in kerberos with superuser principal. Contributed by luhuachao. 2019-07-04 11:21:55 -07:00
Masatake Iwasaki
1c254a8ec7 HDFS-14629. Property value Hard Coded in DNConf.java. Contributed by hemanthboyina. 2019-07-04 23:20:42 +09:00
Weiwei Yang
729cb3aefe HDFS-12748. NameNode memory leak when accessing webhdfs GETHOMEDIRECTORY. Contributed by Weiwei Yang. 2019-07-04 09:37:24 +08:00
paulward24
d8bac50e12 HDFS-14610. HashMap is not thread safe. Field storageMap is typically synchronized by storageMap. However, in one place, field storageMap is not protected with synchronized. (#1015) 2019-07-01 13:54:49 -07:00
paulward24
d203045c30 HDFS-14618. Incorrect synchronization of ArrayList field (ArrayList is thread-unsafe). Contributed by Paul Ward. 2019-06-28 16:06:43 -07:00
Inigo Goiri
4a212242d9 HDFS-14599. HDFS-12487 breaks test TestDiskBalancer.testDiskBalancerWithFedClusterWithOneNameServiceEmpty. Contributed by He Xiaoqiao. 2019-06-27 10:00:05 -07:00
Inigo Goiri
041e0c0564 HDFS-14036. RBF: Add hdfs-rbf-default.xml to HdfsConfiguration by default. Contributed by Takanobu Asanuma. 2019-06-27 09:57:52 -07:00
Anu Engineer
041e7a7dee HDFS-14598. Findbugs warning caused by HDFS-12487. 
Contributed by He Xiaoqiao.
 2019-06-24 19:07:22 -07:00
Inigo Goiri
b76b843c8b HDFS-13371. NPE for FsServerDefaults.getKeyProviderUri() for clientProtocol communication between 2.7 and 3.X. Contributed by Sherwood Zheng. 2019-06-24 17:52:33 -07:00
Inigo Goiri
38a560c6f1
Merge pull request #977 from leosunli/trunk 
HDFS-14541. When evictableMmapped or evictable size is zero, do not throw NoSuchElementException.
 2019-06-24 17:44:54 -07:00
Ajay Kumar
95c94dcca7
HADOOP-16350. Ability to tell HDFS client not to request KMS Information from NameNode. Ccontributed by Greg Senia, Ajay Kumar. 2019-06-24 11:59:07 -07:00
Inigo Goiri
719d57bf46 HDFS-14545. RBF: Router should support GetUserMappingsProtocol. Contributed by Ayush Saxena. 2019-06-24 22:03:04 +05:30
Akira Ajisaka
8627f6532b HDFS-14550. RBF: Failed to get statistics from NameNodes before 2.9.0. Contributed by He Xiaoqiao. 2019-06-24 22:03:04 +05:30
Ayush Saxena
ba221dc50e HDFS-14526. RBF: Update the document of RBF related metrics. Contributed by Takanobu Asanuma. 2019-06-24 22:03:04 +05:30
Ayush Saxena
ade8d3b60e HDFS-14508. RBF: Clean-up and refactor UI components. Contributed by Takanobu Asanuma. 2019-06-24 22:03:04 +05:30
Ayush Saxena
6915d7e13c HDFS-13480. RBF: Separate namenodeHeartbeat and routerHeartbeat to different config key. Contributed by Ayush Saxena. 2019-06-24 22:03:04 +05:30
Ayush Saxena
ddbe08db33 HDFS-13955. RBF: Support secure Namenode in NamenodeHeartbeatService. Contributed by CR Hota. 2019-06-24 22:03:04 +05:30
Inigo Goiri
b062dd462d HDFS-14475. RBF: Expose router security enabled status on the UI. Contributed by CR Hota. 2019-06-24 22:03:04 +05:30
Ayush Saxena
b6fff8c81e HDFS-13787. RBF: Add Snapshot related ClientProtocol APIs. Contributed by Inigo Goiri. 2019-06-24 22:03:04 +05:30
Ayush Saxena
d240eec136 HDFS-14516. RBF: Create hdfs-rbf-site.xml for RBF specific properties. Contributed by Takanobu Asanuma. 2019-06-24 22:03:04 +05:30
Ayush Saxena
9b197c2893 HDFS-13909. RBF: Add Cache pools and directives related ClientProtocol APIs. Contributed by Ayush Saxena. 2019-06-24 22:03:04 +05:30
Ayush Saxena
ffbb6b6557 HDFS-13255. RBF: Fail when try to remove mount point paths. Contributed by Akira Ajisaka. 2019-06-24 22:03:04 +05:30
Ayush Saxena
8e4267650f HDFS-14440. RBF: Optimize the file write process in case of multiple destinations. Contributed by Ayush Saxena. 2019-06-24 22:03:04 +05:30
Brahma Reddy Battula
2636a54ffd HDFS-13995. RBF: Security documentation. Contributed by CR Hota. 2019-06-24 22:03:04 +05:30
Giovanni Matteo Fumarola
393f15176d HDFS-14447. RBF: Router should support RefreshUserMappingsProtocol. Contributed by Shen Yinjie. 2019-06-24 22:03:04 +05:30
Giovanni Matteo Fumarola
3deb5d345f HDFS-14490. RBF: Remove unnecessary quota checks. Contributed by Ayush Saxena. 2019-06-24 22:03:04 +05:30
Ayush Saxena
62fa53a01d HDFS-14210. RBF: ACL commands should work over all the destinations. Contributed by Ayush Saxena. 2019-06-24 22:03:04 +05:30
Giovanni Matteo Fumarola
32841178ba HDFS-14426. RBF: Add delegation token total count as one of the federation metrics. Contributed by Fengnan Li. 2019-06-24 22:03:04 +05:30
Ayush Saxena
203664e6b2 HDFS-14454. RBF: getContentSummary() should allow non-existing folders. Contributed by Inigo Goiri. 2019-06-24 22:03:04 +05:30
Ayush Saxena
b522b52bb1 HDFS-14457. RBF: Add order text SPACE in CLI command 'hdfs dfsrouteradmin'. Contributed by luhuachao. 2019-06-24 22:03:04 +05:30
Brahma Reddy Battula
021a43b1a4 HDFS-13972. RBF: Support for Delegation Token (WebHDFS). Contributed by CR Hota. 2019-06-24 22:03:04 +05:30
Ayush Saxena
de7da9b69e HDFS-14422. RBF: Router shouldn't allow READ operations in safe mode. Contributed by Inigo Goiri. 2019-06-24 22:03:04 +05:30
Ayush Saxena
0f9b8d7a75 HDFS-14369. RBF: Fix trailing / for webhdfs. Contributed by Akira Ajisaka. 2019-06-24 22:03:04 +05:30
Ayush Saxena
dd8c2b92df HDFS-13853. RBF: RouterAdmin update cmd is overwriting the entry not updating the existing. Contributed by Ayush Saxena. 2019-06-24 22:03:04 +05:30
Ayush Saxena
6c42d40504 HDFS-14316. RBF: Support unavailable subclusters for mount points with multiple destinations. Contributed by Inigo Goiri. 2019-06-24 22:03:04 +05:30
Ayush Saxena
0dbd87874a HDFS-14388. RBF: Prevent loading metric system when disabled. Contributed by Inigo Goiri. 2019-06-24 22:03:03 +05:30
Ayush Saxena
9a9fbbe145 HDFS-14351. RBF: Optimize configuration item resolving for monitor namenode. Contributed by He Xiaoqiao and Inigo Goiri. 2019-06-24 22:03:03 +05:30
Giovanni Matteo Fumarola
f539e2a4ee HDFS-14343. RBF: Fix renaming folders spread across multiple subclusters. Contributed by Ayush Saxena. 2019-06-24 22:03:03 +05:30
Giovanni Matteo Fumarola
2a2d5eb441 HDFS-14334. RBF: Use human readable format for long numbers in the Router UI. Contributed by Inigo Goiri. 2019-06-24 22:03:03 +05:30
Inigo Goiri
fcabc8f0e4 HDFS-14335. RBF: Fix heartbeat typos in the Router. Contributed by CR Hota. 2019-06-24 22:03:03 +05:30
Inigo Goiri
6cdf8db55c HDFS-14331. RBF: IOE While Removing Mount Entry. Contributed by Ayush Saxena. 2019-06-24 22:03:03 +05:30
Inigo Goiri
1ce25e702b HDFS-14329. RBF: Add maintenance nodes to federation metrics. Contributed by Ayush Saxena. 2019-06-24 22:03:03 +05:30
Inigo Goiri
7bbe01a196 HDFS-14259. RBF: Fix safemode message for Router. Contributed by Ranith Sadar. 2019-06-24 22:03:03 +05:30
Inigo Goiri
1c7ab59be3 HDFS-14322. RBF: Security manager should not load if security is disabled. Contributed by CR Hota. 2019-06-24 22:03:03 +05:30
Brahma Reddy Battula
c4b1fa91fa HDFS-14052. RBF: Use Router keytab for WebHDFS. Contributed by CR Hota. 2019-06-24 22:03:03 +05:30
Inigo Goiri
de719b08b5 HDFS-14307. RBF: Update tests to use internal Whitebox instead of Mockito. Contributed by CR Hota. 2019-06-24 22:03:03 +05:30
Giovanni Matteo Fumarola
8b8ff5ccbc HDFS-14249. RBF: Tooling to identify the subcluster location of a file. Contributed by Inigo Goiri. 2019-06-24 22:03:03 +05:30
Giovanni Matteo Fumarola
50aee18a84 HDFS-14268. RBF: Fix the location of the DNs in getDatanodeReport(). Contributed by Inigo Goiri. 2019-06-24 22:03:03 +05:30
Inigo Goiri
e2a3c4494b HDFS-14226. RBF: Setting attributes should set on all subclusters' directories. Contributed by Ayush Saxena. 2019-06-24 22:03:03 +05:30
Brahma Reddy Battula
75f8b6ccfa HDFS-13358. RBF: Support for Delegation Token (RPC). Contributed by CR Hota. 2019-06-24 22:03:03 +05:30
Inigo Goiri
7e63e37dc5 HDFS-14230. RBF: Throw RetriableException instead of IOException when no namenodes available. Contributed by Fei Hui. 2019-06-24 22:03:03 +05:30
Giovanni Matteo Fumarola
912b90f91e HDFS-14252. RBF : Exceptions are exposing the actual sub cluster path. Contributed by Ayush Saxena. 2019-06-24 22:03:03 +05:30
Surendra Singh Lilhore
9c4e55685b HDFS-14225. RBF : MiniRouterDFSCluster should configure the failover proxy provider for namespace. Contributed by Ranith Sardar. 2019-06-24 22:03:03 +05:30
Inigo Goiri
9eed3a49df HDFS-14215. RBF: Remove dependency on availability of default namespace. Contributed by Ayush Saxena. 2019-06-24 22:03:03 +05:30