big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
18,572 Commits 2 Branches 2 Tags 264 MiB
0c37e05b10
Commit Graph

6236 Commits

Author SHA1 Message Date
Tsz-Wo Nicholas Sze
346caa2095 HDFS-13227. Add a method to calculate cumulative diff over multiple snapshots in DirectoryDiffList. Contributed by Shashikant Banerjee 2018-03-06 13:35:03 -08:00
Tsz-Wo Nicholas Sze
871d0d39fa HDFS-13223. Reduce DiffListBySkipList memory usage. Contributed by Shashikant Banerjee 2018-03-06 12:23:03 -08:00
Inigo Goiri
7060725662 HDFS-13188. Disk Balancer: Support multiple block pools during block move. Contributed by Bharat Viswanadham. 2018-03-06 09:09:32 -08:00
Arpit Agarwal
a3c304c5dd HDFS-13178. Disk Balancer: Add a force option to DiskBalancer Execute command. Contributed by Bharat Viswanadham. 2018-03-03 15:42:16 -08:00
Tsz-Wo Nicholas Sze
ba82e5c488 HDFS-13173. Replace ArrayList with DirectoryDiffList(SnapshotSkipList) to store DirectoryDiffs. Contributed by Shashikant Banerjee 2018-03-02 17:47:48 -08:00
Xiao Chen
c75105f07b HDFS-13040. Kerberized inotify client fails despite kinit properly. Contributed by Istvan Fajth, Wei-Chiu Chuang, Xiao Chen. 2018-03-02 16:46:04 -08:00
Tsz-Wo Nicholas Sze
60080fbfcf HDFS-13171. Handle Deletion of nodes in SnasphotSkipList. Contributed by Shashikant Banerjee 2018-03-02 10:51:05 -08:00
Tsz-Wo Nicholas Sze
850b2f2567 HDFS-1686. Federation: Add more Balancer tests with federation setting. Contributed by Bharat Viswanadham 2018-03-02 10:42:39 -08:00
Yiqun Lin
55669515f6 HDFS-13210. Fix the typo in MiniDFSCluster class. Contributed by fang zhenyi. 2018-03-02 11:49:12 +08:00
Tsz-Wo Nicholas Sze
96e8f260ab HDFS-13211. Fix a bug in DirectoryDiffList.getMinListForRange. Contributed by Shashikant Banerjee 2018-03-01 14:12:15 -08:00
Yiqun Lin
ececeea8f2 HDFS-13202. Fix the outdated javadocs in HAUtil. Contributed by Chao Sun. 2018-03-01 17:51:43 +08:00
Tsz-Wo Nicholas Sze
81d9446a92 HDFS-13102. Implement SnapshotSkipList class to store Multi level DirectoryDiffs. Contributed by Shashikant Banerjee 2018-02-28 13:42:58 -08:00
Xiaoyu Yao
f20e10b2dd HDFS-13081. Datanode#checkSecureConfig should allow SASL and privileged HTTP. Contributed by Ajay Kumar. 2018-02-28 09:44:39 -08:00
Xiaoyu Yao
2574375bf5 HDFS-13114. CryptoAdmin#ReencryptZoneCommand should resolve Namespace info from path. Contributed by Hanisha Koneru. 2018-02-28 08:37:01 -08:00
Yiqun Lin
a9c14b1119 HDFS-13194. CachePool permissions incorrectly checked. Contributed by Jianfei Jiang. 2018-02-28 10:17:43 +08:00
Inigo Goiri
d86f301d46 HDFS-13199. RBF: Fix the hdfs router page missing label icon issue. Contributed by maobaolong. 2018-02-27 17:39:02 -08:00
Tsz-Wo Nicholas Sze
55c77bf722 HDFS-13143. SnapshotDiff - snapshotDiffReport might be inconsistent if the snapshotDiff calculation happens between a snapshot and the current tree. Contributed by Shashikant Banerjee 2018-02-27 15:28:41 -08:00
Arpit Agarwal
1cc9a58dda HDFS-13181. DiskBalancer: Add an configuration for valid plan hours . Contributed by Bharat Viswanadham. 2018-02-27 10:18:07 -08:00
Inigo Goiri
cd4a97bf80 HDFS-13192. Change the code order in getFileEncryptionInfo to avoid unnecessary call of assignment. Contributed by LiXin Ge. 2018-02-27 08:53:00 -08:00
Inigo Goiri
225d877092 HDFS-13193. Various Improvements for BlockTokenSecretManager. Contributed by BELUGA BEHR. 2018-02-27 08:48:03 -08:00
Yiqun Lin
1e85a995d1 HDFS-13184. RBF: Improve the unit test TestRouterRPCClientRetries. Contributed by Yiqun Lin. 2018-02-27 10:48:52 +08:00
Chao Sun
ae290a4bb4 HDFS-13145. SBN crash when transition to ANN with in-progress edit tailing enabled. Contributed by Chao Sun. 2018-02-26 16:15:00 -08:00
Wei Yan
6ce9f79cc9 HDFS-13187. RBF: Fix Routers information shown in the web UI. Contributed by Inigo Goiri and Wei Yan. 2018-02-26 15:13:41 -08:00
Arpit Agarwal
46c93453e9 HDFS-12781. After Datanode down, In Namenode UI Datanode tab is throwing warning message.. Contributed by Brahma Reddy Battula. 2018-02-26 13:56:34 -08:00
Kihwal Lee
451265a83d HDFS-12070. Failed block recovery leaves files open indefinitely and at risk for data loss. Contributed by Kihwal Lee. 2018-02-26 10:28:04 -06:00
Arpit Agarwal
2fa7963c3d HADOOP-15254. Correct the wrong word spelling 'intialize'. Contributed by fang zhenyi. 2018-02-24 14:41:55 -08:00
Xiaoyu Yao
1e84e46f16 HDFS-13052. WebHDFS: Add support for snasphot diff. Contributed by Lokesh Jain. 2018-02-23 19:35:12 -08:00
Xiao Chen
51088d3233 HDFS-13164. File not closed if streamer fail with DSQuotaExceededException. 2018-02-23 13:49:09 -08:00
Xiaoyu Yao
84a1321f6a HDFS-13136. Avoid taking FSN lock while doing group member lookup for FSD permission check. Contributed by Xiaoyu Yao. 2018-02-22 11:32:32 -08:00
Anu Engineer
121e1e1280 HDFS-13175. Add more information for checking argument in DiskBalancerVolume. 
Contributed by  Lei (Eddy) Xu.
 2018-02-20 19:16:30 -08:00
Inigo Goiri
6f81cc0bee HDFS-13167. DatanodeAdminManager Improvements. Contributed by BELUGA BEHR. 2018-02-20 15:18:27 -08:00
Inigo Goiri
17c592e6cf HDFS-13168. XmlImageVisitor - Prefer Array over LinkedList. Contributed by BELUGA BEHR. 2018-02-20 15:16:01 -08:00
Arpit Agarwal
9028ccaf83 HDFS-13159. TestTruncateQuotaUpdate fails in trunk. Contributed by Nanda kumar. 2018-02-20 11:40:20 -08:00
Yiqun Lin
8896d20b91 HDFS-13119. RBF: Manage unavailable clusters. Contributed by Yiqun Lin. 2018-02-20 09:37:08 +08:00
Tsz-Wo Nicholas Sze
83e2bb98ee HDFS-12998. SnapshotDiff - Provide an iterator-based listing API for calculating snapshotDiff. Contributed by Shashikant Banerjee 2018-02-19 11:42:10 +08:00
Chen Liang
c008d10a11 HDFS-13161. Update comment in start-dfs.sh to mention correct variable for secure datanode user. Contributed by Ajay Kumar. 2018-02-16 16:04:50 -08:00
Arpit Agarwal
4c2119f04e HDFS-13058. Fix dfs.namenode.shared.edits.dir in TestJournalNode. Contributed by Bharat Viswanadham. 2018-02-16 10:29:22 -08:00
Arpit Agarwal
8d5ea7470a HDFS-13153. Enable HDFS diskbalancer by default. Contributed by Ajay Kumar. 2018-02-16 09:38:34 -08:00
Bharat Viswanadham
a1e05e0292
HDFS-13151. Fix the javadoc error in ReplicaInfo 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-02-16 16:22:24 +09:00
Kihwal Lee
47473952e5 HDFS-13112. Token expiration edits may cause log corruption or deadlock. Contributed by Daryn Sharp. 2018-02-15 15:32:42 -06:00
Kihwal Lee
da59acd8ca xattr api cleanup 2018-02-15 11:11:55 -06:00
Tsz-Wo Nicholas Sze
6ea7d78ccb HDFS-13142. Define and Implement a DiifList Interface to store and manage SnapshotDiffs. Contributed by Shashikant Banerjee 2018-02-15 19:33:44 +08:00
Steve Loughran
f20dc0d577 HADOOP-10571. Use Log.*(Object, Throwable) overload to log exceptions. 
Contributed by Andras Bokor.
 2018-02-14 16:20:14 +00:00
Brahma Reddy Battula
35c17351ca HDFS-8693. Addendum patch to execute the command using UGI. Contributed by Brahma Reddy Battula. 2018-02-12 22:14:34 +05:30
Arpit Agarwal
96bb6a51ec HDFS-10453. ReplicationMonitor thread could stuck for long time due to the race between replication and delete of same file in a large cluster.. Contributed by He Xiaoqiao. 2018-02-12 07:17:40 -08:00
Yiqun Lin
25fbec67d1 HDFS-13130. Log object instance get incorrectly in SlowDiskTracker. Contributed by Jianfei Jiang. 2018-02-11 12:02:10 +08:00
Yiqun Lin
543f3abbee HDFS-13099. RBF: Use the ZooKeeper as the default State Store. Contributed by Yiqun Lin. 2018-02-09 13:57:42 +08:00
Steve Loughran
996796f104 HADOOP-15214. Make Hadoop compatible with Guava 21.0. 
Contributed by Igor Dvorzhak
 2018-02-08 10:55:54 -08:00
Xiaoyu Yao
8faf0b50d4 HDFS-13120. Snapshot diff could be corrupted after concat. Contributed by Xiaoyu Yao. 2018-02-08 08:59:48 -08:00
Yongjun Zhang
f491f717e9 HDFS-13115. In getNumUnderConstructionBlocks(), ignore the inodeIds for which the inodes have been deleted. Contributed by Yongjun Zhang. 2018-02-07 12:58:09 -08:00
Jitendra Pandey
b061215ecf HDFS-11701. NPE from Unresolved Host causes permanent DFSInputStream failures. Contributed by Lokesh Jain. 2018-02-07 11:22:36 -08:00
Brahma Reddy Battula
01bd6ab18f HDFS-12935. Get ambiguous result for DFSAdmin command in HA mode when only one namenode is up. Contributed by Jianfei Jiang. 2018-02-07 23:10:33 +05:30
Tsz-Wo Nicholas Sze
4fadcf1631 HDFS-13107. Add Mover Cli Unit Tests for Federated cluster. Contributed by Bharat Viswanadham 2018-02-06 16:59:26 -08:00
Sean Mackrory
b14db95c35 HDFS-13106. Need to exercise all HDFS APIs for EC. Contributed by Haibo Yan. 2018-02-06 16:39:00 -07:00
Tsz-Wo Nicholas Sze
4d1bf33d0f HDFS-12947. Limit the number of Snapshots allowed to be created for a Snapshottable Directory. Contributed by Shashikant Banerjee 2018-02-06 14:46:44 -08:00
Anu Engineer
4304fcd5bd HDFS-12990. Change default NameNode RPC port back to 8020. Contributed by Xiao Chen. 2018-02-06 13:43:45 -08:00
Wei-Chiu Chuang
2021f4bdce HDFS-11187. Optimize disk access for last partial chunk checksum of Finalized replica. Contributed by Wei-Chiu Chuang. 2018-02-02 17:18:42 -08:00
Hanisha Koneru
0ef639235b HDFS-13073. Cleanup code in InterQJournalProtocol.proto. Contributed by Bharat Viswanadham. 2018-02-02 16:28:03 -08:00
Akira Ajisaka
4aef8bd2ef
HDFS-13048. LowRedundancyReplicatedBlocks metric can be negative 2018-02-02 14:34:07 +09:00
Yiqun Lin
b0627c891b HDFS-13068. RBF: Add router admin option to manage safe mode. Contributed by Yiqun Lin. 2018-02-02 11:25:41 +08:00
Anu Engineer
aa45faf0b2 HDFS-12942. Synchronization issue in FSDataSetImpl#moveBlock. Contributed by Ajay Kumar. 2018-02-01 18:03:01 -08:00
Hanisha Koneru
dd50f53997 HDFS-13062. Provide support for JN to use separate journal disk per namespace. Contributed by Bharat Viswanadham. 2018-02-01 12:28:17 -08:00
Xiaoyu Yao
b3ae11d597 HDFS-12997. Move logging to slf4j in BlockPoolSliceStorage and Storage. Contributed by Ajay Kumar. 2018-02-01 10:45:34 -08:00
Inigo Goiri
6ca7204ceb HDFS-13043. RBF: Expose the state of the Routers in the federation. Contributed by Inigo Goiri. 2018-02-01 10:37:14 -08:00
Xiao Chen
ae2177d296 HDFS-12897. getErasureCodingPolicy should handle .snapshot dir better. Contributed by LiXin Ge. 2018-01-31 22:50:25 -08:00
Xiaoyu Yao
af015c0b23 HDFS-13060. Adding a BlacklistBasedTrustedChannelResolver for TrustedChannelResolver. Contributed by Ajay Kumar. 2018-01-31 22:39:51 -08:00
Hanisha Koneru
3ce2190b58 HDFS-13092. Reduce verbosity for ThrottledAsyncChecker#schedule. Contributed by Mukul Kumar Singh. 2018-01-31 11:05:17 -08:00
Xiaoyu Yao
37b7536568 HDFS-13061. SaslDataTransferClient#checkTrustAndSend should not trust a partially trusted channel. Contributed by Ajay Kumar. 2018-01-31 10:47:02 -08:00
Inigo Goiri
5206b2c7ca HDFS-13083. RBF: Fix doc error setting up client. Contributed by tartarus. 2018-01-30 18:27:18 -08:00
Weiwei Yang
2e7331ca26 HDFS-12528. Add an option to not disable short-circuit reads on failures. Contributed by Xiao Chen. 2018-01-31 07:25:03 +08:00
Yiqun Lin
dbb9dded33 HDFS-13044. RBF: Add a safe mode for the Router. Contributed by Inigo Goiri. 2018-01-30 12:12:08 +08:00
Kihwal Lee
fde95d463c HDFS-12574. Add CryptoInputStream to WebHdfsFileSystem read call. Contributed by Rushabh S Shah 2018-01-29 17:23:29 -06:00
Xiao Chen
b63dcd583f HDFS-12974. Exception message is not printed when creating an encryption zone fails with AuthorizationException. Contributed by fang zhenyi. 2018-01-28 22:19:49 -08:00
Xiao Chen
6bc2f7f4b4 HDFS-13065. TestErasureCodingMultipleRacks#testSkewedRack3 is failing. Contributed by Gabor Bota. 2018-01-28 22:12:05 -08:00
Arpit Agarwal
e990904dd5 HDFS-13054. Handling PathIsNotEmptyDirectoryException in DFSClient delete call. Contributed by Nanda kumar. 2018-01-26 13:09:13 -08:00
Yiqun Lin
7721fff744 HDFS-13042. RBF: Heartbeat Router State. Contributed by Inigo Goiri. 2018-01-25 15:51:26 +08:00
Jianfei Jiang
eb2dd08688
HDFS-13063. Fix the incorrect spelling in HDFSHighAvailabilityWithQJM.md 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-01-25 15:30:42 +09:00
Inigo Goiri
95743c672e HDFS-12772. RBF: Federation Router State State Store internal API. Contributed by Inigo Goiri. 2018-01-23 19:15:44 -08:00
Yiqun Lin
d95c13774e HDFS-12963. Error log level in ShortCircuitRegistry#removeShm. Contributed by hu xiaodong. 2018-01-24 10:43:36 +08:00
Hanisha Koneru
6347b2253d HDFS-13023. Journal Sync does not work on a secure cluster. Contributed by Bharat Viswanadham. 2018-01-22 16:02:32 -08:00
Chen Liang
62c9e7fa99 HDFS-13036. Reusing the volume storage ID obtained by replicaInfo. Contributed by liaoyuxiangqin. 2018-01-19 17:51:53 -08:00
Lei Xu
2ed9d61aad HDFS-13039. StripedBlockReader.createBlockReader leaks socket on IOException. (Lei (Eddy) Xu) 2018-01-19 16:40:58 -08:00
Inigo Goiri
130f8bce00 HDFS-13028. RBF: Fix spurious TestRouterRpc#testProxyGetStats. Contributed by Inigo Goiri. 2018-01-19 09:47:40 -08:00
Yiqun Lin
9e4f52d323 HDFS-12973. RBF: Document global quota supporting in federation. Contributed by Yiqun Lin. 2018-01-19 14:18:19 +08:00
Vinayakumar B
09efdfe9e1 HDFS-9049. Make Datanode Netty reverse proxy port to be configurable. Contributed by Vinayakumar B. 2018-01-17 14:16:48 +05:30
Lei Xu
3bd9ea63df HDFS-13004. TestLeaseRecoveryStriped.testLeaseRecovery is failing when safeLength is 0MB or larger than the test file. (Zsolt Venczel via lei) 2018-01-16 15:15:11 -08:00
Yufei Gu
b202935353 HADOOP-15157. Zookeeper authentication related properties to support CredentialProviders. (Contributed by Gergo Repas) 2018-01-16 10:15:22 -08:00
Brahma Reddy Battula
880b9d24ff HDFS-8693. refreshNamenodes does not support adding a new standby to a running DN. Contributed by Ajith S. 2018-01-16 16:21:02 +05:30
Yiqun Lin
9afb8025d6 HDFS-12972. RBF: Display mount table quota info in Web UI and admin command. Contributed by Yiqun Lin. 2018-01-13 13:16:43 +08:00
Inigo Goiri
d5d6a0353b HDFS-12919. RBF: Support erasure coding methods in RouterRpcServer. Contributed by Inigo Goiri. 2018-01-12 16:25:39 -08:00
Arpit Agarwal
b278f7b293 HDFS-12984. BlockPoolSlice can leak in a mini dfs cluster. Contributed by Ajay Kumar. 2018-01-12 08:04:00 -08:00
Yiqun Lin
d98a2e6e23 HDFS-12934. RBF: Federation supports global quota. Contributed by Yiqun Lin. 2018-01-10 13:59:11 +08:00
Inigo Goiri
d9006d8a4e HDFS-12802. RBF: Control MountTableResolver cache size. Contrubuted by Inigo Goiri. 2018-01-09 18:53:25 -08:00
Lei Xu
47563d86fe HDFS-12994. TestReconstructStripedFile.testNNSendsErasureCodingTasks fails due to socket timeout. (Contributed by Lei (Eddy) Xu) 2018-01-09 11:53:49 -08:00
Inigo Goiri
b3290c41a9 HDFS-12808. Add LOG.isDebugEnabled() guard for LOG.debug(...). Contributed by Bharat Viswanadham. 2018-01-08 17:51:09 -08:00
Manoj Govindassamy
73ff09b79a HDFS-12985. NameNode crashes during restart after an OpenForWrite file present in the Snapshot got deleted. 2018-01-08 15:34:00 -08:00
Wei Yan
2ee0d64ace HDFS-12945. Switch to ClientProtocol instead of NamenodeProtocols in NamenodeWebHdfsMethods. Contributed by Wei Yan. 2018-01-08 14:25:39 -08:00
Yiqun Lin
bf5c948995 HDFS-11848. Enhance dfsadmin listOpenFiles command to list files under a given path. Contributed by Yiqun Lin. 2018-01-06 14:31:08 +08:00
Inigo Goiri
83b513ac6d HDFS-12988. RBF: Mount table entries not properly updated in the local cache. Contributed by Inigo Goiri. 2018-01-05 09:11:38 -08:00
Lei Xu
dc735b286b HDFS-12860. StripedBlockUtil#getRangesInternalBlocks throws exception for the block group size larger than 2GB. (Contributed by Lei (Eddy) Xu) 2018-01-04 10:54:56 -08:00
Arpit Agarwal
739d3c394d HDFS-12987. Document - Disabling the Lazy persist file scrubber.. Contributed by Karthik Palanisamy. 2018-01-04 10:32:00 -08:00
Sean Mackrory
45a47198b4 HDFS-12913. TestDNFencingWithReplication sometimes fails because of NameNode in standby state. Contributed by Zsolt Venczel. 2018-01-04 08:37:21 -07:00
Yiqun Lin
2a48b3594c HDFS-12948. DiskBalancer report command top option should only take positive numeric values. Contributed by Shashikant Banerjee. 2018-01-04 10:48:44 +08:00
Xiaoyu Yao
3ba985997d HDFS-12931. Handle InvalidEncryptionKeyException during DistributedFileSystem#getFileChecksum. Contributed by Mukul Kumar Singh. 2018-01-03 14:54:20 -08:00
Manoj Govindassamy
5c28804b95 HDFS-12629. NameNode UI should report total blocks count by type - replicated and erasure coded. 2018-01-02 15:05:27 -08:00
Manoj Govindassamy
42a1c98597 HDFS-11847. Enhance dfsadmin listOpenFiles command to list files blocking datanode decommissioning. 2018-01-02 14:59:36 -08:00
Arpit Agarwal
b4d11337c9 HDFS-12351. Explicitly describe the minimal number of DataNodes required to support an EC policy in EC document.. Contributed by Hanisha Koneru. 2018-01-02 12:54:40 -08:00
Chris Douglas
7fe6f83c8f HADOOP-15117. open(PathHandle) contract test should be exhaustive for default options 2017-12-30 17:58:35 -08:00
Lei Xu
6e3e1b8cde HDFS-12915. Fix findbugs warning in INodeFile$HeaderFormat.getBlockLayoutRedundancy. (Contributed by Chris Douglas) 2017-12-29 12:21:57 -08:00
Xiao Chen
5bf7e594d7 HDFS-9023. When NN is not able to identify DN for replication, reason behind it can be logged. 2017-12-28 11:54:08 -08:00
Manoj Govindassamy
76e664e931 HDFS-12959. Fix TestOpenFilesWithSnapshot redundant configurations. 2017-12-21 15:47:15 -08:00
Chen Liang
826507c41b HDFS-12951. Incorrect javadoc in SaslDataTransferServer.java#receive. Contributed by Mukul Kumar Singh. 2017-12-21 11:20:30 -08:00
Lei Xu
b318bed01a HDFS-12938. TestErasureCodigCLI testAll failing consistently. (Contributed by Ajay Kumar) 2017-12-21 10:28:24 -08:00
Wei Yan
a78db99190 HDFS-12932. Fix confusing LOG message for block replication. Contributed by Chao Sun. 2017-12-20 08:55:46 -08:00
Yiqun Lin
e040c97b77 HDFS-12937. RBF: Add more unit tests for router admin commands. Contributed by Yiqun Lin. 2017-12-19 15:31:34 +08:00
Yiqun Lin
25a36b7452 HDFS-12930. Remove the extra space in HdfsImageViewer.md. Contributed by Rahul Pathak. 2017-12-19 11:23:16 +08:00
Tsz-Wo Nicholas Sze
c7499f2d24 HDFS-12347. TestBalancerRPCDelay#testBalancerRPCDelay fails very frequently. Contributed by Bharat Viswanadham 2017-12-19 10:02:30 +08:00
Erik Krogen
94576b17fb HDFS-12818. Support multiple storages in DataNodeCluster / SimulatedFSDataset. Contributed by Erik Krogen. 2017-12-18 11:36:22 -08:00
Virajith Jalaparti
8239e3afb3 HDFS-12712. [9806] Code style cleanup 2017-12-15 17:51:41 -08:00
Virajith Jalaparti
80c3fec3a1 HDFS-12912. [READ] Fix configuration and implementation of LevelDB-based alias maps 2017-12-15 17:51:41 -08:00
Virajith Jalaparti
c89b29bd42 HDFS-12893. [READ] Support replication of Provided blocks with non-default topologies. 2017-12-15 17:51:41 -08:00
Virajith Jalaparti
0f6aa9564c HDFS-12905. [READ] Handle decommissioning and under-maintenance Datanodes with Provided storage. 2017-12-15 17:51:41 -08:00
Chris Douglas
2298f2d76b HDFS-12874. Documentation for provided storage. Contributed by Virajith Jalaparti 2017-12-15 17:51:41 -08:00
Virajith Jalaparti
71ec170107 HDFS-12887. [READ] Allow Datanodes with Provided volumes to start when blocks with the same id exist locally 2017-12-15 17:51:41 -08:00
Virajith Jalaparti
4531588a94 HDFS-11640. [READ] Datanodes should use a unique identifier when reading from external stores 2017-12-15 17:51:41 -08:00
Virajith Jalaparti
fb996a32a9 HDFS-12894. [READ] Skip setting block count of ProvidedDatanodeStorageInfo on DN registration update 2017-12-15 17:51:40 -08:00
Virajith Jalaparti
9c35be86e1 HDFS-12713. [READ] Refactor FileRegion and BlockAliasMap to separate out HDFS metadata and PROVIDED storage metadata. Contributed by Ewan Higgs 2017-12-15 17:51:40 -08:00
Virajith Jalaparti
a027055dd2 HDFS-12885. Add visibility/stability annotations. Contributed by Chris Douglas 2017-12-15 17:51:40 -08:00
Virajith Jalaparti
b634053c4d HDFS-12591. [READ] Implement LevelDBFileRegionFormat. Contributed by Ewan Higgs. 2017-12-15 17:51:40 -08:00
Virajith Jalaparti
352f994b64 HDFS-12665. [AliasMap] Create a version of the AliasMap that runs in memory in the Namenode (leveldb). Contributed by Ewan Higgs. 2017-12-15 17:51:40 -08:00
Virajith Jalaparti
cc933cba77 HDFS-12685. [READ] FsVolumeImpl exception when scanning Provided storage volume 2017-12-15 17:51:40 -08:00
Virajith Jalaparti
4d59dabb7f HDFS-12809. [READ] Fix the randomized selection of locations in {{ProvidedBlocksBuilder}}. 2017-12-15 17:51:40 -08:00
Virajith Jalaparti
3d3be87e30 HDFS-12778. [READ] Report multiple locations for PROVIDED blocks 2017-12-15 17:51:39 -08:00
Virajith Jalaparti
3b1d30301b HDFS-12775. [READ] Fix reporting of Provided volumes 2017-12-15 17:51:39 -08:00
Virajith Jalaparti
e1a28f95b8 HDFS-12777. [READ] Reduce memory and CPU footprint for PROVIDED volumes. 2017-12-15 17:51:39 -08:00
Virajith Jalaparti
6cd80b2521 HDFS-12779. [READ] Allow cluster id to be specified to the Image generation tool 2017-12-15 17:51:39 -08:00
Virajith Jalaparti
90d1b47a2a HDFS-12776. [READ] Increasing replication for PROVIDED files should create local replicas 2017-12-15 17:51:39 -08:00
Virajith Jalaparti
71d0a82571 HDFS-12607. [READ] Even one dead datanode with PROVIDED storage results in ProvidedStorageInfo being marked as FAILED 2017-12-15 17:51:39 -08:00
Virajith Jalaparti
98f5ed5aa3 HDFS-11902. [READ] Merge BlockFormatProvider and FileRegionProvider. 2017-12-15 17:51:38 -08:00
Virajith Jalaparti
d6a9a89973 HDFS-12605. [READ] TestNameNodeProvidedImplementation#testProvidedDatanodeFailures fails after rebase 2017-12-15 17:51:38 -08:00
Virajith Jalaparti
aca023b72c HDFS-12289. [READ] HDFS-12091 breaks the tests for provided block reads 2017-12-15 17:51:38 -08:00
Virajith Jalaparti
2407c9b93a HDFS-12093. [READ] Share remoteFS between ProvidedReplica instances. 2017-12-15 17:51:38 -08:00
Virajith Jalaparti
663b3c08b1 HDFS-12091. [READ] Check that the replicas served from a ProvidedVolumeImpl belong to the correct external storage 2017-12-15 17:51:38 -08:00
Virajith Jalaparti
546b95f484 HDFS-11673. [READ] Handle failures of Datanode with PROVIDED storage 2017-12-15 17:51:38 -08:00
Virajith Jalaparti
55ade54b8e HDFS-11792. [READ] Test cases for ProvidedVolumeDF and ProviderBlockIteratorImpl 2017-12-15 17:51:38 -08:00
Virajith Jalaparti
89b9faf529 HDFS-11703. [READ] Tests for ProvidedStorageMap 2017-12-15 17:51:37 -08:00
Virajith Jalaparti
aa5ec85f7f HDFS-11663. [READ] Fix NullPointerException in ProvidedBlocksBuilder 2017-12-15 17:51:37 -08:00
Virajith Jalaparti
1108cb7691 HDFS-11653. [READ] ProvidedReplica should return an InputStream that is bounded by its length 2017-12-15 17:51:37 -08:00
Virajith Jalaparti
d65df0f273 HDFS-11190. [READ] Namenode support for data stored in external stores. 2017-12-15 17:51:37 -08:00
Virajith Jalaparti
b668eb9155 HDFS-10675. Datanode support to read from external stores. 2017-12-15 17:51:37 -08:00
Chen Liang
aa503a29d0 HDFS-12917. Fix description errors in testErasureCodingConf.xml. Contributed by chencan. 2017-12-15 13:45:37 -08:00
Lei Xu
949be14b08 HDFS-12927. Update erasure coding doc to address unsupported APIs. (Contributed by Lei (Eddy) Xu) 2017-12-15 10:20:29 -08:00
Lei Xu
1c15b1751c HDFS-12819. Setting/Unsetting EC policy shows warning if the directory is not empty. (Contributed by Lei (Eddy) Xu) 2017-12-15 10:04:43 -08:00
Yiqun Lin
ee028bfdf1 HDFS-12895. RBF: Add ACL support for mount table. Contributed by Yiqun Lin. 2017-12-15 14:09:24 +08:00
Xiao Chen
e1cb278cd0 HDFS-12910. Secure Datanode Starter should log the port when it fails to bind. Contributed by Stephen O'Donnell and Nanda kumar. 2017-12-14 19:22:51 -08:00
Kihwal Lee
f5a72424c0 HDFS-12907. Allow read-only access to reserved raw for non-superusers. Contributed by Rushabh S Shah. 2017-12-14 14:58:47 -06:00
Jason Lowe
80db744ee5 HDFS-12881. Output streams closed with IOUtils suppressing write errors. Contributed by Ajay Kumar 2017-12-14 09:45:44 -06:00
Yiqun Lin
91c96bdf8e HDFS-12883. RBF: Document Router and State Store metrics. Contributed by Yiqun Lin. 2017-12-14 16:00:53 +08:00
Chen Liang
46e18c8da7 HADOOP-14914. Change to a safely casting long to int. Contributed by Ajay Kumar. 2017-12-13 14:56:14 -08:00
Chris Douglas
693169ef34 HDFS-12882. Support full open(PathHandle) contract in HDFS 2017-12-11 20:14:15 -08:00
Wei-Chiu Chuang
55fc2d6485 HDFS-12891. Do not invalidate blocks if toInvalidate is empty. Contributed by Zsolt Venczel. 2017-12-11 16:43:03 -08:00
Inigo Goiri
5cd1056ad7 HDFS-12875. RBF: Complete logic for -readonly option of dfsrouteradmin add command. Contributed by Inigo Goiri. 2017-12-11 15:14:57 -08:00
Manoj Govindassamy
ef7d334d36 HDFS-12825. Fsck report shows config key name for min replication issues (Contributed by Gabor Bota). 2017-12-08 11:46:30 -08:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
f19638333b HADOOP-15059. Undoing the switch of Credentials to PB format as default - done via HADOOP-12563 for supporting 2.x to 3.x upgrades. 2017-12-08 08:00:21 -08:00
Xiao Chen
19e0894209 HADOOP-15056. Fix TestUnbuffer#testUnbufferException failure. Contributed by Jack Bearden. 2017-12-07 21:05:55 -08:00
Wei-Chiu Chuang
d6c31a3e6b HDFS-11915. Sync rbw dir on the first hsync() to avoid file lost on power failure. Contributed by Vinayakumar B. 2017-12-07 14:50:30 -08:00
Lei Xu
67662e2ac9 HDFS-12840. Creating a file with non-default EC policy in a EC zone is not correctly serialized in the editlog. Contributed by Lei (Eddy) Xu. 2017-12-07 11:15:40 -08:00
Akira Ajisaka
6cca5b3bcb
HDFS-12889. Addendum patch to add missing file. 2017-12-07 13:23:37 +09:00
Xiao Chen
56b1ff80dd HDFS-12872. EC Checksum broken when BlockAccessToken is enabled. 2017-12-05 20:48:16 -08:00
Brahma Reddy Battula
6d16a99ae8 HDFS-11751. DFSZKFailoverController daemon exits with wrong status code. Contributed by Bharat Viswanadham 2017-12-05 19:53:46 +05:30
Arpit Agarwal
e00c7f78c1 HADOOP-14976. Set HADOOP_SHELL_EXECNAME explicitly in scripts. 2017-12-04 21:02:04 -08:00
Xiao Chen
404eab4dc0 HDFS-12396. Webhdfs file system should get delegation token from kms provider. Contributed by Rushabh S Shah. 2017-12-04 10:40:11 -08:00
Chris Douglas
42307e3c3a HDFS-11576. Block recovery will fail indefinitely if recovery time > heartbeat interval. Contributed by Lukas Majercak 2017-12-01 22:34:30 -08:00
Wei-Chiu Chuang
0faf506245 HDFS-12836. startTxId could be greater than endTxId when tailing in-progress edit log. Contributed by Chao Sun. 2017-12-01 12:01:21 -08:00
Chris Douglas
53bbef3802 Revert "HDFS-11576. Block recovery will fail indefinitely if recovery time > heartbeat interval. Contributed by Lukas Majercak" 
This reverts commit 5304698dc8.
 2017-12-01 11:19:38 -08:00
Chris Douglas
5304698dc8 HDFS-11576. Block recovery will fail indefinitely if recovery time > heartbeat interval. Contributed by Lukas Majercak 2017-12-01 10:29:30 -08:00
Konstantin V Shvachko
60fd0d7fd7 HDFS-12638. Delete copy-on-truncate block along with the original block, when deleting a file being truncated. Contributed by Konstantin Shvachko. 2017-11-30 18:18:28 -08:00
Tsz-Wo Nicholas Sze
b1c7654ee4 HDFS-12594. snapshotDiff fails if the report exceeds the RPC response limit. Contributed by Shashikant Banerjee 2017-11-30 12:18:29 -08:00
Chris Douglas
0e560f3b8d HDFS-12681. Make HdfsLocatedFileStatus a subtype of LocatedFileStatus 2017-11-29 20:28:06 -08:00
Kihwal Lee
53509f295b HDFS-11754. Make FsServerDefaults cache configurable. Contributed by Mikhail Erofeev. 2017-11-29 15:11:14 -06:00
Wei Yan
301641811d HDFS-12835. Fix the javadoc errors in Router-based federation. 2017-11-29 09:43:03 -08:00
Konstantin V Shvachko
d331762f24 HDFS-12832. INode.getFullPathName may throw ArrayIndexOutOfBoundsException lead to NameNode exit. Contribuited by Konstantin Shvachko. 2017-11-28 17:14:23 -08:00
Surendra Singh Lilhore
30941d99c9 HDFS-12857. StoragePolicyAdmin should support schema based path. Contributed by Surendra Singh Lilhore. 2017-11-28 23:57:03 +05:30
Yiqun Lin
94bed50471 HDFS-12858. Add router admin commands usage in HDFS commands reference doc. Contributed by Yiqun Lin. 2017-11-28 11:48:55 +08:00
Kihwal Lee
738d1a206a HDFS-12754. Lease renewal can hit a deadlock. Contributed by Kuhu Shukla. 2017-11-22 13:01:55 -06:00
Lei Xu
785732c13e HDFS-12847. Regenerate editsStored and editsStored.xml in HDFS tests. Contributed by Lei (Eddy) Xu. 2017-11-22 10:22:32 -08:00
Chen Liang
60fc2a1388 HDFS-12804. Use slf4j instead of log4j in FSEditLog. Contributed by Mukul Kumar Singh. 2017-11-20 12:49:53 -08:00
Manoj Govindassamy
9fb4effd2c HDFS-12730. Verify open files captured in the snapshots across config disable and enable. 2017-11-20 10:28:12 -08:00
Inigo Goiri
e182e77794 HDFS-12801. RBF: Set MountTableResolver as default file resolver. Contributed by Inigo Goiri. 2017-11-16 16:58:47 -08:00
Weiwei Yang
462e25a3b2 HDFS-12814. Add blockId when warning slow mirror/disk in BlockReceiver. Contributed by Jiandan Yang. 2017-11-16 16:19:53 +08:00
Chris Douglas
675e9a8f57 Revert "HDFS-12681. Fold HdfsLocatedFileStatus into HdfsFileStatus." 
This reverts commit b85603e3f8.
 2017-11-15 19:20:37 -08:00
Arpit Agarwal
4908a8970e HDFS-12705. WebHdfsFileSystem exceptions should retain the caused by exception. Contributed by Hanisha Koneru. 2017-11-13 11:30:39 -08:00
Arpit Agarwal
6d201f77c7 HDFS-12498. Journal Syncer is not started in Federated + HA cluster. Contributed by Bharat Viswanadham. 2017-11-10 16:30:38 -08:00
Arpit Agarwal
1d6f8bebe9 HADOOP-14929. Cleanup usage of decodecomponent and use QueryStringDecoder from netty. Contributed by Bharat Viswanadham. 2017-11-10 16:28:12 -08:00
Chen Liang
10a1f557e7 HDFS-12791. NameNode Fsck http Connection can timeout for directories with multiple levels. Contributed by Mukul Kumar Singh. 2017-11-09 18:47:34 -08:00
Akira Ajisaka
3a3566e1d1
HDFS-12732. Correct spellings of ramdomly to randomly in log. Contributed by hu xiaodong. 2017-11-09 15:14:46 +09:00
Brahma Reddy Battula
410d0319cf HDFS-12788. Reset the upload button when file upload fails. Contributed by Brahma Reddy Battula 2017-11-08 14:41:16 +05:30
Weiwei Yang
bb8a6eea52 HDFS-7060. Avoid taking locks when sending heartbeats from the DataNode. Contributed by Jiandan Yang. 2017-11-08 10:22:13 +08:00
Xiaoyu Yao
169cdaa38e HDFS-10528. Add logging to successful standby checkpointing. Contributed by Xiaoyu Yao. 2017-11-03 20:34:40 -07:00
Chris Douglas
b85603e3f8 HDFS-12681. Fold HdfsLocatedFileStatus into HdfsFileStatus. 2017-11-03 14:30:57 -07:00
Kihwal Lee
4d2dce40bb HDFS-12771. Add genstamp and block size to metasave Corrupt blocks list. Contributed by Kuhu Shukla. 2017-11-03 15:16:46 -05:00
Xiao Chen
299d38295d HDFS-11467. Support ErasureCoding section in OIV XML/ReverseXML. Contributed by Huafeng Wang. 2017-11-03 12:05:45 -07:00
Xiao Chen
b00f828d84 HDFS-12725. BlockPlacementPolicyRackFaultTolerant fails with very uneven racks. 2017-11-02 21:53:13 -07:00
Xiao Chen
e565b5277d HDFS-12682. ECAdmin -listPolicies will always show SystemErasureCodingPolicies state as DISABLED. 2017-11-02 21:27:35 -07:00
Weiwei Yang
56b88b0670 HDFS-12744. More logs when short-circuit read is failed and disabled. Contributed by Weiwei Yang. 2017-11-01 16:41:45 +08:00
Yiqun Lin
20304b91cc HDFS-12219. Javadoc for FSNamesystem#getMaxObjects is incorrect. Contributed by Erik Krogen. 2017-11-01 14:37:08 +08:00
Lei Xu
9367c25dbd HDFS-12482. Provide a configuration to adjust the weight of EC recovery tasks to adjust the speed of recovery. (lei) 2017-10-31 21:58:14 -07:00
Andrew Wang
5f681fa821 Revert "HDFS-12499. dfs.namenode.shared.edits.dir property is currently namenode specific key. Contributed by Bharat Viswanadham." 
This reverts commit b922ba7393.
 2017-10-31 10:46:10 -07:00
Arpit Agarwal
b922ba7393 HDFS-12499. dfs.namenode.shared.edits.dir property is currently namenode specific key. Contributed by Bharat Viswanadham. 2017-10-31 10:23:00 -07:00
Inigo Goiri
982bd2a5bf HDFS-12699. TestMountTable fails with Java 7. Contributed by Inigo Goiri. 2017-10-31 10:21:42 -07:00
Chris Douglas
d015e0bbd5 HDFS-7878. API - expose a unique file identifier. 2017-10-31 09:44:01 -07:00
Sean Mackrory
81225430d0 HDFS-206. Support for head in FSShell. Contributed by Gabor Bota. 2017-10-31 08:34:01 -06:00
Chris Douglas
d55a84951a HDFS-12582. Replace HdfsFileStatus constructor with a builder pattern. Contributed by Bharat Viswanadham 2017-10-27 15:36:03 -07:00
Arpit Agarwal
7da15eac7a HDFS-12579. JournalNodeSyncer should use fromUrl field of EditLogManifestResponse to construct servlet Url. Contributed by Hanisha Koneru. 2017-10-25 14:24:22 -07:00
Xiao Chen
f66ad1fab9 HADOOP-14957. ReconfigurationTaskStatus is exposing guava Optional in its public api. Contributed by Haibo Chen, Xiao Chen. 2017-10-25 14:11:30 -07:00
Manoj Govindassamy
075dd45a24 HDFS-12544. SnapshotDiff - support diff generation on any snapshot root descendant directory. 2017-10-25 10:54:40 -07:00
Zhe Zhang
17cd8d0c17 Revert "HDFS-12502. nntop should support a category based on FilesInGetListingOps." 
This reverts commit 60bfee270e.
 2017-10-24 15:06:50 -07:00
Wei-Chiu Chuang
0f2a69127a HDFS-12249. dfsadmin -metaSave to output maintenance mode blocks. Contributed by Wellington Chevreuil. 2017-10-23 15:24:34 -07:00
Arpit Agarwal
96be795656 HDFS-12650. Use slf4j instead of log4j in LeaseManager. Contributed by Ajay Kumar. 2017-10-23 13:20:46 -07:00
Arpit Agarwal
4b00c9a47f HDFS-12683. DFSZKFailOverController re-order logic for logging Exception. Contributed by Bharat Viswanadham. 2017-10-23 10:47:16 -07:00
Xiao Chen
248d9b6fff HDFS-12518. Re-encryption should handle task cancellation and progress better. 2017-10-20 16:25:04 -07:00
Andrew Wang
0477eff8be HDFS-12497. Re-enable TestDFSStripedOutputStreamWithFailure tests. Contributed by Huafeng Wang. 2017-10-20 13:27:21 -07:00
Kai Zheng
ce7cf66e5e HDFS-12448. Make sure user defined erasure coding policy ID will not overflow. Contributed by Huafeng Wang 2017-10-20 09:42:04 +08:00
Inigo Goiri
0f1c037618 HDFS-12620. Backporting HDFS-10467 to branch-2. Contributed by Inigo Goiri. 2017-10-19 18:08:45 -07:00
Wei-Chiu Chuang
4ab0c8f96a HDFS-12619. Do not catch and throw unchecked exceptions if IBRs fail to process. Contributed by Wei-Chiu Chuang. 2017-10-19 06:17:59 -07:00
Zhe Zhang
60bfee270e HDFS-12502. nntop should support a category based on FilesInGetListingOps. 2017-10-18 23:51:24 -07:00
Lei Xu
f27a4ad032 HDFS-12612. DFSStripedOutputStream.close will throw if called a second time with a failed streamer. (Lei (Eddy) Xu) 2017-10-17 15:52:09 -07:00
Lei Xu
31ebccc962 HDFS-12613. Native EC coder should implement release() as idempotent function. (Lei (Eddy) Xu) 2017-10-16 19:44:30 -07:00
Manoj Govindassamy
b406d8e375 HDFS-12614. FSPermissionChecker#getINodeAttrs() throws NPE when INodeAttributesProvider configured. 2017-10-16 17:42:41 -07:00
Wei-Chiu Chuang
8dbc8909c9 HDFS-12485. expunge may fail to remove trash from encryption zone. Contributed by Wei-Chiu Chuang. 2017-10-16 12:57:48 -07:00
Xiao Chen
21bc855587 HDFS-12642. Log block and datanode details in BlockRecoveryWorker. 2017-10-16 10:34:06 -07:00
Xiao Chen
7bd700941d HDFS-12637. Extend TestDistributedFileSystemWithECFile with a random EC policy. Contributed by Takanobu Asanuma. 2017-10-16 09:55:22 -07:00
Xiao Chen
035c6ee587 HDFS-12603. Enable async edit logging by default. Contributed by Andrew Wang. 2017-10-16 09:51:10 -07:00
Xiao Chen
20575ececb HDFS-12659. Update TestDeadDatanode#testNonDFSUsedONDeadNodeReReg to increase heartbeat recheck interval. Contributed by Ajay Kumar. 2017-10-15 19:28:22 -07:00
Manoj Govindassamy
133d7ca76e HADOOP-13055. Implement linkMergeSlash and linkFallback for ViewFileSystem 2017-10-13 17:43:21 -07:00
Arpit Agarwal
8dd1eeb94f HDFS-12553. Add nameServiceId to QJournalProtocol. Contributed by Bharat Viswanadham 2017-10-13 14:22:21 -07:00
Arpit Agarwal
8acdf5c274 HDFS-12542. Update javadoc and documentation for listStatus. Contributed by Ajay Kumar. 2017-10-11 12:29:35 -07:00
Arpit Agarwal
bb0a742aac HDFS-12627. Fix typo in DFSAdmin command output. Contributed by Ajay Kumar. 2017-10-11 12:26:14 -07:00
Akira Ajisaka
dc63a6a52b
HDFS-12622. Fix enumerate in HDFSErasureCoding.md. Contributed by Yiqun Lin. 2017-10-11 18:09:53 +09:00
Andrew Wang
a297fb0886 HDFS-12547. Extend TestQuotaWithStripedBlocks with a random EC policy. Contributed by Takanobu Asanuma. 2017-10-10 17:35:49 -07:00
Manoj Govindassamy
78af6cdc53 HDFS-12573. Divide the total blocks metrics into replicated and erasure coded. Contributed by Takanobu Asanuma. 2017-10-10 14:23:29 -07:00
Jitendra Pandey
f29e55bf5e HADOOP-14910. Upgrade netty-all jar to latest 4.0.x.Final. Contributed by Vinayakumar B. 2017-10-10 11:00:07 -07:00
Andrew Wang
82cd85d868 Revert "HDFS-12603. Enable async edit logging by default." 
This reverts commit afb42aeabf.
 2017-10-09 17:53:56 -07:00
Andrew Wang
7938208233 HDFS-12541. Extend TestSafeModeWithStripedFile with a random EC policy. Contributed by Takanobu Asanuma. 2017-10-09 13:03:32 -07:00
Andrew Wang
afb42aeabf HDFS-12603. Enable async edit logging by default. 2017-10-09 11:20:00 -07:00
Arpit Agarwal
09ad848b64 HDFS-12599. Remove Mockito dependency from DataNodeTestUtils. Contributed by Ted Yu. 2017-10-09 10:29:01 -07:00
Lei Xu
46644319e1 HDFS-12606. When using native decoder, DFSStripedStream.close crashes JVM after being called multiple times. (Lei (Eddy) Xu) 2017-10-09 10:08:30 -07:00
Inigo Goiri
5d63a388d1 HDFS-12577. Rename Router tooling. Contributed by Inigo Goiri. 
(cherry picked from commit 53e8d0d030525e4c7f3875e23807c6dbe778890f)
 2017-10-06 18:50:52 -07:00
Inigo Goiri
81601dac8e HDFS-12273. Federation UI. Contributed by Inigo Goiri. 
(cherry picked from commit adbb2e00c7b85524fd43bd68895d49814c16680a)
 2017-10-06 18:50:52 -07:00
Inigo Goiri
6c69e23dcd HDFS-12580. Rebasing HDFS-10467 after HDFS-12447. Contributed by Inigo Goiri. 
(cherry picked from commit b12440d3479f19138bc66ea59baf41eb89061906)
 2017-10-06 18:50:51 -07:00
Inigo Goiri
ad41c81559 HDFS-12381. [Documentation] Adding configuration keys for the Router. Contributed by Inigo Goiri. 
(cherry picked from commit c2d6aa79055ef72406fa598e1c743b0c994b5da8)
 2017-10-06 18:50:51 -07:00
Inigo Goiri
67785fe006 HDFS-12450. Fixing TestNamenodeHeartbeat and support non-HA. Contributed by Inigo Goiri. 
(cherry picked from commit 928d1e87f9dbe64f89b858ccc1780723f3af58e7)
 2017-10-06 18:50:50 -07:00
Inigo Goiri
1f06b81ecb HDFS-12430. Rebasing HDFS-10467 After HDFS-12269 and HDFS-12218. Contributed by Inigo Goiri. 
(cherry picked from commit 3302e792d469b7e8f3bfa10151e4e1c546589734)
 2017-10-06 18:50:50 -07:00
Inigo Goiri
bc9e588a19 HDFS-12335. Federation Metrics. Contributed by Inigo Goiri. 
(cherry picked from commit 3b19e77752afce87936f5c0d1e6d272fba798d7b)
 2017-10-06 18:50:50 -07:00
Inigo Goiri
cc58e7a983 HDFS-12384. Fixing compilation issue with BanDuplicateClasses. Contributed by Inigo Goiri. 
(cherry picked from commit fabe02c8fafa807198054da0c02b2ebaafda76aa)
 2017-10-06 18:50:49 -07:00
Inigo Goiri
7cb6bdf09e HDFS-10631. Federation State Store ZooKeeper implementation. Contributed by Jason Kace and Inigo Goiri. 
(cherry picked from commit 23c4ddee11ab1300325a6361124ee8ad6f68d7a4)
 2017-10-06 18:50:49 -07:00
Inigo Goiri
346c9fce43 HDFS-12312. Rebasing HDFS-10467 (2). Contributed by Inigo Goiri. 
(cherry picked from commit 90ba6843fb3ac5dc7576535e66a75a5e3433247b)
 2017-10-06 18:50:48 -07:00
Inigo Goiri
67d10087af HDFS-11554. [Documentation] Router-based federation documentation. Contributed by Inigo Goiri. 
(cherry picked from commit ee3260211d94aed223dd6f2386a166eb2c7d67af)
 2017-10-06 18:50:48 -07:00
Inigo Goiri
b3e6bd22e3 HDFS-10646. Federation admin tool. Contributed by Inigo Goiri. 
(cherry picked from commit ae27e31fbcf546481db0b0345772db2e9132372e)
 2017-10-06 18:50:48 -07:00
Inigo Goiri
6f0de27318 HDFS-10880. Federation Mount Table State Store internal API. Contributed by Jason Kace and Inigo Goiri. 
(cherry picked from commit 58b97df661441150d35abd44b3a8606206b46441)
 2017-10-06 18:50:47 -07:00
Inigo Goiri
d8c8107332 HDFS-11826. Federation Namenode Heartbeat. Contributed by Inigo Goiri. 
(cherry picked from commit 928f8dab52191e733984d37f47b69719ccf11313)
 2017-10-06 18:50:47 -07:00
Inigo Goiri
55da7fd7eb HDFS-10687. Federation Membership State Store internal API. Contributed by Jason Kace and Inigo Goiri. 
(cherry picked from commit 95cae08849d23cf12c5d280c29f90908e6a90d40)
 2017-10-06 18:50:47 -07:00
Inigo Goiri
0ec82b8cdf HDFS-12223. Rebasing HDFS-10467. Contributed by Inigo Goiri. 
(cherry picked from commit 47db6e9d8e2c264671c89fdd6cb11a7c762d2cce)
 2017-10-06 18:50:46 -07:00
Inigo Goiri
ca4f209b49 HDFS-11546. Federation Router RPC server. Contributed by Jason Kace and Inigo Goiri. 
(cherry picked from commit 8a9cdebebf26841a0f1e99fb08135f4597f2eba2)
 2017-10-06 18:50:46 -07:00
Inigo Goiri
4bf877b03f HDFS-10630. Federation State Store FS Implementation. Contributed by Jason Kace and Inigo Goiri. 
(cherry picked from commit c6e0bd640cdaf83a660fa050809cad6f1d4c6f4d)
 2017-10-06 18:50:45 -07:00
Inigo
2c740a684a HDFS-10882. Federation State Store Interface API. Contributed by Jason Kace and Inigo Goiri. 
(cherry picked from commit 6d94c90ece1c1d23d4c97e72c54e9991f5dbc481)
 2017-10-06 18:50:45 -07:00
Inigo
533b986633 HDFS-10881. Federation State Store Driver API. Contributed by Jason Kace and Inigo Goiri. 
(cherry picked from commit 0f88e049156dce173afc0dbda864e29190dd2210)
 2017-10-06 18:50:45 -07:00
Inigo
2761bbc91a HDFS-10629. Federation Roter. Contributed by Jason Kace and Inigo Goiri. 
(cherry picked from commit 6821e801724ac38e9737538b2164c9ae88792282)
 2017-10-06 18:50:44 -07:00
Xiao Chen
25f31d9fc4 HADOOP-14521. KMS client needs retry logic. Contributed by Rushabh S Shah. 2017-10-05 19:46:47 -07:00
Andrew Wang
644c2f6924 HDFS-12567. BlockPlacementPolicyRackFaultTolerant fails with racks with very few nodes. 2017-10-05 16:58:43 -07:00
Arpit Agarwal
b6942cbe9b HDFS-12420. Add an option to disallow 'namenode format -force'. Contributed by Ajay Kumar. 2017-10-05 15:26:52 -07:00
Xiaoyu Yao
107c177782 HDFS-12455. WebHDFS - Adding "snapshot enabled" status to ListStatus query result. Contributed by Ajay Kumar. 2017-10-03 13:02:12 -07:00
Arpit Agarwal
b91305119b HDFS-11968. ViewFS: StoragePolicies commands fail with HDFS federation. Contributed by Mukul Kumar Singh. 2017-10-03 11:23:40 -07:00
Anu Engineer
015abcd8ce HDFS-12576. JournalNodes are getting started, even though dfs.namenode.shared.edits.dir is not configured. Contributed by Bharat Viswanadham. 2017-10-02 15:40:00 -07:00
Lei Xu
27ffd43b64 HDFS-12569. Unset EC policy logs empty payload in edit log. (Lei (Eddy) Xu) 2017-10-02 15:35:49 -07:00
Chen Liang
2c62ff7fd4 HDFS-12271. Incorrect statement in Downgrade section of HDFS Rolling Upgrade document. Contributed by Nandakumar. 2017-10-02 13:18:00 -07:00
Chen Liang
5a68c2cef3 HDFS-12552. Use slf4j instead of log4j in FSNamesystem. Contributed by Ajay Kumar. 2017-10-02 11:26:53 -07:00
Lei Xu
f40dbc170e HDFS-12453. TestDataNodeHotSwapVolumes fails in trunk Jenkins runs. (Lei (Eddy) Xu) 2017-09-29 10:46:17 -07:00
Chen Liang
8e1bd114ee HDFS-12560. Remove the extra word it in HdfsUserGuide.md. Contributed by fang zhenyi. 2017-09-28 10:22:27 -07:00
Wei-Chiu Chuang
7c34ceaf4f HDFS-12458. TestReencryptionWithKMS fails regularly. Contributed by Xiao Chen. 2017-09-28 05:10:26 -07:00
Daryn Sharp
0da29cbeea [HDFS-12386] Add fsserver defaults call to WebhdfsFileSystem. (Rushabh Shah via daryn) 2017-09-26 15:15:24 -05:00
Lei Xu
1267ff22ce HDFS-12523. Thread pools in ErasureCodingWorker do not shutdown. (Huafeng Wang via Lei) 2017-09-26 10:08:16 -07:00
Brahma Reddy Battula
9d3e4cccf9 HDFS-5040.Audit log for admin commands/ logging output of all DFS admin commands. Contributed by Kuhu Shukla. 2017-09-26 21:59:22 +05:30
Yiqun Lin
e9b790db9d HDFS-12495. TestPendingInvalidateBlock#testPendingDeleteUnknownBlocks fails intermittently. Contributed by Eric Badger. 2017-09-26 17:21:42 +08:00
Akira Ajisaka
02e2a9b115
HDFS-12304. Remove unused parameter from FsDatasetImpl#addVolume. Contributed by Chen Liang. 2017-09-26 01:25:07 +09:00
Anu Engineer
d0b2c5850b HDFS-12516. Suppress the fsnamesystem lock warning on nn startup. Contributed by Ajay Kumar. 2017-09-24 21:03:06 -07:00
Anu Engineer
415e5a157a HDFS-12375. Fail to start/stop journalnodes using start-dfs.sh/stop-dfs.sh. Contributed by Bharat Viswanadham. 2017-09-23 15:05:25 -07:00
Anu Engineer
40ef9fa171 HADOOP-14904. Fix javadocs issues in Hadoop HDFS. Contributed by Mukul Kumar Singh. 2017-09-23 12:16:48 -07:00
Anu Engineer
2a81af447f HDFS-12536. Add documentation for getconf command with -journalnodes option. Contributed by Bharat Viswanadham. 2017-09-23 10:53:59 -07:00
Anu Engineer
423ebcaf51 HDFS-12320. Add quantiles for transactions batched in Journal sync. Contributed by Hanisha Koneru. 2017-09-23 10:34:13 -07:00
Anu Engineer
36fb90c939 HDFS-12064. Reuse object mapper in HDFS. Contributed by Hanisha Koneru. 2017-09-23 10:11:00 -07:00
Jitendra Pandey
cda3378659 HDFS-12486. GetConf to get journalnodeslist. Contributed by Bharat Viswanadham. 2017-09-22 16:52:47 -07:00
Arpit Agarwal
08fca508e6 HDFS-12530. Processor argument in Offline Image Viewer should be case insensitive. Contributed by Hanisha Koneru. 2017-09-22 09:47:57 -07:00
Arpit Agarwal
b9e423fa8d HDFS-12496. Make QuorumJournalManager timeout properties configurable. Contributed by Ajay Kumar. 2017-09-21 08:44:43 -07:00
Kihwal Lee
6bf921a5c3 HDFS-12371. BlockVerificationFailures and BlocksVerified show up as 0 in Datanode JMX. Contributed by Hanisha Koneru. 2017-09-21 08:42:50 -05:00
Akira Ajisaka
50849ec9eb
HDFS-12526. FSDirectory should use Time.monotonicNow for durations. Contributed by Bharat Viswanadham. 2017-09-21 19:27:48 +09:00
Andrew Wang
a12f09ba3c HDFS-12447. Rename AddECPolicyResponse to AddErasureCodingPolicyResponse. Contributed by SammiChen. 2017-09-20 11:51:17 -07:00
Ming Ma
ce943eb17a HDFS-11035. Better documentation for maintenace mode and upgrade domain. 2017-09-20 09:36:33 -07:00
Ming Ma
230b85d586 HDFS-12473. Change hosts JSON file format. 2017-09-20 09:03:59 -07:00
Andrew Wang
12d9d7bc50 HDFS-12437. Fix test setup in TestLeaseRecoveryStriped. 2017-09-19 16:42:20 -07:00