big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
18,485 Commits 2 Branches 2 Tags 264 MiB
d251a55ff9
Commit Graph

6890 Commits

Author SHA1 Message Date
Anu Engineer
e11e824c9b HDFS-10268. Ozone: end-to-end integration for create/get volumes, buckets and keys. Contributed by Chris Nauroth. 2018-04-25 15:52:56 -07:00
Chris Nauroth
0addb1033e HDFS-10250. Ozone: Add key Persistence. Contributed by Anu Engineer. 2018-04-25 15:52:48 -07:00
Chris Nauroth
c6fd5ea3f0 HDFS-10238. Ozone : Add chunk persistance. Contributed by Anu Engineer. 2018-04-25 15:52:48 -07:00
Anu Engineer
0744d0a947 HDFS-8210. Ozone: Implement storage container manager. Contributed by Chris Nauroth. 2018-04-25 15:52:46 -07:00
Chris Nauroth
e758f90f93 HDFS-10195. Ozone: Add container persistence. Contributed by Anu Engineer. 2018-04-25 15:52:33 -07:00
Chris Nauroth
c18c6801ef HDFS-10202. ozone : Add key commands to CLI. Contributed by Anu Engineer. 2018-04-25 15:52:25 -07:00
Chris Nauroth
f69ce5f3c6 HDFS-10196. Ozone : Enable better error reporting for failed commands in ozone shell. Contributed by Anu Engineer. 2018-04-25 15:52:09 -07:00
Chris Nauroth
fd2fe47b32 HDFS-10179. Ozone: Adding logging support. Contributed by Anu Engineer. 2018-04-25 15:52:07 -07:00
Chris Nauroth
7c55d7feb5 HDFS-9960. OzoneHandler : Add localstorage support for keys. Contributed by Anu Engineer. 2018-04-25 15:52:00 -07:00
Chris Nauroth
4009b66090 HDFS-10180. Ozone: Refactor container Namespace. Contributed by Anu Engineer. 2018-04-25 15:52:00 -07:00
Chris Nauroth
044cc47145 HDFS-9961. Ozone: Add buckets commands to CLI. Contributed by Anu Engineer. 2018-04-25 15:52:00 -07:00
Chris Nauroth
bb47d03906 HDFS-9944. Ozone : Add container dispatcher. Contributed by Anu Engineer. 2018-04-25 15:52:00 -07:00
Chris Nauroth
b9d53ed836 HDFS-9926. ozone : Add volume commands to CLI. Contributed by Anu Engineer. 2018-04-25 15:52:00 -07:00
Chris Nauroth
60fbef08ec HDFS-9925. Ozone: Add Ozone Client lib for bucket handling. Contributed by Anu Engineer. 
Fix build break
 2018-04-25 15:52:00 -07:00
Chris Nauroth
a9879798a6 HDFS-9916. OzoneHandler : Add Key handler. Contributed by Anu Engineer. 2018-04-25 15:52:00 -07:00
Chris Nauroth
93201330b8 HDFS-9916. OzoneHandler : Add Key handler. Contributed by Anu Engineer. 2018-04-25 15:52:00 -07:00
Anu Engineer
2b387d622c HDFS-9920. Stop tracking CHANGES.txt in the HDFS-7240 feature branch. Contributed by Chris Nauroth. 2018-04-25 15:52:00 -07:00
Chris Nauroth
085c62545d HDFS-9891. Ozone: Add container transport client. Contributed by Anu Engineer. 2018-04-25 15:51:58 -07:00
Chris Nauroth
9b158f3b57 HDFS-9873. Ozone: Add container transport server. Contributed by Anu Engineer. 2018-04-25 15:51:53 -07:00
Anu Engineer
3c39c18a6d HDFS-9907. Exclude Ozone protobuf-generated classes from Findbugs analysis. Contributed by Chris Nauroth 2018-04-25 15:51:53 -07:00
Chris Nauroth
0a43e272f1 HDFS-9848. Ozone: Add Ozone Client lib for volume handling. Contributed by Anu Engineer. 2018-04-25 15:51:53 -07:00
Anu Engineer
3481897bde HDFS-9853. Ozone: Add container definitions. Contributed by Anu Engineer. 2018-04-25 15:51:51 -07:00
cnauroth
11a6cc2a4d HDFS-9845. OzoneHandler : Support List and Info Volumes. Contributed by Anu Engineer. 2018-04-25 15:51:39 -07:00
cnauroth
f6ef841f74 HDFS-9834. OzoneHandler : Enable MiniDFSCluster based testing for Ozone. Contributed by Anu Engineer. 2018-04-25 15:51:39 -07:00
Anu Engineer
f6b9c0f8a1 HDFS-8757 : OzoneHandler : Add localStorageHandler support for Buckets. Contributed by Anu Engineer 2018-04-25 15:51:39 -07:00
Anu Engineer
ae109d1de7 HDFS-8527. OzoneHandler: Integration of REST interface and container data pipeline back-end. Contributed by Chris Nauroth 2018-04-25 15:51:36 -07:00
Anu Engineer
a301827c6a HDFS-8695. OzoneHandler : Add Bucket REST Interface. (aengineer) 2018-04-25 15:51:00 -07:00
Anu Engineer
764ff0d505 HDFS-8753. Ozone: Unify StorageContainerConfiguration with ozone-default.xml & ozone-site.xml. Contributed by kanaka kumar avvaru 2018-04-25 15:51:00 -07:00
Arpit Agarwal
507e7274aa HDFS-8717. OzoneHandler : Add common bucket objects. (Contributed by Anu Engineer) 2018-04-25 15:51:00 -07:00
Arpit Agarwal
07213f4eb5 HDFS-8680. OzoneHandler : Add Local StorageHandler support for volumes. (Contributed by Anu Engineer) 2018-04-25 15:51:00 -07:00
Arpit Agarwal
835f872a0a HDFS-8654. OzoneHandler : Add ACL support. (Contributed by Anu Engineer) 2018-04-25 15:51:00 -07:00
Arpit Agarwal
b1fa06aacb HDFS-8644. OzoneHandler : Add volume handler. (Contributed by Anu Engineer) 2018-04-25 15:51:00 -07:00
Arpit Agarwal
2793b4c96f HDFS-8634. OzoneHandler: Add userAuth Interface and Simple userAuth handler. (Contributed by Anu Engineer) 2018-04-25 15:51:00 -07:00
Arpit Agarwal
d65b373ec4 HDFS-8637. OzoneHandler : Add Error Table. (Contributed by Anu Engineer) 2018-04-25 15:51:00 -07:00
cnauroth
2ea29ee062 HDFS-8448. Create REST Interface for Volumes. Contributed by Anu Engineer. 2018-04-25 15:51:00 -07:00
Arpit Agarwal
0a90e6d45c HDFS-8614. OzoneHandler : Add Quota Support. (Contributed by Anu Engineer) 2018-04-25 15:51:00 -07:00
Arpit Agarwal
d41245ca56 HDFS-8456. Introduce STORAGE_CONTAINER_SERVICE as a new NodeType. (Contributed by Arpit Agarwal) 2018-04-25 15:50:52 -07:00
Inigo Goiri
df92a17e02 HDFS-13336. Test cases of TestWriteToReplica failed in windows. Contributed by Xiao Liang. 2018-04-23 19:12:16 -07:00
Bharat Viswanadham
9b5375e0c1 HDFS-13356. Balancer:Set default value of minBlockSize to 10mb. Contributed by Bharat Viswanadham 2018-04-23 16:09:01 -07:00
Lei Xu
42e82f0281 Update CHANGES, RELEASENOTES, and jdiff for 3.0.2 release. 
(cherry picked from commit f6ecb76d0b919b9836600fe28ec9e637b223cd54)

Conflicts:
	hadoop-project-dist/pom.xml
 2018-04-23 14:14:05 -07:00
Chris Douglas
f411de6a79 HDFS-13408. MiniDFSCluster to support being built on randomized base directory. Contributed by Xiao Liang 2018-04-23 11:13:18 -07:00
Arpit Agarwal
c533c77047 HDFS-13433. webhdfs requests can be routed incorrectly in federated cluster. Contributed by Arpit Agarwal. 2018-04-23 10:08:53 -07:00
Arpit Agarwal
1c1ce63cda HDFS-13055. Aggregate usage statistics from datanodes. Contributed by Ajay Kumar. 2018-04-22 08:07:19 -07:00
Sangjin Lee
f40969a141 HDFS-10183. Prevent race condition during class initialization. Contributed by Pavel Avgustinov. 2018-04-20 20:33:10 -07:00
Bharat Viswanadham
bf7694d020 HDFS-13464. Fix javadoc in FsVolumeList#handleVolumeFailures. Contributed by Shashikant Banerjee 2018-04-18 08:23:45 -07:00
Mukul Kumar Singh
bf2f493f81 HDFS-13463. Fix javadoc in FsDatasetImpl#checkAndUpdate. Contributed by Shashikant Banerjee. 2018-04-18 16:11:37 +05:30
Erik Krogen
2d9e791a90
HDFS-12828. OIV ReverseXML Processor fails with escaped characters 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-04-18 14:38:23 +09:00
Inigo Goiri
c9b33514b8 HDFS-13462. Add BIND_HOST configuration for JournalNode's HTTP and RPC Servers. Contributed by Lukas Majercak. 2018-04-17 14:19:23 -07:00
Bharat Viswanadham
a68b04370f HDFS-13129. Add a test for DfsAdmin refreshSuperUserGroupsConfiguration. Contributed by Mukul Kumar Singh 2018-04-17 12:37:30 -07:00
Arpit Agarwal
d907fdc3cd HDFS-13439. Add test case for read block operation when it is moved. Contributed by Ajay Kumar. 2018-04-16 14:16:59 -07:00
Wei-Chiu Chuang
e66e287efe HDFS-13330. ShortCircuitCache#fetchOrCreate never retries. Contributed by Gabor Bota. 2018-04-13 09:17:34 -07:00
Xiao Chen
6503593711 HDFS-13430. Fix TestEncryptionZonesWithKMS failure due to HADOOP-14445. 2018-04-13 09:05:17 -07:00
Bharat Viswanadham
fa8b88ab2b HDFS-13438. Fix javadoc in FsVolumeList#removeVolume. Contributed by Shashikant Banerjee. 2018-04-13 08:56:02 -07:00
Yiqun Lin
0725953efe HDFS-13418. NetworkTopology should be configurable when enable DFSNetworkTopology. Contributed by Tao Jie. 2018-04-13 17:55:45 +08:00
Akira Ajisaka
9b0b9f2af2
HDFS-13436. Fix javadoc of package-info.java 2018-04-13 13:23:51 +09:00
Mukul Kumar Singh
7ed8511ad8 HDFS-13426. Fix javadoc in FsDatasetAsyncDiskService#removeVolume. Contributed by Shashikant Banerjee. 2018-04-12 20:12:31 +05:30
Akira Ajisaka
113af12cfb
HDFS-7101. Potential null dereference in DFSck#doWork(). Contributed by skrho and Ted Yu. 2018-04-12 17:47:37 +09:00
Akira Ajisaka
c7cd362afd
HDFS-13427. Fix the section titles of transparent encryption document. 2018-04-12 10:44:56 +09:00
Xiao Chen
7c9cdad6d0 HDFS-13056. Expose file-level composite CRCs in HDFS which are comparable across different instances/layouts. Contributed by Dennis Huo. 2018-04-10 21:31:48 -07:00
Xiao Chen
e76c2aeb28 HDFS-13363. Record file path when FSDirAclOp throws AclException. Contributed by Gabor Bota. 2018-04-10 11:19:48 -07:00
Rakesh Radhakrishnan
f89594f0b8 HDFS-13328. Abstract ReencryptionHandler recursive logic in separate class. Contributed by Surendra Singh Lilhore. 2018-04-10 23:35:00 +05:30
Weiwei Yang
6729047a8b HDFS-13420. License header is displayed in ArchivalStorage/MemoryStorage html pages. Contributed by Akira Ajisaka. 2018-04-10 22:10:44 +08:00
Surendra Singh Lilhore
70590cd8d9 HDFS-13292. Crypto command should give proper exception when trying to set key on existing EZ directory. Contributed by Ranith Sardar. 2018-04-07 11:23:49 +05:30
Wangda Tan
6cf023f9b7 Added CHANGES/RELEASES/Jdiff for 3.1.0 release 
Change-Id: Ied5067a996151c04d15cad46c46ac98b60c37b39
(cherry picked from commit 2d96570452a72569befdf9cfe9b90c9fa2e0e261)
 2018-04-05 15:52:39 -07:00
Lei Xu
d737bf99d4 HDFS-13350. Negative legacy block ID will confuse Erasure Coding to be considered as striped block. (Contributed by Lei (Eddy) Xu). 2018-04-05 09:59:10 -07:00
Xiao Chen
57374c4737 HADOOP-15317. Improve NetworkTopology chooseRandom's loop. 2018-04-02 22:48:26 -07:00
Konstantin V Shvachko
acfd764fcc HADOOP-15253. Should update maxQueueSize when refresh call queue. Contributed by Tao Jie. 2018-03-30 18:55:35 -07:00
Xiao Chen
2c6cfad5a3 HDFS-13087. Snapshotted encryption zone information should be immutable. Contributed by LiXin Ge. 2018-03-29 15:46:34 -07:00
Wei-Chiu Chuang
ad10cbd91b HDFS-13357. Improve AclException message "Invalid ACL: only directories may have a default ACL.". Contributed by Gabor Bota. 2018-03-29 07:08:22 -07:00
Arpit Agarwal
a991e899fb HDFS-13314. NameNode should optionally exit if it detects FsImage corruption. Contributed by Arpit Agarwal. 2018-03-28 12:49:17 -07:00
Lei Xu
7616683db5 Update 3.0.1 jdiff file and jdiff stable api version 2018-03-23 11:48:36 -07:00
weiy
6e31a09084 HDFS-12512. RBF: Add WebHDFS. 2018-03-23 08:32:25 -07:00
Kihwal Lee
66461ed98b HDFS-13195. DataNode conf page cannot display the current value after reconfig. Contributed by maobaolong. 2018-03-22 12:21:10 -05:00
Konstantin V Shvachko
8d898ab25f HDFS-12884. BlockUnderConstructionFeature.truncateBlock should be of type BlockInfo. Contributed by chencan. 2018-03-21 16:46:03 -07:00
Xiaoyu Yao
389bc6d3da HDFS-11043. TestWebHdfsTimeouts fails. Contributed by Xiaoyu Yao and Chao Sun. 2018-03-21 13:53:35 -07:00
Yongjun Zhang
6c63cc7d30 HDFS-13315. Add a test for the issue reported in HDFS-11481 which is fixed by HDFS-10997. Contributed by Yongjun Zhang. 2018-03-20 23:04:05 -07:00
weiy
6e2b5fa493 HDFS-13215. RBF: Move Router to its own module. Contributed by Wei Yan 2018-03-19 22:13:40 -07:00
Inigo Goiri
08ff1586d5 HDFS-12886. Ignore minReplication for block recovery. Contributed by Lukas Majercak. 2018-03-16 10:29:19 -07:00
Xiaoyu Yao
78b05fde6c HDFS-13280. WebHDFS: Fix NPE in get snasphottable directory list call. Contributed by Lokesh Jain. 2018-03-15 11:02:27 -07:00
Inigo Goiri
e71bc00a47 HDFS-13224. RBF: Resolvers to support mount points across multiple subclusters. Contributed by Inigo Goiri. 2018-03-15 10:32:30 -07:00
Xiaoyu Yao
da777a5498 HDFS-13251. Avoid using hard coded datanode data dirs in unit tests(addendum). Contributed by Ajay Kumar. 2018-03-15 10:18:44 -07:00
Inigo Goiri
6de135169e HDFS-12723. TestReadStripedFileWithMissingBlocks#testReadFileWithMissingBlocks failing consistently. Contributed by Ajay Kumar. 2018-03-15 10:14:35 -07:00
fang zhenyi
5e013d50d1
HADOOP-15305. Replace FileUtils.writeStringToFile(File, String) with (File, String, Charset) to fix deprecation warnings. 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-03-15 16:12:07 +09:00
Xiao Chen
5ff22d4c3a HDFS-13246. FileInputStream redundant closes in readReplicasFromCache. Contributed by liaoyuxiangqin. 2018-03-14 21:59:51 -07:00
Yiqun Lin
3a0f4bc0d5 HDFS-13261. Fix incorrect null value check. Contributed by Jianfei Jiang. 2018-03-15 11:06:04 +08:00
Inigo Goiri
41fc7f80be HDFS-13198. RBF: RouterHeartbeatService throws out CachedStateStore related exceptions when starting router. Contributed by Wei Yan. 2018-03-14 13:14:36 -07:00
Xiaoyu Yao
f83716b7f2 HDFS-13251. Avoid using hard coded datanode data dirs in unit tests. Contributed by Ajay Kumar. 2018-03-14 11:26:58 -07:00
Xiaoyu Yao
0be0f1ce1d HDFS-13275. Adding log for BlockPoolManager#refreshNamenodes failures. Contributed by Ajay Kumar. 2018-03-14 10:38:00 -07:00
Arpit Agarwal
4c57fb0cd9 HDFS-13257. Code cleanup: INode never throws QuotaExceededException. Contributed by Tsz Wo Nicholas Sze. 2018-03-14 09:42:48 -07:00
Xiaoyu Yao
a34d83d2fd HDFS-13249. Document webhdfs support for getting snapshottable directory list. Contributed by Lokesh Jain. 2018-03-14 09:14:26 -07:00
Sammi Chen
ad1b988a82 HDFS-11600. Refactor TestDFSStripedOutputStreamWithFailure test classes. Contributed by Sammi Chen. 2018-03-14 14:39:53 +08:00
Yiqun Lin
76be6cbf6c HDFS-12773. RBF: Improve State Store FS implementation. Contributed by Inigo Goiri. 2018-03-14 11:20:59 +08:00
Arpit Agarwal
9714fc1dd4 HDFS-336. dfsadmin -report should report number of blocks from datanode. Contributed by Bharat Viswanadham. 2018-03-13 16:39:17 -07:00
Arpit Agarwal
39537b7c84 HDFS-13235. DiskBalancer: Update Documentation to add newly added options. Contributed by Bharat Viswanadham. 2018-03-13 16:35:51 -07:00
Xiao Chen
8211a3d469 HDFS-12505. Extend TestFileStatusWithECPolicy with a random EC policy. Contributed by Takanobu Asanuma. 2018-03-13 10:48:45 -07:00
Xiao Chen
f82d38dcb3 Revert "HDFS-12505. Extend TestFileStatusWithECPolicy with a random EC policy. Contributed by Takanobu Asanuma." 
This reverts commit 84c1095586.
 2018-03-13 10:36:16 -07:00
Xiao Chen
3b8dbc2cb7 HDFS-12587. Use Parameterized tests in TestBlockInfoStriped and TestLowRedundancyBlockQueues to test all EC policies. Contributed by Takanobu Asanuma. 2018-03-13 10:14:05 -07:00
Xiao Chen
d6931c30c5 HDFS-13239. Fix non-empty dir warning message when setting default EC policy. Contributed by Bharat Viswanadham. 2018-03-13 10:07:26 -07:00
Xiao Chen
84c1095586 HDFS-12505. Extend TestFileStatusWithECPolicy with a random EC policy. Contributed by Takanobu Asanuma. 2018-03-13 09:58:03 -07:00
Xiaoyu Yao
0355ec20eb HDFS-13141. WebHDFS: Add support for getting snasphottable directory list. Contributed by Lokesh Jain. 2018-03-12 20:41:37 -07:00