big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
20,848 Commits 2 Branches 2 Tags 264 MiB
1a9ba9616a
Commit Graph

42 Commits

Author SHA1 Message Date
Nanda kumar
0ec88ea42b HDFS-13301. Ozone: Remove containerPort, ratisPort and ozoneRestPort from DatanodeID and DatanodeIDProto. Contributed by Shashikant Banerjee. 2018-04-26 05:36:04 -07:00
Anu Engineer
4e61bc431e HDFS-13258. Ozone: restructure Hdsl/Ozone code to separated maven subprojects. 
Contributed by Elek Marton, Mukul Kumar Singh, Xiaoyu Yao, Ajay Kumar, Anu Engineer, Lokesh Jain, Nanda Kumar.
 2018-04-26 05:36:04 -07:00
Anu Engineer
bd33038bf4 HDFS-12159. Ozone: SCM: Add create replication pipeline RPC. Contributed by Anu Engineer. 2018-04-26 05:36:04 -07:00
Chris Nauroth
085c62545d HDFS-9891. Ozone: Add container transport client. Contributed by Anu Engineer. 2018-04-25 15:51:58 -07:00
Xiao Chen
7c9cdad6d0 HDFS-13056. Expose file-level composite CRCs in HDFS which are comparable across different instances/layouts. Contributed by Dennis Huo. 2018-04-10 21:31:48 -07:00
Arpit Agarwal
9714fc1dd4 HDFS-336. dfsadmin -report should report number of blocks from datanode. Contributed by Bharat Viswanadham. 2018-03-13 16:39:17 -07:00
Virajith Jalaparti
352f994b64 HDFS-12665. [AliasMap] Create a version of the AliasMap that runs in memory in the Namenode (leveldb). Contributed by Ewan Higgs. 2017-12-15 17:51:40 -08:00
Virajith Jalaparti
b668eb9155 HDFS-10675. Datanode support to read from external stores. 2017-12-15 17:51:37 -08:00
Chris Douglas
693169ef34 HDFS-12882. Support full open(PathHandle) contract in HDFS 2017-12-11 20:14:15 -08:00
Tsz-Wo Nicholas Sze
b1c7654ee4 HDFS-12594. snapshotDiff fails if the report exceeds the RPC response limit. Contributed by Shashikant Banerjee 2017-11-30 12:18:29 -08:00
Chris Douglas
d015e0bbd5 HDFS-7878. API - expose a unique file identifier. 2017-10-31 09:44:01 -07:00
Xiaoyu Yao
107c177782 HDFS-12455. WebHDFS - Adding "snapshot enabled" status to ListStatus query result. Contributed by Ajay Kumar. 2017-10-03 13:02:12 -07:00
Andrew Wang
a12f09ba3c HDFS-12447. Rename AddECPolicyResponse to AddErasureCodingPolicyResponse. Contributed by SammiChen. 2017-09-20 11:51:17 -07:00
Rakesh Radhakrishnan
200b11368d HDFS-12258. ec -listPolicies should list all policies in system, no matter it's enabled or disabled. Contributed by Wei Zhou. 2017-08-30 12:58:56 +05:30
Xiao Chen
1000a2af04 HDFS-10899. Add functionality to re-encrypt EDEKs. 2017-08-23 17:06:16 -07:00
Chris Douglas
12e44e7bda HDFS-6984. Serialize FileStatus via protobuf. 2017-08-02 12:12:48 -07:00
Lei Xu
45ff4d38e6 HDFS-11647. Add -E option in hdfs "count" command to show erasure policy summarization. Contributed by luhuichun. 2017-06-20 11:55:09 -07:00
Kai Zheng
a62be38a5e HDFS-11793. Allow to enable user defined erasure coding policy. Contributed by Sammi Chen 2017-05-24 18:45:52 +08:00
Chris Douglas
a3954ccab1 HDFS-9807. Add an optional StorageID to writes. Contributed by Ewan Higgs 2017-05-05 12:01:26 -07:00
Kai Zheng
cb672a45a0 HDFS-11605. Allow user to customize new erasure code policies. Contributed by Huafeng Wang 2017-04-28 13:18:50 +08:00
Akira Ajisaka
93fa48fcf2
HDFS-11417. Add datanode admin command to get the storage info. Contributed by Surendra Singh Lilhore. 2017-04-26 17:43:29 +09:00
Chris Douglas
2f73396b59 HDFS-6708. StorageType should be encoded in the block token. Contributed by Ewan Higgs 2017-04-25 23:57:00 -07:00
Andrew Wang
966b1b5b44 HDFS-11565. Use compact identifiers for built-in ECPolicies in HdfsFileStatus. 2017-04-12 12:24:32 -07:00
Chris Nauroth
23b1a7bdf1 HDFS-11163. Mover should move the file blocks to default storage once policy is unset. Contributed by Surendra Singh Lilhore. 2017-04-11 22:03:09 -07:00
Andrew Wang
18432130a7 HADOOP-14104. Client should always ask namenode for kms provider path. Contributed by Rushabh S Shah. 2017-04-04 13:38:25 -07:00
Arpit Agarwal
b5adc5c301 HDFS-10838. Last full block report received time for each DN should be easily discoverable. Contributed by Surendra Singh Lilhore. 2017-03-06 16:39:53 -08:00
Chris Douglas
4ed33e9ca3 HDFS-11026. Convert BlockTokenIdentifier to use Protobuf. Contributed by Ewan Higgs. 2017-02-13 11:29:18 -08:00
Andrew Wang
a2a5d7b5bc HDFS-10759. Change fsimage bool isStriped from boolean to an enum. Contributed by Ewan Higgs. 2017-01-18 13:31:05 -08:00
Andrew Wang
d51f8ba808 HDFS-11312. Fix incompatible tag number change for nonDfsUsed in DatanodeInfoProto. Contributed by Sean Mackrory. 2017-01-11 15:13:27 -08:00
Arpit Agarwal
5f23abfa30 HDFS-9038. DFS reserved space is erroneously counted towards non-DFS used. (Brahma Reddy Battula) 2016-09-06 13:37:21 -07:00
Wei-Chiu Chuang
f0efea490e HDFS-8986. Add option to -du to calculate directory space usage excluding snapshots. Contributed by Xiao Chen. 2016-08-23 04:14:24 -07:00
Lei Xu
a2774debf7 HDFS-9389. Add maintenance states to AdminStates. (Ming Ma via lei) 2016-05-12 15:42:54 -07:00
Zhe Zhang
7600e3c48f HDFS-7866. Erasure coding: NameNode manages multiple erasure coding policies. Contributed by Rui Li. 2016-03-08 22:30:30 -08:00
Kihwal Lee
2fd19b9674 HDFS-8898. Create API and command-line argument to get quota and quota usage without detailed content summary. Contributed by Ming Ma. 2016-01-22 09:10:06 -06:00
Tsz-Wo Nicholas Sze
70d6f20126 HDFS-9575. Use byte array for internal block indices in a striped block. Contributed by jing9 2015-12-22 14:47:57 +08:00
Zhe Zhang
8fd5520246 Merge remote-tracking branch 'apache/trunk' into HDFS-7285 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/hdfs.proto
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSUtil.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelper.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirStatAndListingOp.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/proto/DatanodeProtocol.proto
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockTokenWithDFS.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java

Change-Id: Ic7946c4ea35bed587fe879ce58b959b25ecc0823
 2015-09-29 01:39:16 -07:00
Zhe Zhang
c09dc258a8 HDFS-8882. Erasure Coding: Use datablocks, parityblocks and cell size from ErasureCodingPolicy. Contributed by Vinayakumar B. 
Change-Id: Ic56da0b426f47c63dac440aef6f5fc8554f6cf13
 2015-09-23 13:34:09 -07:00
Haohui Mai
7c5c099324 HDFS-8733. Keep server related definition in hdfs.proto on server side. Contributed by Mingliang Liu. 2015-09-22 20:57:05 -07:00
Zhe Zhang
1080c37300 Merge remote-tracking branch 'apache/trunk' into HDFS-7285 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/Namesystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java

Change-Id: I8511c4d64b0959e79129febc179845a3892fedcc
 2015-09-22 13:27:09 -07:00
Lei Xu
3a9c7076e8 HDFS-9004. Add upgrade domain to DatanodeInfo. Contributed by Ming Ma (via Lei (Eddy) Xu). 
Change-Id: I887c66578eebd61acc34b94f18da6e6851c609f4
 2015-09-19 18:08:09 -07:00
Zhe Zhang
6b6a63bbbd Merge commit '456e901a4c5c639267ee87b8e5f1319f256d20c2' (HDFS-6407. Add sorting and pagination in the datanode tab of the NN Web UI. Contributed by Haohui Mai.) into HDFS-7285-merge 2015-08-24 12:59:26 -07:00
Haohui Mai
fc6182d5ed HDFS-8726. Move protobuf files that define the client-sever protocols to hdfs-client. Contributed by Haohui Mai. 2015-07-08 10:37:10 -07:00