big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
15,755 Commits 2 Branches 2 Tags 264 MiB
a7312715a6
Commit Graph

54 Commits

Author SHA1 Message Date
Andrew Wang
a7312715a6 HDFS-10996. Ability to specify per-file EC policy at create time. Contributed by SammiChen. 2017-04-12 12:27:34 -07:00
Andrew Wang
966b1b5b44 HDFS-11565. Use compact identifiers for built-in ECPolicies in HdfsFileStatus. 2017-04-12 12:24:32 -07:00
Chris Nauroth
23b1a7bdf1 HDFS-11163. Mover should move the file blocks to default storage once policy is unset. Contributed by Surendra Singh Lilhore. 2017-04-11 22:03:09 -07:00
Andrew Wang
18432130a7 HADOOP-14104. Client should always ask namenode for kms provider path. Contributed by Rushabh S Shah. 2017-04-04 13:38:25 -07:00
Arpit Agarwal
b5adc5c301 HDFS-10838. Last full block report received time for each DN should be easily discoverable. Contributed by Surendra Singh Lilhore. 2017-03-06 16:39:53 -08:00
Rakesh Radhakrishnan
82ef9accaf HDFS-11428. Change setErasureCodingPolicy to take a required string EC policy name. Contributed by Andrew Wang. 2017-03-01 16:06:02 +05:30
Chris Douglas
4ed33e9ca3 HDFS-11026. Convert BlockTokenIdentifier to use Protobuf. Contributed by Ewan Higgs. 2017-02-13 11:29:18 -08:00
Andrew Wang
a2a5d7b5bc HDFS-10759. Change fsimage bool isStriped from boolean to an enum. Contributed by Ewan Higgs. 2017-01-18 13:31:05 -08:00
Andrew Wang
d51f8ba808 HDFS-11312. Fix incompatible tag number change for nonDfsUsed in DatanodeInfoProto. Contributed by Sean Mackrory. 2017-01-11 15:13:27 -08:00
Andrew Wang
e69231658d HDFS-11072. Add ability to unset and change directory EC policy. Contributed by Sammi Chen. 2017-01-10 11:32:48 -08:00
Uma Maheswara Rao G
e24a923db5 HDFS-11164: Mover should avoid unnecessary retries if the block is pinned. Contributed by Rakesh R 2016-12-13 17:09:58 -08:00
Arpit Agarwal
5f23abfa30 HDFS-9038. DFS reserved space is erroneously counted towards non-DFS used. (Brahma Reddy Battula) 2016-09-06 13:37:21 -07:00
Chris Nauroth
f0d5382ff3 HDFS-6962. ACL inheritance conflicts with umaskmode. Contributed by Chris Nauroth. 2016-09-06 11:02:39 -07:00
Wei-Chiu Chuang
f0efea490e HDFS-8986. Add option to -du to calculate directory space usage excluding snapshots. Contributed by Xiao Chen. 2016-08-23 04:14:24 -07:00
Eric Yang
c49333becf HDFS-8312. Added permission check for moving file to Trash. (Weiwei Yang via Eric Yang) 2016-08-22 18:29:56 -07:00
Anu Engineer
5628b36c08 HDFS-10559. DiskBalancer: Use SHA1 for Plan ID. Contributed by Xiaobing Zhou. 2016-08-15 20:10:21 -07:00
Lei Xu
9c6a4383ca HDFS-10681. DiskBalancer: query command should report Plan file path apart from PlanID. (Manoj Govindassamy via lei) 2016-08-09 15:59:22 -07:00
Anu Engineer
050677077b HDFS-9720. DiskBalancer : Add configuration parameters. Contributed by Anu Engineer. 2016-06-23 18:18:48 -07:00
Arpit Agarwal
e646c2eb50 HDFS-9681. DiskBalancer: Add QueryPlan implementation. (Contributed by Anu Engineer) 2016-06-23 18:18:48 -07:00
Arpit Agarwal
66f0bb646d HDFS-9647. DiskBalancer: Add getRuntimeSettings. (Contributed by Anu Engineer) 2016-06-23 18:18:48 -07:00
Arpit Agarwal
96fe685b7a HDFS-9645. DiskBalancer: Add Query RPC. (Contributed by Anu Engineer) 2016-06-23 18:18:48 -07:00
Arpit Agarwal
0501d430e2 HDFS-9595. DiskBalancer: Add cancelPlan RPC. (Contributed by Anu Engineer) 2016-06-23 18:18:48 -07:00
Arpit Agarwal
7100c0da35 HDFS-9588. DiskBalancer: Add submitDiskbalancer RPC. (Anu Engineer via Arpit Agarwal) 2016-06-23 18:18:48 -07:00
Kai Zheng
e6cb07520f HDFS-10460. Recompute block checksum for a particular range less than file size on the fly by reconstructing missed block. Contributed by Rakesh R 2016-06-24 17:39:32 +08:00
Colin Patrick Mccabe
5f6b4157a4 HDFS-10328. Add per-cache-pool default replication num configuration (xupeng via cmccabe) 2016-06-20 10:42:50 -07:00
Kai Zheng
d749cf65e1 HDFS-9833. Erasure coding: recomputing block checksum on the fly by reconstructing the missed/corrupt block data. Contributed by Rakesh R. 2016-06-02 12:56:21 +08:00
Lei Xu
a2774debf7 HDFS-9389. Add maintenance states to AdminStates. (Ming Ma via lei) 2016-05-12 15:42:54 -07:00
Lei Xu
0a152103f1 HDFS-3702. Add an option for NOT writing the blocks locally if there is a datanode on the same box as the client. (Contributed by Lei (Eddy) Xu) 2016-04-27 14:22:51 -07:00
Eric Payne
aede8c10ec HDFS-9945. Datanode command for evicting writers. Contributed by Kihwal Lee 2016-04-06 20:20:14 +00:00
Uma Maheswara Rao G
3a4ff7776e HDFS-9694. Make existing DFSClient#getFileChecksum() work for striped blocks. Contributed by Kai Zheng 2016-03-26 19:58:09 -07:00
Arpit Agarwal
a337ceb74e Revert "HDFS-9694. Make existing DFSClient#getFileChecksum() work for striped blocks. Contributed by Kai Zheng" 
This reverts commit e5ff0ea7ba.
 2016-03-26 09:20:01 -07:00
Uma Maheswara Rao G
e5ff0ea7ba HDFS-9694. Make existing DFSClient#getFileChecksum() work for striped blocks. Contributed by Kai Zheng 2016-03-26 00:52:50 -07:00
Zhe Zhang
7600e3c48f HDFS-7866. Erasure coding: NameNode manages multiple erasure coding policies. Contributed by Rui Li. 2016-03-08 22:30:30 -08:00
Arpit Agarwal
27941a1811 HDFS-9534. Add CLI command to clear storage policy from a path. (Contributed by Xiaobing Zhou) 2016-03-02 18:35:28 -08:00
zhezhang
4ae543fdcd HDFS-9731. Erasure Coding: Rename BlockECRecoveryCommand to BlockECReconstructionCommand. Contributed by Rakesh R. 
Change-Id: I405365a8395770e494b92bfe9651f4f0366d8f28
 2016-02-02 12:32:08 -08:00
Eric Yang
10a2bc0dff HDFS-9653. Added blocks pending deletion report to dfsadmin. 
(Weiwei Yang via eyang)
 2016-01-24 14:19:49 -08:00
Kihwal Lee
2fd19b9674 HDFS-8898. Create API and command-line argument to get quota and quota usage without detailed content summary. Contributed by Ming Ma. 2016-01-22 09:10:06 -06:00
Tsz-Wo Nicholas Sze
70d6f20126 HDFS-9575. Use byte array for internal block indices in a striped block. Contributed by jing9 2015-12-22 14:47:57 +08:00
Arpit Agarwal
86c95cb31a HDFS-9414. Refactor reconfiguration of ClientDatanodeProtocol for reusability. (Contributed by Xiaobing Zhou) 2015-12-04 20:24:08 -08:00
Arpit Agarwal
86c92227fc HDFS-4015. Safemode should count and report orphaned blocks. (Contributed by Anu Engineer) 2015-10-23 18:07:17 -07:00
Zhe Zhang
8fd5520246 Merge remote-tracking branch 'apache/trunk' into HDFS-7285 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/hdfs.proto
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSUtil.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelper.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirStatAndListingOp.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/proto/DatanodeProtocol.proto
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockTokenWithDFS.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java

Change-Id: Ic7946c4ea35bed587fe879ce58b959b25ecc0823
 2015-09-29 01:39:16 -07:00
Zhe Zhang
c09dc258a8 HDFS-8882. Erasure Coding: Use datablocks, parityblocks and cell size from ErasureCodingPolicy. Contributed by Vinayakumar B. 
Change-Id: Ic56da0b426f47c63dac440aef6f5fc8554f6cf13
 2015-09-23 13:34:09 -07:00
Haohui Mai
7c5c099324 HDFS-8733. Keep server related definition in hdfs.proto on server side. Contributed by Mingliang Liu. 2015-09-22 20:57:05 -07:00
Zhe Zhang
1080c37300 Merge remote-tracking branch 'apache/trunk' into HDFS-7285 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/Namesystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java

Change-Id: I8511c4d64b0959e79129febc179845a3892fedcc
 2015-09-22 13:27:09 -07:00
Lei Xu
3a9c7076e8 HDFS-9004. Add upgrade domain to DatanodeInfo. Contributed by Ming Ma (via Lei (Eddy) Xu). 
Change-Id: I887c66578eebd61acc34b94f18da6e6851c609f4
 2015-09-19 18:08:09 -07:00
Zhe Zhang
f62237bc2f HDFS-8833. Erasure coding: store EC schema and cell size in INodeFile and eliminate notion of EC zones. 2015-09-09 23:07:55 -07:00
Vinayakumar B
0f0e897bf1 HDFS-7116. Add a command to get the balancer bandwidth (Contributed by Rakesh R) 2015-09-09 22:58:50 +05:30
Zhe Zhang
ab56fcdb12 Merge remote-tracking branch 'apache/trunk' into HDFS-7285 2015-09-01 14:30:25 -07:00
Zhe Zhang
6b6a63bbbd Merge commit '456e901a4c5c639267ee87b8e5f1319f256d20c2' (HDFS-6407. Add sorting and pagination in the datanode tab of the NN Web UI. Contributed by Haohui Mai.) into HDFS-7285-merge 2015-08-24 12:59:26 -07:00
Lei Xu
7087e700e0 HDFS-8924. Add pluggable interface for reading replicas in DFSClient. (Colin Patrick McCabe via Lei Xu) 2015-08-21 17:02:00 -07:00