big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
11,139 Commits 2 Branches 2 Tags 264 MiB
4fdd9abd7e
Commit Graph

94 Commits

Author SHA1 Message Date
Vinayakumar B
0b7af27b9a HDFS-8468. 2 RPC calls for every file read in DFSClient#open(..) resulting in double Audit log entries (Contributed by Vinayakumar B) 2015-06-30 11:02:13 +05:30
Jing Zhao
683332b36d HDFS-8585. Erasure Coding: Remove dataBlockNum and parityBlockNum from StripedBlockProto. Contributed by Yi Liu. 2015-06-12 14:48:53 -07:00
Zhe Zhang
673280df24 HDFS-7621. Erasure Coding: update the Balancer/Mover data migration logic. Contributed by Walter Su. 2015-06-03 11:51:58 -07:00
Zhe Zhang
91c81fdc24 HDFS-8375. Add cellSize as an XAttr to ECZone. Contributed by Vinayakumar B. 2015-05-26 12:02:33 -07:00
Jing Zhao
9da927540f HDFS-8289. Erasure Coding: add ECSchema to HdfsFileStatus. Contributed by Yong Zhang. 2015-05-26 12:01:49 -07:00
Vinayakumar B
04c23d95d0 HDFS-8123. Erasure Coding: Better to move EC related proto messages to a separate erasurecoding proto file (Contrubuted by Rakesh R) 2015-05-26 11:59:11 -07:00
Vinayakumar B
f53e402635 HDFS-8090. Erasure Coding: Add RPC to client-namenode to list all ECSchemas loaded in Namenode. (Contributed by Vinayakumar B) 2015-05-26 11:58:40 -07:00
Vinayakumar B
16484f0030 HDFS-8023. Erasure Coding: retrieve eraure coding schema for a file from NameNode (Contributed by Vinayakumar B) 2015-05-26 11:55:16 -07:00
Jing Zhao
f05c21285e HDFS-7853. Erasure coding: extend LocatedBlocks to support reading from striped files. Contributed by Jing Zhao. 2015-05-26 11:32:25 -07:00
Jing Zhao
9f2f583f40 HDFS-7749. Erasure Coding: Add striped block support in INodeFile. Contributed by Jing Zhao. 2015-05-26 11:07:15 -07:00
Arpit Agarwal
1a495fbb48 HDFS-7645. Rolling upgrade is restoring blocks from trash multiple times (Contributed by Vinayakumar B and Keisuke Ogiwara) 2015-03-30 15:25:16 -07:00
Arpit Agarwal
3d0708bdb0 HDFS-7824. GetContentSummary API and its namenode implementation for Storage Type Quota/Usage. (Contributed by Xiaoyu Yao) 2015-03-26 10:24:11 -07:00
Kihwal Lee
75ead273be HDFS-6841. Use Time.monotonicNow() wherever applicable instead of Time.now(). Contributed by Vinayakumar B 2015-03-20 14:02:09 -05:00
Kihwal Lee
d324164a51 HDFS-7435. PB encoding of block reports is very inefficient. Contributed by Daryn Sharp. 2015-03-13 14:23:37 -05:00
Konstantin V Shvachko
08ac06283a HDFS-7056. Snapshot support for truncate. Contributed by Konstantin Shvachko and Plamen Jeliazkov. 2015-01-13 00:24:23 -08:00
Plamen Jeliazkov
7e9358feb3 HDFS-3107. Introduce truncate. Contributed by Plamen Jeliazkov. 2015-01-12 22:50:04 -08:00
yliu
58c0bb9ed9 HDFS-6606. Optimize HDFS Encrypted Transport performance. (yliu) 2014-10-28 21:11:31 +08:00
arp
bb84f1fccb HDFS-7159. Use block storage policy to set lazy persist preference. (Arpit Agarwal) 2014-09-29 22:27:59 -07:00
arp
032e0eba6b Merge branch 'trunk' into HDFS-6581 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
 2014-09-25 19:50:28 -07:00
Andrew Wang
e96ce6f3e3 HDFS-7077. Separate CipherSuite from crypto protocol version. (wang) 2014-09-25 18:40:40 -07:00
arp
b1000fbba4 Merge branch 'trunk' into HDFS-6581 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java
 2014-09-24 20:08:32 -07:00
Jing Zhao
073bbd805c HDFS-7081. Add new DistributedFileSystem API for getting all the existing storage policies. Contributed by Jing Zhao. 2014-09-24 10:05:40 -07:00
arp
889122bd55 Merge branch 'trunk' into HDFS-6581 2014-09-22 18:00:58 -07:00
Andrew Wang
1737950d0f HDFS-6987. Move CipherSuite xattr information up to the encryption zone root. Contributed by Zhe Zhang. 2014-09-21 21:29:46 -07:00
arp
f8bbf80067 Merge branch 'trunk' into HDFS-6581 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/StorageType.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/HdfsFileStatus.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/HdfsLocatedFileStatus.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelper.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageFormat.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageFormatPBINode.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFileAttributes.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeLayoutVersion.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/FSImageFormatPBSnapshot.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/proto/fsimage.proto
	hadoop-hdfs-project/hadoop-hdfs/src/main/proto/hdfs.proto
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
 2014-09-18 22:26:42 -07:00
arp
7f49537ba1 HDFS-6924. Add new RAM_DISK storage type. (Arpit Agarwal) 2014-08-27 21:47:18 -07:00
arp
a7bcc95358 HDFS-6921. Add LazyPersist flag to FileStatus. (Arpit Agarwal) 2014-08-27 21:47:18 -07:00
Tsz-wo Sze
c92d869d02 Merge r1609845 through r1619277 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1619293 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-21 05:22:10 +00:00
Tsz-wo Sze
9b250d74f0 HDFS-6847. Support storage policy on directories and include storage policy in HdfsFileStatus. Contributed by Jing Zhao 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1618416 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-16 20:58:20 +00:00
Andrew Wang
ac73d416f3 Merge from trunk to branch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1615844 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-05 02:30:54 +00:00
Tsz-wo Sze
dc7744d2e5 Merge r1609845 through r1615019 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1615020 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-01 01:29:49 +00:00
Tsz-wo Sze
b8597e6a10 HDFS-6685. Balancer should preserve storage type of replicas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1615015 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-01 01:05:33 +00:00
Andrew Wang
c3fa0b5884 Merge from trunk to branch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1614553 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-30 06:09:33 +00:00
Tsz-wo Sze
10d6210322 Merge r1609845 through r1614231 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1614234 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-29 00:49:14 +00:00
Tsz-wo Sze
4f9ffc7455 HDFS-6739. Add getDatanodeStorageReport to ClientProtocol. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1614215 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-28 23:43:39 +00:00
Andrew Wang
552b4fb9f9 Merge from trunk to branch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1612928 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-23 19:26:29 +00:00
Tsz-wo Sze
ee13f8bf3e Merge r1609845 through r1612502 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1612505 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-22 08:23:34 +00:00
Tsz-wo Sze
25b0e8471e HDFS-6702. Change DFSClient to pass the StorageType from the namenode to datanodes and change datanode to write block replicas using the specified storage type. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612493 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-22 07:41:24 +00:00
Tsz-wo Sze
4f1fae88f8 HDFS-6670. Add block storage policy support with default HOT, WARM and COLD policies. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1610529 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-14 21:11:22 +00:00
Andrew Wang
d79f27b429 HDFS-6474. Namenode needs to get the actual keys and iv from the KeyProvider. (wang) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1609833 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-11 20:54:47 +00:00
Andrew Wang
dda85637df Merge trunk to branch. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1608603 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-07 20:43:56 +00:00
Andrew Wang
51b97a1396 HDFS-6605.Client server negotiation of cipher suite. (wang) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1607499 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-02 23:08:29 +00:00
Andrew Wang
2efea95213 HDFS-6391. Get the Key/IV from the NameNode for encrypted files in DFSClient. Contributed by Charles Lamb and Andrew Wang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1606220 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-27 20:43:41 +00:00
Jing Zhao
8a83bb7ad6 HDFS-4667. Capture renamed files/directories in snapshot diff report. Contributed by Jing Zhao and Binglin Chang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1604488 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-21 22:59:34 +00:00
Charles Lamb
bdee397e95 HADOOP-6392. Wire crypto streams for encrypted files in DFSClient. (clamb and yliu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1600582 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-05 10:10:48 +00:00
Andrew Wang
d37c31a2db HDFS-6102. Lower the default maximum items per directory to fix PB fsimage loading. Contributed by Andrew Wang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1577426 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-14 04:52:03 +00:00
Kihwal Lee
1a68f13521 HDFS-5585. Provide admin commands for data node upgrade. Contributed by Kihwal Lee. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1568523 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-14 21:20:02 +00:00
Tsz-wo Sze
4f92eb2f61 HDFS-5848. Add rolling upgrade status to heartbeat response. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1563384 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-01 08:58:55 +00:00
Arpit Agarwal
907fb15ee8 HDFS-5480. Update Balancer for HDFS-2832. (Contributed by szetszwo) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1540547 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-10 20:59:32 +00:00
Arpit Agarwal
c2495a7bff Merging r1536572 from trunk to branch HDFS-2832 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1536889 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-29 21:04:31 +00:00