big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
10,968 Commits 2 Branches 2 Tags 264 MiB
dae27f6dd1
Commit Graph

84 Commits

Author SHA1 Message Date
Arpit Agarwal
1a495fbb48 HDFS-7645. Rolling upgrade is restoring blocks from trash multiple times (Contributed by Vinayakumar B and Keisuke Ogiwara) 2015-03-30 15:25:16 -07:00
Arpit Agarwal
3d0708bdb0 HDFS-7824. GetContentSummary API and its namenode implementation for Storage Type Quota/Usage. (Contributed by Xiaoyu Yao) 2015-03-26 10:24:11 -07:00
Kihwal Lee
75ead273be HDFS-6841. Use Time.monotonicNow() wherever applicable instead of Time.now(). Contributed by Vinayakumar B 2015-03-20 14:02:09 -05:00
Kihwal Lee
d324164a51 HDFS-7435. PB encoding of block reports is very inefficient. Contributed by Daryn Sharp. 2015-03-13 14:23:37 -05:00
Konstantin V Shvachko
08ac06283a HDFS-7056. Snapshot support for truncate. Contributed by Konstantin Shvachko and Plamen Jeliazkov. 2015-01-13 00:24:23 -08:00
Plamen Jeliazkov
7e9358feb3 HDFS-3107. Introduce truncate. Contributed by Plamen Jeliazkov. 2015-01-12 22:50:04 -08:00
yliu
58c0bb9ed9 HDFS-6606. Optimize HDFS Encrypted Transport performance. (yliu) 2014-10-28 21:11:31 +08:00
arp
bb84f1fccb HDFS-7159. Use block storage policy to set lazy persist preference. (Arpit Agarwal) 2014-09-29 22:27:59 -07:00
arp
032e0eba6b Merge branch 'trunk' into HDFS-6581 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
 2014-09-25 19:50:28 -07:00
Andrew Wang
e96ce6f3e3 HDFS-7077. Separate CipherSuite from crypto protocol version. (wang) 2014-09-25 18:40:40 -07:00
arp
b1000fbba4 Merge branch 'trunk' into HDFS-6581 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java
 2014-09-24 20:08:32 -07:00
Jing Zhao
073bbd805c HDFS-7081. Add new DistributedFileSystem API for getting all the existing storage policies. Contributed by Jing Zhao. 2014-09-24 10:05:40 -07:00
arp
889122bd55 Merge branch 'trunk' into HDFS-6581 2014-09-22 18:00:58 -07:00
Andrew Wang
1737950d0f HDFS-6987. Move CipherSuite xattr information up to the encryption zone root. Contributed by Zhe Zhang. 2014-09-21 21:29:46 -07:00
arp
f8bbf80067 Merge branch 'trunk' into HDFS-6581 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/StorageType.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/HdfsFileStatus.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/HdfsLocatedFileStatus.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelper.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageFormat.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageFormatPBINode.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFileAttributes.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeLayoutVersion.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/FSImageFormatPBSnapshot.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/proto/fsimage.proto
	hadoop-hdfs-project/hadoop-hdfs/src/main/proto/hdfs.proto
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
 2014-09-18 22:26:42 -07:00
arp
7f49537ba1 HDFS-6924. Add new RAM_DISK storage type. (Arpit Agarwal) 2014-08-27 21:47:18 -07:00
arp
a7bcc95358 HDFS-6921. Add LazyPersist flag to FileStatus. (Arpit Agarwal) 2014-08-27 21:47:18 -07:00
Tsz-wo Sze
c92d869d02 Merge r1609845 through r1619277 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1619293 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-21 05:22:10 +00:00
Tsz-wo Sze
9b250d74f0 HDFS-6847. Support storage policy on directories and include storage policy in HdfsFileStatus. Contributed by Jing Zhao 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1618416 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-16 20:58:20 +00:00
Andrew Wang
ac73d416f3 Merge from trunk to branch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1615844 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-05 02:30:54 +00:00
Tsz-wo Sze
dc7744d2e5 Merge r1609845 through r1615019 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1615020 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-01 01:29:49 +00:00
Tsz-wo Sze
b8597e6a10 HDFS-6685. Balancer should preserve storage type of replicas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1615015 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-01 01:05:33 +00:00
Andrew Wang
c3fa0b5884 Merge from trunk to branch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1614553 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-30 06:09:33 +00:00
Tsz-wo Sze
10d6210322 Merge r1609845 through r1614231 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1614234 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-29 00:49:14 +00:00
Tsz-wo Sze
4f9ffc7455 HDFS-6739. Add getDatanodeStorageReport to ClientProtocol. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1614215 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-28 23:43:39 +00:00
Andrew Wang
552b4fb9f9 Merge from trunk to branch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1612928 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-23 19:26:29 +00:00
Tsz-wo Sze
ee13f8bf3e Merge r1609845 through r1612502 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1612505 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-22 08:23:34 +00:00
Tsz-wo Sze
25b0e8471e HDFS-6702. Change DFSClient to pass the StorageType from the namenode to datanodes and change datanode to write block replicas using the specified storage type. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612493 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-22 07:41:24 +00:00
Tsz-wo Sze
4f1fae88f8 HDFS-6670. Add block storage policy support with default HOT, WARM and COLD policies. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1610529 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-14 21:11:22 +00:00
Andrew Wang
d79f27b429 HDFS-6474. Namenode needs to get the actual keys and iv from the KeyProvider. (wang) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1609833 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-11 20:54:47 +00:00
Andrew Wang
dda85637df Merge trunk to branch. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1608603 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-07 20:43:56 +00:00
Andrew Wang
51b97a1396 HDFS-6605.Client server negotiation of cipher suite. (wang) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1607499 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-02 23:08:29 +00:00
Andrew Wang
2efea95213 HDFS-6391. Get the Key/IV from the NameNode for encrypted files in DFSClient. Contributed by Charles Lamb and Andrew Wang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1606220 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-27 20:43:41 +00:00
Jing Zhao
8a83bb7ad6 HDFS-4667. Capture renamed files/directories in snapshot diff report. Contributed by Jing Zhao and Binglin Chang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1604488 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-21 22:59:34 +00:00
Charles Lamb
bdee397e95 HADOOP-6392. Wire crypto streams for encrypted files in DFSClient. (clamb and yliu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1600582 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-05 10:10:48 +00:00
Andrew Wang
d37c31a2db HDFS-6102. Lower the default maximum items per directory to fix PB fsimage loading. Contributed by Andrew Wang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1577426 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-14 04:52:03 +00:00
Kihwal Lee
1a68f13521 HDFS-5585. Provide admin commands for data node upgrade. Contributed by Kihwal Lee. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1568523 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-14 21:20:02 +00:00
Tsz-wo Sze
4f92eb2f61 HDFS-5848. Add rolling upgrade status to heartbeat response. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1563384 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-01 08:58:55 +00:00
Arpit Agarwal
907fb15ee8 HDFS-5480. Update Balancer for HDFS-2832. (Contributed by szetszwo) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1540547 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-10 20:59:32 +00:00
Arpit Agarwal
c2495a7bff Merging r1536572 from trunk to branch HDFS-2832 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1536889 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-29 21:04:31 +00:00
Andrew Wang
34f08944b7 merge trunk to branch HDFS-4949 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4949@1532952 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-17 02:14:33 +00:00
Arpit Agarwal
168b133337 Merging r1527684 through r1532876 from trunk to branch HDFS-2832 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1532910 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-16 21:07:28 +00:00
Andrew Wang
3fc8792b5c HDFS-5304. Expose if a block replica is cached in getFileBlockLocations. (Contributed by Andrew Wang) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4949@1530802 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-09 21:30:08 +00:00
Brandon Li
8e08046661 HDFS-5306. Datanode https port is not available at the namenode. Contributed by Suresh Srinivas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529562 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-06 03:22:57 +00:00
Arpit Agarwal
eb9f1b6707 HDFS-5232. Protocol changes to transmit StorageUuid. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1525153 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-20 22:06:09 +00:00
Tsz-wo Sze
abf09f090f HDFS-4990. Change BlockPlacementPolicy to choose storages instead of datanodes. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1524444 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-18 15:12:52 +00:00
Tsz-wo Sze
3f070e83b1 HDFS-5009. Include storage information in the LocatedBlock. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1519691 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-03 14:03:35 +00:00
Andrew Wang
fc14a92c6b HDFS-5141. Add cache status information to datanode heartbeat. (Contributed by Andrew Wang) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4949@1519101 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-30 22:15:51 +00:00
Arpit Agarwal
5e27288c1e HDFS-4985. Add storage type to the protocol and expose it in block report and block locations 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832@1516323 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-21 23:07:29 +00:00
Brandon Li
4f68aa0600 HDFS-5043. For HdfsFileStatus, set default value of childrenNum to -1 instead of 0 to avoid confusing applications. Contributed by Brandon Li 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1508694 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-31 00:49:47 +00:00