Commit Graph

266 Commits

Author SHA1 Message Date
Colin Patrick Mccabe
5f6b4157a4 HDFS-10328. Add per-cache-pool default replication num configuration (xupeng via cmccabe) 2016-06-20 10:42:50 -07:00
Akira Ajisaka
1df6f5735c HDFS-10471. DFSAdmin#SetQuotaCommand's help msg is not correct. Contributed by Yiqun Lin. 2016-06-03 04:10:32 +09:00
Kai Zheng
916140604f HADOOP-12911. Upgrade Hadoop MiniKDC with Kerby. Contributed by Jiajia Li 2016-05-28 14:23:39 +08:00
Zhe Zhang
307db61aac HDFS-10172. hdfs erasurecode command should remove the redundant -usage option. Contributed by Yuanbo Liu. 2016-05-10 15:20:17 -07:00
Xiaoyu Yao
dacd1f50fe HDFS-10324. Trash directory in an encryption zone should be pre-created with correct permissions. Contributed by Wei-Chiu Chuang. 2016-05-05 18:08:47 -07:00
Jing Zhao
63e5412f1a HDFS-9427. HDFS should not default to ephemeral ports. Contributed by Xiaobing Zhou. 2016-04-22 15:14:40 -07:00
Kai Zheng
b5d4c7dc76 HADOOP-12924. Configure raw erasure coders for supported codecs. Contributed by Rui Li. 2016-04-20 15:47:01 +08:00
Arpit Agarwal
c9307e48b7 HDFS-10228. TestHDFSCLI fails. (Contributed by Akira AJISAKA) 2016-03-29 13:50:08 -07:00
Lei Xu
4fcfea71bf HDFS-9005. Provide support for upgrade domain script. (Ming Ma via Lei Xu) 2016-03-25 17:09:12 -07:00
Kihwal Lee
2c8496ebf3 HDFS-8791. block ID-based DN storage layout can be very slow for datanode on ext4. Contributed by Chris Trezzo. 2016-03-01 15:04:59 -06:00
Zhe Zhang
def754ec06 HDFS-9816. Erasure Coding: allow to use multiple EC policies in striping related tests [Part 3]. Contributed by Rui Li.
Change-Id: I64b57bab4722cdc6e1e3148c3a3a401370249afe
2016-02-23 14:05:17 -08:00
zhezhang
dbe49c1bd6 HDFS-9244. Support nested encryption zones.
Change-Id: I43a13035a8b27956e90967ee82058efb647c3415
2016-02-08 16:30:51 -08:00
Arpit Agarwal
86c92227fc HDFS-4015. Safemode should count and report orphaned blocks. (Contributed by Anu Engineer) 2015-10-23 18:07:17 -07:00
Zhe Zhang
1080c37300 Merge remote-tracking branch 'apache/trunk' into HDFS-7285
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/Namesystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java

Change-Id: I8511c4d64b0959e79129febc179845a3892fedcc
2015-09-22 13:27:09 -07:00
Zhe Zhang
607bda2a21 HDFS-9097. Erasure coding: update EC command "-s" flag to "-p" when specifying policy. Contributed by Zhe Zhang.
Change-Id: I82919e55c4a0d8646d25a53aac757098a56b3029
2015-09-17 14:52:37 -07:00
Vinayakumar B
ce69c9b54c HDFS-8953. DataNode Metrics logging (Contributed by Kanaka Kumar Avvaru) 2015-09-16 00:18:29 +05:30
Vinayakumar B
a440567491 HDFS-9067. o.a.h.hdfs.server.datanode.fsdataset.impl.TestLazyWriter is failing in trunk (Contributed by Surendra Singh Lilhore) 2015-09-15 17:30:46 +05:30
Zhe Zhang
f62237bc2f HDFS-8833. Erasure coding: store EC schema and cell size in INodeFile and eliminate notion of EC zones. 2015-09-09 23:07:55 -07:00
Andrew Wang
e1feaf6db0 HADOOP-12358. Add -safely flag to rm to prompt when deleting many files. Contributed by Xiaoyu Yao. 2015-09-04 13:42:55 -07:00
Andrew Wang
afc88b396f HDFS-328. Improve fs -setrep error message for invalid replication factors. Contributed by Daniel Templeton. 2015-09-02 13:46:00 -07:00
Colin Patrick Mccabe
c006c3a1e6 HDFS-2070. Add more unit tests for FsShell getmerge (Daniel Templeton via Colin P. McCabe) 2015-09-02 12:36:14 -07:00
Zhe Zhang
ab56fcdb12 Merge remote-tracking branch 'apache/trunk' into HDFS-7285 2015-09-01 14:30:25 -07:00
Colin Patrick Mccabe
a4d9acc51d HDFS-8846. Add a unit test for INotify functionality across a layout version upgrade (Zhe Zhang via Colin P. McCabe) 2015-08-25 14:29:53 -07:00
Zhe Zhang
6b6a63bbbd Merge commit '456e901a4c5c639267ee87b8e5f1319f256d20c2' (HDFS-6407. Add sorting and pagination in the datanode tab of the NN Web UI. Contributed by Haohui Mai.) into HDFS-7285-merge 2015-08-24 12:59:26 -07:00
Arpit Agarwal
a88f31ebf3 HDFS-8880. NameNode metrics logging. (Arpit Agarwal) 2015-08-17 16:55:46 -07:00
Zhe Zhang
1d37a88121 HDFS-8854. Erasure coding: add ECPolicy to replace schema+cellSize in hadoop-hdfs. Contributed by Walter Su. 2015-08-13 10:04:45 -07:00
cnauroth
6cc8e38db5 HADOOP-12258. Need translate java.nio.file.NoSuchFileException to FileNotFoundException to avoid regression. Contributed by Zhihai Xu. 2015-08-12 16:44:53 -07:00
Colin Patrick Mccabe
c95993cbaf HDFS-8844. TestHDFSCLI does not cleanup the test directory (Masatake Iwasaki via Colin P. McCabe) 2015-08-04 12:22:06 -07:00
Aaron T. Myers
49dfad9429 HDFS-6440. Support more than 2 NameNodes. Contributed by Jesse Yates. 2015-06-23 17:26:11 -07:00
Walter Su
b7d6ea8e91 HDFS-8571. Fix TestErasureCodingCli test. Contributed by Vinayakumar B. 2015-06-11 10:26:07 +08:00
Vinayakumar B
9a18598e2d HDFS-8408. Revisit and refactor ErasureCodingInfo (Contributed by Vinayakumar B) 2015-05-26 12:07:15 -07:00
Vinayakumar B
bba15e06d2 HDFS-8364. Erasure coding: fix some minor bugs in EC CLI (Contributed by Walter Su) 2015-05-26 12:01:51 -07:00
Vinayakumar B
4392325546 HDFS-8242. Erasure Coding: XML based end-to-end test for ECCli commands (Contributed by Rakesh R) 2015-05-26 11:59:58 -07:00
Akira Ajisaka
449e4426a5 HDFS-8325. Misspelling of threshold in log4j.properties for tests. Contributed by Brahma Reddy Battula. 2015-05-07 15:14:20 +09:00
Arpit Agarwal
0d5b0143cc HDFS-8205. CommandFormat#parse() should not parse option as value of option. (Contributed by Peter Shi and Xiaoyu Yao) 2015-04-27 12:23:34 -07:00
cnauroth
fef596df03 HDFS-8055. NullPointerException when topology script is missing. Contributed by Anu Engineer. 2015-04-14 10:19:30 -07:00
yliu
3df61f303a HDFS-7997. The first non-existing xattr should also throw IOException. (zhouyingchao via yliu) 2015-03-31 21:17:44 +08:00
Tsz-Wo Nicholas Sze
cf51ff2fe8 recommit "HDFS-7769. TestHDFSCLI should not create files in hdfs project root dir."
(cherry picked from commit 7c6b6547ee)
2015-02-27 14:30:41 -08:00
Tsz-Wo Nicholas Sze
2954e65467 Revert "HDFS-7769. TestHDFSCLI should not create files in hdfs project root dir."
This reverts commit 7c6b6547ee.

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
2015-02-27 18:25:32 +08:00
cnauroth
8c6ae0d619 HDFS-7814. Fix usage string of storageType parameter for "dfsadmin -setSpaceQuota/clrSpaceQuota". Contributed by Xiaoyu Yao. 2015-02-20 12:21:46 -08:00
Kihwal Lee
7ae5255a16 HDFS-7788. Post-2.6 namenode may not start up with an image containing inodes created with an old release. Contributed by Rushabh Shah. 2015-02-20 09:09:02 -06:00
Akira Ajisaka
46b6d23e8f HDFS-7684. The host:port settings of the deamons should be trimmed before use. Contributed by Anu Engineer. 2015-02-12 17:40:30 -08:00
Arpit Agarwal
5dae97a584 HDFS-7723. Quota By Storage Type namenode implemenation. (Contributed by Xiaoyu Yao) 2015-02-11 10:41:50 -08:00
Tsz-Wo Nicholas Sze
7c6b6547ee HDFS-7769. TestHDFSCLI should not create files in hdfs project root dir. 2015-02-10 17:48:57 -08:00
Konstantin V Shvachko
b73956fdad HDFS-7058. Tests for truncate CLI. Contributed by Dasha Boudnik. 2015-02-09 21:26:59 -08:00
Colin Patrick Mccabe
02340a24f2 HDFS-7718. Store KeyProvider in ClientContext to avoid leaking key provider threads when using FileContext (Arun Suresh via Colin P. McCabe) 2015-02-09 20:23:23 -08:00
Allen Wittenauer
30b797ee9d HADOOP-8934. Shell command ls should include sort options (Jonathan Allen via aw) 2015-02-09 12:50:44 -08:00
Tsz-Wo Nicholas Sze
350b520c64 HDFS-7751. Fix TestHDFSCLI for quota with storage type. Contributed by Xiaoyu Yao 2015-02-07 15:40:47 -08:00
Arpit Agarwal
8de80ff712 HDFS-7720. Quota by Storage Type API, tools and ClientNameNode Protocol changes. (Contributed by Xiaoyu Yao) 2015-02-06 17:01:19 -08:00
Allen Wittenauer
fb06c00837 HADOOP-7713. dfs -count -q should label output column (Jonathan Allen via aw) 2015-02-05 07:44:49 -08:00
Jing Zhao
2848db814a HDFS-3689. Add support for variable length block. Contributed by Jing Zhao. 2015-01-27 12:58:10 -08:00
Arpit Agarwal
d34074e237 HDFS-7575. Upgrade should generate a unique storage ID for each volume. (Contributed by Arpit Agarwal) 2015-01-22 14:08:20 -08:00
Konstantin V Shvachko
08ac06283a HDFS-7056. Snapshot support for truncate. Contributed by Konstantin Shvachko and Plamen Jeliazkov. 2015-01-13 00:24:23 -08:00
Plamen Jeliazkov
7e9358feb3 HDFS-3107. Introduce truncate. Contributed by Plamen Jeliazkov. 2015-01-12 22:50:04 -08:00
Colin Patrick Mccabe
8fa265a290 HDFS-7443. Datanode upgrade to BLOCKID_BASED_LAYOUT fails if duplicate block files are present in the same volume (cmccabe) 2014-12-19 13:18:03 -08:00
Andrew Wang
b0b9084433 HDFS-7530. Allow renaming of encryption zone roots. Contributed by Charles Lamb. 2014-12-18 14:06:53 -08:00
cnauroth
ffe942b82c HDFS-7384. getfacl command and getAclStatus output should be in sync. Contributed by Vinayakumar B. 2014-12-08 10:23:09 -08:00
Allen Wittenauer
ba879a5dad HADOOP-8989. hadoop fs -find feature (Jonathan Allen via aw) 2014-11-13 11:52:38 -08:00
Byron Wong
28051e4155 HADOOP-6857. FsShell should report raw disk usage including replication factor. Contributed by Byron Wong. 2014-10-25 12:31:41 -07:00
arp
9e9e9cf711 HDFS-7153. Add storagePolicy to NN edit log during file creation. (Arpit Agarwal) 2014-09-30 18:10:13 -07:00
Jing Zhao
911979c8ab Merge changes from trunk 2014-09-17 13:12:45 -07:00
Andrew Wang
ea4e2e843e HDFS-6705. Create an XAttr that disallows the HDFS admin from accessing a file. (clamb via wang) 2014-09-17 11:23:47 -07:00
Jing Zhao
db41a1b7b9 Merge changes from trunk 2014-09-09 22:47:50 -07:00
Andrew Wang
04915a0814 HDFS-6951. Correctly persist raw namespace xattrs to edit log and fsimage. Contributed by Charles Lamb. 2014-09-08 16:59:30 -07:00
Jing Zhao
54767d3ff4 Fix TestOfflineEditsViewer after merging. 2014-09-05 12:05:22 -07:00
Jing Zhao
300cb12bef Merge trunk into HDFS-6584 2014-09-05 11:37:22 -07:00
Jing Zhao
6104520369 HDFS-6886. Use single editlog record for creating file + overwrite. Contributed by Yi Liu. 2014-09-04 18:54:38 -07:00
Tsz-wo Sze
c92d869d02 Merge r1609845 through r1619277 from trunk.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1619293 13f79535-47bb-0310-9956-ffa450edef68
2014-08-21 05:22:10 +00:00
Andrew Wang
0cc08f6da4 Merge from trunk to branch.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1618700 13f79535-47bb-0310-9956-ffa450edef68
2014-08-18 18:41:31 +00:00
Tsz-wo Sze
cb75b6b07d Merge r1609845 through r1618416 from trunk.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1618417 13f79535-47bb-0310-9956-ffa450edef68
2014-08-16 21:02:21 +00:00
Allen Wittenauer
a34dafe325 HADOOP-8944. Shell command fs -count should include human readable option (Jonathan Allen via aw)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617775 13f79535-47bb-0310-9956-ffa450edef68
2014-08-13 17:30:26 +00:00
Tsz-wo Sze
b50cb5e056 Merge r1609845 through r1616427 from trunk.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1616428 13f79535-47bb-0310-9956-ffa450edef68
2014-08-07 07:38:23 +00:00
Andrew Wang
ac73d416f3 Merge from trunk to branch
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1615844 13f79535-47bb-0310-9956-ffa450edef68
2014-08-05 02:30:54 +00:00
Colin McCabe
1ba3f89714 HDFS-6482. Use block ID-based block layout on datanodes (James Thomas via Colin Patrick McCabe)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1615223 13f79535-47bb-0310-9956-ffa450edef68
2014-08-01 20:41:05 +00:00
Charles Lamb
9ca2f34c8f HDFS-6730. Create a .RAW extended attribute namespace. (clamb)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1614535 13f79535-47bb-0310-9956-ffa450edef68
2014-07-30 01:35:04 +00:00
Andrew Wang
415223548d HDFS-6771. Require specification of an encryption key when creating an encryption zone. (wang)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1614519 13f79535-47bb-0310-9956-ffa450edef68
2014-07-29 23:39:38 +00:00
Tsz-wo Sze
10d6210322 Merge r1609845 through r1614231 from trunk.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1614234 13f79535-47bb-0310-9956-ffa450edef68
2014-07-29 00:49:14 +00:00
Andrew Wang
e670641bbf Merge from trunk to branch
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1613334 13f79535-47bb-0310-9956-ffa450edef68
2014-07-24 23:55:15 +00:00
Andrew Wang
e171254d56 Name node cannot start if the path of a file under construction contains .snapshot. (wang)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1613329 13f79535-47bb-0310-9956-ffa450edef68
2014-07-24 23:42:06 +00:00
Uma Maheswara Rao G
5343b43fd9 HDFS-6422. getfattr in CLI doesn't throw exception or return non-0 return code when xattr doesn't exist. (Charles Lamb via umamahesh)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612922 13f79535-47bb-0310-9956-ffa450edef68
2014-07-23 19:05:11 +00:00
Andrew Wang
69b75fca7a HDFS-6720. Remove KeyProvider in EncryptionZoneManager. (wang)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1612632 13f79535-47bb-0310-9956-ffa450edef68
2014-07-22 17:57:06 +00:00
Tsz-wo Sze
0a736fbc17 HDFS-6679. Bump NameNodeLayoutVersion and update editsStored test files. Contributed by vinayakumarb
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-6584@1612266 13f79535-47bb-0310-9956-ffa450edef68
2014-07-21 13:38:06 +00:00
Andrew Wang
dda85637df Merge trunk to branch.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1608603 13f79535-47bb-0310-9956-ffa450edef68
2014-07-07 20:43:56 +00:00
Steve Loughran
b6fbaba8e9 HADOOP-9361: Strictly define FileSystem APIs - HDFS portion
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1607597 13f79535-47bb-0310-9956-ffa450edef68
2014-07-03 12:09:45 +00:00
Charles Lamb
2a3bccddd9 HDFS-6625. Remove the Delete Encryption Zone function (clamb)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1607507 13f79535-47bb-0310-9956-ffa450edef68
2014-07-03 00:58:45 +00:00
Charles Lamb
31617733ac HDFS-6389. Rename restrictions for encryption zones. (clamb)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1606253 13f79535-47bb-0310-9956-ffa450edef68
2014-06-27 21:45:18 +00:00
Charles Lamb
2676365799 HDFS-6387. HDFS CLI admin tool for creating & deleting an encryption zone. (clamb)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1605518 13f79535-47bb-0310-9956-ffa450edef68
2014-06-25 17:42:03 +00:00
Vinayakumar B
e8ca648005 HDFS-6507. Improve DFSAdmin to support HA cluster better. (Contributd by Zesheng Wu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1604692 13f79535-47bb-0310-9956-ffa450edef68
2014-06-23 05:16:05 +00:00
Andrew Wang
d417e49ce4 HDFS-6492. Support create-time xattrs and atomically setting multiple xattrs. (wang)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1603971 13f79535-47bb-0310-9956-ffa450edef68
2014-06-19 17:37:31 +00:00
Konstantin Boudnik
2f4f261cda HDFS-6471. Make moveFromLocal CLI testcases to be non-disruptive (Contributed by Dasha Boudnik)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1602064 13f79535-47bb-0310-9956-ffa450edef68
2014-06-12 05:48:26 +00:00
Konstantin Boudnik
fb7ea6d8bc HDFS-6297. Add CLI testcases to reflect new features of dfs and dfsadmin (Contributed by Dasha Boudnik)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1601059 13f79535-47bb-0310-9956-ffa450edef68
2014-06-07 05:22:34 +00:00
Uma Maheswara Rao G
ac23a55547 Merge HDFS-2006 HDFS XAttrs branch to Trunk
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1596575 13f79535-47bb-0310-9956-ffa450edef68
2014-05-21 13:57:33 +00:00
Andrew Wang
8f48760663 HDFS-6345. DFS.listCacheDirectives() should allow filtering based on cache directive ID. (wang)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1595086 13f79535-47bb-0310-9956-ffa450edef68
2014-05-16 01:18:03 +00:00
Uma Maheswara Rao G
8b0acf4b1f HDFS-6337. Setfacl testcase is failing due to dash character in username in TestAclCLI. Contributed by Uma Maheswara Rao G.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1592489 13f79535-47bb-0310-9956-ffa450edef68
2014-05-05 11:53:33 +00:00
Andrew Wang
9445859930 HDFS-6295. Add decommissioning state and node state filtering to dfsadmin. Contributed by Andrew Wang.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1592438 13f79535-47bb-0310-9956-ffa450edef68
2014-05-05 03:38:29 +00:00
Jing Zhao
9dab514b22 HDFS-6038. Allow JournalNode to handle editlog produced by new release with future layoutversion. Contributed by Jing Zhao.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1579813 13f79535-47bb-0310-9956-ffa450edef68
2014-03-20 23:06:06 +00:00
Chris Nauroth
12711177d3 HDFS-6063. TestAclCLI fails intermittently when running test 24: copyFromLocal. Contributed by Chris Nauroth.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1575171 13f79535-47bb-0310-9956-ffa450edef68
2014-03-07 05:44:57 +00:00
Haohui Mai
9192f8446d HDFS-6058. Fix TestHDFSCLI failures after HADOOP-8691 change. Contributed by Akira Ajisaka and Haohui Mai.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1574792 13f79535-47bb-0310-9956-ffa450edef68
2014-03-06 08:13:40 +00:00
Jing Zhao
f9ebdea446 Merging r1573814 through r1574170 from trunk.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-5535@1574171 13f79535-47bb-0310-9956-ffa450edef68
2014-03-04 18:22:26 +00:00
Chris Nauroth
2cfe554980 HDFS-6039. Uploading a File under a Dir with default acls throws "Duplicated ACLFeature". Contributed by Chris Nauroth.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1573879 13f79535-47bb-0310-9956-ffa450edef68
2014-03-04 06:08:25 +00:00