big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
13,523 Commits 2 Branches 2 Tags 264 MiB
d4274c64bc
Commit Graph

5500 Commits

Author SHA1 Message Date
Andrew Wang
010e6ac328 HDFS-2173. saveNamespace should not throw IOE when only one storage directory fails to write VERSION file. Contributed by Andras Bokor. 2016-05-18 14:13:41 -07:00
Lei Xu
cf552aa87b HDFS-10360. DataNode may format directory and lose blocks if current/VERSION is missing. (Wei-Chiu Chuang via lei) 2016-05-18 09:38:37 -07:00
Arpit Agarwal
f4d8fde822 HDFS-9926. MiniDFSCluster leaks dependency Mockito via DataNodeTestUtils. (Contributed by Josh Elser) 2016-05-18 09:20:57 -07:00
Xiaoyu Yao
dd99f5fc79 HDFS-10383. Safely close resources in DFSTestUtil. Contributed by Mingliang Liu. 2016-05-17 16:12:00 -07:00
Yongjun Zhang
16c07cc68a HDFS-10381, DataStreamer DataNode exclusion log message should be warning. (John Zhuge via Yongjun Zhang) 2016-05-17 15:57:33 -07:00
Junping Du
f6ef876fe1 MAPREDUCE-6657. Job history server can fail on startup when NameNode is in start phase. Contributed by Haibo Chen. 2016-05-17 14:41:47 -07:00
Yongjun Zhang
e24fe2641b HDFS-9732, Improve DelegationTokenIdentifier.toString() for better logging. Contributed by Yongjun Zhang. 2016-05-17 14:03:08 -07:00
Kihwal Lee
1356cbe994 Revert "HDFS-10417. Improve error message from checkBlockLocalPathAccess. Contributed by Tianyin Xu." 
This reverts commit 0942954e8a.
 2016-05-17 15:20:03 -05:00
Kihwal Lee
0942954e8a HDFS-10417. Improve error message from checkBlockLocalPathAccess. Contributed by Tianyin Xu. 2016-05-17 15:06:25 -05:00
Colin Patrick Mccabe
7cd5ae62f6 HDFS-10404. Fix formatting of CacheAdmin command usage help text (Yiqun Lin via cmccabe) 2016-05-17 11:10:48 -07:00
Kihwal Lee
4a5819dae2 HDFS-10303. DataStreamer#ResponseProcessor calculates packet ack latency incorrectly. Contributed by Surendra Singh Lilhore. 2016-05-17 08:53:31 -05:00
Akira Ajisaka
9fe5828f05 HDFS-10242. Cannot create space quota of zero. Contributed by Takashi Ohnishi. 2016-05-17 15:53:09 +09:00
Sangjin Lee
61f46be071 HDFS-10208. Addendum for HDFS-9579: to handle the case when client machine can't resolve network path (Ming Ma via sjlee) 2016-05-16 18:49:47 -07:00
Allen Wittenauer
730bc746f9 HADOOP-12930. Dynamic subcommands for hadoop shell scripts (aw) 
This commit contains the following JIRA issues:

    HADOOP-12931. bin/hadoop work for dynamic subcommands
    HADOOP-12932. bin/yarn work for dynamic subcommands
    HADOOP-12933. bin/hdfs work for dynamic subcommands
    HADOOP-12934. bin/mapred work for dynamic subcommands
    HADOOP-12935. API documentation for dynamic subcommands
    HADOOP-12936. modify hadoop-tools to take advantage of dynamic subcommands
    HADOOP-13086. enable daemonization of dynamic commands
    HADOOP-13087. env var doc update for dynamic commands
    HADOOP-13088. fix shellprofiles in hadoop-tools to allow replacement
    HADOOP-13089. hadoop distcp adds client opts twice when dynamic
    HADOOP-13094. hadoop-common unit tests for dynamic commands
    HADOOP-13095. hadoop-hdfs unit tests for dynamic commands
    HADOOP-13107. clean up how rumen is executed
    HADOOP-13108. dynamic subcommands need a way to manipulate arguments
    HADOOP-13110. add a streaming subcommand to mapred
    HADOOP-13111. convert hadoop gridmix to be dynamic
    HADOOP-13115. dynamic subcommand docs should talk about exit vs. continue program flow
    HADOOP-13117. clarify daemonization and security vars for dynamic commands
    HADOOP-13120. add a --debug message when dynamic commands have been used
    HADOOP-13121. rename sub-project shellprofiles to match the rest of Hadoop
    HADOOP-13129. fix typo in dynamic subcommand docs
    HADOOP-13151. Underscores should be escaped in dynamic subcommands document
    HADOOP-13153. fix typo in debug statement for dynamic subcommands
 2016-05-16 17:54:45 -07:00
Lei Xu
6a6e74acf5 HDFS-10410. RedundantEditLogInputStream.LOG is set to wrong class. (John Zhuge via lei) 2016-05-16 17:05:46 -07:00
Kai Zheng
ad9441122f HDFS-8449. Add tasks count metrics to datanode for ECWorker. Contributed by Bo Li. 2016-05-15 23:39:09 -07:00
Andrew Wang
45788204ae HDFS-10333. Intermittent org.apache.hadoop.hdfs.TestFileAppend failure in trunk. Contributed by Yiqun Lin. 2016-05-15 22:15:05 -07:00
Andrew Wang
3c5c57af28 HADOOP-13142. Change project version from 3.0.0 to 3.0.0-alpha1. 2016-05-12 18:27:28 -07:00
Lei Xu
a2774debf7 HDFS-9389. Add maintenance states to AdminStates. (Ming Ma via lei) 2016-05-12 15:42:54 -07:00
Andrew Wang
ca5613af91 Revert "Update project version to 3.0.0-alpha1-SNAPSHOT." 
This reverts commit 6b53802cba.
 2016-05-12 15:32:45 -07:00
Andrew Wang
6b53802cba Update project version to 3.0.0-alpha1-SNAPSHOT. 2016-05-12 11:05:05 -07:00
Tsz-Wo Nicholas Sze
7251bb922b HDFS-10346. Implement asynchronous setPermission/setOwner for DistributedFileSystem. Contributed by Xiaobing Zhou 2016-05-11 17:53:40 -07:00
Colin Patrick Mccabe
687233f20d HADOOP-13065. Add a new interface for retrieving FS and FC Statistics (Mingliang Liu via cmccabe) 2016-05-11 13:45:39 -07:00
Zhe Zhang
307db61aac HDFS-10172. hdfs erasurecode command should remove the redundant -usage option. Contributed by Yuanbo Liu. 2016-05-10 15:20:17 -07:00
Colin Patrick Mccabe
f0252ad252 HDFS-10377. CacheReplicationMonitor shutdown log message should use INFO level (Yiqun Lin via cmccabe) 2016-05-10 11:15:05 -07:00
Kihwal Lee
b9e5a32fa1 HDFS-10372. Fix for failing TestFsDatasetImpl#testCleanShutdownOfVolume. Contributed by Rushabh Shah. 2016-05-09 15:50:09 -05:00
Akira Ajisaka
411fb4b2b7 HDFS-10337. OfflineEditsViewer stats option should print 0 instead of null for the count of operations. Contributed by Yiqun Lin. 2016-05-08 15:49:13 -07:00
Masatake Iwasaki
6957e45699 HDFS-10374. typo in HDFSHighAvailabilityWithQJM doc. Contributed by Takashi Ohnishi. 2016-05-07 11:26:42 +09:00
Masatake Iwasaki
2835f146b5 HDFS-2043. TestHFlush failing intermittently. Contributed by Lin Yiqun. 2016-05-06 15:36:47 +09:00
Xiaoyu Yao
dacd1f50fe HDFS-10324. Trash directory in an encryption zone should be pre-created with correct permissions. Contributed by Wei-Chiu Chuang. 2016-05-05 18:08:47 -07:00
Ravi Prakash
d8faf47f32 HADOOP-13051. Test for special characters in path being respected during globPaths. Contributed by Harsh J. 2016-05-05 14:18:39 -07:00
Allen Wittenauer
35cf503149 HADOOP-13077. Handle special characters in passwords in httpfs.sh (Xiao Chen via aw) 2016-05-05 11:33:06 -07:00
Ming Ma
1268cf5fbe HDFS-10320. Rack failures may result in NN terminate. (Xiao Chen via mingma) 2016-05-04 17:02:26 -07:00
Arpit Agarwal
6d77d6eab7 HDFS-9902. Support different values of dfs.datanode.du.reserved per storage type. (Contributed by Brahma Reddy Battula) 2016-05-03 16:52:43 -07:00
Xiaoyu Yao
45a753ccf7 HDFS-10344. DistributedFileSystem#getTrashRoots should skip encryption zone that does not have .Trash. Contributed by Xiaoyu Yao. 2016-05-02 19:30:47 -07:00
Andrew Wang
3ff0510ffd Fix hadoop-hdfs-native-client compilation on Windows. Contributed by Brahma Reddy Battula. 2016-05-02 18:15:51 -07:00
Tsz-Wo Nicholas Sze
1b9f18623a HADOOP-12957. Limit the number of outstanding async calls. Contributed by Xiaobing Zhou 2016-05-02 11:16:30 -07:00
Ravi Prakash
2753185a01 HADOOP-12563. Updated utility (dtutil) to create/modify token files. Contributed by Matthew Paduano 2016-04-29 22:32:27 -07:00
Kihwal Lee
7da540d03e HDFS-10347. Namenode report bad block method doesn't log the bad block or datanode. Contributed by Rushabh Shah. 2016-04-29 15:50:43 -05:00
Tsz-Wo Nicholas Sze
4da6f69ca1 HDFS-10335 Mover$Processor#chooseTarget() always chooses the first matching target storage group. Contributed by Mingliang Liu 2016-04-29 11:22:33 -07:00
Kihwal Lee
af9b000535 HDFS-10260. TestFsDatasetImpl#testCleanShutdownOfVolume often fails. Contributed by Rushabh Shah. 2016-04-29 09:58:44 -05:00
Kihwal Lee
6243eabb48 HDFS-9958. BlockManager#createLocatedBlocks can throw NPE for corruptBlocks on failed storages. Contributed by Kuhu Shukla 2016-04-28 12:44:53 -05:00
Lei Xu
8bd0bca0b1 HDFS-3702. Fix missing imports from HDFS-3702 trunk patch. 2016-04-27 15:37:41 -07:00
Lei Xu
0a152103f1 HDFS-3702. Add an option for NOT writing the blocks locally if there is a datanode on the same box as the client. (Contributed by Lei (Eddy) Xu) 2016-04-27 14:22:51 -07:00
Allen Wittenauer
7b1c37a13a HADOOP-12892. fix/rewrite create-release (aw) 2016-04-27 08:38:22 -07:00
Kihwal Lee
919a1d824a HDFS-10330. Add Corrupt Blocks Information in Metasave output. Contributed by Kuhu Shukla. 2016-04-27 08:19:48 -05:00
Tsz-Wo Nicholas Sze
fc94810d3f HDFS-10224. Implement asynchronous rename for DistributedFileSystem. Contributed by Xiaobing Zhou 2016-04-26 17:10:13 -07:00
Andrew Wang
6be22ddbf1 HDFS-10297. Increase default balance bandwidth and concurrent moves. Contributed by John Zhuge. 2016-04-26 13:04:18 -07:00
Kihwal Lee
c6a2430b47 HDFS-10329. Bad initialisation of StringBuffer in RequestHedgingProxyProvider. Contributed by Lin Yiqun. 2016-04-26 11:45:29 -05:00
Zhe Zhang
5865fe2bf0 HDFS-9869. Erasure Coding: Rename replication-based names in BlockManager to more generic [part-2]. Contributed by Rakesh R. 2016-04-25 22:01:54 -07:00