big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
9,447 Commits 2 Branches 2 Tags 264 MiB
f5b1631179
Commit Graph

3246 Commits

Author SHA1 Message Date
Konstantin V Shvachko
203c63030f HDFS-7333. Improve logging in Storage.tryLock(). Contributed by Konstantin Shvachko. 2014-11-05 09:53:23 -08:00
Konstantin V Shvachko
6e8722e49c HDFS-7335. Redundant checkOperation() in FSN.analyzeFileState(). Contributed by Milan Desai. 2014-11-05 09:32:32 -08:00
yliu
73e601259f HDFS-7218. FSNamesystem ACL operations should write to audit log on failure. (clamb via yliu) 2014-11-05 15:49:37 +08:00
Jing Zhao
5bd3a569f9 HDFS-7233. NN logs unnecessary org.apache.hadoop.hdfs.protocol.UnresolvedPathException. Contributed by Rushabh S Shah. 2014-11-04 16:02:07 -08:00
Haohui Mai
99d710348a HDFS-7355. TestDataNodeVolumeFailure#testUnderReplicationAfterVolFailure fails on Windows, because we cannot deny access to the file owner. Contributed by Chris Nauroth. 2014-11-04 15:18:51 -08:00
Haohui Mai
d0449bd2fd HDFS-7334. Fix periodic failures of TestCheckpoint#testTooManyEditReplayFailures. Contributed by Charles Lamb. 2014-11-04 14:34:52 -08:00
Jing Zhao
3dfd6e68fe HDFS-7340. Make rollingUpgrade start/finalize idempotent. Contributed by Jing Zhao. 2014-11-04 10:16:37 -08:00
cnauroth
2bb327eb93 HDFS-7328. TestTraceAdmin assumes Unix line endings. Contributed by Chris Nauroth. 2014-11-03 19:59:52 -08:00
Haohui Mai
35d353e0f6 HDFS-7147. Update archival storage user documentation. Contributed by Tsz Wo Nicholas Sze. 2014-11-03 15:10:22 -08:00
Suresh Srinivas
237890feab HDFS-7324. haadmin command usage prints incorrect command name. Contributed by Brahma Reddy Battula. 2014-11-03 13:27:09 -08:00
Kihwal Lee
67f13b58e4 HADOOP-10987. Provide an iterator-based listing API for FileSystem. Contributed by Kihwal Lee. 2014-11-03 08:20:22 -06:00
Tsz-Wo Nicholas Sze
36ccf097a9 HDFS-7276. Limit the number of byte arrays used by DFSOutputStream and provide a mechanism for recycling arrays. 2014-11-01 11:22:13 -07:00
Colin Patrick Mccabe
7b026c50f1 HDFS-6917. Add an hdfs debug command to validate blocks, call recoverlease, etc. (cmccabe) 2014-10-31 13:15:17 -07:00
Haohui Mai
3f030c04e8 HDFS-7315. DFSTestUtil.readFileBuffer opens extra FSDataInputStream. Contributed by Plamen Jeliazkov. 2014-10-31 11:47:41 -07:00
Ravi Prakash
c7f81dad30 HDFS-7309. XMLUtils.mangleXmlString doesn't seem to handle less than sign. (Colin Patrick McCabe via raviprak) 2014-10-31 11:22:25 -07:00
cnauroth
5573b3476a HDFS-7313. Support optional configuration of AES cipher suite on DataTransferProtocol. Contributed by Chris Nauroth. 2014-10-30 22:48:25 -07:00
cnauroth
c637d6d9d5 HDFS-6385. Show when block deletion will start after NameNode startup in WebUI. Contributed by Chris Nauroth. 2014-10-30 22:26:35 -07:00
Colin Patrick Mccabe
a9331fe9b0 HDFS-7035. Make adding a new data directory to the DataNode an atomic operation and improve error handling (Lei Xu via Colin P. McCabe) 2014-10-30 17:31:23 -07:00
Andrew Wang
c2866ac34d HDFS-3342. SocketTimeoutException in BlockSender.sendChunks could have a better error message. Contributed by Yongjun Zhang. 2014-10-30 11:04:10 -07:00
Jing Zhao
b811212f23 HDFS-7230. Add rolling downgrade documentation. Contributed by Tsz Wo Nicholas Sze. 2014-10-30 10:47:05 -07:00
Tao Luo
0126cf16b7 HDFS-7263. Snapshot read can reveal future bytes for appended files. Contributed by Tao Luo. 2014-10-29 20:21:12 -07:00
Jing Zhao
6f5f604a79 HDFS-7305. NPE seen in wbhdfs FS while running SLive. Contributed by Jing Zhao. 2014-10-29 16:25:48 -07:00
Kihwal Lee
3ae84e1ba8 HDFS-7300. HDFS-7300. The getMaxNodesPerRack() method in 
BlockPlacementPolicyDefault is flawed. contributed by Kihwal Lee
 2014-10-29 17:23:34 -05:00
Colin Patrick Mccabe
d33e07dc49 HDFS-7287. The OfflineImageViewer (OIV) can output invalid XML depending on the filename (Ravi Prakash via Colin P. McCabe) 2014-10-29 12:31:59 -07:00
Brandon Li
72a556d3b0 HADOOP-11195. Move Id-Name mapping in NFS to the hadoop-common area for better maintenance. Contributed by Yongjun Zhang 2014-10-29 11:05:29 -07:00
Haohui Mai
675bca2968 HDFS-7280. Use netty 4 in WebImageViewer. Contributed by Haohui Mai. 2014-10-28 16:53:53 -07:00
Colin Patrick Mccabe
ac9ab037e9 HDFS-7235. DataNode#transferBlock should report blocks that don't exist using reportBadBlock (yzhang via cmccabe) 2014-10-28 16:41:22 -07:00
cnauroth
c6f04f391b HDFS-7291. Persist in-memory replicas with appropriate unbuffered copy API on POSIX and Windows. Contributed by Xiaoyu Yao. 2014-10-28 14:06:27 -07:00
Kihwal Lee
e226b5b40d HDFS-7213. processIncrementalBlockReport performance degradation. 
Contributed by Eric Payne.
 2014-10-28 14:54:05 -05:00
Kihwal Lee
371a3b87ed HDFS-6663. Admin command to track file and locations from block id. 
Contributed by Chen He.
 2014-10-28 14:26:04 -05:00
yliu
58c0bb9ed9 HDFS-6606. Optimize HDFS Encrypted Transport performance. (yliu) 2014-10-28 21:11:31 +08:00
Harsh J
0398db19b2 HDFS-6741. Improve permission denied message when FSPermissionChecker#checkOwner fails. Contributed by Stephen Chu and Harsh J. (harsh) 2014-10-28 12:22:37 +05:30
Colin Patrick Mccabe
518a7f4af3 HDFS-7282. Fix intermittent TestShortCircuitCache and TestBlockReaderFactory failures resulting from TemporarySocketDirectory GC (Jinghui Wang via Colin P. McCabe) 2014-10-27 17:44:07 -07:00
Haohui Mai
00b4e44a2e HDFS-5928. Show namespace and namenode ID on NN dfshealth page. Contributed by Siqi Li. 2014-10-27 16:22:19 -07:00
Colin Patrick Mccabe
baf794dc40 HDFS-7278. Add a command that allows sysadmins to manually trigger full block reports from a DN (cmccabe) 2014-10-27 09:53:16 -07:00
cnauroth
463aec1171 HDFS-6934. Move checksum computation off the hot path when writing to RAM disk. Contributed by Chris Nauroth. 2014-10-27 09:38:30 -07:00
Harsh J
0058eadbd3 HDFS-6538. Comment format error in ShortCircuitRegistry javadoc. Contributed by David Luo. (harsh) 2014-10-27 19:24:50 +05:30
Byron Wong
28051e4155 HADOOP-6857. FsShell should report raw disk usage including replication factor. Contributed by Byron Wong. 2014-10-25 12:31:41 -07:00
Colin Patrick Mccabe
a52eb4bc5f HDFS-6988. Improve HDFS-6581 eviction configuration (Xiaoyu Yao via Colin P. McCabe) 2014-10-24 13:08:59 -07:00
Jitendra Pandey
86ac0d40b8 HDFS-6904. Added files missing in previous commit. 2014-10-24 12:01:09 -07:00
Jitendra Pandey
e2be333744 HDFS-6904. YARN unable to renew delegation token fetched via webhdfs due to incorrect service port. 2014-10-24 11:35:19 -07:00
Haohui Mai
b3d8a642a9 HDFS-7283. Bump DataNode OOM log from WARN to ERROR. Contributed by Stephen Chu. 2014-10-24 10:50:16 -07:00
Haohui Mai
0942c99eba HDFS-7258. CacheReplicationMonitor rescan schedule log should use DEBUG level instead of INFO level. Contributed by Xiaoyu Yao. 2014-10-23 23:58:32 -07:00
yliu
57dec28807 HDFS-7243. HDFS concat operation should not be allowed in Encryption Zone. (clamb via yliu) 2014-10-24 11:12:18 +08:00
Colin Patrick Mccabe
5b56ac4c72 HDFS-7223. Tracing span description of IPC client is too long (iwasakims via cmccabe) 2014-10-23 19:14:00 -07:00
Haohui Mai
670879ef41 HDFS-7257. Add the time of last HA state transition to NN's /jmx page. Contributed by Charles Lamb. 2014-10-23 16:52:58 -07:00
Andrew Wang
86cad007d7 HDFS-7222. Expose DataNode network errors as a metric. (Charles Lamb via wang) 2014-10-23 12:53:01 -07:00
Andrew Wang
8c5b23b547 HDFS-7165. Separate block metrics for files with replication count 1. (Zhe Zhang via wang) 2014-10-23 12:28:02 -07:00
Haohui Mai
f729ecf9d2 HDFS-7277. Remove explicit dependency on netty 3.2 in BKJournal. Contributed by Haohui Mai. 2014-10-22 17:31:57 -07:00
Andrew Wang
a36399e09c HDFS-6824. Additional user documentation for HDFS encryption. 2014-10-22 14:38:47 -07:00
Colin Patrick Mccabe
7b0f9bb258 HDFS-6877. Avoid calling checkDisk when an HDFS volume is removed during a write. (Lei Xu via Colin P. McCabe) 2014-10-22 13:38:26 -07:00
Colin Patrick Mccabe
66e8187ea1 HDFS-7254. Add documentation for hot swaping DataNode drives (Lei Xu via Colin P. McCabe) 2014-10-22 13:27:51 -07:00
Jing Zhao
1c8d191117 HDFS-7228. Fix TestDNFencing.testQueueingWithAppend. Contributed by Yongjun Zhang. 2014-10-22 10:48:15 -07:00
Allen Wittenauer
4baca311ff HDFS-7204. balancer doesn't run as a daemon (aw) 2014-10-21 14:26:08 -07:00
Andrew Wang
ac56b0637e HDFS-7221. TestDNFencingWithReplication fails consistently. Contributed by Charles Lamb. 2014-10-21 12:55:18 -07:00
Brandon Li
4e134a02a4 HDFS-7215.Add JvmPauseMonitor to NFS gateway. Contributed by Brandon Li 2014-10-21 11:49:10 -07:00
Colin Patrick Mccabe
7aab5fa1bd HDFS-7154. Fix returning value of starting reconfiguration task (Lei Xu via Colin P. McCabe) 2014-10-20 18:33:11 -07:00
Colin Patrick Mccabe
4799570dfd HDFS-7266. HDFS Peercache enabled check should not lock on object (awang via cmccabe) 2014-10-20 18:24:53 -07:00
Benoy Antony
e4d6a87854 HDFS-7184. Allow data migration tool to run as a daemon. (Benoy Antony) 2014-10-20 12:15:58 -07:00
Tsz-Wo Nicholas Sze
2e140523d3 HDFS-7260. Change DFSOutputStream.MAX_PACKETS to be configurable. 2014-10-17 18:30:09 -07:00
Vinayakumar B
368743140d HDFS-7252. small refinement to the use of isInAnEZ in FSNamesystem. (Yi Liu via vinayakumarb) 2014-10-17 18:03:44 +05:30
Vinayakumar B
1c3ff0b7c8 HDFS-7242. Code improvement for FSN#checkUnreadableBySuperuser. (Contributed by Yi Liu) 2014-10-17 17:48:20 +05:30
Vinayakumar B
cba1f9e389 HDFS-6995. Block should be placed in the client's 'rack-local' node if 'client-local' node is not available (vinayakumarb) 
Adding missed test file
 2014-10-17 17:31:01 +05:30
Tsz-Wo Nicholas Sze
2894433332 HDFS-5089. When a LayoutVersion support SNAPSHOT, it must support FSIMAGE_NAME_OPTIMIZATION. 2014-10-15 20:56:59 -07:00
Tsz-Wo Nicholas Sze
41980c56d3 HDFS-7208. NN doesn't schedule replication when a DN storage fails. Contributed by Ming Ma 2014-10-15 20:44:24 -07:00
Jing Zhao
18620649f9 HDFS-7185. The active NameNode will not accept an fsimage sent from the standby during rolling upgrade. Contributed by Jing Zhao. 2014-10-15 10:27:30 -07:00
Haohui Mai
128ace10cd HDFS-7190. Bad use of Preconditions in startFileInternal(). Contributed by Dawson Choong. 2014-10-14 23:42:19 -07:00
Haohui Mai
0260231667 HDFS-7201. Fix typos in hdfs-default.xml. Contributed by Dawson Choong. 2014-10-14 17:36:07 -07:00
Jing Zhao
7dcad84143 HDFS-7228. Add an SSD policy into the default BlockStoragePolicySuite. Contributed by Jing Zhao. 2014-10-14 10:22:34 -07:00
Tsz-Wo Nicholas Sze
f6d0b8892a HDFS-7237. The command "hdfs namenode -rollingUpgrade" throws ArrayIndexOutOfBoundsException. 2014-10-13 18:28:34 -07:00
cnauroth
1770bb942f HDFS-7090. Use unbuffered writes when persisting in-memory replicas. Contributed by Xiaoyu Yao. 2014-10-13 10:56:32 -07:00
Haohui Mai
53100318ea HDFS-6544. Broken Link for GFS in package.html. Contributed by Suraj Nayak M. 2014-10-13 10:33:19 -07:00
Jing Zhao
98ac9f26c5 HDFS-7236. Fix TestOpenFilesWithSnapshot#testOpenFilesWithMultipleSnapshots. Contributed by Yongjun Zhang. 2014-10-13 10:27:36 -07:00
Andrew Wang
d3d3d47202 HDFS-7209. Populate EDEK cache when creating encryption zone. (Yi Liu via wang) 2014-10-10 13:40:37 -07:00
Colin Patrick Mccabe
1ceb326933 HDFS-7198. Fix "unchecked conversion" warning in DFSClient#getPathTraceScope (cmccabe) 2014-10-10 13:03:29 -07:00
Aaron T. Myers
cbd21fd13b HDFS-7026. Introduce a string constant for "Failed to obtain user group info...". Contributed by Yongjun Zhang. 2014-10-09 18:52:48 -07:00
Andrew Wang
8d94114495 HADOOP-11174. Delegation token for KMS should only be got once if it already exists. (Yi Liu via wang) 2014-10-09 13:21:19 -07:00
Kihwal Lee
db71bb54bc HDFS-7217. Better batching of IBRs. Contributed by Kihwal Lee. 2014-10-09 12:07:51 -05:00
Aaron T. Myers
2a51494ce1 HADOOP-11161. Expose close method in KeyProvider to give clients of Provider implementations a hook to release resources. Contribued by Arun Suresh. 2014-10-08 18:01:51 -07:00
Colin Patrick Mccabe
d996235285 HDFS-7202. Should be able to omit package name of SpanReceiver on "hadoop trace -add" (iwasakims via cmccabe) 2014-10-08 17:55:46 -07:00
Kihwal Lee
853cb704ed HDFS-7203. Concurrent appending to the same file can cause data 
corruption. Contributed by Kihwal Lee.
 2014-10-08 15:05:34 -05:00
cnauroth
1efd9c9825 HDFS-7112. LazyWriter should use either async IO or one thread per physical disk. Contributed by Xiaoyu Yao. 2014-10-07 20:25:19 -07:00
cnauroth
9b8a35aff6 HDFS-7128. Decommission slows way down when it gets towards the end. Contributed by Ming Ma. 2014-10-07 15:01:09 -07:00
Kihwal Lee
99d7a452be HADOOP-11007. Reinstate building of ant tasks support. Contributed by 
Jason Lowe.
 2014-10-07 16:20:16 -05:00
Colin Patrick Mccabe
8dc6abf2f4 HDFS-7194 Fix findbugs "inefficient new String constructor" warning in DFSClient#PATH (yzhang via cmccabe) 2014-10-06 15:29:09 -07:00
Vinayakumar B
ed841dd9a9 HDFS-6995. Block should be placed in the client's 'rack-local' node if 'client-local' node is not available (vinayakumarb) 2014-10-06 14:31:15 +05:30
Colin Patrick Mccabe
7f6ed7fe36 HDFS-7055. Add tracing to DFSInputStream (cmccabe) 2014-10-03 13:35:43 -07:00
Andrew Wang
d2d5a0ea03 HDFS-7179. DFSClient should instantiate a KeyProvider, not a KeyProviderCryptoExtension. (wang) 2014-10-02 13:50:05 -07:00
Colin Patrick Mccabe
03db9cc839 HDFS-7162. Wrong path when deleting through fuse-dfs a file which already exists in trash (cmccabe) 2014-10-02 12:42:19 -07:00
arp
c2fa5d1c42 HDFS-7178. Additional unit test for replica write with full disk. (Arpit Agarwal) 2014-10-02 09:28:26 -07:00
Colin Patrick Mccabe
dd1b8f2ed8 HDFS-7176. The namenode usage message doesn't include "-rollingupgrade started" (cmccabe) 2014-10-01 14:53:44 -07:00
Haohui Mai
1f5b42ac48 HDFS-7158. Reduce the memory usage of WebImageViewer. Contributed by Haohui Mai. 2014-10-01 10:53:38 -07:00
arp
e431d054cb Merge branch 'trunk' into HDFS-6581 2014-10-01 08:52:21 -07:00
Kihwal Lee
3f25d916d5 HDFS-6754. TestNamenodeCapacityReport.testXceiverCount may sometimes fail due to lack of retry. Contributed by Mit Desai. 2014-10-01 09:53:01 -05:00
arp
a45ad330fa HDFS-7171. Fix Jenkins failures in HDFS-6581 branch. (Arpit Agarwal) 2014-09-30 18:25:20 -07:00
arp
65ef35bbab Merge branch 'trunk' into HDFS-6581 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/resources/editsStored.xml
 2014-09-30 18:20:13 -07:00
arp
9e9e9cf711 HDFS-7153. Add storagePolicy to NN edit log during file creation. (Arpit Agarwal) 2014-09-30 18:10:13 -07:00
arp
dde2ed13c2 Merge branch 'trunk' into HDFS-6581 2014-09-30 18:05:54 -07:00
Jing Zhao
cdf1af0e5a HDFS-7167. NPE while running Mover if the given path is for a file. Contributed by Jing Zhao. 2014-09-30 11:13:35 -07:00
arp
5e8b697352 HDFS-7129. Metrics to track usage of memory for writes. (Contributed by Xiaoyu Yao) 2014-09-30 00:53:18 -07:00