big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
11,040 Commits 2 Branches 2 Tags 264 MiB
eb612b0b70
Commit Graph

3808 Commits

Author SHA1 Message Date
Zhe Zhang
9af1f4779b Addendum fix for HDFS-7749 to be compatible with HDFS-7993 2015-05-26 11:07:16 -07:00
Jing Zhao
9f2f583f40 HDFS-7749. Erasure Coding: Add striped block support in INodeFile. Contributed by Jing Zhao. 2015-05-26 11:07:15 -07:00
Jing Zhao
ba93714920 HDFS-7716. Erasure Coding: extend BlockInfo to handle EC info. Contributed by Jing Zhao. 2015-05-26 11:07:11 -07:00
Jing Zhao
ec48013426 Fix Compilation Error in TestAddBlockgroup.java after the merge 2015-05-26 11:03:34 -07:00
Zhe Zhang
0c1da5a030 HDFS-7652. Process block reports for erasure coded blocks. Contributed by Zhe Zhang 2015-05-26 11:03:34 -07:00
Zhe Zhang
bc2833b1c9 HDFS-7339. Allocating and persisting block groups in NameNode. Contributed by Zhe Zhang 2015-05-26 11:03:34 -07:00
Vinayakumar B
f166e67a23 HDFS-7347. Configurable erasure coding policy for individual files and directories ( Contributed by Zhe Zhang ) 2015-05-26 11:03:34 -07:00
Haohui Mai
ada233b7cd HDFS-8377. Support HTTP/2 in datanode. Contributed by Duo Zhang. 2015-05-24 22:30:32 -07:00
Vinayakumar B
0c6638c2ea HDFS-8268. Port conflict log for data node server is not sufficient (Contributed by Mohammad Shahid Khan) 2015-05-22 16:15:15 +05:30
Arpit Agarwal
cf2b5694d6 HDFS-8454. Remove unnecessary throttling in TestDatanodeDeath. (Arpit Agarwal) 2015-05-21 20:43:57 -07:00
Xiaoyu Yao
05e04f34f2 HDFS-8451. DFSClient probe for encryption testing interprets empty URI property for enabled. Contributed by Steve Loughran. 2015-05-21 11:58:00 -07:00
Haohui Mai
2b6bcfdafa HDFS-8421. Move startFile() and related functions into FSDirWriteFileOp. Contributed by Haohui Mai. 2015-05-21 08:08:28 -07:00
Akira Ajisaka
0e4f1081c7 HADOOP-10366. Add whitespaces between classes for values in core-default.xml to fit better in browser. Contributed by kanaka kumar avvaru. 2015-05-21 17:52:03 +09:00
Akira Ajisaka
dc8434ab2b HDFS-4383. Document the lease limits. Contributed by Arshad Mohammad. 2015-05-21 17:30:43 +09:00
Kihwal Lee
8860e352c3 HDFS-8404. Pending block replication can get stuck using older genstamp. Contributed by Nathan Roberts. 2015-05-19 13:05:15 -05:00
Colin Patrick Mccabe
470c87dbc6 HADOOP-11970. Replace uses of ThreadLocal<Random> with JDK7 ThreadLocalRandom (Sean Busbey via Colin P. McCabe) 2015-05-19 10:50:15 -07:00
Kihwal Lee
de30d66b26 HDFS-8131. Implement a space balanced block placement policy. Contributed by Liu Shaohui. 2015-05-19 08:04:38 -05:00
Vinayakumar B
d4a2830b63 HADOOP-11103. Clean up RemoteException (Contributed by Sean Busbey) 2015-05-19 14:41:05 +05:30
Vinayakumar B
93972a332a HDFS-6348. SecondaryNameNode not terminating properly on runtime exceptions (Contributed by Rakesh R) 2015-05-19 12:24:25 +05:30
Ravi Prakash
cdfae446ad HDFS-4185. Add a metric for number of active leases (Rakesh R via raviprak) 2015-05-18 12:37:21 -07:00
Tsz-Wo Nicholas Sze
0c590e1c09 HDFS-8405. Fix a typo in NamenodeFsck. Contributed by Takanobu Asanuma 2015-05-19 02:57:54 +08:00
Arpit Agarwal
a2190bf15d HDFS-8345. Storage policy APIs must be exposed via the FileSystem interface. (Arpit Agarwal) 2015-05-18 11:36:29 -07:00
Arpit Agarwal
e453989a57 HDFS-8157. Writes to RAM DISK reserve locked memory for block files. (Arpit Agarwal) 2015-05-16 09:05:35 -07:00
Haohui Mai
ac742c762d HDFS-8403. Eliminate retries in TestFileCreation#testOverwriteOpenForWrite. Contributed by Arpit Agarwal. 2015-05-15 19:12:40 -07:00
Haohui Mai
e5afac5896 HDFS-8394. Move getAdditionalBlock() and related functionalities into a separate class. Contributed by Haohui Mai. 2015-05-15 19:09:59 -07:00
Jing Zhao
8f37873342 HDFS-8397. Refactor the error handling code in DataStreamer. Contributed by Tsz Wo Nicholas Sze. 2015-05-15 16:14:54 -07:00
Vinayakumar B
3bef7c80a9 HDFS-6888. Allow selectively audit logging ops (Contributed by Chen He) 2015-05-15 11:07:51 +05:30
Akira Ajisaka
ee7beda6e3 HDFS-8350. Remove old webhdfs.xml and other outdated documentation stuff. Contributed by Brahma Reddy Battula. 2015-05-15 13:54:35 +09:00
Akira Ajisaka
cbc01ed08e HDFS-8371. Fix test failure in TestHdfsConfigFields for spanreceiver properties. Contributed by Ray Chiang. 2015-05-15 12:14:47 +09:00
Vinayakumar B
def9136e02 HDFS-8150. Make getFileChecksum fail for blocks under construction (Contributed by J.Andreina) 2015-05-14 15:54:51 +05:30
Haohui Mai
b2c85db86c HDFS-7728. Avoid updating quota usage while loading edits. Contributed by Jing Zhao. 2015-05-13 21:50:35 -07:00
Arpit Agarwal
54fa9b4217 HDFS-8243. Files written by TestHostsFiles and TestNameNodeMXBean are causing Release Audit Warnings. (Contributed by Ruth Wisniewski) 2015-05-13 19:43:53 -07:00
Colin Patrick Mccabe
281d47a969 HDFS-8380. Always call addStoredBlock on blocks which have been shifted from one storage to another (cmccabe) 2015-05-13 14:29:05 -07:00
Kihwal Lee
f9a46a00d2 HDFS-8358. TestTraceAdmin fails. Contributed by Masatake Iwasaki. 2015-05-13 14:15:27 -05:00
Tsz-Wo Nicholas Sze
cdec12d1b8 HDFS-8143. Mover should exit after some retry when failed to move blocks. Contributed by surendra singh lilhore 2015-05-13 11:57:49 -07:00
Vinayakumar B
065d8f2a34 HDFS-6300. Prevent multiple balancers from running simultaneously (Contributed by Rakesh R) 2015-05-13 17:27:34 +05:30
Akira Ajisaka
2463666ecb HDFS-6184. Capture NN's thread dump when it fails over. Contributed by Ming Ma. 2015-05-13 11:37:22 +09:00
yliu
6d5da94841 HDFS-8255. Rename getBlockReplication to getPreferredBlockReplication. (Contributed by Zhe Zhang) 2015-05-12 21:29:22 +08:00
Vinayakumar B
987abc99b0 HDFS-8362. Java Compilation Error in TestHdfsConfigFields.java (Contributed by Arshad Mohammad) 2015-05-12 12:09:13 +05:30
Kihwal Lee
ea11590aad HDFS-7916. 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for infinite loop. Contributed by Rushabh Shah. 2015-05-11 14:31:58 -05:00
Akira Ajisaka
1dd79ffaca HDFS-8241. Remove unused NameNode startup option -finalize. Contributed by Brahma Reddy Battula. 2015-05-12 00:18:18 +09:00
Junping Du
7e543c27fa YARN-3587. Fix the javadoc of DelegationTokenSecretManager in yarn, etc. projects. Contributed by Gabor Liptak. 2015-05-11 05:27:07 -07:00
Akira Ajisaka
3fa2efc09f HDFS-8351. Remove namenode -finalize option from document. (aajisaka) 2015-05-11 15:34:44 +09:00
Haohui Mai
4536399d47 HDFS-8357. Consolidate parameters of INode.CleanSubtree() into a parameter objects. Contributed by Li Lu. 2015-05-09 22:51:58 -07:00
Haohui Mai
02a4a22b9c HDFS-8327. Compute storage type quotas in INodeFile.computeQuotaDeltaForTruncate(). Contributed by Haohui Mai. 2015-05-08 23:09:25 -07:00
Haohui Mai
00fe1ed3a4 HDFS-6757. Simplify lease manager with INodeID. Contributed by Haohui Mai. 2015-05-08 23:04:31 -07:00
Arpit Agarwal
59995cec4a HDFS-8097. TestFileTruncate is failing intermittently. (Contributed by Rakesh R) 2015-05-08 16:28:24 -07:00
Allen Wittenauer
c945c20483 HADOOP-11590. Update sbin commands and documentation to use new --slaves option (aw) 2015-05-08 15:27:25 -07:00
Xiaoyu Yao
d0e75e60fb HDFS-8326. Documentation about when checkpoints are run is out of date. (Misty Stanley-Jones via xyao) 2015-05-08 14:46:37 -07:00
Kihwal Lee
2d4ae3d18b HDFS-8245. Standby namenode doesn't process DELETED_BLOCK if the addblock request is in edit log. Contributed by Rushabh S Shah. 2015-05-08 16:36:41 -05:00
Colin Patrick Mccabe
f523e963e4 HDFS-8113. Add check for null BlockCollection pointers in BlockInfoContiguous structures (Chengbing Liu via Colin P. McCabe) 2015-05-08 13:50:51 -07:00
Kihwal Lee
7a7960be41 HDFS-7433. Optimize performance of DatanodeManager's node map. Contributed by Daryn Sharp. 2015-05-08 15:18:29 -05:00
Colin Patrick Mccabe
8f7c2364d7 HDFS-8284. Update documentation about how to use HTrace with HDFS (Masatake Iwasaki via Colin P. McCabe) 2015-05-08 12:30:03 -07:00
Yongjun Zhang
730f9930a4 HDFS-8311. DataStreamer.transfer() should timeout the socket InputStream. (Esteban Gutierrez via Yongjun Zhang) 2015-05-08 00:11:46 -07:00
cnauroth
26f61d41df HDFS-5640. Add snapshot methods to FileContext. Contributed by Rakesh R. 2015-05-08 11:20:55 -07:00
Arun Suresh
3cefc02af7 HDFS-7559. Create unit test to automatically compare HDFS related classes and hdfs-default.xml. (Ray Chiang via asuresh) 2015-05-08 10:29:38 -07:00
Arpit Agarwal
a2d40bced9 HDFS-8340. Fix NFS documentation of nfs.wtmax. (Contributed by Ajith S) 2015-05-08 10:15:26 -07:00
Haohui Mai
f4ebbc6afc HDFS-8346. libwebhdfs build fails during link due to unresolved external symbols. Contributed by Chris Nauroth. 2015-05-08 09:58:17 -07:00
Kihwal Lee
6f622672b6 HDFS-7894. Rolling upgrade readiness is not updated in jmx until query command is issued. Contributed by Brahma Reddy Battula. 2015-05-08 09:30:38 -05:00
Uma Maheswara Rao G
7b1ea9c481 HDFS-6285. tidy an error log inside BlockReceiver. Contributed by Liang Xie. 2015-05-08 17:22:52 +05:30
Vinayakumar B
c648317a68 HDFS-3384. DataStreamer thread should be closed immediatly when failed to setup a PipelineForAppendOrRecovery (Contributed by Uma Maheswara Rao G) 2015-05-08 17:18:14 +05:30
Vinayakumar B
e01c1a979c HDFS-8226. Non-HA rollback compatibility broken (Contributed by J.Andreina) 2015-05-08 15:24:36 +05:30
Vinayakumar B
4c6816faf8 HDFS-8209. Support different number of datanode directories in MiniDFSCluster. (Contributed by surendra singh lilhore) 2015-05-08 15:03:44 +05:30
Tsuyoshi Ozawa
888e630990 HDFS-8207. Improper log message when blockreport interval compared with initial delay. Contributed by Brahma Reddy Battula and Ashish Singhi. 2015-05-08 18:13:09 +09:00
Vinayakumar B
6232235df3 HDFS-8175. Provide information on snapshotDiff for supporting the comparison between snapshot and current status (Contributed by J.Andreina) 2015-05-08 14:37:50 +05:30
Vinayakumar B
e642eff0e0 HDFS-8187. Remove usage of '-setStoragePolicy' and '-getStoragePolicy' using dfsadmin cmd (as it is not been supported) (Contributed by J.Andreina) 2015-05-08 14:16:30 +05:30
Uma Maheswara Rao G
4b00ae5c11 HDFS-8108. Fsck should provide the info on mandatory option to be used along with -blocks ,-locations and -racks. Contributed by J.Andreina. 2015-05-08 14:00:45 +05:30
Akira Ajisaka
fd605997ee HDFS-8222. Remove usage of "dfsadmin -upgradeProgress" from document which is no longer supported. Contributed by J.Andreina. 2015-05-08 16:27:40 +09:00
Vinayakumar B
26da2ddf38 HDFS-7998. HDFS Federation : Command mentioned to add a NN to existing federated cluster is wrong (Contributed by Ajith S) 2015-05-08 12:33:49 +05:30
Akira Ajisaka
c5afe444df HDFS-8349. Remove .xml and documentation references to dfs.webhdfs.enabled. Contributed by Ray Chiang. 2015-05-08 15:57:13 +09:00
Uma Maheswara Rao G
e16f4b7f70 HDFS-8332. DFS client API calls should check filesystem closed. Contributed by Rakesh R. 2015-05-08 12:26:47 +05:30
Vinayakumar B
e5e492a963 HDFS-6291. FSImage may be left unclosed in BootstrapStandby#doRun() ( Contributed by Sanghyun Yun) 2015-05-08 11:25:24 +05:30
Uma Maheswara Rao G
2ea0f2fc93 HDFS-8174. Update replication count to live rep count in fsck report. Contributed by J.Andreina 2015-05-08 11:01:51 +05:30
Vinayakumar B
66988476d0 HDFS-8067. haadmin prints out stale help messages (Contributed by Ajith S) 2015-05-08 10:45:21 +05:30
Vinayakumar B
c7c26a1e4a HDFS-8257. Namenode rollingUpgrade option is incorrect in document (Contributed by J.Andreina) 2015-05-08 10:35:25 +05:30
Haohui Mai
4d9f9e546f HDFS-8037. CheckAccess in WebHDFS silently accepts malformed FsActions parameters. Contributed by Walter Su. 2015-05-07 15:25:26 -07:00
Haohui Mai
767b91cd83 HDFS-8321. CacheDirectives and CachePool operations should throw RetriableException in safemode. Contributed by Haohui Mai. 2015-05-07 15:17:11 -07:00
Tsz-Wo Nicholas Sze
f9427f1760 HDFS-7980. Incremental BlockReport will dramatically slow down namenode startup. Contributed by Walter Su 2015-05-07 11:36:35 -07:00
Akira Ajisaka
449e4426a5 HDFS-8325. Misspelling of threshold in log4j.properties for tests. Contributed by Brahma Reddy Battula. 2015-05-07 15:14:20 +09:00
cnauroth
6633a8474d HDFS-7833. DataNode reconfiguration does not recalculate valid volumes required, based on configured failed volumes tolerated. Contributed by Lei (Eddy) Xu. 2015-05-06 21:11:30 -07:00
Konstantin V Shvachko
c75cfa29cf HDFS-2484. checkLease should throw FileNotFoundException when file does not exist. Contributed by Rakesh R. 2015-05-06 16:06:04 -07:00
Haohui Mai
4da8490b51 HDFS-8314. Move HdfsServerConstants#IO_FILE_BUFFER_SIZE and SMALL_BUFFER_SIZE to the users. Contributed by Li Lu. 2015-05-05 15:41:22 -07:00
Xiaoyu Yao
0100b15501 HDFS-8219. setStoragePolicy with folder behavior is different after cluster restart. (surendra singh lilhore via Xiaoyu Yao) 2015-05-05 13:41:14 -07:00
Colin Patrick Mccabe
ffce9a3413 HDFS-7847. Modify NNThroughputBenchmark to be able to operate on a remote NameNode (Charles Lamb via Colin P. McCabe) 2015-05-05 11:34:58 -07:00
Colin Patrick Mccabe
24d3a2d4fd HDFS-7758. Retire FsDatasetSpi#getVolumes() and use FsDatasetSpi#getVolumeRefs() instead (Lei (Eddy) Xu via Colin P. McCabe) 2015-05-05 11:08:59 -07:00
Colin Patrick Mccabe
fcd4cb7516 HDFS-8305: HDFS INotify: the destination field of RenameOp should always end with the file name (cmccabe) 2015-05-05 10:50:09 -07:00
Vinayakumar B
318081ccd7 HDFS-7916. 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for infinite loop (Contributed by Vinayakumar B) 
Reverted earlier commit
 2015-05-05 11:05:37 +05:30
Haohui Mai
0d6aa5d609 HDFS-8237. Move all protocol classes used by ClientProtocol to hdfs-client. Contributed by Haohui Mai. 2015-05-04 15:00:29 -07:00
Colin Patrick Mccabe
3fe79e1db8 HDFS-7397. Add more detail to the documentation for the conf key "dfs.client.read.shortcircuit.streams.cache.size" (Brahma Reddy Battula via Colin P. McCabe) 2015-05-04 12:50:29 -07:00
cnauroth
c4578760b6 HDFS-8290. WebHDFS calls before namesystem initialization can cause NullPointerException. Contributed by Chris Nauroth. 2015-05-04 11:35:04 -07:00
Xiaoyu Yao
a319771d1d HDFS-8309. Skip unit test using DataNodeTestUtils#injectDataDirFailure() on Windows. (xyao) 2015-05-02 22:15:24 -07:00
Haohui Mai
6ae2a0d048 HDFS-8249. Separate HdfsConstants into the client and the server side class. Contributed by Haohui Mai. 2015-05-02 10:03:23 -07:00
Arpit Agarwal
6f541edce0 HDFS-8229. LAZY_PERSIST file gets deleted after NameNode restart. (Contributed by Surendra Singh Lilhore) 2015-05-01 16:42:19 -07:00
Tsz-Wo Nicholas Sze
d3d019c337 HDFS-8086. Move LeaseRenewer to the hdfs.client.impl package. Contributed by Takanobu 2015-05-01 15:12:18 -07:00
Arpit Agarwal
64d30a6186 HDFS-8276. LazyPersistFileScrubber should be disabled if scrubber interval configured zero. (Contributed by Surendra Singh Lilhore) 2015-05-01 14:43:09 -07:00
Colin Patrick Mccabe
b82567d455 HDFS-8213. DFSClient should use hdfs.client.htrace HTrace configuration prefix rather than hadoop.htrace (cmccabe) 2015-05-01 11:19:40 -07:00
Yongjun Zhang
279958b772 HDFS-7281. Missing block is marked as corrupted block (Ming Ma via Yongjun Zhang) 2015-05-01 08:42:00 -07:00
Jing Zhao
98a6176628 HDFS-8300. Fix unit test failures and findbugs warning caused by HDFS-8283. Contributed by Jing Zhao. 2015-04-30 19:27:07 -07:00
Andrew Wang
87e9978235 HDFS-8292. Move conditional in fmt_time from dfs-dust.js to status.html. Contributed by Charles Lamb. 2015-04-30 18:23:13 -07:00
Haohui Mai
c55d609053 HDFS-8200. Refactor FSDirStatAndListingOp. Contributed by Haohui Mai. 2015-04-30 13:41:46 -07:00