big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
12,152 Commits 2 Branches 2 Tags 264 MiB
960201b79b
Commit Graph

3362 Commits

Author SHA1 Message Date
Zhe Zhang
6c17d31528 Merge remote-tracking branch 'apache/trunk' into HDFS-7285 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java

Change-Id: I53ec1c426dc988d6c4a2c87b00caef49c4057010
 2015-09-30 08:39:13 -07:00
yliu
06abc57a90 HADOOP-12448. TestTextCommand: use mkdirs rather than mkdir to create test directory. (Contributed by Colin Patrick McCabe and Chris Nauroth) 2015-09-30 10:56:22 +08:00
Haohui Mai
8703301b46 HDFS-9174. Fix findbugs warnings in FSOutputSummer.tracer and DirectoryScanner$ReportCompiler.currentThread. Contributed by Yi Liu. 2015-09-29 14:56:06 -07:00
Colin Patrick Mccabe
a2efb945e3 Add HADOOP-12447 to CHANGES.txt 2015-09-29 09:37:13 -07:00
Colin Patrick Mccabe
850d679acb HADOOP-12447. Clean up some htrace integration issues (cmccabe) 2015-09-29 09:25:11 -07:00
Zhe Zhang
8fd5520246 Merge remote-tracking branch 'apache/trunk' into HDFS-7285 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/hdfs.proto
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSUtil.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelper.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirStatAndListingOp.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/proto/DatanodeProtocol.proto
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockTokenWithDFS.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java

Change-Id: Ic7946c4ea35bed587fe879ce58b959b25ecc0823
 2015-09-29 01:39:16 -07:00
yliu
d6fa34e014 HDFS-8859. Improve DataNode ReplicaMap memory footprint to save about 45%. (yliu) 2015-09-29 16:20:35 +08:00
Ming Ma
5c3b663bf9 HADOOP-12440. TestRPC#testRPCServerShutdown did not produce the desired thread states before shutting down. (Xiao Chen via mingma) 2015-09-28 18:12:51 -07:00
Kihwal Lee
3abbdc929b HADOOP-12446. Undeprecate createNonRecursive(). Contributed by Ted Yu. 2015-09-28 17:31:51 -05:00
Colin Patrick Mccabe
892ade689f HDFS-9080. Update htrace version to 4.0.1 (cmccabe) 2015-09-28 07:42:12 -07:00
Vinayakumar B
f0f984e4e6 HADOOP-11984. Enable parallel JUnit tests in pre-commit (Contributed by Chris Nauroth) 2015-09-27 14:12:07 +05:30
Vinayakumar B
861b52db24 HADOOP-12442. Display help if the command option to 'hdfs dfs' is not valid (Contributed by nijel) 2015-09-26 21:36:17 +05:30
cnauroth
7fe521b1dd HADOOP-11918. Listing an empty s3a root directory throws FileNotFound. Contributed by Lei (Eddy) Xu. 2015-09-25 22:33:53 -07:00
Yongjun Zhang
3187dbd7a8 HDFS-9123. Copying from the root to a subdirectory should be forbidden. (Wei-Chiu Chuang via Yongjun Zhang) 2015-09-24 21:36:38 -07:00
Zhihai Xu
9cc9f8126b Update CHANGES.txt to reflect commit of HADOOP-8436 to 2.8.0 2015-09-24 12:52:07 -07:00
Zhihai Xu
52c1f272ec HADOOP-12252. LocalDirAllocator should not throw NPE with empty string configuration. Contributed by Zhihai Xu 2015-09-24 11:51:22 -07:00
Arpit Agarwal
df31c446bf HADOOP-12437. Allow SecurityUtil to lookup alternate hostnames. (Contributed by Arpit Agarwal) 2015-09-24 11:41:48 -07:00
Haohui Mai
0ef7ff47d5 HADOOP-12438. Reset RawLocalFileSystem.useDeprecatedFileStatus in TestLocalFileSystem. Contributed by Chris Nauroth. 2015-09-23 19:36:44 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
1f707ecffd Release process for 2.6.1: Set the release date for 2.6.1 2015-09-23 15:10:50 -07:00
Vinayakumar B
a2c76e5f26 HDFS-9013. Deprecate NameNodeMXBean#getNNStarted in branch2 and remove from trunk (Contributed by Surendra Singh Lilhore) 2015-09-23 15:28:41 +05:30
Zhe Zhang
1080c37300 Merge remote-tracking branch 'apache/trunk' into HDFS-7285 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/Namesystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java

Change-Id: I8511c4d64b0959e79129febc179845a3892fedcc
 2015-09-22 13:27:09 -07:00
Tsuyoshi Ozawa
dfd807afab HADOOP-12428. Fix inconsistency between log-level guards and statements. Contributed by Jagadesh Kiran N and Jackie Chang. 2015-09-22 12:54:29 +09:00
Steve Loughran
c39ddc306d HADOOP-12360. Create StatsD metrics2 sink. (Dave Marion via stevel) 2015-09-19 12:41:42 +01:00
Zhihai Xu
88d89267ff HADOOP-12404. Disable caching for JarURLConnection to avoid sharing JarFile with other users when loading resource from URL in Configuration class. Contributed by Zhihai Xu 2015-09-18 10:40:04 -07:00
Zhihai Xu
3732d5238a Revert "Disable caching for JarURLConnection to avoid sharing JarFile with other users when loading resource from URL in Configuration class. Contributed by Zhihai Xu" 
This reverts commit e690a32e57.
 2015-09-18 10:30:40 -07:00
Zhihai Xu
e690a32e57 Disable caching for JarURLConnection to avoid sharing JarFile with other users when loading resource from URL in Configuration class. Contributed by Zhihai Xu 2015-09-18 08:24:53 -07:00
Wangda Tan
6b97fa6652 HADOOP-12386. RetryPolicies.RETRY_FOREVER should be able to specify a retry interval. (Sunil G via wangda) 2015-09-17 17:46:28 -07:00
Jason Lowe
58d1a02b8d MAPREDUCE-6481. LineRecordReader may give incomplete record and wrong position/key information for uncompressed input sometimes. Contributed by Zhihai Xu 2015-09-17 14:30:18 +00:00
Haohui Mai
0832b38d48 HADOOP-12417. TestWebDelegationToken failing with port in use. Contributed by Mingliang Liu. 2015-09-16 18:38:37 -07:00
Eric Yang
2ffe2db95e HADOOP-12374. Updated expunge command description. 
(WeiWei Yang via eyang)
 2015-09-15 20:03:24 -07:00
Akira Ajisaka
77aaf4cf7d HADOOP-12416. Trash messages should be handled by Logger instead of being delivered on System.out. Contributed by Mingliang Liu. 2015-09-16 10:20:24 +09:00
Vinayakumar B
ce69c9b54c HDFS-8953. DataNode Metrics logging (Contributed by Kanaka Kumar Avvaru) 2015-09-16 00:18:29 +05:30
cnauroth
b2017d9b03 HADOOP-12413. AccessControlList should avoid calling getGroupNames in isUserInList with empty groups. Contributed by Zhihai Xu. 2015-09-15 10:41:50 -07:00
Vinayakumar B
a440567491 HDFS-9067. o.a.h.hdfs.server.datanode.fsdataset.impl.TestLazyWriter is failing in trunk (Contributed by Surendra Singh Lilhore) 2015-09-15 17:30:46 +05:30
Steve Loughran
7269906254 HADOOP-12087. [JDK8] Fix javadoc errors caused by incorrect or illegal tags. (Akira AJISAKA via stevel). 2015-09-13 14:25:26 +01:00
Steve Loughran
3f685cd571 HADOOP-12407. Test failing: hadoop.ipc.TestSaslRPC. (stevel) 2015-09-12 18:56:42 +01:00
Steve Loughran
4992f075c8 syncing branch-2 and trunk CHANGES.TXT to be closer together 2015-09-12 18:51:37 +01:00
Steve Loughran
8c054412e1 fix trunk/hadoop-common CHANGES.TXT to be the reference across trunk & branch-2 2015-09-12 18:13:54 +01:00
Robert Kanter
9538af0e1a HADOOP-12348. MetricsSystemImpl creates MetricsSourceAdapter with wrong time unit parameter. (zxu via rkanter) 2015-09-11 15:20:17 -07:00
Steve Loughran
ca0827a862 HADOOP-12324. Better exception reporting in SaslPlainServer. (Mike Yoder via stevel) 2015-09-11 15:58:53 +01:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
fbbb7ff1ed Updating all CHANGES.txt files to move entires from future releases into 2.6.1 
section given the large number of backports to 2.6.1.
 2015-09-10 13:45:57 -07:00
Vinayakumar B
94cf7ab9d2 HDFS-8929. Add a metric to expose the timestamp of the last journal (Contributed by surendra singh lilhore) 2015-09-09 11:03:57 +05:30
Zhihai Xu
d9c1fab2ec HADOOP-12388. Fix components' version information in the web page About the Cluster. Contributed by Jun Gong. 2015-09-08 13:06:36 -07:00
Vinayakumar B
090d26652c HADOOP-12384. Add '-direct' flag option for fs copy so that user can choose not to create '._COPYING_' file (Contributed by J.Andreina) 2015-09-08 19:11:18 +05:30
Andrew Wang
e1feaf6db0 HADOOP-12358. Add -safely flag to rm to prompt when deleting many files. Contributed by Xiaoyu Yao. 2015-09-04 13:42:55 -07:00
Arpit Agarwal
524ba8708b HDFS-9009. Send metrics logs to NullAppender by default. (Arpit Agarwal) 2015-09-03 11:14:47 -07:00
Tsuyoshi Ozawa
0ebc658105 HADOOP-12213. Interrupted exception can occur when Client#stop is called. Contributed by Kuhu Shukla. 2015-09-03 23:32:42 +09:00
Tsuyoshi Ozawa
355eaaa33d HADOOP-10318. Incorrect reference to nodeFile in RumenToSLSConverter error message. Contributed by Wei Yan. 2015-09-03 22:48:53 +09:00
Jian He
09c64ba1ba YARN-4101. RM should print alert messages if Zookeeper and Resourcemanager gets connection issue. Contributed by Xuan Gong 2015-09-02 17:45:23 -07:00
Akira Ajisaka
65ccf2b125 HDFS-8388. Time and Date format need to be in sync in NameNode UI page. Contributed by Surendra Singh Lilhore. 2015-09-02 14:28:38 +09:00
Zhe Zhang
53358fe680 Merge remote-tracking branch 'apache/trunk' into HDFS-7285 2015-09-01 14:48:37 -07:00
Zhe Zhang
ab56fcdb12 Merge remote-tracking branch 'apache/trunk' into HDFS-7285 2015-09-01 14:30:25 -07:00
Andrew Wang
0eb9b19325 HADOOP-12369. Point hadoop-project/pom.xml java.security.krb5.conf within target folder. 2015-09-01 10:57:32 -07:00
Tsuyoshi Ozawa
dd149adeac HADOOP-10365. BufferedOutputStream in FileUtil#unpackEntries() should be closed in finally block. Contributed by Kiran Kumar M R and Sanghyun Yun. 2015-09-02 02:01:51 +09:00
Akira Ajisaka
2e251a7674 HADOOP-12359. hadoop fs -getmerge doc is wrong. Contributed by Jagadesh Kiran N. 2015-09-01 20:55:33 +09:00
yliu
f4d96be6c6 HADOOP-12367. Move TestFileUtil's test resources to resources folder. (wang via yliu) 2015-09-01 16:20:56 +08:00
Andrew Wang
7ad3556ed3 HADOOP-12368. Mark ViewFileSystemBaseTest and ViewFsBaseTest as abstract. 2015-08-31 18:17:14 -07:00
Lei Xu
6ab2d19f5c HADOOP-12346. Increase some default timeouts / retries for S3a connector. (Sean Mackrory via Lei (Eddy) Xu) 2015-08-29 09:59:30 -07:00
Arpit Agarwal
bdbe53c676 HDFS-8983. NameNode support for protected directories. (Contributed by Arpit Agarwal) 2015-08-29 09:52:37 -07:00
Andrew Wang
90fe7bcc3b HADOOP-12362. Set hadoop.tmp.dir and hadoop.log.dir in pom. Contributed by Charlie Helin. 2015-08-27 10:17:00 -07:00
Andrew Wang
50fa383704 Revert "MAPREDUCE-6455. Unable to use surefire > 2.18. (Charlie Helin via kasha)" 
This reverts commit 61bf9cae6f.
 2015-08-27 10:15:19 -07:00
Walter Su
067ec8c2b1 HDFS-8838. Erasure Coding: Tolerate datanode failures in DFSStripedOutputStream when the data length is small. Contributed by Tsz Wo Nicholas Sze. 2015-08-27 09:11:23 +08:00
Jing Zhao
f44b599003 HDFS-8896. DataNode object isn't GCed when shutdown, because it has GC root in ShutdownHookManager. Contributed by Walter Su. 2015-08-26 16:27:34 -07:00
Harsh J
af78767870 HADOOP-12352. Delay in checkpointing Trash can leave trash for 2 intervals before deleting. Contributed by Casey Brotherton. 2015-08-25 11:21:19 +05:30
Xiaoyu Yao
48774d0a45 HADOOP-12325. RPC Metrics : Add the ability track and log slow RPCs. Contributed by Anu Engineer 2015-08-24 14:31:24 -07:00
Zhe Zhang
6b6a63bbbd Merge commit '456e901a4c5c639267ee87b8e5f1319f256d20c2' (HDFS-6407. Add sorting and pagination in the datanode tab of the NN Web UI. Contributed by Haohui Mai.) into HDFS-7285-merge 2015-08-24 12:59:26 -07:00
Xuan
37e1c3d82a YARN-221. NM should provide a way for AM to tell it not to aggregate 
logs. Contributed by Ming Ma
 2015-08-22 16:25:24 -07:00
Karthik Kambatla
61bf9cae6f MAPREDUCE-6455. Unable to use surefire > 2.18. (Charlie Helin via kasha) 2015-08-22 00:39:04 -07:00
Xiaoyu Yao
caa636bf10 HADOOP-12347. Fix mismatch parameter name in javadocs of AuthToken#setMaxInactives. Contributed by Xiaoyu Yao 2015-08-21 16:32:57 -07:00
Vinod Kumar Vavilapalli
33474939a0 Creating 2.6.2 entries in CHANGES.txt files. 2015-08-20 17:21:06 -07:00
Akira Ajisaka
36b1a1e784 HADOOP-12061. Incorrect command in single cluster setup document. Contributed by Kengo Seki. 2015-08-20 11:09:45 +09:00
Robert Kanter
1e06299df8 HADOOP-12317. Applications fail on NM restart on some linux distro because NM container recovery declares AM container as LOST (adhoot via rkanter) 2015-08-19 19:00:51 -07:00
Arpit Agarwal
9c3571ea60 HDFS-8911. NameNode Metric : Add Editlog counters as a JMX metric. (Contributed by Anu Engineer) 2015-08-19 09:42:35 -07:00
Tsz-Wo Nicholas Sze
7ecbfd44aa HDFS-8826. In Balancer, add an option to specify the source node list so that balancer only selects blocks to move from those nodes. 2015-08-18 19:25:50 -07:00
Chris Douglas
30e342a5d3 HDFS-8435. Support CreateFlag in WebHDFS. Contributed by Jakob Homan 2015-08-18 17:32:48 -07:00
Benoy Antony
71aedfabf3 hadoop-12050. Enable MaxInactiveInterval for hadoop http auth token. Contributed by Huizhi Lu. 2015-08-18 13:43:34 -07:00
Arpit Agarwal
a88f31ebf3 HDFS-8880. NameNode metrics logging. (Arpit Agarwal) 2015-08-17 16:55:46 -07:00
Xiaoyu Yao
a7862d5fe4 HDFS-8883. NameNode Metrics : Add FSNameSystem lock Queue Length. Contributed by Anu Engineer. 2015-08-17 10:15:56 -07:00
Vinayakumar B
fc508b41a9 HADOOP-7139. Allow appending to existing SequenceFiles (Contributed by kanaka kumar avvaru) 
Moved to 2.6.1
 2015-08-14 15:55:29 +05:30
Vinayakumar B
05ed69058f HADOOP-11333. Fix deadlock in DomainSocketWatcher when the notification pipe is full (zhaoyunjiong via cmccabe) 
Moved to 2.6.1
 2015-08-14 12:53:46 +05:30
Vinayakumar B
e7aa81394d HADOOP-10786. Fix UGI#reloginFromKeytab on Java 8. Contributed by Stephen Chu. 
Moved CHANGES.txt entry to 2.6.1
 2015-08-14 12:23:51 +05:30
Vinayakumar B
24a11e3996 HADOOP-10786. Fix UGI#reloginFromKeytab on Java 8. Contributed by Stephen Chu. 
Moved CHANGES.txt entry to 2.6.1

(cherry picked from commit d6050f06a3b7e049541b1cb4597c388abf00a5be)
 2015-08-14 12:09:56 +05:30
Arpit Agarwal
6b1cefc561 HADOOP-12322. Typos in rpcmetrics.java. (Contributed by Anu Engineer) 2015-08-13 21:17:30 -07:00
Allen Wittenauer
b73181f187 HADOOP-12244. recover broken rebase during precommit (aw) 2015-08-13 12:29:19 -07:00
Zhe Zhang
1d37a88121 HDFS-8854. Erasure coding: add ECPolicy to replace schema+cellSize in hadoop-hdfs. Contributed by Walter Su. 2015-08-13 10:04:45 -07:00
yliu
53bef9c5b9 HADOOP-12295. Improve NetworkTopology#InnerNode#remove logic. (yliu) 2015-08-13 16:45:20 +08:00
cnauroth
6cc8e38db5 HADOOP-12258. Need translate java.nio.file.NoSuchFileException to FileNotFoundException to avoid regression. Contributed by Zhihai Xu. 2015-08-12 16:44:53 -07:00
Aaron T. Myers
820f864a26 HADOOP-12318. Expose underlying LDAP exceptions in SaslPlainServer. Contributed by Mike Yoder. 2015-08-12 15:24:16 -07:00
Andrew Wang
1ea1a8334e HDFS-8887. Expose storage type and storage ID in BlockLocation. 2015-08-11 23:25:33 -07:00
Xuan
3ae716fa69 YARN-3999. RM hangs on draing events. Contributed by Jian He 2015-08-11 18:25:11 -07:00
Jian He
f59612edd7 HADOOP-11932. MetricsSinkAdapter may hang when being stopped. Contributed by Brahma Reddy Battula 2015-08-05 16:12:45 -07:00
cnauroth
3da0bedaec HADOOP-12304. Applications using FileContext fail with the default file system configured to be wasb/s3/etc. Contributed by Chris Nauroth. 2015-08-05 14:56:41 -07:00
Colin Patrick Mccabe
4ab49a4365 HADOOP-12302. Fix native compilation on Windows after HADOOP-7824 (Vinayakumar B via Colin P. McCabe) 2015-08-05 09:46:51 -07:00
Colin Patrick Mccabe
6f83274afc HADOOP-12280. Skip unit tests based on maven profile rather than NativeCodeLoader.isNativeCodeLoaded (Masatake Iwasaki via Colin P. McCabe) 2015-08-05 09:42:38 -07:00
Akira Ajisaka
52f3525586 HADOOP-12274. Remove direct download link from BULIDING.txt. Contributed by Caleb Severn. 2015-08-05 17:45:55 +09:00
Lei Xu
d5403747b5 HADOOP-12269. Update aws-sdk dependency to 1.10.6 (Thomas Demoor via Lei (Eddy) Xu) 2015-08-04 18:51:52 -07:00
Zhihai Xu
a628a6f126 HADOOP-12268. AbstractContractAppendTest#testRenameFileBeingAppended misses rename operation. Contributed by Zhihai Xu 2015-07-31 21:32:45 -07:00
Colin Patrick Mccabe
454da959c7 HADOOP-12183. Annotate the HTrace span created by FsShell with the command-line arguments passed by the user (Masatake Iwasaki via Colin P. McCabe) 2015-07-31 15:14:18 -07:00
Colin Patrick Mccabe
21d10ccc6e HADOOP-7824. NativeIO.java flags and identifiers must be set correctly for each platform, not hardcoded to their Linux values (Martin Walsh via Colin P. McCabe) 2015-07-31 15:01:34 -07:00
Allen Wittenauer
a890a31529 HADOOP-10854. unit tests for the shell scripts (aw) 2015-07-31 14:34:48 -07:00