big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
12,393 Commits 2 Branches 2 Tags 264 MiB
053a511919
Commit Graph

3438 Commits

Author SHA1 Message Date
Zhe Zhang
8def51a708 HDFS-9229. Expose size of NameNode directory as a metric. Contributed by Surendra Singh Lilhore. 
Change-Id: I985627a5d1400249d72d084283ef366d5ac6e07b
 2015-10-29 11:23:08 -07:00
Jian He
c41699965e YARN-4288. Fixed RMProxy to retry on IOException from local host. Contributed by Junping Du 2015-10-29 00:00:48 -07:00
Walter Su
5eca6dece6 HADOOP-12327. Initialize output buffers with ZERO bytes in erasure coder. Contributed by Kai Zheng. 2015-10-29 15:04:34 +08:00
Vinayakumar B
7d2d16f4ee HDFS-8545. Refactor FS#getUsed() to use ContentSummary and add an API to fetch the total file length from a specific path (Contributed by J.Andreina) 2015-10-29 11:28:17 +05:30
Sangjin Lee
b0c818b60d Set the release date for 2.6.2 2015-10-28 20:43:35 -07:00
cnauroth
2d10cb8e00 HADOOP-12519. hadoop-azure tests should avoid creating a metrics configuration fiale in the module root directory. Contributed by Chris Nauroth. 2015-10-28 09:44:10 -07:00
Steve Loughran
a04b16970b HADOOP-12514. Make static fields in GenericTestUtils for assertExceptionContains() package-private and final. (Mingliang Liu via stevel) 2015-10-28 10:36:43 +00:00
Steve Loughran
ff2b2bea91 HADOOP-12385. Include nested stack trace in SaslRpcClient.getServerToken() (stevel) 2015-10-28 10:25:22 +00:00
yliu
c201cf951d HADOOP-12040. Adjust inputs order for the decode API in raw erasure coder. (Kai Zheng via yliu) 2015-10-28 16:18:23 +08:00
cnauroth
bf8e452982 HDFS-9311. Support optional offload of NameNode HA service health checks to a separate RPC server. Contributed by Chris Nauroth. 2015-10-27 23:07:14 -07:00
cnauroth
1f7ecb0c84 HADOOP-11685. StorageException complaining " no lease ID" during HBase distributed log splitting. Contributed by Duo Xu. 2015-10-27 22:56:22 -07:00
cnauroth
73822de7c3 HADOOP-12520. Use XInclude in hadoop-azure test configuration to isolate Azure Storage account keys for service integration tests. Contributed by Chris Nauroth. 2015-10-27 22:48:56 -07:00
Allen Wittenauer
1396867b52 HADOOP-12494. fetchdt stores the token based on token kind instead of token service (HeeSoo Kim via aw) 2015-10-27 12:02:43 -07:00
Zhihai Xu
ed9806ea40 HADOOP-12178. NPE during handling of SASL setup if problem with SASL resolver class. Contributed by Steve Loughran 2015-10-27 09:51:26 -07:00
Akira Ajisaka
bcb2386e39 HADOOP-12515. Mockito dependency is missing in hadoop-kafka module. Contributed by Kai Zheng. 2015-10-27 20:17:16 +09:00
Tsuyoshi Ozawa
ea6b183a1a HADOOP-12457. [JDK8] Fix a failure of compiling common by javadoc. Contributed by Akira AJISAKA. 2015-10-27 10:57:45 +09:00
Jing Zhao
a01a209fbe HADOOP-12472. Make GenericTestUtils.assertExceptionContains robust. Contributed by Steve Loughran. 2015-10-26 14:03:15 -07:00
Tsuyoshi Ozawa
123b3db743 HADOOP-12513. Dockerfile lacks initial 'apt-get update'. Contributed by Akihiro Suda. 2015-10-26 23:18:52 +09:00
Tsuyoshi Ozawa
ce60b4fc8b YARN-3528. Tests with 12345 as hard-coded port break jenkins. Contributed by Brahma Reddy Battula. 2015-10-26 16:45:11 +09:00
Jitendra Pandey
600ad7bf41 HDFS-9184. Logging HDFS operation's caller context into audit logs. Contributed by Mingliang Liu. 2015-10-23 12:15:01 -07:00
Jonathan Eagles
f8adeb712d YARN-4009. CORS support for ResourceManager REST API. ( Varun Vasudev via jeagles) 2015-10-23 10:34:08 -05:00
Tsuyoshi Ozawa
35a303dfbe HADOOP-7266. Deprecate metrics v1. Contributed by Akira AJISAKA. 2015-10-23 23:47:51 +09:00
Tsuyoshi Ozawa
513ec3de19 HADOOP-9692. Improving log message when SequenceFile reader throws EOFException on zero-length file. (Zhe Zhang and Chu Tong via ozawa) 2015-10-23 06:50:50 +09:00
Tsuyoshi Ozawa
039a1f9e96 Revert HADOOP-9692. 
This reverts commit 381610da62.
This reverts commit b5ca649bff.
 2015-10-23 06:48:26 +09:00
cnauroth
cb282d5b89 HADOOP-12484. Single File Rename Throws Incorrectly In Potential Race Condition Scenarios. Contributed by Gaurav Kanade. 2015-10-22 14:29:57 -07:00
Junping Du
0fce5f9a49 YARN-4243. Add retry on establishing Zookeeper conenction in EmbeddedElectorService#serviceInit. Contributed by Xuan Gong. 2015-10-22 13:41:09 -07:00
cnauroth
47641fcbc9 HADOOP-12334. Change Mode Of Copy Operation of HBase WAL Archiving to bypass Azure Storage Throttling after retries. Contributed by Gaurav Kanade. 2015-10-22 12:21:32 -07:00
Allen Wittenauer
4c0bae240b HADOOP-12436. GlobPattern regex library has performance issues with wildcard characters (Matthew Paduano via aw) 2015-10-22 08:19:56 -07:00
Tsuyoshi Ozawa
381610da62 HADOOP-9692. Improving log message when SequenceFile reader throws EOFException on zero-length file. Contributed by Zhe Zhang and Chu Tong. 2015-10-22 11:59:12 +09:00
Tsuyoshi Ozawa
b5ca649bff HADOOP-9692. SequenceFile reader throws EOFException on zero-length file. Contributed by Zhe Zhang and Chu Tong. 2015-10-22 11:55:25 +09:00
Andrew Wang
d1cdce77fa HADOOP-10406. TestIPC.testIpcWithReaderQueuing may fail. Contributed by Xiao Chen. 2015-10-21 15:21:01 -07:00
Allen Wittenauer
d759b4bd74 HADOOP-11791. Update src/site/markdown/releases to include old versions of Hadoop (aw) 2015-10-21 11:30:59 -07:00
Ming Ma
e27c2ae8ba HDFS-8647. Abstract BlockManager's rack policy into BlockPlacementPolicy. (Brahma Reddy Battula via mingma) 2015-10-21 08:06:58 -07:00
Allen Wittenauer
b37c41fd6e HADOOP-12495. Fix posix_spawn error on OS X (aw) 2015-10-21 08:00:26 -07:00
Kihwal Lee
01b103f4ff HADOOP-12418. TestRPC.testRPCInterruptedSimple fails intermittently. Contributed Kihwal Lee. 2015-10-20 15:09:15 -05:00
Allen Wittenauer
ded012e1d6 HADOOP-12493. bash unit tests are failing (aw) 2015-10-19 19:02:20 -07:00
Kihwal Lee
6144e0137b HADOOP-12464. Interrupted client may try to fail-over and retry. Contributed by Kihwal Lee. 2015-10-19 10:52:29 -05:00
Vinayakumar B
7f0e1eb43d HADOOP-12460. Add overwrite option for 'get' shell command (Contributed by Jagadesh Kiran N) 2015-10-19 12:16:16 +05:30
yliu
476a251e5e HADOOP-12483. Maintain wrapped SASL ordering for postponed IPC responses. (Daryn Sharp via yliu) 2015-10-19 09:54:11 +08:00
Steve Loughran
0ab3f9d564 HADOOP-10941. Proxy user verification NPEs if remote host is unresolvable (Benoy Antony via stevel). 2015-10-18 14:05:28 +01:00
Steve Loughran
e286512a71 HADOOP-12450. UserGroupInformation should not log at WARN level if no groups are found. (Elliott Clark via stevel) 2015-10-18 12:23:43 +01:00
Steve Loughran
bafeb6c7bc HADOOP-11628. SPNEGO auth does not work with CNAMEs in JDK8. (Daryn Sharp via stevel). 2015-10-18 11:45:41 +01:00
cnauroth
a121fa1d39 HADOOP-12481. JWTRedirectAuthenticationHandler doesn't Retain Original Query String. Contributed by Larry McCay. 2015-10-15 16:44:59 -07:00
cnauroth
fdd7406224 HADOOP-12479. ProtocMojo does not log the reason for a protoc compilation failure. Contributed by Chris Nauroth. 2015-10-15 15:54:48 -07:00
cnauroth
5f3f0e0f70 HADOOP-12475. Move attribution to 2.8.0 section of CHANGES.txt. 2015-10-15 15:53:57 -07:00
Sangjin Lee
8d2d3eb7bb HADOOP-12475. Replace guava Cache with ConcurrentHashMap for caching Connection in ipc Client (Walter Su via sjlee) 2015-10-15 11:43:07 -07:00
Allen Wittenauer
dc45a7a7c4 Revert "HADOOP-12436. GlobPattern regex library has performance issues with wildcard characters (Matthew Paduano via aw)" 
This reverts commit 0d77e85f0a.
 2015-10-15 08:26:06 -07:00
Steve Loughran
dfa7848463 HADOOP-12478. Shell.getWinUtilsPath() has been renamed Shell.getWinutilsPath(). (stevel) 2015-10-14 20:25:47 +01:00
Allen Wittenauer
ab13aa30cb fix CHANGES.txt name typo 2015-10-14 11:48:28 -07:00
Allen Wittenauer
56dc7773f8 HADOOP-12364. Deleting pid file after stop is causing the daemons to keep restarting (Sigi Li via aw) 2015-10-14 11:16:10 -07:00
Allen Wittenauer
0d77e85f0a HADOOP-12436. GlobPattern regex library has performance issues with wildcard characters (Matthew Paduano via aw) 2015-10-14 08:59:41 -07:00
Colin Patrick Mccabe
d286032b71 HDFS-9187. Fix null pointer error in Globber when FS was not constructed via FileSystem#createFileSystem (cmccabe) 2015-10-13 16:11:46 -07:00
Steve Loughran
c59af2fdf8 HADOOP-10775. Shell operations to fail with meaningful errors on windows if winutils.exe not found. (stevel) 2015-10-13 21:49:28 +01:00
Tsuyoshi Ozawa
33fefa33e5 HADOOP-11515. Upgrade jsch lib to jsch-0.1.51 to avoid problems running on java7. (stevel and ozawa) 2015-10-14 05:31:43 +09:00
Haohui Mai
d1e1925bf6 HDFS-9145. Tracking methods that hold FSNamesytemLock for too long. Contributed by Mingliang Liu. 2015-10-13 13:24:15 -07:00
Steve Loughran
59e9980117 HADOOP-12449. TestDNS and TestNetUtils failing if no network. (stevel) 2015-10-13 21:12:08 +01:00
Haohui Mai
7f00fcac77 HADOOP-12474. MiniKMS should use random ports for Jetty server by default. Contributed by Mingliang Liu. 2015-10-13 13:07:57 -07:00
Colin Patrick Mccabe
6716f15964 HADOOP-12344. Improve validateSocketPathSecurity0 error message (Casey Brotherton via Colin P. McCabe) 2015-10-13 12:18:03 -07:00
Jitendra Pandey
84cbd72afd HDFS-8855. Webhdfs client leaks active NameNode connections. Contributed by Xiaobing Zhou. 2015-10-13 09:12:04 -07:00
yliu
e617cf6dd1 HADOOP-10300. Allowed deferred sending of call responses. (Daryn Sharp via yliu) 2015-10-12 16:09:14 +08:00
Haohui Mai
7e2c971fed Revert "HADOOP-12469. distcp shout not ignore the ignoreFailures option. Contributed by Mingliang Liu." 
This reverts commit fa5a54cac7.
 2015-10-10 03:24:59 -07:00
Haohui Mai
fa5a54cac7 HADOOP-12469. distcp shout not ignore the ignoreFailures option. Contributed by Mingliang Liu. 2015-10-09 22:38:06 -07:00
Akira Ajisaka
e1bf8b3df6 HADOOP-11104. org.apache.hadoop.metrics2.lib.MetricsRegistry needs numerical parameter checking. Contributed by Ray Chiang. 2015-10-09 11:10:29 +09:00
Andrew Wang
66e2cfa1a0 HDFS-8632. Add InterfaceAudience annotation to the erasure coding classes. Contributed by Rakesh R. 2015-10-07 18:12:26 -07:00
Akira Ajisaka
7fbf69bf47 HADOOP-12465. Incorrect javadoc in WritableUtils.java. Contributed by Jagadesh Kiran N. 2015-10-08 06:08:28 +09:00
Uma Mahesh
8b7339312c HDFS-9182. Cleanup the findbugs and other issues after HDFS EC merged to trunk. Contributed by Uma Maheswara Rao G. 2015-10-06 22:22:32 -07:00
Steve Loughran
6d5713a8a8 HADOOP-12284. UserGroupInformation doAs can throw misleading exception (Aaron Dosset via stevel) 2015-10-06 21:39:53 +01:00
cnauroth
5f6edb30c2 HADOOP-12350. WASB Logging: Improve WASB Logging around deletes, reads and writes. Contributed by Dushyanth. 2015-10-05 20:11:41 -07:00
Allen Wittenauer
b59e434116 HADOOP-10949. metrics2 sink plugin for Apache Kafka (Babak Behzad via aw) 2015-10-05 14:02:07 -07:00
Colin Patrick Mccabe
3e1752f9fb HADOOP-12452. Fix tracing documention reflecting the update to htrace-4 (Masatake Iwasaki via Colin P. McCabe) 2015-10-05 11:35:31 -07:00
Vinod Kumar Vavilapalli
f1c19b9365 HADOOP-12441. Fixed shell-kill command behaviour to work correctly on some Linux distributions after HADOOP-12317. Contributed by Wangda Tan. 2015-10-05 10:56:55 -07:00
Tsuyoshi Ozawa
30e2f836a2 HADOOP-11098. [JDK8] Max Non Heap Memory default changed between JDK7 and 8. 2015-10-05 08:15:06 +09:00
Harsh J
c918f7be5e HADOOP-12458. Retries is typoed to spell Retires in parts of hadoop-yarn and hadoop-common. Contributed by Neelesh Srinivas Salian. 2015-10-03 18:37:58 +05:30
Jason Lowe
fdf02d1f26 YARN-3619. ContainerMetrics unregisters during getMetrics and leads to ConcurrentModificationException. Contributed by Zhihai Xu 2015-10-02 20:20:31 +00:00
Zhihai Xu
fd026f535c HADOOP-8437. getLocalPathForWrite should throw IOException for invalid paths. Contributed by Brahma Reddy Battula 2015-10-01 11:56:49 -07:00
Akira Ajisaka
ecbfd68974 HADOOP-10296. Incorrect null check in SwiftRestClient#buildException(). Contributed by Rahul Palamuttam and Kanaka Kumar Avvaru. 2015-10-01 18:25:31 +02:00
Zhe Zhang
6c17d31528 Merge remote-tracking branch 'apache/trunk' into HDFS-7285 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java

Change-Id: I53ec1c426dc988d6c4a2c87b00caef49c4057010
 2015-09-30 08:39:13 -07:00
yliu
06abc57a90 HADOOP-12448. TestTextCommand: use mkdirs rather than mkdir to create test directory. (Contributed by Colin Patrick McCabe and Chris Nauroth) 2015-09-30 10:56:22 +08:00
Haohui Mai
8703301b46 HDFS-9174. Fix findbugs warnings in FSOutputSummer.tracer and DirectoryScanner$ReportCompiler.currentThread. Contributed by Yi Liu. 2015-09-29 14:56:06 -07:00
Colin Patrick Mccabe
a2efb945e3 Add HADOOP-12447 to CHANGES.txt 2015-09-29 09:37:13 -07:00
Colin Patrick Mccabe
850d679acb HADOOP-12447. Clean up some htrace integration issues (cmccabe) 2015-09-29 09:25:11 -07:00
Zhe Zhang
8fd5520246 Merge remote-tracking branch 'apache/trunk' into HDFS-7285 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/hdfs.proto
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSUtil.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelper.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirStatAndListingOp.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/proto/DatanodeProtocol.proto
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockTokenWithDFS.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java

Change-Id: Ic7946c4ea35bed587fe879ce58b959b25ecc0823
 2015-09-29 01:39:16 -07:00
yliu
d6fa34e014 HDFS-8859. Improve DataNode ReplicaMap memory footprint to save about 45%. (yliu) 2015-09-29 16:20:35 +08:00
Ming Ma
5c3b663bf9 HADOOP-12440. TestRPC#testRPCServerShutdown did not produce the desired thread states before shutting down. (Xiao Chen via mingma) 2015-09-28 18:12:51 -07:00
Kihwal Lee
3abbdc929b HADOOP-12446. Undeprecate createNonRecursive(). Contributed by Ted Yu. 2015-09-28 17:31:51 -05:00
Colin Patrick Mccabe
892ade689f HDFS-9080. Update htrace version to 4.0.1 (cmccabe) 2015-09-28 07:42:12 -07:00
Vinayakumar B
f0f984e4e6 HADOOP-11984. Enable parallel JUnit tests in pre-commit (Contributed by Chris Nauroth) 2015-09-27 14:12:07 +05:30
Vinayakumar B
861b52db24 HADOOP-12442. Display help if the command option to 'hdfs dfs' is not valid (Contributed by nijel) 2015-09-26 21:36:17 +05:30
cnauroth
7fe521b1dd HADOOP-11918. Listing an empty s3a root directory throws FileNotFound. Contributed by Lei (Eddy) Xu. 2015-09-25 22:33:53 -07:00
Yongjun Zhang
3187dbd7a8 HDFS-9123. Copying from the root to a subdirectory should be forbidden. (Wei-Chiu Chuang via Yongjun Zhang) 2015-09-24 21:36:38 -07:00
Zhihai Xu
9cc9f8126b Update CHANGES.txt to reflect commit of HADOOP-8436 to 2.8.0 2015-09-24 12:52:07 -07:00
Zhihai Xu
52c1f272ec HADOOP-12252. LocalDirAllocator should not throw NPE with empty string configuration. Contributed by Zhihai Xu 2015-09-24 11:51:22 -07:00
Arpit Agarwal
df31c446bf HADOOP-12437. Allow SecurityUtil to lookup alternate hostnames. (Contributed by Arpit Agarwal) 2015-09-24 11:41:48 -07:00
Haohui Mai
0ef7ff47d5 HADOOP-12438. Reset RawLocalFileSystem.useDeprecatedFileStatus in TestLocalFileSystem. Contributed by Chris Nauroth. 2015-09-23 19:36:44 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
1f707ecffd Release process for 2.6.1: Set the release date for 2.6.1 2015-09-23 15:10:50 -07:00
Vinayakumar B
a2c76e5f26 HDFS-9013. Deprecate NameNodeMXBean#getNNStarted in branch2 and remove from trunk (Contributed by Surendra Singh Lilhore) 2015-09-23 15:28:41 +05:30
Zhe Zhang
1080c37300 Merge remote-tracking branch 'apache/trunk' into HDFS-7285 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/Namesystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java

Change-Id: I8511c4d64b0959e79129febc179845a3892fedcc
 2015-09-22 13:27:09 -07:00
Tsuyoshi Ozawa
dfd807afab HADOOP-12428. Fix inconsistency between log-level guards and statements. Contributed by Jagadesh Kiran N and Jackie Chang. 2015-09-22 12:54:29 +09:00
Steve Loughran
c39ddc306d HADOOP-12360. Create StatsD metrics2 sink. (Dave Marion via stevel) 2015-09-19 12:41:42 +01:00
Zhihai Xu
88d89267ff HADOOP-12404. Disable caching for JarURLConnection to avoid sharing JarFile with other users when loading resource from URL in Configuration class. Contributed by Zhihai Xu 2015-09-18 10:40:04 -07:00