big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
14,748 Commits 2 Branches 2 Tags 264 MiB
4e403def80
Commit Graph

4063 Commits

Author SHA1 Message Date
Chris Nauroth
1d330fbaf6 HADOOP-13150. Avoid use of toString() in output of HDFS ACL shell commands. Contributed by Chris Nauroth. 2016-10-06 13:19:16 -07:00
Chris Nauroth
2d46c3f6b7 HADOOP-13323. Downgrade stack trace on FS load from Warn to debug. Contributed by Steve Loughran. 2016-10-06 10:57:01 -07:00
Brahma Reddy Battula
35b9d7de9f HADOOP-13690. Fix typos in core-default.xml. Contributed by Yiqun Lin 2016-10-06 18:06:08 +05:30
Brahma Reddy Battula
b90fc70d67 HDFS-10963. Reduce log level when network topology cannot find enough datanodes. Contributed by Xiao chen 2016-10-06 17:47:31 +05:30
Wei-Chiu Chuang
c5ca216915 HDFS-10683. Make class Token$PrivateToken private. Contributed by John Zhuge. 2016-10-05 17:36:50 -07:00
Mingliang Liu
00160f71b6 HADOOP-13628. Support to retrieve specific property from configuration via REST API. Contributed by Weiwei Yang 2016-10-05 16:50:01 -04:00
Akira Ajisaka
9a61b52fbe HADOOP-13685. Document -safely option of rm command. Contributed by Xiaoyu Yao. 2016-10-05 20:41:08 +09:00
Xiao Chen
853d65a157 HDFS-10918. Add a tool to get FileEncryptionInfo from CLI. Contributed by Xiao Chen. 2016-10-03 16:01:54 -07:00
Xiao Chen
89bd6d29a6 HADOOP-13317. Add logs to KMS server-side to improve supportability. Contributed by Suraj Acharya. 2016-09-30 17:51:39 -07:00
Wei-Chiu Chuang
57aec2b46b HADOOP-12974. Create a CachingGetSpaceUsed implementation that uses df. Contributed by Elliott Clark. 2016-09-30 12:58:37 -07:00
Chris Nauroth
1e0ea27e96 Revert "HADOOP-13081. add the ability to create multiple UGIs/subjects from one kerberos login. Contributed by Sergey Shelukhin." 
This reverts commit 0458a2af6e.
 2016-09-29 13:59:09 -07:00
Kihwal Lee
236ac773c9 HADOOP-13537. Support external calls in the RPC call queue. Contributed by Daryn Sharp. 2016-09-29 13:27:30 -05:00
Steve Loughran
ee0c722dc8 HADOOP-13164 Optimize S3AFileSystem::deleteUnnecessaryFakeDirectories. Contributed by Rajesh Balamohan. 2016-09-29 17:01:00 +01:00
Steve Loughran
1518cb9532 HADOOP-13663 Index out of range in SysInfoWindows. Contributed by Inigo Goiri 2016-09-29 11:35:00 +01:00
Kihwal Lee
e19b37ead2 HADOOP-11780. Prevent IPC reader thread death. Contributed by Daryn Sharp. 2016-09-28 08:24:21 -05:00
Mingliang Liu
9a44a832a9 HADOOP-13658. Replace config key literal strings with names I: hadoop common. Contributed by Chen Liang 2016-09-27 17:36:53 -07:00
Wangda Tan
875062b5bc HADOOP-13544. JDiff reports unncessarily show unannotated APIs and cause confusion while our javadocs only show annotated and public APIs. (vinodkv via wangda) 2016-09-27 11:26:45 -07:00
Brahma Reddy Battula
edf0d0f8b2 HDFS-9444. Add utility to find set of available ephemeral ports to ServerSocketUtil. Contributed by Masatake Iwasaki 2016-09-27 07:04:37 +05:30
Xiao Chen
fa397e74fe HADOOP-13638. KMS should set UGI's Configuration object properly. Contributed by Wei-Chiu Chuang. 2016-09-26 13:00:57 -07:00
Konstantin V Shvachko
6eb700ee74 HDFS-10866. Fix Eclipse Java compile errors related to generic parameters. Contributed by Konstantin Shvachko. 2016-09-23 14:10:11 -07:00
Brahma Reddy Battula
bbdf350ff9 HDFS-10886. Replace fs.default.name with fs.defaultFS in viewfs document. 2016-09-23 19:21:34 +05:30
Mingliang Liu
e80386d69d HADOOP-13601. Fix a log message typo in AbstractDelegationTokenSecretManager. Contributed by Mehran Hassani. 2016-09-20 13:20:01 -07:00
Kai Zheng
734d54c1a8 HDFS-10861. Refactor StripeReaders and use ECChunk version decode API. Contributed by Sammi Chen 2016-09-21 21:34:48 +08:00
Xiao Chen
ea839bd48e HDFS-10489. Deprecate dfs.encryption.key.provider.uri for HDFS encryption zones. Contributed by Xiao Chen. 2016-09-17 22:25:39 -07:00
Kai Zheng
58bae35447 YARN-5163. Migrate TestClientToAMTokens and TestClientRMTokens tests from the old RPC engine. Contributed by Wei Zhou and Kai Zheng 2016-09-18 08:43:36 +08:00
Andrew Wang
8a40953058 HDFS-10823. Implement HttpFSFileSystem#listStatusIterator. 2016-09-16 15:37:36 -07:00
Wei-Chiu Chuang
f6f3a447bf HADOOP-13580. If user is unauthorized, log "unauthorized" instead of "Invalid signed text:". Contributed by Wei-Chiu Chuang. 2016-09-16 14:53:30 -07:00
Sangjin Lee
fcbac00212 Revert "HADOOP-13410. RunJar adds the content of the jar twice to the classpath (Yuanbo Liu via sjlee)" 
This reverts commit 4d3ea92f4f.
 2016-09-15 10:18:56 -07:00
Akira Ajisaka
7cad7b7046 HADOOP-13616. Broken code snippet area in Hadoop Benchmarking. Contributed by Kai Sasaki. 2016-09-15 23:42:34 +09:00
Steve Loughran
53a12fa721 HADOOP-13606 swift FS to add a service load metadata file. Contributed by Steve Loughran 2016-09-14 15:45:04 +01:00
Kai Zheng
ea0c2b8b05 HADOOP-13218. Migrate other Hadoop side tests to prepare for removing WritableRPCEngine. Contributed by Wei Zhou and Kai Zheng 2016-09-15 10:46:00 +08:00
Jing Zhao
08d8e0ba25 HADOOP-13546. Override equals and hashCode of the default retry policy to avoid connection leakage. Contributed by Xiaobing Zhou. 2016-09-13 11:12:52 -07:00
Yongjun Zhang
d2466ec3e0 HDFS-10657. testAclCLI.xml setfacl test should expect mask r-x. (John Zhuge via Yongjun Zhang) 2016-09-12 21:25:06 -07:00
Mingliang Liu
59d59667a8 HADOOP-13588. ConfServlet should respect Accept request header. Contributed by Weiwei Yang 2016-09-12 16:17:08 -07:00
Allen Wittenauer
58ed4fa544 HADOOP-13341. Deprecate HADOOP_SERVERNAME_OPTS; replace with (command)_(subcommand)_OPTS 
This commit includes the following changes:

	HADOOP-13356. Add a function to handle command_subcommand_OPTS
	HADOOP-13355. Handle HADOOP_CLIENT_OPTS in a function
	HADOOP-13554. Add an equivalent of hadoop_subcmd_opts for secure opts
	HADOOP-13562. Change hadoop_subcommand_opts to use only uppercase
	HADOOP-13358. Modify HDFS to use hadoop_subcommand_opts
	HADOOP-13357. Modify common to use hadoop_subcommand_opts
	HADOOP-13359. Modify YARN to use hadoop_subcommand_opts
	HADOOP-13361. Modify hadoop_verify_user to be consistent with hadoop_subcommand_opts (ie more granularity)
	HADOOP-13564. modify mapred to use hadoop_subcommand_opts
	HADOOP-13563. hadoop_subcommand_opts should print name not actual content during debug
	HADOOP-13360. Documentation for HADOOP_subcommand_OPTS

This closes apache/hadoop#126
 2016-09-12 11:10:00 -07:00
Arpit Agarwal
a99bf26a08 HDFS-10830. FsDatasetImpl#removeVolumes crashes with IllegalMonitorStateException when vol being removed is in use. (Arpit Agarwal and Manoj Govindassamy) 2016-09-10 18:22:25 -07:00
Chris Douglas
9f192cc5ac MAPREDUCE-6628. Potential memory leak in CryptoOutputStream. Contributed by Mariappan Asokan 2016-09-09 11:12:44 -07:00
Kihwal Lee
d4d076876a HADOOP-10940. RPC client does no bounds checking of responses. Contributed by Daryn Sharp. 2016-09-09 10:39:35 -05:00
Chris Douglas
011f3b24d4 HDFS-10742. Measure lock time in FsDatasetImpl. Contributed by Chen Liang 2016-09-08 17:53:40 -07:00
Chris Douglas
1192781a78 HADOOP-13519. Make Path Serializable. Contributed by Steve Loughran 2016-09-08 17:46:12 -07:00
Allen Wittenauer
cae331186d Revert "YARN-5567. Fix script exit code checking in NodeHealthScriptRunner#reportHealthStatus. (Yufei Gu via rchiang)" 
This reverts commit 05ede00386.
 2016-09-08 12:40:30 -07:00
Zhe Zhang
401db4fc65 HDFS-8901. Use ByteBuffer in striping positional read. Contributed by Sammi Chen and Kai Zheng. 2016-09-08 11:54:33 -07:00
Kai Zheng
d355573f56 Revert "HADOOP-13218. Migrate other Hadoop side tests to prepare for removing WritableRPCEngine. Contributed by Wei Zhou and Kai Zheng" 
This reverts commit 62a9667136
 2016-09-08 05:50:17 +08:00
Anu Engineer
f414d5e118 HADOOP-13388. Clean up TestLocalFileSystemPermission. Contributed by Andras Bokor. 2016-09-07 11:09:41 -07:00
Xiao Chen
680be58aac HADOOP-13558. UserGroupInformation created from a Subject incorrectly tries to renew the Kerberos ticket. Contributed by Xiao Chen. 2016-09-06 20:54:17 -07:00
Chris Nauroth
f0d5382ff3 HDFS-6962. ACL inheritance conflicts with umaskmode. Contributed by Chris Nauroth. 2016-09-06 11:02:39 -07:00
Chris Douglas
d37dc5d1b8 HDFS-9847. HDFS configuration should accept time units. Contributed by Yiqun Lin 2016-09-06 10:38:04 -07:00
Andrew Wang
f388a20943 Add CHANGES and release notes for 3.0.0-alpha1 to site 2016-09-06 09:51:55 -07:00
Kihwal Lee
39d1b1d747 HADOOP-13549. Eliminate intermediate buffer for server-side PB encoding. Contributed by Daryn Sharp. 2016-09-06 10:02:42 -05:00
Kai Zheng
62a9667136 HADOOP-13218. Migrate other Hadoop side tests to prepare for removing WritableRPCEngine. Contributed by Wei Zhou and Kai Zheng 2016-09-07 17:05:33 +08:00
Kihwal Lee
23abb09c1f HADOOP-13547. Optimize IPC client protobuf decoding. Contributed by Daryn Sharp. 2016-09-02 11:03:18 -05:00
Kihwal Lee
76cd81f4b6 HADOOP-13465. Design Server.Call to be extensible for unified call queue. Contributed by Daryn Sharp. 2016-09-01 15:44:44 -05:00
Chris Douglas
3069df7551 FsPermission string constructor does not recognize sticky bit. Contributed by Atul Sikaria 2016-09-01 11:32:26 -07:00
Mingliang Liu
dcd21d083a HADOOP-13375. o.a.h.security.TestGroupsCaching.testBackgroundRefreshCounters seems flaky. (Contributed by Weiwei Yang) 2016-09-01 11:03:06 -07:00
Ray Chiang
05ede00386 YARN-5567. Fix script exit code checking in NodeHealthScriptRunner#reportHealthStatus. (Yufei Gu via rchiang) 2016-08-29 15:55:33 -07:00
Mingliang Liu
6fcb04c178 HADOOP-13559. Remove close() within try-with-resources in ChecksumFileSystem/ChecksumFs classes. (Contributed by Aaron Fabbri) 2016-08-29 13:04:28 -07:00
Anu Engineer
e1ad598cef HADOOP-7363. TestRawLocalFileSystemContract is needed. Contributed by Andras Bokor. 2016-08-29 09:26:46 -07:00
Jason Lowe
92d8f37155 HADOOP-13552. RetryInvocationHandler logs all remote exceptions. Contributed by Jason Lowe 2016-08-29 15:55:38 +00:00
Kihwal Lee
81485dbfc1 Revert "HADOOP-13465. Design Server.Call to be extensible for unified call queue. Contributed by Daryn Sharp." 
This reverts commit d288a0ba83.
 2016-08-25 16:04:54 -05:00
Kihwal Lee
d288a0ba83 HADOOP-13465. Design Server.Call to be extensible for unified call queue. Contributed by Daryn Sharp. 2016-08-25 11:44:13 -05:00
Albert Chu
52b25fe6ae HADOOP-13533. Do not require user to set HADOOP_SSH_OPTS to a non-null string, allow setting of an empty string. (Albert Chu via aw) closes #121 
Signed-off-by: Allen Wittenauer <aw@apache.org>
 2016-08-25 07:11:06 -07:00
Allen Wittenauer
601599b036 HADOOP-13532. Fix typo in hadoop_connect_to_hosts error message (Albert Chu via aw) closes #120 2016-08-25 06:58:42 -07:00
Akira Ajisaka
ab3b727b5f HADOOP-13534. Remove unused TrashPolicy#getInstance and initialize code. Contributed by Yiqun Lin. 2016-08-25 13:54:57 +09:00
Xiao Chen
3476156807 HADOOP-13396. Allow pluggable audit loggers in KMS. Contributed by Xiao Chen 2016-08-24 10:14:46 -07:00
Akira Ajisaka
092b4d5bfd HADOOP-13538. Deprecate getInstance and initialize methods with Path in TrashPolicy. Contributed by Yiqun Lin. 2016-08-24 16:47:52 +09:00
Chris Douglas
c37346d0e3 HADOOP-12726. Unsupported FS operations should throw UnsupportedOperationException. Contributed by Daniel Templeton. 2016-08-23 14:12:52 -07:00
Masatake Iwasaki
8aae8d6bf0 HADOOP-13497. fix wrong command in CredentialProviderAPI.md. Contributed by Yuanbo Liu. 2016-08-24 00:03:27 +09:00
Wei-Chiu Chuang
f0efea490e HDFS-8986. Add option to -du to calculate directory space usage excluding snapshots. Contributed by Xiao Chen. 2016-08-23 04:14:24 -07:00
Eric Yang
c49333becf HDFS-8312. Added permission check for moving file to Trash. (Weiwei Yang via Eric Yang) 2016-08-22 18:29:56 -07:00
Xiao Chen
4070caad70 HADOOP-13526. Add detailed logging in KMS for the authentication failure of proxy user. Contributed by Suraj Acharya. 2016-08-22 18:09:35 -07:00
Xiao Chen
f4d4d3474c HADOOP-13487. Hadoop KMS should load old delegation tokens from Zookeeper on startup. Contributed by Xiao Chen. 2016-08-22 14:42:13 -07:00
Mingliang Liu
115ecb52a8 HADOOP-13527. Add Spark to CallerContext LimitedPrivate scope. (Contributed by Weiqing Yang) 2016-08-21 09:40:29 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
99603e9022 HADOOP-13428. Fixed hadoop-common build files to generate jdiff by working around a jdiff-bug. Contributed by Wangda Tan. 2016-08-19 19:08:53 -07:00
Chris Nauroth
763f0497bb HADOOP-13252. Tune S3A provider plugin mechanism. Contributed by Steve Loughran. 2016-08-19 10:48:10 -07:00
Wei-Chiu Chuang
03a9343d57 HADOOP-12765. HttpServer2 should switch to using the non-blocking SslSelectChannelConnector to prevent performance degradation when handling SSL connections. Contributed by Min Shen. 2016-08-19 09:22:49 -07:00
Akira Ajisaka
8179f9a493 HDFS-10645. Make block report size as a metric and add this metric to datanode web ui. Contributed by Yuanbo Liu. 2016-08-19 16:15:00 +09:00
Kai Zheng
dbcaf999d9 HADOOP-13504. Refactor jni_common to conform to C89 restrictions imposed by Visual Studio 2010. Contributed by Sammi Chen 2016-08-19 11:18:15 +08:00
Jing Zhao
c5c3e81b49 HADOOP-13503. Improve SaslRpcClient failure logging. Contributed by Xiaobing Zhou. 2016-08-18 14:55:26 -07:00
Mingliang Liu
0f51eae0c0 HADOOP-13512. ReloadingX509TrustManager should keep reloading in case of exception. (Contributed by Mingliang Liu) 2016-08-18 14:10:37 -07:00
Junping Du
0da69c324d YARN-4676. Automatic and Asynchronous Decommissioning Nodes Status Tracking. Contributed by Diniel Zhi. 
(cherry picked from commit d464483bf7f0b3e3be3ba32cd6c3eee546747ab5)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceTrackerService.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNode.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
 2016-08-18 07:27:23 -07:00
Steve Loughran
040c185d62 HADOOP-13405 doc for fs.s3a.acl.default indicates incorrect values. Contributed by Shen Yinjie 2016-08-18 14:36:55 +01:00
Anu Engineer
c57523163f HADOOP-11786. Fix Javadoc typos in org.apache.hadoop.fs.FileSystem. Contributed by Andras Bokor. 2016-08-17 15:00:14 -07:00
Mingliang Liu
23532716fc HADOOP-13470. GenericTestUtils$LogCapturer is flaky. (Contributed by Mingliang Liu) 2016-08-16 17:33:04 -07:00
Mingliang Liu
27a6e09c4e Revert "HADOOP-13470. GenericTestUtils$LogCapturer is flaky. (Contributed by Mingliang Liu)" 
This reverts commit 9336a0495f.
 2016-08-16 16:25:37 -07:00
Andrew Wang
4b689e7a75 HADOOP-13494. ReconfigurableBase can log sensitive information. Contributed by Sean Mackrory. 2016-08-16 15:01:18 -07:00
Masatake Iwasaki
b8a446ba57 HADOOP-13419. Fix javadoc warnings by JDK8 in hadoop-common package. Contributed by Kai Sasaki. 2016-08-16 13:30:40 +09:00
Mingliang Liu
9336a0495f HADOOP-13470. GenericTestUtils$LogCapturer is flaky. (Contributed by Mingliang Liu) 2016-08-15 20:24:54 -07:00
Mingliang Liu
382d615260 HDFS-10678. Documenting NNThroughputBenchmark tool. (Contributed by Mingliang Liu) 2016-08-15 20:22:14 -07:00
Mingliang Liu
4bcbef39f7 HDFS-10724. Document caller context config keys. (Contributed by Mingliang Liu) 2016-08-15 20:20:33 -07:00
Mingliang Liu
12ad63d723 HDFS-10725. Caller context should always be constructed by a builder. (Contributed by Mingliang Liu) 2016-08-15 20:14:05 -07:00
Xiao Chen
9daa9979a1 HADOOP-13437. KMS should reload whitelist and default key ACLs when hot-reloading. Contributed by Xiao Chen. 2016-08-15 18:14:45 -07:00
Varun Saxena
d714030b5d HADOOP-13333. testConf.xml ls comparators in wrong order (Vrushali C via Varun Saxena) 2016-08-16 03:03:44 +05:30
Kai Zheng
b5af9be72c HDFS-8668. Erasure Coding: revisit buffer used for encoding and decoding. Contributed by Sammi Chen 2016-08-13 13:52:37 +08:00
Sangjin Lee
4d3ea92f4f HADOOP-13410. RunJar adds the content of the jar twice to the classpath (Yuanbo Liu via sjlee) 2016-08-11 19:56:58 -07:00
Wei-Chiu Chuang
db719ef125 HADOOP-13190. Mention LoadBalancingKMSClientProvider in KMS HA documentation. Contributed by Wei-Chiu Chuang. 2016-08-11 12:27:09 -07:00
Wei-Chiu Chuang
d892ae9576 HADOOP-13441. Document LdapGroupsMapping keystore password properties. Contributed by Yuanbo Liu. 2016-08-11 11:57:43 -07:00
Kai Zheng
8fbb57fbd9 HADOOP-11588. Benchmark framework and test for erasure coders. Contributed by Rui Li 2016-08-12 15:05:52 +08:00
Xiao Chen
e83be44af5 HADOOP-13461. NPE in KeyProvider.rollNewVersion. Contributed by Colm O hEigeartaigh. 2016-08-10 16:28:02 -07:00
Chris Nauroth
4d4fe07c3c HADOOP-13476. CredentialProviderFactory fails at class loading from libhdfs (JNI). Contributed by Aaron Fabbri. 2016-08-10 09:54:31 -07:00
Wei-Chiu Chuang
d00d3add9e HDFS-8224. Schedule a block for scanning if its metadata file is corrupt. Contributed by Rushabh S Shah. 2016-08-09 18:11:34 -07:00