big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
20,136 Commits 2 Branches 2 Tags 264 MiB
e5287a4fe0
Commit Graph

4893 Commits

Author SHA1 Message Date
Wei-Chiu Chuang
bff3d7b0cf HADOOP-15382. Log kinit output in credential renewal thread. Contributed by Gabor Bota. 2018-04-27 10:05:55 -07:00
Ewan Higgs
84ecfe3ceb
HADOOP-14188. Remove the usage of org.mockito.internal.util.reflection.Whitebox 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-04-27 19:21:37 +09:00
Wangda Tan
13d389bf51 HADOOP-15411. AuthenticationFilter should use Configuration.getPropsWithPrefix instead of iterator. (Suma Shivaprasad via wangda) 
Change-Id: Ia19458a43aceb4ffdb1e8eccc519655235d92ba9
 2018-04-26 13:54:40 -07:00
Daryn Sharp
bbc8dfcb8b HADOOP-15402. Prevent double logout of UGI's LoginContext 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-04-26 13:54:40 -07:00
Owen O'Malley
2adda92de1
HDFS-7240 git commit fixup 
Signed-off-by: Owen O'Malley <omalley@apache.org>
 2018-04-26 13:52:17 -07:00
Mukul Kumar Singh
8cc13ff962 HDFS-13425. Ozone: Clean-up of ozone related change from hadoop-common-project. Contributed by Lokesh Jain. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
ec6c8742e5 HDFS-13415. Ozone: Remove cblock code from HDFS-7240. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
8b832f3c35 HDFS-13405. Ozone: Rename HDSL to HDDS. 
Contributed by Ajay Kumar, Elek Marton, Mukul Kumar Singh, Shashikant Banerjee and Anu Engineer.
 2018-04-26 05:36:04 -07:00
Anu Engineer
4f37ff7616 HDFS-13341. Ozone: Move ozone specific ServiceRuntimeInfo utility to the framework. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
4e61bc431e HDFS-13258. Ozone: restructure Hdsl/Ozone code to separated maven subprojects. 
Contributed by Elek Marton, Mukul Kumar Singh, Xiaoyu Yao, Ajay Kumar, Anu Engineer, Lokesh Jain, Nanda Kumar.
 2018-04-26 05:36:04 -07:00
Anu Engineer
740a06cdd7 HDFS-12719. Ozone: Fix checkstyle, javac, whitespace issues in HDFS-7240 branch. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Anu Engineer
06c6582694 HDFS-12585. Ozone: KSM UI: Add description for configs in UI. Contributed by Ajay Kumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
0b3270a8de HDFS-12513. Ozone: Create UI page to show Ozone configs by tag. Contributed by Ajay Kumar. 2018-04-26 05:36:04 -07:00
Chris Douglas
6d3d01d7ad HADOOP-12077. Provide a multi-URI replication Inode for ViewFs. Contributed by Gera Shegalov 2018-04-26 05:36:04 -07:00
Anu Engineer
dcc21a4f93 HDFS-12005. Ozone: Web interface for SCM. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
acfae2716a HDFS-12034. Ozone: Web interface for KSM. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Yiqun Lin
8cf0c864c4 HDFS-12170. Ozone: OzoneFileSystem: KSM should maintain key creation time and modification time. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Rakesh Radhakrishnan
665f7c8b69 HDFS-11794. Add ec sub command -listCodec to show currently supported ec codecs. Contributed by SammiChen. 2018-04-26 05:36:04 -07:00
Robert Kanter
7ab08a9c37 HADOOP-15390. Yarn RM logs flooded by DelegationTokenRenewer trying to renew KMS tokens (xiaochen via rkanter) 2018-04-23 15:44:15 -07:00
Lei Xu
42e82f0281 Update CHANGES, RELEASENOTES, and jdiff for 3.0.2 release. 
(cherry picked from commit f6ecb76d0b919b9836600fe28ec9e637b223cd54)

Conflicts:
	hadoop-project-dist/pom.xml
 2018-04-23 14:14:05 -07:00
Virajith Jalaparti
827d8593a5 HDFS-13311. RBF: TestRouterAdminCLI#testCreateInvalidEntry fails on Windows. Contributed by Inigo Goiri 2018-04-16 15:55:00 -07:00
Bharat Viswanadham
ec1e8c1a8c HADOOP-15379. Make IrqHandler.bind() public. Contributed by Ajay Kumar 2018-04-12 21:51:20 -07:00
Aaron Fabbri
0d898b7bb8
HADOOP-12502 SetReplication OutOfMemoryError. Contributed by Vinayakumar B. 2018-04-11 17:19:56 -07:00
Xiao Chen
7c9cdad6d0 HDFS-13056. Expose file-level composite CRCs in HDFS which are comparable across different instances/layouts. Contributed by Dennis Huo. 2018-04-10 21:31:48 -07:00
Xiao Chen
583fa6ed48 HADOOP-14445. Delegation tokens are not shared between KMS instances. Contributed by Xiao Chen and Rushabh S Shah. 2018-04-10 15:38:25 -07:00
Jason Lowe
e81397545a HADOOP-15357. Configuration.getPropsWithPrefix no longer does variable substitution. Contributed by Jim Brennan 2018-04-10 16:44:55 -05:00
Xiaoyu Yao
8ab776d61e HADOOP-15340. Provide meaningful RPC server name for RpcMetrics. Contributed by Elek Marton. 2018-04-10 11:42:54 -07:00
Ewan Higgs
cef8eb7981
HADOOP-15376. Remove double semi colons on imports that make Clover fall over. 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-04-10 23:58:26 +09:00
Bharat
0006346abe HADOOP-15328. Fix the typo in HttpAuthentication.md. Contributed by fang zhenyi 2018-04-09 16:37:49 -07:00
Mukul Kumar Singh
0b345b7653 HADOOP-15366. Add a helper shutdown routine in HadoopExecutor to ensure clean shutdown. Contributed by Shashikant Banerjee. 2018-04-07 16:29:01 +05:30
Wangda Tan
6cf023f9b7 Added CHANGES/RELEASES/Jdiff for 3.1.0 release 
Change-Id: Ied5067a996151c04d15cad46c46ac98b60c37b39
(cherry picked from commit 2d96570452a72569befdf9cfe9b90c9fa2e0e261)
 2018-04-05 15:52:39 -07:00
Wei Yan
3121e8c293 HDFS-13353. RBF: TestRouterWebHDFSContractCreate failed. Contributed by Takanobu Asanuma. 2018-04-05 12:00:52 -07:00
Miklos Szegedi
e52539b46f HADOOP-14855. Hadoop scripts may errantly believe a daemon is still running, preventing it from starting. Contributed by Robert Kanter. 2018-04-04 15:35:58 -07:00
Xiao Chen
1077392eaa HADOOP-15355. TestCommonConfigurationFields is broken by HADOOP-15312. Contributed by LiXin Ge. 2018-04-02 23:10:08 -07:00
Xiao Chen
57374c4737 HADOOP-15317. Improve NetworkTopology chooseRandom's loop. 2018-04-02 22:48:26 -07:00
Steve Loughran
dde1579096 HADOOP-15146. Remove DataOutputByteBuffer. 
Contributed by BELUGA BEHR.
 2018-04-02 15:52:40 +01:00
Konstantin V Shvachko
acfd764fcc HADOOP-15253. Should update maxQueueSize when refresh call queue. Contributed by Tao Jie. 2018-03-30 18:55:35 -07:00
Konstantin V Shvachko
2216bde322 HADOOP-12862. LDAP Group Mapping over SSL can not specify trust store. Contributed by Wei-Chiu Chuang and Konstantin Shvachko. 2018-03-29 17:16:23 -07:00
Wei-Chiu Chuang
3fe41c65d8 HADOOP-15312. Undocumented KeyProvider configuration keys. Contributed by LiXin Ge. 2018-03-27 13:33:41 -07:00
Xiao Chen
c22d62b338 HADOOP-15313. TestKMS should close providers. 2018-03-26 15:59:32 -07:00
Xiaoyu Yao
22194f3d21 HADOOP-15339. Support additional key/value propereties in JMX bean registration. Contributed by Elek Marton. 2018-03-26 10:45:29 -07:00
Anu Engineer
28790b81ec HADOOP-15295. Remove redundant logging related to tags from Configuration. Contributed by Ajay Kumar. 2018-03-23 21:17:18 -07:00
Lei Xu
f9427b73a2 Update releasenotes and changelogs for 3.0.1 release 2018-03-23 11:46:41 -07:00
Jitendra Pandey
4bea96f9a8 HADOOP-14067. VersionInfo should load version-info.properties from its own classloader. Contributed by Thejas M Nair. 2018-03-22 14:09:06 -07:00
Yufei Gu
268c29a5f5 HADOOP-15331. Fix a race condition causing parsing error of java.io.BufferedInputStream in class org.apache.hadoop.conf.Configuration. Contributed by Miklos Szegedi. 2018-03-22 11:04:37 -07:00
fang zhenyi
a3ed6f40e1
HADOOP-15330. Remove jdk1.7 profile from hadoop-annotations module 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-03-21 14:01:26 +09:00
Yufei Gu
138c1ed566 HADOOP-15062. TestCryptoStreamsWithOpensslAesCtrCryptoCodec fails on Debian 9. Contributed by Miklos Szegedi. 2018-03-20 15:19:18 -07:00
Inigo Goiri
fe224ff972 HDFS-13268. TestWebHdfsFileContextMainOperations fails on Windows. Contributed by Xiao Liang. 2018-03-20 11:19:46 -07:00
Inigo Goiri
e65ff1c8be HDFS-13296. GenericTestUtils generates paths with drive letter in Windows and fail webhdfs related test cases. Contributed by Xiao Liang. 2018-03-19 17:19:30 -07:00
Chris Douglas
3fc3fa9711 HADOOP-14667. Flexible Visual Studio support. Contributed by Allen Wittenauer 2018-03-19 16:05:55 -07:00
Xiao Chen
21c6661461 HADOOP-15234. Throw meaningful message on null when initializing KMSWebApp. Contributed by fang zhenyi. 2018-03-15 20:17:16 -07:00
Steve Loughran
1976e0066e HADOOP-15209. DistCp to eliminate needless deletion of files under already-deleted directories. 
Contributed by Steve Loughran.
 2018-03-15 18:05:14 +00:00
Aaron Fabbri
e840b4adf4
HADOOP-15278 log s3a at info. Contributed by Steve Loughran. 2018-03-14 15:47:56 -07:00
Wangda Tan
252c2b4d52 Revert "HADOOP-13707. If kerberos is enabled while HTTP SPNEGO is not configured, some links cannot be accessed. Contributed by Yuanbo Liu" 
Change-Id: I946a466a43d56c73bb0135384e73cb8513595461
(cherry picked from commit 80ee5248b2dda1cb8d122d4f362f2f8cf02b9467)
 2018-03-14 13:36:53 -07:00
Takanobu Asanuma
fea16a440d
HADOOP-15294. TestUGILoginFromKeytab fails on Java9 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-03-14 13:38:07 +09:00
Inigo Goiri
427fd027a3 HADOOP-15308. TestConfiguration fails on Windows because of paths. Contributed by Xiao Liang. 2018-03-13 20:01:07 -07:00
Chris Douglas
9d6994da19 HADOOP-15311. HttpServer2 needs a way to configure the acceptor/selector count. Contributed by Erik Krogen 2018-03-13 13:55:18 -07:00
Chris Douglas
45d1b0fdcc HADOOP-14696. parallel tests don't work for Windows. Contributed by Allen Wittenauer 2018-03-12 20:05:39 -07:00
Steve Loughran
dd05871b8b HADOOP-15297. Make S3A etag => checksum feature optional. 
Contributed by Steve Loughran.
 2018-03-12 14:01:42 +00:00
Owen O'Malley
fa6a8b78d4 Revert "HADOOP-13119. Add ability to secure log servlet using proxy users. Contribute by Yuanbo Liu." 
This reverts commit a847903b6e.

Change-Id: I3122a2142f5bdf8507dece930e447556a43cd9ae
(cherry picked from commit 8fad3ec76070ccfcd3ed80feaba4355077bc6f5c)
 2018-03-09 22:46:41 -08:00
Owen O'Malley
3a8dade9b1 Revert "HADOOP-14077. Add ability to access jmx via proxy. Contributed by Yuanbo Liu." 
This reverts commit 172b23af33.

(cherry picked from commit d0d2d4c51e9534e08893ae14cf3fff7b2ee70b1d)
 2018-03-09 22:46:30 -08:00
Takanobu Asanuma
99ab511cba
HADOOP-15293. TestLogLevel fails on Java 9 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-03-09 10:20:35 -08:00
Steve Loughran
3f7bd46797 HADOOP-15277. Remove .FluentPropertyBeanIntrospector from CLI operation log output. 
Contributed by Steve Loughran.
 2018-03-09 10:44:07 +00:00
Xiao Chen
a906a22645 HADOOP-15280. TestKMS.testWebHDFSProxyUserKerb and TestKMS.testWebHDFSProxyUserSimple fail in trunk. Contributed by Bharat Viswanadham. 2018-03-08 10:17:02 -08:00
Takanobu Asanuma
e6f99e205b
HADOOP-15291. TestMiniKdc fails on Java 9 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-03-06 06:01:33 -08:00
Steve Loughran
12ecb55ffe HADOOP-15289. FileStatus.readFields() assertion incorrect. 
Contributed by Steve Loughran.
 2018-03-06 10:05:12 +00:00
Takanobu Asanuma
4971276631
HADOOP-15271. Remove unicode multibyte characters from JavaDoc 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-03-05 15:07:58 -08:00
Ajay Kumar
3dc30bc24e
HADOOP-15286. Remove unused imports from TestKMSWithZK.java 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-03-04 07:12:52 -08:00
Ray Chiang
923e1773e1 HADOOP-14652. Update metrics-core version to 3.2.4. (rchiang) 2018-03-01 15:36:10 -08:00
Nanda kumar
55d04a6db1
HADOOP-15275. Incorrect javadoc for return type of RetryPolicy#shouldRetry 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-03-01 19:03:47 +09:00
Xiaoyu Yao
f20e10b2dd HDFS-13081. Datanode#checkSecureConfig should allow SASL and privileged HTTP. Contributed by Ajay Kumar. 2018-02-28 09:44:39 -08:00
Arpit Agarwal
ac42dfc888 MAPREDUCE-7061. SingleCluster setup document needs to be updated. Contributed by Bharat Viswanadham. 2018-02-27 10:27:18 -08:00
Arpit Agarwal
137f0d324a HADOOP-14959. DelegationTokenAuthenticator.authenticate() to wrap network exceptions. Contributed by Ajay Kumar. 2018-02-27 10:15:43 -08:00
Arpit Agarwal
28f644bf25 HADOOP-15178. Generalize NetUtils#wrapException to handle other subclasses with String Constructor. Contributed by Ajay Kumar. 2018-02-27 07:38:29 -08:00
Arpit Agarwal
78a10029ec HADOOP-15265. Exclude json-smart explicitly in hadoop-auth avoid being pulled in transitively. Contributed by Nishant Bangarwa. 2018-02-26 13:56:53 -08:00
Arpit Agarwal
2fa7963c3d HADOOP-15254. Correct the wrong word spelling 'intialize'. Contributed by fang zhenyi. 2018-02-24 14:41:55 -08:00
Arpit Agarwal
033f9c68ea HADOOP-15255. Upper/Lower case conversion support for group names in LdapGroupsMapping. Contributed by Nanda kumar. 2018-02-23 15:37:17 -08:00
Kihwal Lee
59cf758877 HADOOP-9747. Reduce unnecessary UGI synchronization. Contributed by Daryn Sharp. 2018-02-23 13:10:56 -06:00
Anu Engineer
3688e491d5 HADOOP-15007. Stabilize and document Configuration <tag> element. Contributed by Ajay Kumar. 2018-02-23 10:26:22 -08:00
Sean Mackrory
2bc3351eaf HADOOP-6852. apparent bug in concatenated-bzip2 support (decoding). Contributed by Zsolt Venczel. 2018-02-21 12:57:14 -07:00
Arpit Agarwal
b0d3c877e3 HADOOP-12897. KerberosAuthenticator.authenticate to include URL on IO failures. Contributed by Ajay Kumar. 2018-02-20 18:18:58 -08:00
Robert Kanter
324e5a7cf2 HADOOP-15235. Authentication Tokens should use HMAC instead of MAC (rkanter) 2018-02-20 17:24:37 -08:00
Steve Loughran
1d37cf675c HADOOP-15070. add test to verify FileSystem and paths differentiate on user info. 
Contributed by Steve Loughran.
 2018-02-19 20:43:40 +00:00
fang zhenyi
4d4dde5112
HADOOP-15223. Replace Collections.EMPTY* with empty* when available 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-02-18 22:19:39 +09:00
Wangda Tan
31db977cb3 HADOOP-14875. Create end user documentation from the compatibility guidelines. (Daniel Templeton via wangda) 
Change-Id: I5d66d0f034874623af22aecae0a0b858b65396ed
 2018-02-17 12:28:55 +08:00
Jason Lowe
0898ff42e9 HADOOP-15206. BZip2 drops and duplicates records when input split size is small. Contributed by Aki Tanaka 2018-02-16 14:49:00 -06:00
Steve Loughran
9a013b255f HADOOP-15176. Enhance IAM Assumed Role support in S3A client. 
Contributed by Steve Loughran

(cherry picked from commit 96c047fbb98c2378eed9693a724d4cbbd03c00fd)
 2018-02-15 15:57:10 +00:00
Anu Engineer
8f66affd62 HADOOP-15204. Add Configuration API for parsing storage sizes. Contributed by Anu Engineer. 2018-02-14 13:11:37 -08:00
Xiao Chen
1f20f432d2 Revert "HADOOP-12897. KerberosAuthenticator.authenticate to include URL on IO failures. Contributed by Ajay Kumar." 
This reverts commit 332269de06.
 2018-02-14 10:25:05 -08:00
Steve Loughran
f20dc0d577 HADOOP-10571. Use Log.*(Object, Throwable) overload to log exceptions. 
Contributed by Andras Bokor.
 2018-02-14 16:20:14 +00:00
Arpit Agarwal
332269de06 HADOOP-12897. KerberosAuthenticator.authenticate to include URL on IO failures. Contributed by Ajay Kumar. 2018-02-13 10:14:16 -08:00
Chris Douglas
0c5d7d71a8 HADOOP-15195. With SELinux enabled, directories mounted with start-build-env.sh may not be accessible. Contributed by Grigori Rybkine 2018-02-12 21:07:15 -08:00
Chris Douglas
9cc6d1dfb3 Revert "HADOOP-15195. With SELinux enabled, directories mounted with start-build-env.sh may not be accessible. Contributed by Grigori Rybkine" 
This reverts commit 5b88cb3398.
 2018-02-12 21:06:10 -08:00
Chris Douglas
5b88cb3398 HADOOP-15195. With SELinux enabled, directories mounted with start-build-env.sh may not be accessible. Contributed by Grigori Rybkine 2018-02-12 21:00:47 -08:00
Wangda Tan
60f9e60b3b Preparing for 3.2.0 development 
Change-Id: I6d0e01f3d665d26573ef2b957add1cf0cddf7938
 2018-02-11 11:17:38 +08:00
Steve Loughran
996796f104 HADOOP-15214. Make Hadoop compatible with Guava 21.0. 
Contributed by Igor Dvorzhak
 2018-02-08 10:55:54 -08:00
Anu Engineer
4304fcd5bd HDFS-12990. Change default NameNode RPC port back to 8020. Contributed by Xiao Chen. 2018-02-06 13:43:45 -08:00
Kihwal Lee
2dd960de98 HADOOP-15212. Add independent secret manager method for logging expired tokens. Contributed by Daryn Sharp. 2018-02-06 15:33:18 -06:00
Jason Lowe
33e6cdbd16 YARN-7876. Localized jars that are expanded after localization are not fully copied. Contributed by Miklos Szegedi 2018-02-05 12:30:20 -06:00
Hanisha Koneru
51cb6c5380 HADOOP-15168. Add kdiag tool to hadoop command. Contributed by Bharat Viswanadham. 2018-02-02 12:51:27 -08:00
Jason Lowe
460d77bd64 HADOOP-15170. Add symlink support to FileUtil#unTarUsingJava. Contributed by Ajay Kumar 2018-02-02 11:33:26 -06:00
Xiao Chen
09dd709d6e HADOOP-15197. Remove tomcat from the Hadoop-auth test bundle. 2018-02-01 15:33:52 -08:00
Xiaoyu Yao
af015c0b23 HDFS-13060. Adding a BlacklistBasedTrustedChannelResolver for TrustedChannelResolver. Contributed by Ajay Kumar. 2018-01-31 22:39:51 -08:00
Eric Yang
0bee3849e3 YARN-7816. Allow same application name submitted by multiple users. (Contributed by Gour Saha) 2018-01-31 20:51:40 -05:00
Jason Lowe
7288b08330 YARN-2185. Use pipes when localizing archives. Contributed by Miklos Szegedi 2018-01-31 09:39:43 -06:00
Jason Lowe
901d15a30b Revert "YARN-2185. Use pipes when localizing archives. Contributed by Miklos Szegedi" 
This reverts commit 1b0f265db1.
 2018-01-30 08:34:39 -06:00
Xiao Chen
b63dcd583f HDFS-12974. Exception message is not printed when creating an encryption zone fails with AuthorizationException. Contributed by fang zhenyi. 2018-01-28 22:19:49 -08:00
Haibo Chen
649ef7ac33 YARN-7064. Use cgroup to get container resource utilization. (Miklos Szegedi via Haibo Chen) 2018-01-26 16:27:31 -08:00
Jason Lowe
1b0f265db1 YARN-2185. Use pipes when localizing archives. Contributed by Miklos Szegedi 2018-01-26 13:25:20 -06:00
Chris Douglas
56872cff92 MapFile.fix creates a wrong index file in case of block-compressed data file. Contributed by Grigori Rybkine 2018-01-26 09:18:30 -08:00
Wei Yan
0c139d5bcf HDFS-13049. RBF: Inconsistent Router OPTS config in branch-2 and branch-3. Contributed by Wei Yan. 2018-01-25 11:36:40 -08:00
Billie Rinaldi
e307edcb47 YARN-7540 and YARN-7605. Convert yarn app cli to call yarn api services and implement doAs for Api Service REST API. Contributed by Eric Yang and Jian He 2018-01-23 17:54:39 -08:00
Hanisha Koneru
6347b2253d HDFS-13023. Journal Sync does not work on a secure cluster. Contributed by Bharat Viswanadham. 2018-01-22 16:02:32 -08:00
Hanisha Koneru
3fde0f1db5 HADOOP-15121. Encounter NullPointerException when using DecayRpcScheduler. Contributed by Tao Jie. 2018-01-22 15:54:44 -08:00
Masahiro Tanaka
6e27b20b4d
HADOOP-15181. Typo in SecureMode.md 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-01-22 17:12:36 +09:00
Vrushali C
c191538ed1 HADOOP-15166 CLI MiniCluster fails with ClassNotFoundException o.a.h.yarn.server.timelineservice.collector.TimelineCollectorManager. Contributed by Gera Shegalov 2018-01-19 16:15:55 -08:00
Steve Loughran
d689b2d99c HADOOP-15114. Add closeStreams(...) to IOUtils (addendum). 
Contributed by Ajay Kumar.
 2018-01-19 14:54:13 +00:00
Steve Loughran
e5a1ad6e24 HADOOP-14788. Credentials readTokenStorageFile to stop wrapping IOEs in IOEs. 
Contributed by Ajay Kumar.
 2018-01-19 14:48:45 +00:00
Steve Loughran
de630708d1 HADOOP-15123. KDiag tries to load krb5.conf from KRB5CCNAME instead of KRB5_CONFIG. 
Contributed by Vipin Rathor.

(cherry picked from commit 1ef906e29e0989aafcb35c51ad2acbb262b3c8e7)
(cherry picked from commit f61edab1d0ea08b6d752ecdfb6068103822012ec)
 2018-01-18 12:35:57 +00:00
Steve Loughran
f274fe33ea Revert "HADOOP-13974. S3Guard CLI to support list/purge of pending multipart commits." 
This reverts commit 35ad9b1dd2.
 2018-01-18 12:35:57 +00:00
Brahma Reddy Battula
08332e12d0 HADOOP-15150. in FsShell, UGI params should be overidden through env vars(-D arg). Contributed by Brahma Reddy Battula. 2018-01-18 10:54:32 +05:30
Aaron Fabbri
268ab4e027
HADOOP-15141 Support IAM Assumed roles in S3A. Contributed by Steve Loughran. 2018-01-17 00:05:24 -08:00
Yufei Gu
b202935353 HADOOP-15157. Zookeeper authentication related properties to support CredentialProviders. (Contributed by Gergo Repas) 2018-01-16 10:15:22 -08:00
Arpit Agarwal
7016dd44e0 HDFS-13016. globStatus javadoc refers to glob pattern as "regular expression". Contributed by Mukul Kumar Singh. 2018-01-13 07:19:30 -08:00
Arpit Agarwal
addbcd8cd4 HADOOP-15114. Add closeStreams(...) to IOUtils. Contributed by Ajay Kumar. 2018-01-11 16:50:21 -08:00
Steve Loughran
2e0a451a8d HADOOP-15033. Use java.util.zip.CRC32C for Java 9 and above 
Contributed by Dmitry Chuyko,
 2018-01-11 10:47:50 +00:00
Miklos Szegedi
12d0645990 HADOOP-15060. TestShellBasedUnixGroupsMapping.testFiniteGroupResolutionTime flaky. Contributed by Miklos Szegedi. 2018-01-10 10:52:26 -08:00
Haibo Chen
2f6c038be6 YARN-7602. NM should reference the singleton JvmMetrics instance. 2018-01-03 09:41:26 -08:00
Arpit Agarwal
4ad39ec308 Revert "HADOOP-10054. ViewFsFileStatus.toString() is broken. Contributed by Hanisha Koneru." 
This reverts commit 37efa67e37.
 2018-01-02 17:34:29 -08:00
Arpit Agarwal
d82874851e HADOOP-15155. Error in javadoc of ReconfigurableBase#reconfigureProperty. Contributed by Ajay Kumar. 2018-01-02 17:22:55 -08:00
Arpit Agarwal
dfe0cd8655 HADOOP-15152. Typo in javadoc of ReconfigurableBase#reconfigurePropertyImpl. Contributed by Nanda kumar. 2018-01-02 10:50:13 -08:00
Chris Douglas
7fe6f83c8f HADOOP-15117. open(PathHandle) contract test should be exhaustive for default options 2017-12-30 17:58:35 -08:00
Xiao Chen
81127616c5 HADOOP-15149. CryptoOutputStream should implement StreamCapabilities. 2017-12-29 13:41:15 -08:00
Jitendra Pandey
d31c9d8c49 HADOOP-15143. NPE due to Invalid KerberosTicket in UGI. Contributed by Mukul Kumar Singh. 2017-12-27 23:17:07 -08:00
Steve Loughran
c8ff0cc304 HADOOP-13282. S3 blob etags to be made visible in S3A status/getFileChecksum() calls. 
Contributed by Steve Loughran
 2017-12-21 14:58:58 +00:00
Junping Du
a7f8caf58e Add 2.8.3 release jdiff files. 
(cherry picked from commit c89f99aade575ab1f6a9836df719cda272293d90)
 2017-12-18 22:39:01 -08:00
Steve Loughran
35ad9b1dd2 HADOOP-13974. S3Guard CLI to support list/purge of pending multipart commits. 
Contributed by Aaron Fabbri
 2017-12-18 21:19:06 +00:00
Chris Douglas
5e81f32d11 HADOOP-15106. FileSystem::open(PathHandle) should throw a specific exception on validation failure 2017-12-16 10:53:10 -08:00
Virajith Jalaparti
b668eb9155 HDFS-10675. Datanode support to read from external stores. 2017-12-15 17:51:37 -08:00
Xiaoyu Yao
37efa67e37 HADOOP-10054. ViewFsFileStatus.toString() is broken. Contributed by Hanisha Koneru. 2017-12-14 10:58:16 -08:00
Jason Lowe
f8af0e2feb HADOOP-15085. Output streams closed with IOUtils suppressing write errors. Contributed by Jim Brennan 2017-12-14 10:18:08 -06:00
Yiqun Lin
91c96bdf8e HDFS-12883. RBF: Document Router and State Store metrics. Contributed by Yiqun Lin. 2017-12-14 16:00:53 +08:00
Kai Zheng
f86c81d923 HADOOP-15039. Move SemaphoredDelegatingExecutor to hadoop-common. Contributed by Genmao Yu 2017-12-14 11:14:52 +08:00
Andrew Wang
d447152d49 Update CHANGES, RELEASENOTES, jdiff for 3.0.0 release. 2017-12-13 13:05:32 -08:00
Chris Douglas
693169ef34 HDFS-12882. Support full open(PathHandle) contract in HDFS 2017-12-11 20:14:15 -08:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
f19638333b HADOOP-15059. Undoing the switch of Credentials to PB format as default - done via HADOOP-12563 for supporting 2.x to 3.x upgrades. 2017-12-08 08:00:21 -08:00
Xiao Chen
19e0894209 HADOOP-15056. Fix TestUnbuffer#testUnbufferException failure. Contributed by Jack Bearden. 2017-12-07 21:05:55 -08:00
Sunil G
daa1cdd062 YARN-7564. Cleanup to fix checkstyle issues of YARN-5881 branch. Contributed by Sunil G. 2017-12-07 18:56:54 -08:00
Wangda Tan
5e798b1a0d YARN-6471. Support to add min/max resource configuration for a queue. (Sunil G via wangda) 
Change-Id: I9213f5297a6841fab5c573e85ee4c4e5f4a0b7ff
 2017-12-07 18:56:54 -08:00
Sean Mackrory
acb92904d0 HADOOP-15098. TestClusterTopology#testChooseRandom fails intermittently. Contributed by Zsolt Venczel. 2017-12-07 12:30:58 -07:00
Lei Xu
67662e2ac9 HDFS-12840. Creating a file with non-default EC policy in a EC zone is not correctly serialized in the editlog. Contributed by Lei (Eddy) Xu. 2017-12-07 11:15:40 -08:00
Arpit Agarwal
e00c7f78c1 HADOOP-14976. Set HADOOP_SHELL_EXECNAME explicitly in scripts. 2017-12-04 21:02:04 -08:00
Xiao Chen
404eab4dc0 HDFS-12396. Webhdfs file system should get delegation token from kms provider. Contributed by Rushabh S Shah. 2017-12-04 10:40:11 -08:00
Chris Douglas
42307e3c3a HDFS-11576. Block recovery will fail indefinitely if recovery time > heartbeat interval. Contributed by Lukas Majercak 2017-12-01 22:34:30 -08:00
Chris Douglas
f9d195dfe9 HADOOP-14600. LocatedFileStatus constructor forces RawLocalFS to exec a process to get the permissions. Contributed by Ping Liu 2017-12-01 22:12:26 -08:00
Chris Douglas
53bbef3802 Revert "HDFS-11576. Block recovery will fail indefinitely if recovery time > heartbeat interval. Contributed by Lukas Majercak" 
This reverts commit 5304698dc8.
 2017-12-01 11:19:38 -08:00
Chris Douglas
5304698dc8 HDFS-11576. Block recovery will fail indefinitely if recovery time > heartbeat interval. Contributed by Lukas Majercak 2017-12-01 10:29:30 -08:00
Chris Douglas
0780fdb1eb HDFS-12877. Add open(PathHandle) with default buffersize 2017-11-30 15:13:16 -08:00
Robert Kanter
75a3ab88f5 HADOOP-13493. Compatibility Docs should clarify the policy for what takes precedence when a conflict is found (templedf via rkanter) 2017-11-30 07:39:15 -08:00
Chris Douglas
0e560f3b8d HDFS-12681. Make HdfsLocatedFileStatus a subtype of LocatedFileStatus 2017-11-29 20:28:06 -08:00
Xiao Chen
d162252d7a HADOOP-15067. GC time percentage reported in JvmMetrics should be a gauge, not counter. Contributed by Misha Dmitriev. 2017-11-23 09:01:28 -08:00
Steve Loughran
de8b6ca5ef HADOOP-13786 Add S3A committer for zero-rename commits to S3 endpoints. 
Contributed by Steve Loughran and Ryan Blue.
 2017-11-22 15:28:12 +00:00
Robert Kanter
f2efaf013f HADOOP-14982. Clients using FailoverOnNetworkExceptionRetry can go into a loop if they're used without authenticating with kerberos in HA env (pbacsko via rkanter) 2017-11-16 11:11:19 -08:00
Chris Douglas
675e9a8f57 Revert "HDFS-12681. Fold HdfsLocatedFileStatus into HdfsFileStatus." 
This reverts commit b85603e3f8.
 2017-11-15 19:20:37 -08:00
Xiao Chen
b1941b200d HADOOP-15023. ValueQueue should also validate (int) (lowWatermark * numValues) > 0 on construction. 2017-11-15 16:44:06 -08:00
Daniel Templeton
69043ba8b5 HADOOP-14876. Create downstream developer docs from the compatibility guidelines 2017-11-15 10:03:29 -08:00
Arun Suresh
b07e68b02a Addendum patch for Configuration fix. (Jason Lowe via asuresh) 2017-11-13 14:03:50 -08:00
Eric Yang
1b68b8ff2c HADOOP-15008. Fixed period unit calculation for Hadoop Metrics V2. (Contribute by Erik Krogen) 2017-11-13 12:42:43 -05:00
Akira Ajisaka
975a57a688
HADOOP-15031. Fix javadoc issues in Hadoop Common. Contributed by Mukul Kumar Singh. 2017-11-13 23:12:23 +09:00
Chris Douglas
796a0d3a5c HADOOP-8522. ResetableGzipOutputStream creates invalid gzip files when finish() and resetState() are used. Contributed by Mike Percy 2017-11-10 17:41:29 -08:00
Xiao Chen
3c6adda291 HADOOP-14960. Add GC time percentage monitor/alerter. Contributed by Misha Dmitriev. 2017-11-09 21:06:06 -08:00
Arun Suresh
5eb7dbe9b3 Fixing Job History Server Configuration parsing. (Jason Lowe via asuresh) 2017-11-09 15:15:51 -08:00
John Zhuge
6c32ddad30 HADOOP-14872. CryptoInputStream should implement unbuffer. Contributed by John Zhuge. 2017-11-09 10:16:12 -08:00
John Zhuge
bf6a660232 HADOOP-15012. Add readahead, dropbehind, and unbuffer to StreamCapabilities. Contributed by John Zhuge. 2017-11-09 10:16:12 -08:00
Mingliang Liu
b6d8f87b8e HADOOP-15015. TestConfigurationFieldsBase to use SLF4J for logging. Contributed by Steve Loughran 2017-11-03 21:01:56 -07:00
Xiaoyu Yao
59d78a5088 HADOOP-14987. Improve KMSClientProvider log around delegation token checking. Contributed by Xiaoyu Yao and Xiao Chen. 2017-11-03 16:10:37 -07:00
Chris Douglas
b85603e3f8 HDFS-12681. Fold HdfsLocatedFileStatus into HdfsFileStatus. 2017-11-03 14:30:57 -07:00
Chris Douglas
d015e0bbd5 HDFS-7878. API - expose a unique file identifier. 2017-10-31 09:44:01 -07:00
Sean Mackrory
81225430d0 HDFS-206. Support for head in FSShell. Contributed by Gabor Bota. 2017-10-31 08:34:01 -06:00
Jason Lowe
2fae63aa60 HADOOP-14919. BZip2 drops records when reading data in splits. Contributed by Jason Lowe 2017-10-31 09:30:13 -05:00
Junping Du
a25b5aa0cf Add several jdiff xml files for 2.8.2 release. 2017-10-26 19:30:48 -07:00
Junping Du
1ad863e220 Update CHANGES.md and RELEASENOTES for 2.8.2 release. 2017-10-26 19:13:58 -07:00
Xiao Chen
f66ad1fab9 HADOOP-14957. ReconfigurationTaskStatus is exposing guava Optional in its public api. Contributed by Haibo Chen, Xiao Chen. 2017-10-25 14:11:30 -07:00
Akira Ajisaka
4c970088af
HADOOP-14030. PreCommit TestKDiag failure. Contributed by Wei-Chiu Chuang. 2017-10-25 17:17:21 +09:00
Varun Saxena
67e7673750 HADOOP-9657. NetUtils.wrapException to have special handling for 0.0.0.0 addresses and :0 ports. Contributed by Varun Saxena. 2017-10-25 03:06:12 +05:30
John Zhuge
c904d60c38 Revert "HADOOP-14954. MetricsSystemImpl#init should increment refCount when already initialized. Contributed by John Zhuge." 
This reverts commit 14b3c2695b.
 2017-10-23 15:32:25 -07:00
Naganarasimha
6b8122458e HADOOP-14966. Handle JDK-8071638 for hadoop-common. Contributed by Bibin A Chundatt. 2017-10-23 23:42:06 +05:30
Arpit Agarwal
4b00c9a47f HDFS-12683. DFSZKFailOverController re-order logic for logging Exception. Contributed by Bharat Viswanadham. 2017-10-23 10:47:16 -07:00
John Zhuge
14b3c2695b HADOOP-14954. MetricsSystemImpl#init should increment refCount when already initialized. Contributed by John Zhuge. 2017-10-20 16:13:41 -07:00
Kai Zheng
ce7cf66e5e HDFS-12448. Make sure user defined erasure coding policy ID will not overflow. Contributed by Huafeng Wang 2017-10-20 09:42:04 +08:00
Wei-Chiu Chuang
97c70c7ac6 HADOOP-14880. [KMS] Document&test missing KMS client side configs. Contributed by Gabor Bota. 2017-10-19 06:02:13 -07:00
Junping Du
b016f08f67 HADOOP-14958. Fix source-level compatibility after HADOOP-11252. Contributed by Junping Du. 2017-10-18 15:06:30 -07:00
Xiao Chen
86ee0c5e4e HADOOP-14944. Add JvmMetrics to KMS. 2017-10-17 19:06:45 -07:00
Lei Xu
31ebccc962 HDFS-12613. Native EC coder should implement release() as idempotent function. (Lei (Eddy) Xu) 2017-10-16 19:44:30 -07:00
Wei-Chiu Chuang
e906108fc9 HADOOP-14948. Document missing config key hadoop.treat.subject.external. Contributed by Ajay Kumar. 2017-10-16 16:42:59 -07:00
Xiao Chen
b7ff624c76 HADOOP-14949. TestKMS#testACLs fails intermittently. 2017-10-16 14:19:31 -07:00
Manoj Govindassamy
133d7ca76e HADOOP-13055. Implement linkMergeSlash and linkFallback for ViewFileSystem 2017-10-13 17:43:21 -07:00
Robert Kanter
e163f41850 HADOOP-14938. Configuration.updatingResource map should be initialized lazily (misha@cloudera.com via rkanter) 2017-10-13 13:52:58 -07:00
Anu Engineer
075358eb6f HADOOP-13102. Update GroupsMapping documentation to reflect the new changes. Contributed by Esther Kundin. 2017-10-11 15:58:20 -07:00
Arun Suresh
b6c2c9058e HADOOP-13556. Change Configuration.getPropsWithPrefix to use getProps instead of iterator. (Larry McCay via asuresh) 2017-10-11 15:21:21 -07:00