big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
20,753 Commits 2 Branches 2 Tags 264 MiB
76efeacd5f
Commit Graph

1859 Commits

Author SHA1 Message Date
Eric Yang
77721f39e2 HADOOP-15593. Fixed NPE in UGI spawnAutoRenewalThreadForUserCreds. 
Contributed by Gabor Bota
 2018-07-26 18:35:36 -04:00
Mukul Kumar Singh
5f0b924360 HADOOP-15395. DefaultImpersonationProvider fails to parse proxy user config if username has . in it. Contributed by Ajay Kumar. 2018-07-25 21:09:11 +05:30
Xiao Chen
81d59506e5 HADOOP-15609. Retry KMS calls when SSLHandshakeException occurs. Contributed by Kitti Nanasi. 2018-07-24 21:45:43 -07:00
Gera Shegalov
6bec03cfc8 HADOOP-15612. Improve exception when tfile fails to load LzoCodec. (gera) 2018-07-24 14:32:30 -07:00
Kihwal Lee
ccf2db7fc2 HADOOP-15614. TestGroupsCaching.testExceptionOnBackgroundRefreshHandled reliably fails. Contributed by Weiwei Yang. 2018-07-19 11:13:37 -05:00
Wei-Chiu Chuang
d215357718 HDFS-13485. DataNode WebHDFS endpoint throws NPE. Contributed by Siyao Meng. 2018-07-16 15:45:55 -07:00
Sean Mackrory
4f3f9391b0 HADOOP-15316. GenericTestUtils can exceed maxSleepTime. Contributed by Adam Antal. 2018-07-12 17:24:01 +02:00
Xiaoyu Yao
498e3bfb6b HADOOP-15571. Multiple FileContexts created with the same configuration object should be allowed to have different umask. Contributed by Vinod Kumar Vavilapalli. 2018-07-06 11:56:09 -07:00
Eric E Payne
d36f6b9e93 HADOOP-15548: Randomize local dirs. Contributed by Jim Brennan. 2018-06-29 18:18:32 +00:00
Akira Ajisaka
2b2399d623
HADOOP-15495. Upgrade commons-lang version to 3.7 in hadoop-common-project and hadoop-tools. Contributed by Takanobu Asanuma. 2018-06-28 14:37:22 +09:00
Inigo Goiri
abc3e4bad9 HADOOP-15458. TestLocalFileSystem#testFSOutputStreamBuilder fails on Windows. Contributed by Xiao Liang. 2018-06-25 09:50:27 -07:00
Steve Loughran
1ba4e62304
HADOOP-14396. Add builder interface to FileContext. 
Contributed by  Lei (Eddy) Xu.
 2018-06-25 14:38:57 +01:00
Eric E Payne
6432128622 YARN-8444: NodeResourceMonitor crashes on bad swapFree value. Contributed by Jim Brennan. 2018-06-22 17:15:29 +00:00
Eric Yang
2d87592fc6 HADOOP-15527. Improve delay check for stopping processes. 
Fixed script location.
               Contributed by Vinod Kumar Vavilapalli
 2018-06-19 13:38:13 -04:00
Chris Douglas
980031bb04 HADOOP-13186. Multipart Uploader API. Contributed by Ewan Higgs 2018-06-17 11:54:26 -07:00
Surendra Singh Lilhore
3905fdb793 HADOOP-15523. Shell command timeout given is in seconds whereas it is taken as millisec while scheduling. Contributed by Bilwa S T. 2018-06-17 12:12:01 +05:30
Yiqun Lin
8e7548d33b HDFS-13641. Add metrics for edit log tailing. Contributed by Chao Sun. 2018-06-13 20:05:55 +08:00
Eric Yang
108da85320 HADOOP-15527. Improve delay check for stopping processes. 
Contributed by Vinod Kumar Vavilapalli
 2018-06-12 20:40:32 -04:00
Inigo Goiri
aeaf9fec62 HADOOP-15532. TestBasicDiskValidator fails with NoSuchFileException. Contributed by Giovanni Matteo Fumarola. 2018-06-12 14:16:14 -07:00
Sammi Chen
18201b882a HADOOP-15499. Performance severe drops when running RawErasureCoderBenchmark with NativeRSRawErasureCoder. Contributed by Sammi Chen. 2018-06-11 13:53:37 +08:00
Steve Loughran
ef0118b91e
HADOOP-15520. Add tests for various org.apache.hadoop.util classes. 
Contributed by Arash Nabili
 2018-06-09 15:33:38 +01:00
Inigo Goiri
12be8bad7d HADOOP-15516. Add test cases to cover FileUtil#readLink. Contributed by Giovanni Matteo Fumarola. 2018-06-07 13:34:52 -07:00
Inigo Goiri
db81f85e84 HADOOP-15513. Add additional test cases to cover some corner cases for FileUtil#symlink. Contributed by Giovanni Matteo Fumarola. 2018-06-06 11:44:17 -07:00
Chen Liang
1b0d4f4606 HDFS-13547. Add ingress port based sasl resolver. Contributed by Chen Liang. 2018-06-05 11:51:29 -07:00
Xiao Chen
6d5e87aec2 HADOOP-15507. Add MapReduce counters about EC bytes read. 2018-06-04 21:13:47 -07:00
Haibo Chen
04cf699dd5 YARN-8388. TestCGroupElasticMemoryController.testNormalExit() hangs on Linux. (Miklos Szegedi via Haibo Chen) 2018-06-04 14:23:08 -07:00
Haibo Chen
4880d890ee YARN-8375. TestCGroupElasticMemoryController fails surefire build. (Miklos Szegedi via Haibo Chen) 2018-06-01 16:29:13 -07:00
Takanobu Asanuma
d1e2b80980
MAPREDUCE-7098. Upgrade common-langs version to 3.7 in hadoop-mapreduce-project 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-05-31 18:31:10 +09:00
Kihwal Lee
e3236a9680 Additional check when unpacking archives. Contributed by Wilfred Spiegelenburg. 2018-05-29 14:15:12 -05:00
Inigo Goiri
3c75f8e493 HADOOP-15497. TestTrash should use proper test path to avoid failing on Windows. Contributed by Anbang Hu. 2018-05-29 09:11:08 -07:00
Johan Gustavsson
d14e26b31f
HADOOP-15477. Make unjar in RunJar overrideable 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-05-28 17:29:59 +09:00
Steve Loughran
bddfe796f2 HADOOP-15494. TestRawLocalFileSystemContract fails on Windows. 
Contributed by Anbang Hu.
 2018-05-25 11:12:47 +01:00
Robert Kanter
aa23d49fc8 HADOOP-15457. Add Security-Related HTTP Response Header in WEBUIs. (kanwaljeets via rkanter) 2018-05-23 10:24:09 -07:00
Akira Ajisaka
745f203e57
Additional check when unpacking archives. Contributed by Jason Lowe and Akira Ajisaka. 2018-05-23 17:16:23 +09:00
Kihwal Lee
bcc8e76bad HADOOP-15450. Avoid fsync storm triggered by DiskChecker and handle disk full situation. Contributed by Arpit Agarwal. 2018-05-22 11:20:51 -05:00
Mukul Kumar Singh
57c2feb0d3 HADOOP-15474. Rename properties introduced for <tags>. Contributed by Zsolt Venczel. 2018-05-22 13:33:31 +05:30
Xiao Chen
89f5911392 HADOOP-15154. Abstract new method assertCapability for StreamCapabilities testing. Contributed by Zsolt Venczel. 2018-05-18 11:29:20 -07:00
Inigo Goiri
1da8d4190d HADOOP-15454. TestRollingFileSystemSinkWithLocal fails on Windows. Contributed by Xiao Liang. 2018-05-10 09:41:16 -07:00
Xiao Chen
a3a1552c33 Revert "HADOOP-14445. Delegation tokens are not shared between KMS instances. Contributed by Xiao Chen and Rushabh S Shah." 
This reverts commit 583fa6ed48.

 Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/kms/KMSTokenRenewer.java
	hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/TestKMS.java
 2018-05-07 13:32:27 -07:00
Daniel Templeton
87c23ef643 HDFS-13481. TestRollingFileSystemSinkWithHdfs#testFlushThread: test failed intermittently 
(Contributed by Gabor Bota via Daniel Templeton)

Change-Id: I9921981dfa69669fe7912dd2a31ae8b638283204
 2018-05-02 17:13:40 -07:00
Steve Loughran
8f42dafcf8 HADOOP-15250. Split-DNS MultiHomed Server Network Cluster Network IPC Client Bind Addr Wrong 
Contributed by Ajay Kumar
 2018-05-01 22:32:40 +01:00
Ewan Higgs
84ecfe3ceb
HADOOP-14188. Remove the usage of org.mockito.internal.util.reflection.Whitebox 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-04-27 19:21:37 +09:00
Owen O'Malley
2adda92de1
HDFS-7240 git commit fixup 
Signed-off-by: Owen O'Malley <omalley@apache.org>
 2018-04-26 13:52:17 -07:00
Anu Engineer
0b3270a8de HDFS-12513. Ozone: Create UI page to show Ozone configs by tag. Contributed by Ajay Kumar. 2018-04-26 05:36:04 -07:00
Aaron Fabbri
0d898b7bb8
HADOOP-12502 SetReplication OutOfMemoryError. Contributed by Vinayakumar B. 2018-04-11 17:19:56 -07:00
Xiao Chen
7c9cdad6d0 HDFS-13056. Expose file-level composite CRCs in HDFS which are comparable across different instances/layouts. Contributed by Dennis Huo. 2018-04-10 21:31:48 -07:00
Xiao Chen
583fa6ed48 HADOOP-14445. Delegation tokens are not shared between KMS instances. Contributed by Xiao Chen and Rushabh S Shah. 2018-04-10 15:38:25 -07:00
Jason Lowe
e81397545a HADOOP-15357. Configuration.getPropsWithPrefix no longer does variable substitution. Contributed by Jim Brennan 2018-04-10 16:44:55 -05:00
Xiaoyu Yao
8ab776d61e HADOOP-15340. Provide meaningful RPC server name for RpcMetrics. Contributed by Elek Marton. 2018-04-10 11:42:54 -07:00
Ewan Higgs
cef8eb7981
HADOOP-15376. Remove double semi colons on imports that make Clover fall over. 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-04-10 23:58:26 +09:00
Wei Yan
3121e8c293 HDFS-13353. RBF: TestRouterWebHDFSContractCreate failed. Contributed by Takanobu Asanuma. 2018-04-05 12:00:52 -07:00
Steve Loughran
dde1579096 HADOOP-15146. Remove DataOutputByteBuffer. 
Contributed by BELUGA BEHR.
 2018-04-02 15:52:40 +01:00
Xiaoyu Yao
22194f3d21 HADOOP-15339. Support additional key/value propereties in JMX bean registration. Contributed by Elek Marton. 2018-03-26 10:45:29 -07:00
Anu Engineer
28790b81ec HADOOP-15295. Remove redundant logging related to tags from Configuration. Contributed by Ajay Kumar. 2018-03-23 21:17:18 -07:00
Yufei Gu
268c29a5f5 HADOOP-15331. Fix a race condition causing parsing error of java.io.BufferedInputStream in class org.apache.hadoop.conf.Configuration. Contributed by Miklos Szegedi. 2018-03-22 11:04:37 -07:00
Inigo Goiri
fe224ff972 HDFS-13268. TestWebHdfsFileContextMainOperations fails on Windows. Contributed by Xiao Liang. 2018-03-20 11:19:46 -07:00
Inigo Goiri
e65ff1c8be HDFS-13296. GenericTestUtils generates paths with drive letter in Windows and fail webhdfs related test cases. Contributed by Xiao Liang. 2018-03-19 17:19:30 -07:00
Steve Loughran
1976e0066e HADOOP-15209. DistCp to eliminate needless deletion of files under already-deleted directories. 
Contributed by Steve Loughran.
 2018-03-15 18:05:14 +00:00
Wangda Tan
252c2b4d52 Revert "HADOOP-13707. If kerberos is enabled while HTTP SPNEGO is not configured, some links cannot be accessed. Contributed by Yuanbo Liu" 
Change-Id: I946a466a43d56c73bb0135384e73cb8513595461
(cherry picked from commit 80ee5248b2dda1cb8d122d4f362f2f8cf02b9467)
 2018-03-14 13:36:53 -07:00
Inigo Goiri
427fd027a3 HADOOP-15308. TestConfiguration fails on Windows because of paths. Contributed by Xiao Liang. 2018-03-13 20:01:07 -07:00
Chris Douglas
9d6994da19 HADOOP-15311. HttpServer2 needs a way to configure the acceptor/selector count. Contributed by Erik Krogen 2018-03-13 13:55:18 -07:00
Chris Douglas
45d1b0fdcc HADOOP-14696. parallel tests don't work for Windows. Contributed by Allen Wittenauer 2018-03-12 20:05:39 -07:00
Owen O'Malley
fa6a8b78d4 Revert "HADOOP-13119. Add ability to secure log servlet using proxy users. Contribute by Yuanbo Liu." 
This reverts commit a847903b6e.

Change-Id: I3122a2142f5bdf8507dece930e447556a43cd9ae
(cherry picked from commit 8fad3ec76070ccfcd3ed80feaba4355077bc6f5c)
 2018-03-09 22:46:41 -08:00
Owen O'Malley
3a8dade9b1 Revert "HADOOP-14077. Add ability to access jmx via proxy. Contributed by Yuanbo Liu." 
This reverts commit 172b23af33.

(cherry picked from commit d0d2d4c51e9534e08893ae14cf3fff7b2ee70b1d)
 2018-03-09 22:46:30 -08:00
Takanobu Asanuma
99ab511cba
HADOOP-15293. TestLogLevel fails on Java 9 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-03-09 10:20:35 -08:00
Arpit Agarwal
28f644bf25 HADOOP-15178. Generalize NetUtils#wrapException to handle other subclasses with String Constructor. Contributed by Ajay Kumar. 2018-02-27 07:38:29 -08:00
Arpit Agarwal
033f9c68ea HADOOP-15255. Upper/Lower case conversion support for group names in LdapGroupsMapping. Contributed by Nanda kumar. 2018-02-23 15:37:17 -08:00
Kihwal Lee
59cf758877 HADOOP-9747. Reduce unnecessary UGI synchronization. Contributed by Daryn Sharp. 2018-02-23 13:10:56 -06:00
Anu Engineer
3688e491d5 HADOOP-15007. Stabilize and document Configuration <tag> element. Contributed by Ajay Kumar. 2018-02-23 10:26:22 -08:00
Arpit Agarwal
b0d3c877e3 HADOOP-12897. KerberosAuthenticator.authenticate to include URL on IO failures. Contributed by Ajay Kumar. 2018-02-20 18:18:58 -08:00
Steve Loughran
1d37cf675c HADOOP-15070. add test to verify FileSystem and paths differentiate on user info. 
Contributed by Steve Loughran.
 2018-02-19 20:43:40 +00:00
Steve Loughran
9a013b255f HADOOP-15176. Enhance IAM Assumed Role support in S3A client. 
Contributed by Steve Loughran

(cherry picked from commit 96c047fbb98c2378eed9693a724d4cbbd03c00fd)
 2018-02-15 15:57:10 +00:00
Anu Engineer
8f66affd62 HADOOP-15204. Add Configuration API for parsing storage sizes. Contributed by Anu Engineer. 2018-02-14 13:11:37 -08:00
Steve Loughran
f20dc0d577 HADOOP-10571. Use Log.*(Object, Throwable) overload to log exceptions. 
Contributed by Andras Bokor.
 2018-02-14 16:20:14 +00:00
Chris Douglas
0c5d7d71a8 HADOOP-15195. With SELinux enabled, directories mounted with start-build-env.sh may not be accessible. Contributed by Grigori Rybkine 2018-02-12 21:07:15 -08:00
Chris Douglas
9cc6d1dfb3 Revert "HADOOP-15195. With SELinux enabled, directories mounted with start-build-env.sh may not be accessible. Contributed by Grigori Rybkine" 
This reverts commit 5b88cb3398.
 2018-02-12 21:06:10 -08:00
Chris Douglas
5b88cb3398 HADOOP-15195. With SELinux enabled, directories mounted with start-build-env.sh may not be accessible. Contributed by Grigori Rybkine 2018-02-12 21:00:47 -08:00
Anu Engineer
4304fcd5bd HDFS-12990. Change default NameNode RPC port back to 8020. Contributed by Xiao Chen. 2018-02-06 13:43:45 -08:00
Jason Lowe
33e6cdbd16 YARN-7876. Localized jars that are expanded after localization are not fully copied. Contributed by Miklos Szegedi 2018-02-05 12:30:20 -06:00
Jason Lowe
460d77bd64 HADOOP-15170. Add symlink support to FileUtil#unTarUsingJava. Contributed by Ajay Kumar 2018-02-02 11:33:26 -06:00
Chris Douglas
56872cff92 MapFile.fix creates a wrong index file in case of block-compressed data file. Contributed by Grigori Rybkine 2018-01-26 09:18:30 -08:00
Hanisha Koneru
3fde0f1db5 HADOOP-15121. Encounter NullPointerException when using DecayRpcScheduler. Contributed by Tao Jie. 2018-01-22 15:54:44 -08:00
Steve Loughran
d689b2d99c HADOOP-15114. Add closeStreams(...) to IOUtils (addendum). 
Contributed by Ajay Kumar.
 2018-01-19 14:54:13 +00:00
Steve Loughran
e5a1ad6e24 HADOOP-14788. Credentials readTokenStorageFile to stop wrapping IOEs in IOEs. 
Contributed by Ajay Kumar.
 2018-01-19 14:48:45 +00:00
Brahma Reddy Battula
08332e12d0 HADOOP-15150. in FsShell, UGI params should be overidden through env vars(-D arg). Contributed by Brahma Reddy Battula. 2018-01-18 10:54:32 +05:30
Yufei Gu
b202935353 HADOOP-15157. Zookeeper authentication related properties to support CredentialProviders. (Contributed by Gergo Repas) 2018-01-16 10:15:22 -08:00
Arpit Agarwal
addbcd8cd4 HADOOP-15114. Add closeStreams(...) to IOUtils. Contributed by Ajay Kumar. 2018-01-11 16:50:21 -08:00
Steve Loughran
2e0a451a8d HADOOP-15033. Use java.util.zip.CRC32C for Java 9 and above 
Contributed by Dmitry Chuyko,
 2018-01-11 10:47:50 +00:00
Haibo Chen
2f6c038be6 YARN-7602. NM should reference the singleton JvmMetrics instance. 2018-01-03 09:41:26 -08:00
Chris Douglas
7fe6f83c8f HADOOP-15117. open(PathHandle) contract test should be exhaustive for default options 2017-12-30 17:58:35 -08:00
Xiao Chen
81127616c5 HADOOP-15149. CryptoOutputStream should implement StreamCapabilities. 2017-12-29 13:41:15 -08:00
Jitendra Pandey
d31c9d8c49 HADOOP-15143. NPE due to Invalid KerberosTicket in UGI. Contributed by Mukul Kumar Singh. 2017-12-27 23:17:07 -08:00
Steve Loughran
c8ff0cc304 HADOOP-13282. S3 blob etags to be made visible in S3A status/getFileChecksum() calls. 
Contributed by Steve Loughran
 2017-12-21 14:58:58 +00:00
Chris Douglas
5e81f32d11 HADOOP-15106. FileSystem::open(PathHandle) should throw a specific exception on validation failure 2017-12-16 10:53:10 -08:00
Virajith Jalaparti
b668eb9155 HDFS-10675. Datanode support to read from external stores. 2017-12-15 17:51:37 -08:00
Jason Lowe
f8af0e2feb HADOOP-15085. Output streams closed with IOUtils suppressing write errors. Contributed by Jim Brennan 2017-12-14 10:18:08 -06:00
Chris Douglas
693169ef34 HDFS-12882. Support full open(PathHandle) contract in HDFS 2017-12-11 20:14:15 -08:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
f19638333b HADOOP-15059. Undoing the switch of Credentials to PB format as default - done via HADOOP-12563 for supporting 2.x to 3.x upgrades. 2017-12-08 08:00:21 -08:00
Sean Mackrory
acb92904d0 HADOOP-15098. TestClusterTopology#testChooseRandom fails intermittently. Contributed by Zsolt Venczel. 2017-12-07 12:30:58 -07:00
Arpit Agarwal
e00c7f78c1 HADOOP-14976. Set HADOOP_SHELL_EXECNAME explicitly in scripts. 2017-12-04 21:02:04 -08:00