big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
20,438 Commits 2 Branches 2 Tags 264 MiB
97bd49fc36
Commit Graph

4928 Commits

Author SHA1 Message Date
Thomas Marquardt
ce03a93f78 HADOOP-15446. ABFS: tune imports & javadocs; stabilise tests. 
Contributed by Steve Loughran and Da Zhou.
 2018-09-17 19:54:01 +00:00
Steve Loughran
f044deedbb HADOOP-15407. HADOOP-15540. Support Windows Azure Storage - Blob file system "ABFS" in Hadoop: Core Commit. 
Contributed by Shane Mainali, Thomas Marquardt, Zichen Sun, Georgi Chalakov, Esfandiar Manii, Amit Singh, Dana Kaban, Da Zhou, Junhua Gu, Saher Ahwal, Saurabh Pant, James Baker, Shaoyu Zhang, Lawrence Chen, Kevin Chen and Steve Loughran
 2018-09-17 19:54:01 +00:00
Brahma Reddy Battula
ef5c776a42 HADOOP-15733. Correct the log when Invalid emptier Interval configured. Contributed by Ayush Saxena 2018-09-14 07:32:27 +05:30
Hanisha Koneru
291dcf2247 HDDS-423. Introduce an ozone specific log4j.properties. Contributed by Elek, Marton. 2018-09-13 16:37:44 -07:00
Takanobu Asanuma
5d084d7eca HADOOP-12760. sun.misc.Cleaner has moved to a new location in OpenJDK 9. Contributed by Akira Ajisaka. 2018-09-13 20:42:09 +09:00
Steve Loughran
d7c0a08a1c
HADOOP-15426 Make S3guard client resilient to DDB throttle events and network failures (Contributed by Steve Loughran) 2018-09-12 21:04:49 -07:00
bibinchundatt
c44088ac19 YARN-8739. Fix jenkins issues for Node Attributes branch. Contributed by Sunil Govindan. 2018-09-12 16:01:01 +05:30
Naganarasimha
2475fb0a1e YARN-6856. [YARN-3409] Support CLI for Node Attributes Mapping. Contributed by Naganarasimha G R. 2018-09-12 16:00:59 +05:30
Mingliang Liu
87f63b6479 HADOOP-14833. Remove s3a user:secret authentication. Contributed by Steve Loughran 2018-09-11 17:18:42 -07:00
Allen Wittenauer
4b8c2b1c1c
HADOOP-14671. Upgrade Apache Yetus to 0.8.0 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-09-11 08:55:34 -07:00
Inigo Goiri
527288ef89 HDFS-13836. RBF: Handle mount table znode with null value. Contributed by yanghuafeng. 2018-09-06 16:47:54 -07:00
Vrushali C
962089ab59 HADOOP-15657 Registering MutableQuantiles via Metric annotation. Contributed by Sushil Ks 2018-09-05 20:53:21 -07:00
Giovanni Matteo Fumarola
9af96d4ed4 HADOOP-15707. Add IsActiveServlet to be used for Load Balancers. Contributed by Lukas Majercak. 2018-09-05 10:50:25 -07:00
Wei-Chiu Chuang
e780556ae9 HADOOP-15696. KMS performance regression due to too many open file descriptors after Jetty migration. Contributed by Wei-Chiu Chuang. 2018-09-05 09:52:35 -07:00
Steve Loughran
9e96ac666d
HADOOP-10219. ipc.Client.setupIOstreams() needs to check for ClientCache.stopClient requested shutdowns. 
Contributed by Kihwal Lee and Lukas Majercak.
 2018-09-04 16:46:12 +01:00
Daniel Templeton
f2c2a68ec2 HADOOP-15706. Typo in compatibility doc: SHOUD -> SHOULD 
(Contributed by Laszlo Kollar via Daniel Templeton)

Change-Id: I6e2459d0700df7f3bad4eac8297a11690191c3ba
 2018-08-30 09:12:36 -07:00
Steve Loughran
2e6c1109dc
HADOOP-15667. FileSystemMultipartUploader should verify that UploadHandle has non-0 length. 
Contributed by Ewan Higgs
 2018-08-30 14:33:16 +01:00
Kitti Nanasi
781437c219 HADOOP-15698. KMS log4j is not initialized properly at startup. Contributed by Kitti Nanasi. 2018-08-29 22:07:49 -07:00
Daniel Templeton
d53a10b0a5 HADOOP-15705. Typo in the definition of "stable" in the interface classification 
Change-Id: I3eae2143400a534903db4f186400561fc8d2bd56
 2018-08-29 13:59:32 -07:00
John Zhuge
f9c6fd9471
HADOOP-15633. fs.TrashPolicyDefault: Can't create trash directory. Contributed by Fei Hui. 2018-08-27 09:22:59 -07:00
Konstantin V Shvachko
a4121c71c2 HDFS-13848. Refactor NameNode failover proxy providers. Contributed by Konstantin Shvachko. 2018-08-24 18:27:30 -07:00
Takanobu Asanuma
bed8cb6979 HADOOP-14314. The OpenSolaris taxonomy link is dead in InterfaceClassification.md. Contributed by Rui Gao. 2018-08-23 21:54:38 +09:00
Steve Loughran
34577d2c21
HADOOP-15679. ShutdownHookManager shutdown time needs to be configurable & extended. 
Contributed by Steve Loughran.
 2018-08-20 18:36:24 -07:00
Giovanni Matteo Fumarola
79c97f6a0b HADOOP-14624. Add GenericTestUtils.DelayAnswer that accept slf4j logger API. Contributed by Ian Pickering and Wenxin He. 2018-08-17 14:40:00 -07:00
Xiao Chen
60ffec9f79 HADOOP-9214. Create a new touch command to allow modifying atime and mtime. Contributed by Hrishikesh Gadre. 2018-08-17 11:18:09 -07:00
Jason Lowe
a17eed1b87 HDFS-13822. speedup libhdfs++ build (enable parallel build). Contributed by Allen Wittenauer and Pradeep Ambati 2018-08-17 12:25:36 -05:00
Xiao Chen
8d7c93186e HADOOP-15674. Test failure TestSSLHttpServer.testExcludedCiphers with TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 cipher suite. Contributed by Szilard Nemeth. 2018-08-17 10:09:23 -07:00
Márton Elek
77b015000a HADOOP-8807. Update README and website to reflect HADOOP-8662. Contributed by Andras Bokor. 2018-08-17 11:10:29 +02:00
Xiao Chen
edeb2a356a HADOOP-15655. Enhance KMS client retry behavior. Contributed by Kitti Nanasi. 2018-08-16 22:42:03 -07:00
Akira Ajisaka
3e3963b035
HADOOP-15552. Move logging APIs over to slf4j in hadoop-tools - Part2. Contributed by Ian Pickering. 2018-08-16 00:31:59 +09:00
Márton Elek
7e822ec246 HDDS-324. Addendum: remove the q letter which is accidentally added to the hadoop-functions.sh. 2018-08-14 20:54:33 +02:00
Xiaoyu Yao
b4031a8f1b HDDS-324. Use pipeline name as Ratis groupID to allow datanode to report pipeline info. Contributed by Mukul Kumar Singh. 2018-08-13 12:47:49 -07:00
Xiao Chen
b94c8874e2 HADOOP-15638. KMS Accept Queue Size default changed from 500 to 128 in Hadoop 3.x. Contributed by Wei-Chiu Chuang. 2018-08-13 10:40:31 -07:00
Steve Loughran
da9a39eed1
HADOOP-15583. Stabilize S3A Assumed Role support. 
Contributed by Steve Loughran.
 2018-08-08 22:57:24 -07:00
Wangda Tan
49c687608b Make 3.1.1 awared by other branches 2018-08-08 13:02:12 -07:00
Xiao Chen
9499df7b81 HDFS-13658. Expose HighestPriorityLowRedundancy blocks statistics. Contributed by Kitti Nanasi. 2018-08-08 10:40:20 -07:00
Ewan Higgs
2ec97abb2e HADOOP-15576. S3A Multipart Uploader to work with S3Guard and encryption Originally contributed by Ewan Higgs with refinements by Steve Loughran. 2018-08-08 13:50:23 +02:00
Mukul Kumar Singh
6ed8593d18 HDDS-301. ozone command shell does not contain subcommand to run ozoneFS commands. Contributed by Nilotpal Nandi. 2018-08-07 16:09:53 +05:30
Yiqun Lin
2e4e02b4df HDFS-13792. Fix FSN read/write lock metrics name. Contributed by Chao Sun. 2018-08-07 09:32:51 +08:00
Arpit Agarwal
f2e29acbfa HADOOP-15476. fix logging for split-dns multihome . Contributed by Ajay Kumar. 2018-08-01 12:32:01 -07:00
Chen Liang
e8f952ef06 HADOOP-15637. LocalFs#listLocatedStatus does not filter out hidden .crc files. Contributed by Erik Krogen. 2018-07-30 10:25:07 -07:00
Eric Yang
77721f39e2 HADOOP-15593. Fixed NPE in UGI spawnAutoRenewalThreadForUserCreds. 
Contributed by Gabor Bota
 2018-07-26 18:35:36 -04:00
Xiao Chen
be150a17b1 HDFS-13622. mkdir should print the parent directory in the error message when parent directories do not exist. Contributed by Shweta. 2018-07-26 10:24:32 -07:00
Yiqun Lin
9089790cab HADOOP-15611. Log more details for FairCallQueue. Contributed by Ryan Wu. 2018-07-26 18:08:28 +08:00
Mukul Kumar Singh
5f0b924360 HADOOP-15395. DefaultImpersonationProvider fails to parse proxy user config if username has . in it. Contributed by Ajay Kumar. 2018-07-25 21:09:11 +05:30
Xiao Chen
81d59506e5 HADOOP-15609. Retry KMS calls when SSLHandshakeException occurs. Contributed by Kitti Nanasi. 2018-07-24 21:45:43 -07:00
Daniel Templeton
849c45db18 HDFS-13448. HDFS Block Placement - Ignore Locality for First Block Replica 
(Contributed by BELUGA BEHR via Daniel Templeton)

Change-Id: I965d1cfa642ad24296038b83e3d5c9983545267d
 2018-07-24 16:05:27 -07:00
Gera Shegalov
6bec03cfc8 HADOOP-15612. Improve exception when tfile fails to load LzoCodec. (gera) 2018-07-24 14:32:30 -07:00
bibinchundatt
8461278833 YARN-8544. [DS] AM registration fails when hadoop authorization is enabled. Contributed by Bibin A Chundatt. 2018-07-24 13:09:17 +05:30
Haibo Chen
17e26163ec HADOOP-15586. Fix wrong log statement in AbstractService. (Szilard Nemeth via Haibo Chen) 2018-07-23 11:18:25 -07:00
Xiao Chen
993ec026d1 HADOOP-15596. Stack trace should not be printed out when running hadoop key commands. Contributed by Kitti Nanasi. 2018-07-20 19:46:46 -07:00
Haibo Chen
e6873dfde0 YARN-7300. DiskValidator is not used in LocalDirAllocator. (Szilard Nemeth via Haibo Chen) 2018-07-19 16:27:11 -07:00
Kihwal Lee
ccf2db7fc2 HADOOP-15614. TestGroupsCaching.testExceptionOnBackgroundRefreshHandled reliably fails. Contributed by Weiwei Yang. 2018-07-19 11:13:37 -05:00
Wei-Chiu Chuang
d215357718 HDFS-13485. DataNode WebHDFS endpoint throws NPE. Contributed by Siyao Meng. 2018-07-16 15:45:55 -07:00
Xiao Chen
d2874e0417 HDFS-13690. Improve error message when creating encryption zone while KMS is unreachable. Contributed by Kitti Nanasi. 2018-07-16 13:19:53 -07:00
Wei-Chiu Chuang
0c7a578927 HADOOP-15598. DataChecksum calculate checksum is contented on hashtable synchronization. Contributed by Prasanth Jayachandran. 2018-07-16 11:32:45 -07:00
Akira Ajisaka
88625f5cd9
HADOOP-15531. Use commons-text instead of commons-lang in some classes to fix deprecation warnings. Contributed by Takanobu Asanuma. 2018-07-13 11:42:12 -04:00
Sean Mackrory
4f3f9391b0 HADOOP-15316. GenericTestUtils can exceed maxSleepTime. Contributed by Adam Antal. 2018-07-12 17:24:01 +02:00
Wei-Chiu Chuang
162228e8db HDFS-13723. Occasional "Should be different group" error in TestRefreshUserMappings#testGroupMappingRefresh. Contributed by Siyao Meng. 2018-07-11 10:02:08 -07:00
Akira Ajisaka
4a08ddfa68
HADOOP-15568. fix some typos in the .sh comments. Contributed by Steve Loughran. 2018-07-09 15:43:38 -04:00
Xiao Chen
895845e9b0 HADOOP-15581. Set default jetty log level to INFO in KMS. Contributed by Kitti Nanasi. 2018-07-09 12:06:50 -07:00
Xiao Chen
def9d94a40 HADOOP-15591. KMSClientProvider should log KMS DT acquisition at INFO level. Contributed by Kitti Nanasi. 2018-07-09 12:01:52 -07:00
Yiqun Lin
7a68ac607c HDFS-13674. Improve documentation on Metrics. Contributed by Chao Sun. 2018-07-09 14:27:34 +08:00
Arpit Agarwal
061b168529 HDDS-167. Rename KeySpaceManager to OzoneManager. Contributed by Arpit Agarwal. 2018-07-06 12:09:05 -07:00
Xiaoyu Yao
498e3bfb6b HADOOP-15571. Multiple FileContexts created with the same configuration object should be allowed to have different umask. Contributed by Vinod Kumar Vavilapalli. 2018-07-06 11:56:09 -07:00
Andrew Wang
f51da9c4d1 HADOOP-15554. Improve JIT performance for Configuration parsing. Contributed by Todd Lipcon. 2018-07-02 18:31:21 +02:00
Nanda kumar
d40121845e HADOOP-15574: Suppress build error if there are no docs after excluding private annotations. Contributed by Takanobu Asanuma. 2018-07-01 17:11:46 +05:30
Eric E Payne
d36f6b9e93 HADOOP-15548: Randomize local dirs. Contributed by Jim Brennan. 2018-06-29 18:18:32 +00:00
Eric Yang
384764cdea YARN-8409. Fixed NPE in ActiveStandbyElectorBasedElectorService. 
Contributed by Chandni Singh
 2018-06-28 12:39:49 -04:00
Bharat Viswanadham
d3fa83a44b HDFS-13705:The native ISA-L library loading failure should be made warning rather than an error message. Contributed by Shashikant Banerjee. 2018-06-28 09:21:56 -07:00
Akira Ajisaka
ddbff7c8d3
HADOOP-14313. Replace/improve Hadoop's byte[] comparator. Contributed by Vikas Vishwakarma. 2018-06-28 14:58:40 +09:00
Akira Ajisaka
2b2399d623
HADOOP-15495. Upgrade commons-lang version to 3.7 in hadoop-common-project and hadoop-tools. Contributed by Takanobu Asanuma. 2018-06-28 14:37:22 +09:00
Todd Lipcon
7a3c6e9c3c HADOOP-15550. Avoid static initialization of ObjectMappers 2018-06-25 15:36:45 -07:00
Inigo Goiri
abc3e4bad9 HADOOP-15458. TestLocalFileSystem#testFSOutputStreamBuilder fails on Windows. Contributed by Xiao Liang. 2018-06-25 09:50:27 -07:00
Steve Loughran
1ba4e62304
HADOOP-14396. Add builder interface to FileContext. 
Contributed by  Lei (Eddy) Xu.
 2018-06-25 14:38:57 +01:00
Eric E Payne
6432128622 YARN-8444: NodeResourceMonitor crashes on bad swapFree value. Contributed by Jim Brennan. 2018-06-22 17:15:29 +00:00
Todd Lipcon
59de967954 HADOOP-15549. Upgrade to commons-configuration 2.1 regresses task CPU consumption 2018-06-21 10:32:52 -07:00
Todd Lipcon
43541a1890 HADOOP-15551. Avoid use of Arrays.stream in Configuration.addTags 2018-06-20 16:43:10 -07:00
Xiao Chen
32f867a6a9 HDFS-13682. Cannot create encryption zone after KMS auth token expires. 2018-06-20 15:58:26 -07:00
Eric Yang
2d87592fc6 HADOOP-15527. Improve delay check for stopping processes. 
Fixed script location.
               Contributed by Vinod Kumar Vavilapalli
 2018-06-19 13:38:13 -04:00
Chris Douglas
980031bb04 HADOOP-13186. Multipart Uploader API. Contributed by Ewan Higgs 2018-06-17 11:54:26 -07:00
Surendra Singh Lilhore
3905fdb793 HADOOP-15523. Shell command timeout given is in seconds whereas it is taken as millisec while scheduling. Contributed by Bilwa S T. 2018-06-17 12:12:01 +05:30
Yiqun Lin
8e7548d33b HDFS-13641. Add metrics for edit log tailing. Contributed by Chao Sun. 2018-06-13 20:05:55 +08:00
Eric Yang
108da85320 HADOOP-15527. Improve delay check for stopping processes. 
Contributed by Vinod Kumar Vavilapalli
 2018-06-12 20:40:32 -04:00
Inigo Goiri
aeaf9fec62 HADOOP-15532. TestBasicDiskValidator fails with NoSuchFileException. Contributed by Giovanni Matteo Fumarola. 2018-06-12 14:16:14 -07:00
Xiao Chen
a16623df21 HADOOP-15307. NFS: flavor AUTH_SYS should use VerifierNone. Contributed by Gabor Bota. 2018-06-12 11:16:24 -07:00
Rohith Sharma K S
2df73dace0 YARN-8405. RM zk-state-store.parent-path ACLs has been changed since HADOOP-14773. Contributed by Íñigo Goiri. 2018-06-12 17:13:09 +05:30
Yongjun Zhang
7c3dc39083 Update CHANGES, RELEASENOTES, and jdiff for 3.0.3 release. 
(cherry picked from commit 5a82f10e3254bc0745c2dda6bcef888d3ff0d1c3)
 2018-06-11 15:13:18 -07:00
Sammi Chen
18201b882a HADOOP-15499. Performance severe drops when running RawErasureCoderBenchmark with NativeRSRawErasureCoder. Contributed by Sammi Chen. 2018-06-11 13:53:37 +08:00
Steve Loughran
ef0118b91e
HADOOP-15520. Add tests for various org.apache.hadoop.util classes. 
Contributed by Arash Nabili
 2018-06-09 15:33:38 +01:00
Inigo Goiri
12be8bad7d HADOOP-15516. Add test cases to cover FileUtil#readLink. Contributed by Giovanni Matteo Fumarola. 2018-06-07 13:34:52 -07:00
Steve Loughran
f494f0b896
HADOOP-15512. Clean up Shell from JDK7 workarounds. 
Contributed by Zsolt Venczel.
 2018-06-07 15:55:49 +01:00
Inigo Goiri
db81f85e84 HADOOP-15513. Add additional test cases to cover some corner cases for FileUtil#symlink. Contributed by Giovanni Matteo Fumarola. 2018-06-06 11:44:17 -07:00
Xiao Chen
ba4011d64f HADOOP-15217. FsUrlConnection does not handle paths with spaces. Contributed by Joseph Fourny and Zsolt Venczel. 2018-06-05 21:18:31 -07:00
Chen Liang
1b0d4f4606 HDFS-13547. Add ingress port based sasl resolver. Contributed by Chen Liang. 2018-06-05 11:51:29 -07:00
Xiao Chen
6d5e87aec2 HADOOP-15507. Add MapReduce counters about EC bytes read. 2018-06-04 21:13:47 -07:00
Haibo Chen
04cf699dd5 YARN-8388. TestCGroupElasticMemoryController.testNormalExit() hangs on Linux. (Miklos Szegedi via Haibo Chen) 2018-06-04 14:23:08 -07:00
Haibo Chen
4880d890ee YARN-8375. TestCGroupElasticMemoryController fails surefire build. (Miklos Szegedi via Haibo Chen) 2018-06-01 16:29:13 -07:00
Daniel Templeton
cba3194998 HDFS-13636. Cross-Site Scripting vulnerability in HttpServer2 
(Contributed by Haibo Yan via Daniel Templeton)

Change-Id: I28edde8125dd20d8d270f0e609d1c04d8173c8b7
 2018-06-01 14:42:39 -07:00
Wei-Chiu Chuang
32671d8713 HADOOP-14783. [KMS] Add missing configuration properties into kms-default.xml. Contributed by Chetna Chaudhari. 2018-05-31 16:09:33 -07:00