big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
23,077 Commits 2 Branches 2 Tags 264 MiB
35f093f5b3
Commit Graph

5055 Commits

Author SHA1 Message Date
Akira Ajisaka
c62b440421
HADOOP-15693. Typo in Secure Mode documentation. Contributed by Thomas Levine. 2018-09-26 10:30:43 +09:00
Akira Ajisaka
93b0f540ed
HADOOP-15783. [JDK10] TestSFTPFileSystem.testGetModifyTime fails. 2018-09-25 16:03:28 +09:00
Sunil G
d060cbea48 HDFS-13937. Multipart Uploader APIs to be marked as private/unstable in 3.2.0. Contributed by Steve Loughran. 2018-09-24 21:19:47 +05:30
Sean Mackrory
0def61482b Merge branch 'HADOOP-15407' into trunk 2018-09-22 21:19:12 -06:00
Ewan Higgs
429a07e08c HADOOP-15764. [JDK10] Migrate from sun.net.dns.ResolverConfiguration to the replacement. Contributed by Akira Ajisaka. 2018-09-20 15:13:55 +02:00
Ewan Higgs
3da94a36e2 HADOOP-15756. [JDK10] Migrate from sun.net.util.IPAddressUtil to the replacement. Contributed by Akira Ajisaka. 2018-09-20 14:53:21 +02:00
Steve Loughran
646874c326
HADOOP-15748. S3 listing inconsistency can raise NPE in globber. 
Contributed by Steve Loughran.
 2018-09-20 13:04:52 +01:00
Vinayakumar B
7ad27e97f0 HADOOP-15736. Trash : Negative Value For Deletion Interval Leads To Abnormal Behaviour. Contributed by Ayush Saxena. 2018-09-20 09:31:35 +05:30
Chen Liang
a30b4f9e71 HADOOP-15726. Create utility to limit frequency of log statements. Contributed by Erik Krogen. 2018-09-19 13:22:37 -07:00
Jason Lowe
e71f61ecb8 HADOOP-15755. StringUtils#createStartupShutdownMessage throws NPE when args is null. Contributed by Lokesh Jain and Dinesh Chitlangia 2018-09-18 15:57:33 -05:00
Steve Loughran
1cf38a38da
HADOOP-15744. AbstractContractAppendTest fails against HDFS on HADOOP-15407 branch. 
Contributed by Steve Loughran.
 2018-09-18 10:56:56 +01:00
Yiqun Lin
ee051ef9fe HADOOP-15742. Log if ipc backoff is enabled in CallQueueManager. Contributed by Ryan Wu. 2018-09-18 11:10:33 +08:00
Sean Mackrory
9475fd902a HADOOP-15745. ABFS: Add ABFS configuration to ConfigRedactor. 2018-09-17 19:54:01 +00:00
Thomas Marquardt
81dc4a995c HADOOP-15663. ABFS: Simplify configuration. 
Contributed by Da Zhou.
 2018-09-17 19:54:01 +00:00
Thomas Marquardt
ce03a93f78 HADOOP-15446. ABFS: tune imports & javadocs; stabilise tests. 
Contributed by Steve Loughran and Da Zhou.
 2018-09-17 19:54:01 +00:00
Steve Loughran
f044deedbb HADOOP-15407. HADOOP-15540. Support Windows Azure Storage - Blob file system "ABFS" in Hadoop: Core Commit. 
Contributed by Shane Mainali, Thomas Marquardt, Zichen Sun, Georgi Chalakov, Esfandiar Manii, Amit Singh, Dana Kaban, Da Zhou, Junhua Gu, Saher Ahwal, Saurabh Pant, James Baker, Shaoyu Zhang, Lawrence Chen, Kevin Chen and Steve Loughran
 2018-09-17 19:54:01 +00:00
Brahma Reddy Battula
ef5c776a42 HADOOP-15733. Correct the log when Invalid emptier Interval configured. Contributed by Ayush Saxena 2018-09-14 07:32:27 +05:30
Hanisha Koneru
291dcf2247 HDDS-423. Introduce an ozone specific log4j.properties. Contributed by Elek, Marton. 2018-09-13 16:37:44 -07:00
Takanobu Asanuma
5d084d7eca HADOOP-12760. sun.misc.Cleaner has moved to a new location in OpenJDK 9. Contributed by Akira Ajisaka. 2018-09-13 20:42:09 +09:00
Steve Loughran
d7c0a08a1c
HADOOP-15426 Make S3guard client resilient to DDB throttle events and network failures (Contributed by Steve Loughran) 2018-09-12 21:04:49 -07:00
bibinchundatt
c44088ac19 YARN-8739. Fix jenkins issues for Node Attributes branch. Contributed by Sunil Govindan. 2018-09-12 16:01:01 +05:30
Naganarasimha
2475fb0a1e YARN-6856. [YARN-3409] Support CLI for Node Attributes Mapping. Contributed by Naganarasimha G R. 2018-09-12 16:00:59 +05:30
Mingliang Liu
87f63b6479 HADOOP-14833. Remove s3a user:secret authentication. Contributed by Steve Loughran 2018-09-11 17:18:42 -07:00
Allen Wittenauer
4b8c2b1c1c
HADOOP-14671. Upgrade Apache Yetus to 0.8.0 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-09-11 08:55:34 -07:00
Inigo Goiri
527288ef89 HDFS-13836. RBF: Handle mount table znode with null value. Contributed by yanghuafeng. 2018-09-06 16:47:54 -07:00
Vrushali C
962089ab59 HADOOP-15657 Registering MutableQuantiles via Metric annotation. Contributed by Sushil Ks 2018-09-05 20:53:21 -07:00
Giovanni Matteo Fumarola
9af96d4ed4 HADOOP-15707. Add IsActiveServlet to be used for Load Balancers. Contributed by Lukas Majercak. 2018-09-05 10:50:25 -07:00
Wei-Chiu Chuang
e780556ae9 HADOOP-15696. KMS performance regression due to too many open file descriptors after Jetty migration. Contributed by Wei-Chiu Chuang. 2018-09-05 09:52:35 -07:00
Steve Loughran
9e96ac666d
HADOOP-10219. ipc.Client.setupIOstreams() needs to check for ClientCache.stopClient requested shutdowns. 
Contributed by Kihwal Lee and Lukas Majercak.
 2018-09-04 16:46:12 +01:00
Daniel Templeton
f2c2a68ec2 HADOOP-15706. Typo in compatibility doc: SHOUD -> SHOULD 
(Contributed by Laszlo Kollar via Daniel Templeton)

Change-Id: I6e2459d0700df7f3bad4eac8297a11690191c3ba
 2018-08-30 09:12:36 -07:00
Steve Loughran
2e6c1109dc
HADOOP-15667. FileSystemMultipartUploader should verify that UploadHandle has non-0 length. 
Contributed by Ewan Higgs
 2018-08-30 14:33:16 +01:00
Daniel Templeton
d53a10b0a5 HADOOP-15705. Typo in the definition of "stable" in the interface classification 
Change-Id: I3eae2143400a534903db4f186400561fc8d2bd56
 2018-08-29 13:59:32 -07:00
John Zhuge
f9c6fd9471
HADOOP-15633. fs.TrashPolicyDefault: Can't create trash directory. Contributed by Fei Hui. 2018-08-27 09:22:59 -07:00
Konstantin V Shvachko
a4121c71c2 HDFS-13848. Refactor NameNode failover proxy providers. Contributed by Konstantin Shvachko. 2018-08-24 18:27:30 -07:00
Takanobu Asanuma
bed8cb6979 HADOOP-14314. The OpenSolaris taxonomy link is dead in InterfaceClassification.md. Contributed by Rui Gao. 2018-08-23 21:54:38 +09:00
Steve Loughran
34577d2c21
HADOOP-15679. ShutdownHookManager shutdown time needs to be configurable & extended. 
Contributed by Steve Loughran.
 2018-08-20 18:36:24 -07:00
Giovanni Matteo Fumarola
79c97f6a0b HADOOP-14624. Add GenericTestUtils.DelayAnswer that accept slf4j logger API. Contributed by Ian Pickering and Wenxin He. 2018-08-17 14:40:00 -07:00
Xiao Chen
60ffec9f79 HADOOP-9214. Create a new touch command to allow modifying atime and mtime. Contributed by Hrishikesh Gadre. 2018-08-17 11:18:09 -07:00
Jason Lowe
a17eed1b87 HDFS-13822. speedup libhdfs++ build (enable parallel build). Contributed by Allen Wittenauer and Pradeep Ambati 2018-08-17 12:25:36 -05:00
Xiao Chen
8d7c93186e HADOOP-15674. Test failure TestSSLHttpServer.testExcludedCiphers with TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 cipher suite. Contributed by Szilard Nemeth. 2018-08-17 10:09:23 -07:00
Márton Elek
77b015000a HADOOP-8807. Update README and website to reflect HADOOP-8662. Contributed by Andras Bokor. 2018-08-17 11:10:29 +02:00
Xiao Chen
edeb2a356a HADOOP-15655. Enhance KMS client retry behavior. Contributed by Kitti Nanasi. 2018-08-16 22:42:03 -07:00
Akira Ajisaka
3e3963b035
HADOOP-15552. Move logging APIs over to slf4j in hadoop-tools - Part2. Contributed by Ian Pickering. 2018-08-16 00:31:59 +09:00
Márton Elek
7e822ec246 HDDS-324. Addendum: remove the q letter which is accidentally added to the hadoop-functions.sh. 2018-08-14 20:54:33 +02:00
Xiaoyu Yao
b4031a8f1b HDDS-324. Use pipeline name as Ratis groupID to allow datanode to report pipeline info. Contributed by Mukul Kumar Singh. 2018-08-13 12:47:49 -07:00
Steve Loughran
da9a39eed1
HADOOP-15583. Stabilize S3A Assumed Role support. 
Contributed by Steve Loughran.
 2018-08-08 22:57:24 -07:00
Wangda Tan
49c687608b Make 3.1.1 awared by other branches 2018-08-08 13:02:12 -07:00
Xiao Chen
9499df7b81 HDFS-13658. Expose HighestPriorityLowRedundancy blocks statistics. Contributed by Kitti Nanasi. 2018-08-08 10:40:20 -07:00
Ewan Higgs
2ec97abb2e HADOOP-15576. S3A Multipart Uploader to work with S3Guard and encryption Originally contributed by Ewan Higgs with refinements by Steve Loughran. 2018-08-08 13:50:23 +02:00
Mukul Kumar Singh
6ed8593d18 HDDS-301. ozone command shell does not contain subcommand to run ozoneFS commands. Contributed by Nilotpal Nandi. 2018-08-07 16:09:53 +05:30
Yiqun Lin
2e4e02b4df HDFS-13792. Fix FSN read/write lock metrics name. Contributed by Chao Sun. 2018-08-07 09:32:51 +08:00
Arpit Agarwal
f2e29acbfa HADOOP-15476. fix logging for split-dns multihome . Contributed by Ajay Kumar. 2018-08-01 12:32:01 -07:00
Chen Liang
e8f952ef06 HADOOP-15637. LocalFs#listLocatedStatus does not filter out hidden .crc files. Contributed by Erik Krogen. 2018-07-30 10:25:07 -07:00
Eric Yang
77721f39e2 HADOOP-15593. Fixed NPE in UGI spawnAutoRenewalThreadForUserCreds. 
Contributed by Gabor Bota
 2018-07-26 18:35:36 -04:00
Xiao Chen
be150a17b1 HDFS-13622. mkdir should print the parent directory in the error message when parent directories do not exist. Contributed by Shweta. 2018-07-26 10:24:32 -07:00
Yiqun Lin
9089790cab HADOOP-15611. Log more details for FairCallQueue. Contributed by Ryan Wu. 2018-07-26 18:08:28 +08:00
Mukul Kumar Singh
5f0b924360 HADOOP-15395. DefaultImpersonationProvider fails to parse proxy user config if username has . in it. Contributed by Ajay Kumar. 2018-07-25 21:09:11 +05:30
Xiao Chen
81d59506e5 HADOOP-15609. Retry KMS calls when SSLHandshakeException occurs. Contributed by Kitti Nanasi. 2018-07-24 21:45:43 -07:00
Daniel Templeton
849c45db18 HDFS-13448. HDFS Block Placement - Ignore Locality for First Block Replica 
(Contributed by BELUGA BEHR via Daniel Templeton)

Change-Id: I965d1cfa642ad24296038b83e3d5c9983545267d
 2018-07-24 16:05:27 -07:00
Gera Shegalov
6bec03cfc8 HADOOP-15612. Improve exception when tfile fails to load LzoCodec. (gera) 2018-07-24 14:32:30 -07:00
bibinchundatt
8461278833 YARN-8544. [DS] AM registration fails when hadoop authorization is enabled. Contributed by Bibin A Chundatt. 2018-07-24 13:09:17 +05:30
Haibo Chen
17e26163ec HADOOP-15586. Fix wrong log statement in AbstractService. (Szilard Nemeth via Haibo Chen) 2018-07-23 11:18:25 -07:00
Xiao Chen
993ec026d1 HADOOP-15596. Stack trace should not be printed out when running hadoop key commands. Contributed by Kitti Nanasi. 2018-07-20 19:46:46 -07:00
Haibo Chen
e6873dfde0 YARN-7300. DiskValidator is not used in LocalDirAllocator. (Szilard Nemeth via Haibo Chen) 2018-07-19 16:27:11 -07:00
Kihwal Lee
ccf2db7fc2 HADOOP-15614. TestGroupsCaching.testExceptionOnBackgroundRefreshHandled reliably fails. Contributed by Weiwei Yang. 2018-07-19 11:13:37 -05:00
Wei-Chiu Chuang
d215357718 HDFS-13485. DataNode WebHDFS endpoint throws NPE. Contributed by Siyao Meng. 2018-07-16 15:45:55 -07:00
Xiao Chen
d2874e0417 HDFS-13690. Improve error message when creating encryption zone while KMS is unreachable. Contributed by Kitti Nanasi. 2018-07-16 13:19:53 -07:00
Wei-Chiu Chuang
0c7a578927 HADOOP-15598. DataChecksum calculate checksum is contented on hashtable synchronization. Contributed by Prasanth Jayachandran. 2018-07-16 11:32:45 -07:00
Akira Ajisaka
88625f5cd9
HADOOP-15531. Use commons-text instead of commons-lang in some classes to fix deprecation warnings. Contributed by Takanobu Asanuma. 2018-07-13 11:42:12 -04:00
Sean Mackrory
4f3f9391b0 HADOOP-15316. GenericTestUtils can exceed maxSleepTime. Contributed by Adam Antal. 2018-07-12 17:24:01 +02:00
Wei-Chiu Chuang
162228e8db HDFS-13723. Occasional "Should be different group" error in TestRefreshUserMappings#testGroupMappingRefresh. Contributed by Siyao Meng. 2018-07-11 10:02:08 -07:00
Akira Ajisaka
4a08ddfa68
HADOOP-15568. fix some typos in the .sh comments. Contributed by Steve Loughran. 2018-07-09 15:43:38 -04:00
Xiao Chen
def9d94a40 HADOOP-15591. KMSClientProvider should log KMS DT acquisition at INFO level. Contributed by Kitti Nanasi. 2018-07-09 12:01:52 -07:00
Yiqun Lin
7a68ac607c HDFS-13674. Improve documentation on Metrics. Contributed by Chao Sun. 2018-07-09 14:27:34 +08:00
Arpit Agarwal
061b168529 HDDS-167. Rename KeySpaceManager to OzoneManager. Contributed by Arpit Agarwal. 2018-07-06 12:09:05 -07:00
Xiaoyu Yao
498e3bfb6b HADOOP-15571. Multiple FileContexts created with the same configuration object should be allowed to have different umask. Contributed by Vinod Kumar Vavilapalli. 2018-07-06 11:56:09 -07:00
Andrew Wang
f51da9c4d1 HADOOP-15554. Improve JIT performance for Configuration parsing. Contributed by Todd Lipcon. 2018-07-02 18:31:21 +02:00
Eric E Payne
d36f6b9e93 HADOOP-15548: Randomize local dirs. Contributed by Jim Brennan. 2018-06-29 18:18:32 +00:00
Eric Yang
384764cdea YARN-8409. Fixed NPE in ActiveStandbyElectorBasedElectorService. 
Contributed by Chandni Singh
 2018-06-28 12:39:49 -04:00
Bharat Viswanadham
d3fa83a44b HDFS-13705:The native ISA-L library loading failure should be made warning rather than an error message. Contributed by Shashikant Banerjee. 2018-06-28 09:21:56 -07:00
Akira Ajisaka
ddbff7c8d3
HADOOP-14313. Replace/improve Hadoop's byte[] comparator. Contributed by Vikas Vishwakarma. 2018-06-28 14:58:40 +09:00
Akira Ajisaka
2b2399d623
HADOOP-15495. Upgrade commons-lang version to 3.7 in hadoop-common-project and hadoop-tools. Contributed by Takanobu Asanuma. 2018-06-28 14:37:22 +09:00
Todd Lipcon
7a3c6e9c3c HADOOP-15550. Avoid static initialization of ObjectMappers 2018-06-25 15:36:45 -07:00
Inigo Goiri
abc3e4bad9 HADOOP-15458. TestLocalFileSystem#testFSOutputStreamBuilder fails on Windows. Contributed by Xiao Liang. 2018-06-25 09:50:27 -07:00
Steve Loughran
1ba4e62304
HADOOP-14396. Add builder interface to FileContext. 
Contributed by  Lei (Eddy) Xu.
 2018-06-25 14:38:57 +01:00
Eric E Payne
6432128622 YARN-8444: NodeResourceMonitor crashes on bad swapFree value. Contributed by Jim Brennan. 2018-06-22 17:15:29 +00:00
Todd Lipcon
59de967954 HADOOP-15549. Upgrade to commons-configuration 2.1 regresses task CPU consumption 2018-06-21 10:32:52 -07:00
Todd Lipcon
43541a1890 HADOOP-15551. Avoid use of Arrays.stream in Configuration.addTags 2018-06-20 16:43:10 -07:00
Xiao Chen
32f867a6a9 HDFS-13682. Cannot create encryption zone after KMS auth token expires. 2018-06-20 15:58:26 -07:00
Eric Yang
2d87592fc6 HADOOP-15527. Improve delay check for stopping processes. 
Fixed script location.
               Contributed by Vinod Kumar Vavilapalli
 2018-06-19 13:38:13 -04:00
Chris Douglas
980031bb04 HADOOP-13186. Multipart Uploader API. Contributed by Ewan Higgs 2018-06-17 11:54:26 -07:00
Surendra Singh Lilhore
3905fdb793 HADOOP-15523. Shell command timeout given is in seconds whereas it is taken as millisec while scheduling. Contributed by Bilwa S T. 2018-06-17 12:12:01 +05:30
Yiqun Lin
8e7548d33b HDFS-13641. Add metrics for edit log tailing. Contributed by Chao Sun. 2018-06-13 20:05:55 +08:00
Eric Yang
108da85320 HADOOP-15527. Improve delay check for stopping processes. 
Contributed by Vinod Kumar Vavilapalli
 2018-06-12 20:40:32 -04:00
Inigo Goiri
aeaf9fec62 HADOOP-15532. TestBasicDiskValidator fails with NoSuchFileException. Contributed by Giovanni Matteo Fumarola. 2018-06-12 14:16:14 -07:00
Rohith Sharma K S
2df73dace0 YARN-8405. RM zk-state-store.parent-path ACLs has been changed since HADOOP-14773. Contributed by Íñigo Goiri. 2018-06-12 17:13:09 +05:30
Yongjun Zhang
7c3dc39083 Update CHANGES, RELEASENOTES, and jdiff for 3.0.3 release. 
(cherry picked from commit 5a82f10e3254bc0745c2dda6bcef888d3ff0d1c3)
 2018-06-11 15:13:18 -07:00
Sammi Chen
18201b882a HADOOP-15499. Performance severe drops when running RawErasureCoderBenchmark with NativeRSRawErasureCoder. Contributed by Sammi Chen. 2018-06-11 13:53:37 +08:00
Steve Loughran
ef0118b91e
HADOOP-15520. Add tests for various org.apache.hadoop.util classes. 
Contributed by Arash Nabili
 2018-06-09 15:33:38 +01:00
Inigo Goiri
12be8bad7d HADOOP-15516. Add test cases to cover FileUtil#readLink. Contributed by Giovanni Matteo Fumarola. 2018-06-07 13:34:52 -07:00
Steve Loughran
f494f0b896
HADOOP-15512. Clean up Shell from JDK7 workarounds. 
Contributed by Zsolt Venczel.
 2018-06-07 15:55:49 +01:00
Inigo Goiri
db81f85e84 HADOOP-15513. Add additional test cases to cover some corner cases for FileUtil#symlink. Contributed by Giovanni Matteo Fumarola. 2018-06-06 11:44:17 -07:00
Xiao Chen
ba4011d64f HADOOP-15217. FsUrlConnection does not handle paths with spaces. Contributed by Joseph Fourny and Zsolt Venczel. 2018-06-05 21:18:31 -07:00
Chen Liang
1b0d4f4606 HDFS-13547. Add ingress port based sasl resolver. Contributed by Chen Liang. 2018-06-05 11:51:29 -07:00
Xiao Chen
6d5e87aec2 HADOOP-15507. Add MapReduce counters about EC bytes read. 2018-06-04 21:13:47 -07:00
Haibo Chen
04cf699dd5 YARN-8388. TestCGroupElasticMemoryController.testNormalExit() hangs on Linux. (Miklos Szegedi via Haibo Chen) 2018-06-04 14:23:08 -07:00
Haibo Chen
4880d890ee YARN-8375. TestCGroupElasticMemoryController fails surefire build. (Miklos Szegedi via Haibo Chen) 2018-06-01 16:29:13 -07:00
Daniel Templeton
cba3194998 HDFS-13636. Cross-Site Scripting vulnerability in HttpServer2 
(Contributed by Haibo Yan via Daniel Templeton)

Change-Id: I28edde8125dd20d8d270f0e609d1c04d8173c8b7
 2018-06-01 14:42:39 -07:00
Takanobu Asanuma
d1e2b80980
MAPREDUCE-7098. Upgrade common-langs version to 3.7 in hadoop-mapreduce-project 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-05-31 18:31:10 +09:00
Kihwal Lee
e3236a9680 Additional check when unpacking archives. Contributed by Wilfred Spiegelenburg. 2018-05-29 14:15:12 -05:00
Inigo Goiri
3c75f8e493 HADOOP-15497. TestTrash should use proper test path to avoid failing on Windows. Contributed by Anbang Hu. 2018-05-29 09:11:08 -07:00
Karthik Palanisamy
61df174e8b
HADOOP-15449. Increase default timeout of ZK session to avoid frequent NameNode failover 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-05-28 19:41:07 +09:00
Johan Gustavsson
d14e26b31f
HADOOP-15477. Make unjar in RunJar overrideable 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-05-28 17:29:59 +09:00
Xiao Chen
02322de3f9 HADOOP-15473. Configure serialFilter in KeyProvider to avoid UnrecoverableKeyException caused by JDK-8189997. Contributed by Gabor Bota. 2018-05-25 09:10:51 -07:00
Steve Loughran
bddfe796f2 HADOOP-15494. TestRawLocalFileSystemContract fails on Windows. 
Contributed by Anbang Hu.
 2018-05-25 11:12:47 +01:00
Arpit Agarwal
51ce02bb54 HADOOP-15486. Make NetworkTopology#netLock fair. Contributed by Nanda kumar. 2018-05-23 10:30:12 -07:00
Robert Kanter
aa23d49fc8 HADOOP-15457. Add Security-Related HTTP Response Header in WEBUIs. (kanwaljeets via rkanter) 2018-05-23 10:24:09 -07:00
Sammi Chen
34e8b9f9a8 HDFS-13540. DFSStripedInputStream should only allocate new buffers when reading. Contributed by Xiao Chen. 2018-05-23 19:10:09 +08:00
Akira Ajisaka
745f203e57
Additional check when unpacking archives. Contributed by Jason Lowe and Akira Ajisaka. 2018-05-23 17:16:23 +09:00
Miklos Szegedi
3e5f7ea986 YARN-8310. Handle old NMTokenIdentifier, AMRMTokenIdentifier, and ContainerTokenIdentifier formats. Contributed by Robert Kanter. 2018-05-22 18:10:33 -07:00
Kihwal Lee
bcc8e76bad HADOOP-15450. Avoid fsync storm triggered by DiskChecker and handle disk full situation. Contributed by Arpit Agarwal. 2018-05-22 11:20:51 -05:00
Mukul Kumar Singh
57c2feb0d3 HADOOP-15474. Rename properties introduced for <tags>. Contributed by Zsolt Venczel. 2018-05-22 13:33:31 +05:30
Rushabh Shah
73e9120ad7 Skip the proxy user check if the ugi has not been initialized. Contributed by Daryn Sharp 2018-05-21 12:33:00 -05:00
Xiao Chen
89f5911392 HADOOP-15154. Abstract new method assertCapability for StreamCapabilities testing. Contributed by Zsolt Venczel. 2018-05-18 11:29:20 -07:00
Inigo Goiri
8783613696 HDFS-13586. Fsync fails on directories on Windows. Contributed by Lukas Majercack. 2018-05-17 19:26:44 -07:00
Inigo Goiri
53b807a6a8 HDFS-13560. Insufficient system resources exist to complete the requested service for some tests on Windows. Contributed by Anbang Hu. 2018-05-17 17:03:23 -07:00
Steve Loughran
07d8505f75 HADOOP-15466. Correct units in adl.http.timeout. 
Contributed by Sean Mackrory,
 2018-05-15 16:19:03 +01:00
Sammi Chen
f3f544b004 Add 2.9.1 release notes and changes documents 2018-05-14 15:14:02 +08:00
sammichen
66c9905b57 Revert "Add 2.9.1 release notes and changes documents" 
This reverts commit e4dc346d65.
 2018-05-14 14:38:40 +08:00
littlezhou
e4dc346d65 Add 2.9.1 release notes and changes documents 2018-05-14 14:24:01 +08:00
Rushabh Shah
a5449d3631 HADOOP-15441. Log kms url and token service at debug level. Contributed by Gabor Bota 2018-05-12 12:19:13 -05:00
Haibo Chen
274eee3284 MAPREDUCE-7095. Race conditions in closing FadvisedChunkedFile. (Miklos Szegedi via Haibo Chen) 2018-05-10 10:46:55 -07:00
Inigo Goiri
1da8d4190d HADOOP-15454. TestRollingFileSystemSinkWithLocal fails on Windows. Contributed by Xiao Liang. 2018-05-10 09:41:16 -07:00
Sean Mackrory
1cfe7506f7 HADOOP-15356. Make HTTP timeout configurable in ADLS connector. Contributed by Atul Sikaria and Sean Mackrory. 2018-05-09 15:15:51 -06:00
Anu Engineer
3a43ac2851 HDDS-1. Remove SCM Block DB. Contributed by Xiaoyu Yao. 2018-05-07 14:58:52 -07:00
Xiao Chen
a3a1552c33 Revert "HADOOP-14445. Delegation tokens are not shared between KMS instances. Contributed by Xiao Chen and Rushabh S Shah." 
This reverts commit 583fa6ed48.

 Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/kms/KMSTokenRenewer.java
	hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/TestKMS.java
 2018-05-07 13:32:27 -07:00
Rushabh Shah
4cdbdce752 HADOOP-14841 Kms client should disconnect if unable to get output stream from connection. Contributed by Rushabh S Shah 2018-05-04 15:36:13 -05:00
Daniel Templeton
87c23ef643 HDFS-13481. TestRollingFileSystemSinkWithHdfs#testFlushThread: test failed intermittently 
(Contributed by Gabor Bota via Daniel Templeton)

Change-Id: I9921981dfa69669fe7912dd2a31ae8b638283204
 2018-05-02 17:13:40 -07:00
Steve Loughran
fe649bb305 HADOOP-12071. conftest is not documented. 
Contributed by Kengo Seki.
 2018-05-02 13:33:56 +01:00
BELUGA BEHR
33768724ff
HADOOP-15377. Improve debug messages in MetricsConfig.java 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-05-02 17:09:22 +09:00
Steve Loughran
8f42dafcf8 HADOOP-15250. Split-DNS MultiHomed Server Network Cluster Network IPC Client Bind Addr Wrong 
Contributed by Ajay Kumar
 2018-05-01 22:32:40 +01:00
Wei-Chiu Chuang
bff3d7b0cf HADOOP-15382. Log kinit output in credential renewal thread. Contributed by Gabor Bota. 2018-04-27 10:05:55 -07:00
Ewan Higgs
84ecfe3ceb
HADOOP-14188. Remove the usage of org.mockito.internal.util.reflection.Whitebox 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-04-27 19:21:37 +09:00
Wangda Tan
13d389bf51 HADOOP-15411. AuthenticationFilter should use Configuration.getPropsWithPrefix instead of iterator. (Suma Shivaprasad via wangda) 
Change-Id: Ia19458a43aceb4ffdb1e8eccc519655235d92ba9
 2018-04-26 13:54:40 -07:00
Daryn Sharp
bbc8dfcb8b HADOOP-15402. Prevent double logout of UGI's LoginContext 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-04-26 13:54:40 -07:00
Owen O'Malley
2adda92de1
HDFS-7240 git commit fixup 
Signed-off-by: Owen O'Malley <omalley@apache.org>
 2018-04-26 13:52:17 -07:00
Mukul Kumar Singh
8cc13ff962 HDFS-13425. Ozone: Clean-up of ozone related change from hadoop-common-project. Contributed by Lokesh Jain. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh
ec6c8742e5 HDFS-13415. Ozone: Remove cblock code from HDFS-7240. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
8b832f3c35 HDFS-13405. Ozone: Rename HDSL to HDDS. 
Contributed by Ajay Kumar, Elek Marton, Mukul Kumar Singh, Shashikant Banerjee and Anu Engineer.
 2018-04-26 05:36:04 -07:00
Anu Engineer
4f37ff7616 HDFS-13341. Ozone: Move ozone specific ServiceRuntimeInfo utility to the framework. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
4e61bc431e HDFS-13258. Ozone: restructure Hdsl/Ozone code to separated maven subprojects. 
Contributed by Elek Marton, Mukul Kumar Singh, Xiaoyu Yao, Ajay Kumar, Anu Engineer, Lokesh Jain, Nanda Kumar.
 2018-04-26 05:36:04 -07:00
Anu Engineer
740a06cdd7 HDFS-12719. Ozone: Fix checkstyle, javac, whitespace issues in HDFS-7240 branch. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Anu Engineer
06c6582694 HDFS-12585. Ozone: KSM UI: Add description for configs in UI. Contributed by Ajay Kumar. 2018-04-26 05:36:04 -07:00
Anu Engineer
0b3270a8de HDFS-12513. Ozone: Create UI page to show Ozone configs by tag. Contributed by Ajay Kumar. 2018-04-26 05:36:04 -07:00
Chris Douglas
6d3d01d7ad HADOOP-12077. Provide a multi-URI replication Inode for ViewFs. Contributed by Gera Shegalov 2018-04-26 05:36:04 -07:00
Anu Engineer
dcc21a4f93 HDFS-12005. Ozone: Web interface for SCM. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer
acfae2716a HDFS-12034. Ozone: Web interface for KSM. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Yiqun Lin
8cf0c864c4 HDFS-12170. Ozone: OzoneFileSystem: KSM should maintain key creation time and modification time. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Rakesh Radhakrishnan
665f7c8b69 HDFS-11794. Add ec sub command -listCodec to show currently supported ec codecs. Contributed by SammiChen. 2018-04-26 05:36:04 -07:00
Robert Kanter
7ab08a9c37 HADOOP-15390. Yarn RM logs flooded by DelegationTokenRenewer trying to renew KMS tokens (xiaochen via rkanter) 2018-04-23 15:44:15 -07:00
Lei Xu
42e82f0281 Update CHANGES, RELEASENOTES, and jdiff for 3.0.2 release. 
(cherry picked from commit f6ecb76d0b919b9836600fe28ec9e637b223cd54)

Conflicts:
	hadoop-project-dist/pom.xml
 2018-04-23 14:14:05 -07:00
Virajith Jalaparti
827d8593a5 HDFS-13311. RBF: TestRouterAdminCLI#testCreateInvalidEntry fails on Windows. Contributed by Inigo Goiri 2018-04-16 15:55:00 -07:00
Bharat Viswanadham
ec1e8c1a8c HADOOP-15379. Make IrqHandler.bind() public. Contributed by Ajay Kumar 2018-04-12 21:51:20 -07:00
Aaron Fabbri
0d898b7bb8
HADOOP-12502 SetReplication OutOfMemoryError. Contributed by Vinayakumar B. 2018-04-11 17:19:56 -07:00
Xiao Chen
7c9cdad6d0 HDFS-13056. Expose file-level composite CRCs in HDFS which are comparable across different instances/layouts. Contributed by Dennis Huo. 2018-04-10 21:31:48 -07:00
Xiao Chen
583fa6ed48 HADOOP-14445. Delegation tokens are not shared between KMS instances. Contributed by Xiao Chen and Rushabh S Shah. 2018-04-10 15:38:25 -07:00
Jason Lowe
e81397545a HADOOP-15357. Configuration.getPropsWithPrefix no longer does variable substitution. Contributed by Jim Brennan 2018-04-10 16:44:55 -05:00
Xiaoyu Yao
8ab776d61e HADOOP-15340. Provide meaningful RPC server name for RpcMetrics. Contributed by Elek Marton. 2018-04-10 11:42:54 -07:00
Ewan Higgs
cef8eb7981
HADOOP-15376. Remove double semi colons on imports that make Clover fall over. 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-04-10 23:58:26 +09:00
Bharat
0006346abe HADOOP-15328. Fix the typo in HttpAuthentication.md. Contributed by fang zhenyi 2018-04-09 16:37:49 -07:00
Mukul Kumar Singh
0b345b7653 HADOOP-15366. Add a helper shutdown routine in HadoopExecutor to ensure clean shutdown. Contributed by Shashikant Banerjee. 2018-04-07 16:29:01 +05:30
Wangda Tan
6cf023f9b7 Added CHANGES/RELEASES/Jdiff for 3.1.0 release 
Change-Id: Ied5067a996151c04d15cad46c46ac98b60c37b39
(cherry picked from commit 2d96570452a72569befdf9cfe9b90c9fa2e0e261)
 2018-04-05 15:52:39 -07:00
Wei Yan
3121e8c293 HDFS-13353. RBF: TestRouterWebHDFSContractCreate failed. Contributed by Takanobu Asanuma. 2018-04-05 12:00:52 -07:00
Miklos Szegedi
e52539b46f HADOOP-14855. Hadoop scripts may errantly believe a daemon is still running, preventing it from starting. Contributed by Robert Kanter. 2018-04-04 15:35:58 -07:00
Xiao Chen
1077392eaa HADOOP-15355. TestCommonConfigurationFields is broken by HADOOP-15312. Contributed by LiXin Ge. 2018-04-02 23:10:08 -07:00
Xiao Chen
57374c4737 HADOOP-15317. Improve NetworkTopology chooseRandom's loop. 2018-04-02 22:48:26 -07:00
Steve Loughran
dde1579096 HADOOP-15146. Remove DataOutputByteBuffer. 
Contributed by BELUGA BEHR.
 2018-04-02 15:52:40 +01:00
Konstantin V Shvachko
acfd764fcc HADOOP-15253. Should update maxQueueSize when refresh call queue. Contributed by Tao Jie. 2018-03-30 18:55:35 -07:00
Konstantin V Shvachko
2216bde322 HADOOP-12862. LDAP Group Mapping over SSL can not specify trust store. Contributed by Wei-Chiu Chuang and Konstantin Shvachko. 2018-03-29 17:16:23 -07:00
Wei-Chiu Chuang
3fe41c65d8 HADOOP-15312. Undocumented KeyProvider configuration keys. Contributed by LiXin Ge. 2018-03-27 13:33:41 -07:00
Xiao Chen
c22d62b338 HADOOP-15313. TestKMS should close providers. 2018-03-26 15:59:32 -07:00
Xiaoyu Yao
22194f3d21 HADOOP-15339. Support additional key/value propereties in JMX bean registration. Contributed by Elek Marton. 2018-03-26 10:45:29 -07:00
Anu Engineer
28790b81ec HADOOP-15295. Remove redundant logging related to tags from Configuration. Contributed by Ajay Kumar. 2018-03-23 21:17:18 -07:00
Lei Xu
f9427b73a2 Update releasenotes and changelogs for 3.0.1 release 2018-03-23 11:46:41 -07:00
Jitendra Pandey
4bea96f9a8 HADOOP-14067. VersionInfo should load version-info.properties from its own classloader. Contributed by Thejas M Nair. 2018-03-22 14:09:06 -07:00
Yufei Gu
268c29a5f5 HADOOP-15331. Fix a race condition causing parsing error of java.io.BufferedInputStream in class org.apache.hadoop.conf.Configuration. Contributed by Miklos Szegedi. 2018-03-22 11:04:37 -07:00
Yufei Gu
138c1ed566 HADOOP-15062. TestCryptoStreamsWithOpensslAesCtrCryptoCodec fails on Debian 9. Contributed by Miklos Szegedi. 2018-03-20 15:19:18 -07:00
Inigo Goiri
fe224ff972 HDFS-13268. TestWebHdfsFileContextMainOperations fails on Windows. Contributed by Xiao Liang. 2018-03-20 11:19:46 -07:00
Inigo Goiri
e65ff1c8be HDFS-13296. GenericTestUtils generates paths with drive letter in Windows and fail webhdfs related test cases. Contributed by Xiao Liang. 2018-03-19 17:19:30 -07:00
Chris Douglas
3fc3fa9711 HADOOP-14667. Flexible Visual Studio support. Contributed by Allen Wittenauer 2018-03-19 16:05:55 -07:00
Steve Loughran
1976e0066e HADOOP-15209. DistCp to eliminate needless deletion of files under already-deleted directories. 
Contributed by Steve Loughran.
 2018-03-15 18:05:14 +00:00
Aaron Fabbri
e840b4adf4
HADOOP-15278 log s3a at info. Contributed by Steve Loughran. 2018-03-14 15:47:56 -07:00
Wangda Tan
252c2b4d52 Revert "HADOOP-13707. If kerberos is enabled while HTTP SPNEGO is not configured, some links cannot be accessed. Contributed by Yuanbo Liu" 
Change-Id: I946a466a43d56c73bb0135384e73cb8513595461
(cherry picked from commit 80ee5248b2dda1cb8d122d4f362f2f8cf02b9467)
 2018-03-14 13:36:53 -07:00
Takanobu Asanuma
fea16a440d
HADOOP-15294. TestUGILoginFromKeytab fails on Java9 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-03-14 13:38:07 +09:00
Inigo Goiri
427fd027a3 HADOOP-15308. TestConfiguration fails on Windows because of paths. Contributed by Xiao Liang. 2018-03-13 20:01:07 -07:00
Chris Douglas
9d6994da19 HADOOP-15311. HttpServer2 needs a way to configure the acceptor/selector count. Contributed by Erik Krogen 2018-03-13 13:55:18 -07:00
Chris Douglas
45d1b0fdcc HADOOP-14696. parallel tests don't work for Windows. Contributed by Allen Wittenauer 2018-03-12 20:05:39 -07:00
Steve Loughran
dd05871b8b HADOOP-15297. Make S3A etag => checksum feature optional. 
Contributed by Steve Loughran.
 2018-03-12 14:01:42 +00:00
Owen O'Malley
fa6a8b78d4 Revert "HADOOP-13119. Add ability to secure log servlet using proxy users. Contribute by Yuanbo Liu." 
This reverts commit a847903b6e.

Change-Id: I3122a2142f5bdf8507dece930e447556a43cd9ae
(cherry picked from commit 8fad3ec76070ccfcd3ed80feaba4355077bc6f5c)
 2018-03-09 22:46:41 -08:00
Owen O'Malley
3a8dade9b1 Revert "HADOOP-14077. Add ability to access jmx via proxy. Contributed by Yuanbo Liu." 
This reverts commit 172b23af33.

(cherry picked from commit d0d2d4c51e9534e08893ae14cf3fff7b2ee70b1d)
 2018-03-09 22:46:30 -08:00
Takanobu Asanuma
99ab511cba
HADOOP-15293. TestLogLevel fails on Java 9 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-03-09 10:20:35 -08:00
Steve Loughran
3f7bd46797 HADOOP-15277. Remove .FluentPropertyBeanIntrospector from CLI operation log output. 
Contributed by Steve Loughran.
 2018-03-09 10:44:07 +00:00
Steve Loughran
12ecb55ffe HADOOP-15289. FileStatus.readFields() assertion incorrect. 
Contributed by Steve Loughran.
 2018-03-06 10:05:12 +00:00
Takanobu Asanuma
4971276631
HADOOP-15271. Remove unicode multibyte characters from JavaDoc 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-03-05 15:07:58 -08:00
Nanda kumar
55d04a6db1
HADOOP-15275. Incorrect javadoc for return type of RetryPolicy#shouldRetry 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-03-01 19:03:47 +09:00
Xiaoyu Yao
f20e10b2dd HDFS-13081. Datanode#checkSecureConfig should allow SASL and privileged HTTP. Contributed by Ajay Kumar. 2018-02-28 09:44:39 -08:00
Arpit Agarwal
ac42dfc888 MAPREDUCE-7061. SingleCluster setup document needs to be updated. Contributed by Bharat Viswanadham. 2018-02-27 10:27:18 -08:00
Arpit Agarwal
137f0d324a HADOOP-14959. DelegationTokenAuthenticator.authenticate() to wrap network exceptions. Contributed by Ajay Kumar. 2018-02-27 10:15:43 -08:00
Arpit Agarwal
28f644bf25 HADOOP-15178. Generalize NetUtils#wrapException to handle other subclasses with String Constructor. Contributed by Ajay Kumar. 2018-02-27 07:38:29 -08:00
Arpit Agarwal
2fa7963c3d HADOOP-15254. Correct the wrong word spelling 'intialize'. Contributed by fang zhenyi. 2018-02-24 14:41:55 -08:00
Arpit Agarwal
033f9c68ea HADOOP-15255. Upper/Lower case conversion support for group names in LdapGroupsMapping. Contributed by Nanda kumar. 2018-02-23 15:37:17 -08:00
Kihwal Lee
59cf758877 HADOOP-9747. Reduce unnecessary UGI synchronization. Contributed by Daryn Sharp. 2018-02-23 13:10:56 -06:00
Anu Engineer
3688e491d5 HADOOP-15007. Stabilize and document Configuration <tag> element. Contributed by Ajay Kumar. 2018-02-23 10:26:22 -08:00
Sean Mackrory
2bc3351eaf HADOOP-6852. apparent bug in concatenated-bzip2 support (decoding). Contributed by Zsolt Venczel. 2018-02-21 12:57:14 -07:00
Arpit Agarwal
b0d3c877e3 HADOOP-12897. KerberosAuthenticator.authenticate to include URL on IO failures. Contributed by Ajay Kumar. 2018-02-20 18:18:58 -08:00
Steve Loughran
1d37cf675c HADOOP-15070. add test to verify FileSystem and paths differentiate on user info. 
Contributed by Steve Loughran.
 2018-02-19 20:43:40 +00:00
fang zhenyi
4d4dde5112
HADOOP-15223. Replace Collections.EMPTY* with empty* when available 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-02-18 22:19:39 +09:00
Wangda Tan
31db977cb3 HADOOP-14875. Create end user documentation from the compatibility guidelines. (Daniel Templeton via wangda) 
Change-Id: I5d66d0f034874623af22aecae0a0b858b65396ed
 2018-02-17 12:28:55 +08:00
Jason Lowe
0898ff42e9 HADOOP-15206. BZip2 drops and duplicates records when input split size is small. Contributed by Aki Tanaka 2018-02-16 14:49:00 -06:00
Steve Loughran
9a013b255f HADOOP-15176. Enhance IAM Assumed Role support in S3A client. 
Contributed by Steve Loughran

(cherry picked from commit 96c047fbb98c2378eed9693a724d4cbbd03c00fd)
 2018-02-15 15:57:10 +00:00
Anu Engineer
8f66affd62 HADOOP-15204. Add Configuration API for parsing storage sizes. Contributed by Anu Engineer. 2018-02-14 13:11:37 -08:00
Steve Loughran
f20dc0d577 HADOOP-10571. Use Log.*(Object, Throwable) overload to log exceptions. 
Contributed by Andras Bokor.
 2018-02-14 16:20:14 +00:00
Chris Douglas
0c5d7d71a8 HADOOP-15195. With SELinux enabled, directories mounted with start-build-env.sh may not be accessible. Contributed by Grigori Rybkine 2018-02-12 21:07:15 -08:00
Chris Douglas
9cc6d1dfb3 Revert "HADOOP-15195. With SELinux enabled, directories mounted with start-build-env.sh may not be accessible. Contributed by Grigori Rybkine" 
This reverts commit 5b88cb3398.
 2018-02-12 21:06:10 -08:00
Chris Douglas
5b88cb3398 HADOOP-15195. With SELinux enabled, directories mounted with start-build-env.sh may not be accessible. Contributed by Grigori Rybkine 2018-02-12 21:00:47 -08:00
Wangda Tan
60f9e60b3b Preparing for 3.2.0 development 
Change-Id: I6d0e01f3d665d26573ef2b957add1cf0cddf7938
 2018-02-11 11:17:38 +08:00
Steve Loughran
996796f104 HADOOP-15214. Make Hadoop compatible with Guava 21.0. 
Contributed by Igor Dvorzhak
 2018-02-08 10:55:54 -08:00
Anu Engineer
4304fcd5bd HDFS-12990. Change default NameNode RPC port back to 8020. Contributed by Xiao Chen. 2018-02-06 13:43:45 -08:00
Kihwal Lee
2dd960de98 HADOOP-15212. Add independent secret manager method for logging expired tokens. Contributed by Daryn Sharp. 2018-02-06 15:33:18 -06:00
Jason Lowe
33e6cdbd16 YARN-7876. Localized jars that are expanded after localization are not fully copied. Contributed by Miklos Szegedi 2018-02-05 12:30:20 -06:00
Hanisha Koneru
51cb6c5380 HADOOP-15168. Add kdiag tool to hadoop command. Contributed by Bharat Viswanadham. 2018-02-02 12:51:27 -08:00
Jason Lowe
460d77bd64 HADOOP-15170. Add symlink support to FileUtil#unTarUsingJava. Contributed by Ajay Kumar 2018-02-02 11:33:26 -06:00
Xiaoyu Yao
af015c0b23 HDFS-13060. Adding a BlacklistBasedTrustedChannelResolver for TrustedChannelResolver. Contributed by Ajay Kumar. 2018-01-31 22:39:51 -08:00
Eric Yang
0bee3849e3 YARN-7816. Allow same application name submitted by multiple users. (Contributed by Gour Saha) 2018-01-31 20:51:40 -05:00
Jason Lowe
7288b08330 YARN-2185. Use pipes when localizing archives. Contributed by Miklos Szegedi 2018-01-31 09:39:43 -06:00
Jason Lowe
901d15a30b Revert "YARN-2185. Use pipes when localizing archives. Contributed by Miklos Szegedi" 
This reverts commit 1b0f265db1.
 2018-01-30 08:34:39 -06:00
Xiao Chen
b63dcd583f HDFS-12974. Exception message is not printed when creating an encryption zone fails with AuthorizationException. Contributed by fang zhenyi. 2018-01-28 22:19:49 -08:00
Haibo Chen
649ef7ac33 YARN-7064. Use cgroup to get container resource utilization. (Miklos Szegedi via Haibo Chen) 2018-01-26 16:27:31 -08:00
Jason Lowe
1b0f265db1 YARN-2185. Use pipes when localizing archives. Contributed by Miklos Szegedi 2018-01-26 13:25:20 -06:00
Chris Douglas
56872cff92 MapFile.fix creates a wrong index file in case of block-compressed data file. Contributed by Grigori Rybkine 2018-01-26 09:18:30 -08:00
Wei Yan
0c139d5bcf HDFS-13049. RBF: Inconsistent Router OPTS config in branch-2 and branch-3. Contributed by Wei Yan. 2018-01-25 11:36:40 -08:00
Billie Rinaldi
e307edcb47 YARN-7540 and YARN-7605. Convert yarn app cli to call yarn api services and implement doAs for Api Service REST API. Contributed by Eric Yang and Jian He 2018-01-23 17:54:39 -08:00
Hanisha Koneru
6347b2253d HDFS-13023. Journal Sync does not work on a secure cluster. Contributed by Bharat Viswanadham. 2018-01-22 16:02:32 -08:00
Hanisha Koneru
3fde0f1db5 HADOOP-15121. Encounter NullPointerException when using DecayRpcScheduler. Contributed by Tao Jie. 2018-01-22 15:54:44 -08:00
Masahiro Tanaka
6e27b20b4d
HADOOP-15181. Typo in SecureMode.md 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-01-22 17:12:36 +09:00
Vrushali C
c191538ed1 HADOOP-15166 CLI MiniCluster fails with ClassNotFoundException o.a.h.yarn.server.timelineservice.collector.TimelineCollectorManager. Contributed by Gera Shegalov 2018-01-19 16:15:55 -08:00
Steve Loughran
d689b2d99c HADOOP-15114. Add closeStreams(...) to IOUtils (addendum). 
Contributed by Ajay Kumar.
 2018-01-19 14:54:13 +00:00
Steve Loughran
e5a1ad6e24 HADOOP-14788. Credentials readTokenStorageFile to stop wrapping IOEs in IOEs. 
Contributed by Ajay Kumar.
 2018-01-19 14:48:45 +00:00
Steve Loughran
de630708d1 HADOOP-15123. KDiag tries to load krb5.conf from KRB5CCNAME instead of KRB5_CONFIG. 
Contributed by Vipin Rathor.

(cherry picked from commit 1ef906e29e0989aafcb35c51ad2acbb262b3c8e7)
(cherry picked from commit f61edab1d0ea08b6d752ecdfb6068103822012ec)
 2018-01-18 12:35:57 +00:00
Steve Loughran
f274fe33ea Revert "HADOOP-13974. S3Guard CLI to support list/purge of pending multipart commits." 
This reverts commit 35ad9b1dd2.
 2018-01-18 12:35:57 +00:00
Brahma Reddy Battula
08332e12d0 HADOOP-15150. in FsShell, UGI params should be overidden through env vars(-D arg). Contributed by Brahma Reddy Battula. 2018-01-18 10:54:32 +05:30
Aaron Fabbri
268ab4e027
HADOOP-15141 Support IAM Assumed roles in S3A. Contributed by Steve Loughran. 2018-01-17 00:05:24 -08:00
Yufei Gu
b202935353 HADOOP-15157. Zookeeper authentication related properties to support CredentialProviders. (Contributed by Gergo Repas) 2018-01-16 10:15:22 -08:00
Arpit Agarwal
7016dd44e0 HDFS-13016. globStatus javadoc refers to glob pattern as "regular expression". Contributed by Mukul Kumar Singh. 2018-01-13 07:19:30 -08:00
Arpit Agarwal
addbcd8cd4 HADOOP-15114. Add closeStreams(...) to IOUtils. Contributed by Ajay Kumar. 2018-01-11 16:50:21 -08:00
Steve Loughran
2e0a451a8d HADOOP-15033. Use java.util.zip.CRC32C for Java 9 and above 
Contributed by Dmitry Chuyko,
 2018-01-11 10:47:50 +00:00
Miklos Szegedi
12d0645990 HADOOP-15060. TestShellBasedUnixGroupsMapping.testFiniteGroupResolutionTime flaky. Contributed by Miklos Szegedi. 2018-01-10 10:52:26 -08:00
Haibo Chen
2f6c038be6 YARN-7602. NM should reference the singleton JvmMetrics instance. 2018-01-03 09:41:26 -08:00
Arpit Agarwal
4ad39ec308 Revert "HADOOP-10054. ViewFsFileStatus.toString() is broken. Contributed by Hanisha Koneru." 
This reverts commit 37efa67e37.
 2018-01-02 17:34:29 -08:00
Arpit Agarwal
d82874851e HADOOP-15155. Error in javadoc of ReconfigurableBase#reconfigureProperty. Contributed by Ajay Kumar. 2018-01-02 17:22:55 -08:00
Arpit Agarwal
dfe0cd8655 HADOOP-15152. Typo in javadoc of ReconfigurableBase#reconfigurePropertyImpl. Contributed by Nanda kumar. 2018-01-02 10:50:13 -08:00
Chris Douglas
7fe6f83c8f HADOOP-15117. open(PathHandle) contract test should be exhaustive for default options 2017-12-30 17:58:35 -08:00
Xiao Chen
81127616c5 HADOOP-15149. CryptoOutputStream should implement StreamCapabilities. 2017-12-29 13:41:15 -08:00
Jitendra Pandey
d31c9d8c49 HADOOP-15143. NPE due to Invalid KerberosTicket in UGI. Contributed by Mukul Kumar Singh. 2017-12-27 23:17:07 -08:00
Steve Loughran
c8ff0cc304 HADOOP-13282. S3 blob etags to be made visible in S3A status/getFileChecksum() calls. 
Contributed by Steve Loughran
 2017-12-21 14:58:58 +00:00
Junping Du
a7f8caf58e Add 2.8.3 release jdiff files. 
(cherry picked from commit c89f99aade575ab1f6a9836df719cda272293d90)
 2017-12-18 22:39:01 -08:00
Steve Loughran
35ad9b1dd2 HADOOP-13974. S3Guard CLI to support list/purge of pending multipart commits. 
Contributed by Aaron Fabbri
 2017-12-18 21:19:06 +00:00
Chris Douglas
5e81f32d11 HADOOP-15106. FileSystem::open(PathHandle) should throw a specific exception on validation failure 2017-12-16 10:53:10 -08:00
Virajith Jalaparti
b668eb9155 HDFS-10675. Datanode support to read from external stores. 2017-12-15 17:51:37 -08:00
Xiaoyu Yao
37efa67e37 HADOOP-10054. ViewFsFileStatus.toString() is broken. Contributed by Hanisha Koneru. 2017-12-14 10:58:16 -08:00
Jason Lowe
f8af0e2feb HADOOP-15085. Output streams closed with IOUtils suppressing write errors. Contributed by Jim Brennan 2017-12-14 10:18:08 -06:00
Yiqun Lin
91c96bdf8e HDFS-12883. RBF: Document Router and State Store metrics. Contributed by Yiqun Lin. 2017-12-14 16:00:53 +08:00
Kai Zheng
f86c81d923 HADOOP-15039. Move SemaphoredDelegatingExecutor to hadoop-common. Contributed by Genmao Yu 2017-12-14 11:14:52 +08:00
Andrew Wang
d447152d49 Update CHANGES, RELEASENOTES, jdiff for 3.0.0 release. 2017-12-13 13:05:32 -08:00
Chris Douglas
693169ef34 HDFS-12882. Support full open(PathHandle) contract in HDFS 2017-12-11 20:14:15 -08:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
f19638333b HADOOP-15059. Undoing the switch of Credentials to PB format as default - done via HADOOP-12563 for supporting 2.x to 3.x upgrades. 2017-12-08 08:00:21 -08:00
Xiao Chen
19e0894209 HADOOP-15056. Fix TestUnbuffer#testUnbufferException failure. Contributed by Jack Bearden. 2017-12-07 21:05:55 -08:00
Sunil G
daa1cdd062 YARN-7564. Cleanup to fix checkstyle issues of YARN-5881 branch. Contributed by Sunil G. 2017-12-07 18:56:54 -08:00
Wangda Tan
5e798b1a0d YARN-6471. Support to add min/max resource configuration for a queue. (Sunil G via wangda) 
Change-Id: I9213f5297a6841fab5c573e85ee4c4e5f4a0b7ff
 2017-12-07 18:56:54 -08:00
Sean Mackrory
acb92904d0 HADOOP-15098. TestClusterTopology#testChooseRandom fails intermittently. Contributed by Zsolt Venczel. 2017-12-07 12:30:58 -07:00
Lei Xu
67662e2ac9 HDFS-12840. Creating a file with non-default EC policy in a EC zone is not correctly serialized in the editlog. Contributed by Lei (Eddy) Xu. 2017-12-07 11:15:40 -08:00
Arpit Agarwal
e00c7f78c1 HADOOP-14976. Set HADOOP_SHELL_EXECNAME explicitly in scripts. 2017-12-04 21:02:04 -08:00
Xiao Chen
404eab4dc0 HDFS-12396. Webhdfs file system should get delegation token from kms provider. Contributed by Rushabh S Shah. 2017-12-04 10:40:11 -08:00
Chris Douglas
42307e3c3a HDFS-11576. Block recovery will fail indefinitely if recovery time > heartbeat interval. Contributed by Lukas Majercak 2017-12-01 22:34:30 -08:00
Chris Douglas
f9d195dfe9 HADOOP-14600. LocatedFileStatus constructor forces RawLocalFS to exec a process to get the permissions. Contributed by Ping Liu 2017-12-01 22:12:26 -08:00
Chris Douglas
53bbef3802 Revert "HDFS-11576. Block recovery will fail indefinitely if recovery time > heartbeat interval. Contributed by Lukas Majercak" 
This reverts commit 5304698dc8.
 2017-12-01 11:19:38 -08:00
Chris Douglas
5304698dc8 HDFS-11576. Block recovery will fail indefinitely if recovery time > heartbeat interval. Contributed by Lukas Majercak 2017-12-01 10:29:30 -08:00
Chris Douglas
0780fdb1eb HDFS-12877. Add open(PathHandle) with default buffersize 2017-11-30 15:13:16 -08:00
Robert Kanter
75a3ab88f5 HADOOP-13493. Compatibility Docs should clarify the policy for what takes precedence when a conflict is found (templedf via rkanter) 2017-11-30 07:39:15 -08:00
Chris Douglas
0e560f3b8d HDFS-12681. Make HdfsLocatedFileStatus a subtype of LocatedFileStatus 2017-11-29 20:28:06 -08:00
Xiao Chen
d162252d7a HADOOP-15067. GC time percentage reported in JvmMetrics should be a gauge, not counter. Contributed by Misha Dmitriev. 2017-11-23 09:01:28 -08:00
Steve Loughran
de8b6ca5ef HADOOP-13786 Add S3A committer for zero-rename commits to S3 endpoints. 
Contributed by Steve Loughran and Ryan Blue.
 2017-11-22 15:28:12 +00:00
Robert Kanter
f2efaf013f HADOOP-14982. Clients using FailoverOnNetworkExceptionRetry can go into a loop if they're used without authenticating with kerberos in HA env (pbacsko via rkanter) 2017-11-16 11:11:19 -08:00
Chris Douglas
675e9a8f57 Revert "HDFS-12681. Fold HdfsLocatedFileStatus into HdfsFileStatus." 
This reverts commit b85603e3f8.
 2017-11-15 19:20:37 -08:00
Xiao Chen
b1941b200d HADOOP-15023. ValueQueue should also validate (int) (lowWatermark * numValues) > 0 on construction. 2017-11-15 16:44:06 -08:00
Daniel Templeton
69043ba8b5 HADOOP-14876. Create downstream developer docs from the compatibility guidelines 2017-11-15 10:03:29 -08:00
Arun Suresh
b07e68b02a Addendum patch for Configuration fix. (Jason Lowe via asuresh) 2017-11-13 14:03:50 -08:00
Eric Yang
1b68b8ff2c HADOOP-15008. Fixed period unit calculation for Hadoop Metrics V2. (Contribute by Erik Krogen) 2017-11-13 12:42:43 -05:00
Akira Ajisaka
975a57a688
HADOOP-15031. Fix javadoc issues in Hadoop Common. Contributed by Mukul Kumar Singh. 2017-11-13 23:12:23 +09:00
Chris Douglas
796a0d3a5c HADOOP-8522. ResetableGzipOutputStream creates invalid gzip files when finish() and resetState() are used. Contributed by Mike Percy 2017-11-10 17:41:29 -08:00
Xiao Chen
3c6adda291 HADOOP-14960. Add GC time percentage monitor/alerter. Contributed by Misha Dmitriev. 2017-11-09 21:06:06 -08:00
Arun Suresh
5eb7dbe9b3 Fixing Job History Server Configuration parsing. (Jason Lowe via asuresh) 2017-11-09 15:15:51 -08:00
John Zhuge
6c32ddad30 HADOOP-14872. CryptoInputStream should implement unbuffer. Contributed by John Zhuge. 2017-11-09 10:16:12 -08:00
John Zhuge
bf6a660232 HADOOP-15012. Add readahead, dropbehind, and unbuffer to StreamCapabilities. Contributed by John Zhuge. 2017-11-09 10:16:12 -08:00
Mingliang Liu
b6d8f87b8e HADOOP-15015. TestConfigurationFieldsBase to use SLF4J for logging. Contributed by Steve Loughran 2017-11-03 21:01:56 -07:00
Xiaoyu Yao
59d78a5088 HADOOP-14987. Improve KMSClientProvider log around delegation token checking. Contributed by Xiaoyu Yao and Xiao Chen. 2017-11-03 16:10:37 -07:00
Chris Douglas
b85603e3f8 HDFS-12681. Fold HdfsLocatedFileStatus into HdfsFileStatus. 2017-11-03 14:30:57 -07:00
Chris Douglas
d015e0bbd5 HDFS-7878. API - expose a unique file identifier. 2017-10-31 09:44:01 -07:00
Sean Mackrory
81225430d0 HDFS-206. Support for head in FSShell. Contributed by Gabor Bota. 2017-10-31 08:34:01 -06:00
Jason Lowe
2fae63aa60 HADOOP-14919. BZip2 drops records when reading data in splits. Contributed by Jason Lowe 2017-10-31 09:30:13 -05:00
Junping Du
a25b5aa0cf Add several jdiff xml files for 2.8.2 release. 2017-10-26 19:30:48 -07:00
Junping Du
1ad863e220 Update CHANGES.md and RELEASENOTES for 2.8.2 release. 2017-10-26 19:13:58 -07:00
Xiao Chen
f66ad1fab9 HADOOP-14957. ReconfigurationTaskStatus is exposing guava Optional in its public api. Contributed by Haibo Chen, Xiao Chen. 2017-10-25 14:11:30 -07:00
Akira Ajisaka
4c970088af
HADOOP-14030. PreCommit TestKDiag failure. Contributed by Wei-Chiu Chuang. 2017-10-25 17:17:21 +09:00
Varun Saxena
67e7673750 HADOOP-9657. NetUtils.wrapException to have special handling for 0.0.0.0 addresses and :0 ports. Contributed by Varun Saxena. 2017-10-25 03:06:12 +05:30
John Zhuge
c904d60c38 Revert "HADOOP-14954. MetricsSystemImpl#init should increment refCount when already initialized. Contributed by John Zhuge." 
This reverts commit 14b3c2695b.
 2017-10-23 15:32:25 -07:00
Naganarasimha
6b8122458e HADOOP-14966. Handle JDK-8071638 for hadoop-common. Contributed by Bibin A Chundatt. 2017-10-23 23:42:06 +05:30
Arpit Agarwal
4b00c9a47f HDFS-12683. DFSZKFailOverController re-order logic for logging Exception. Contributed by Bharat Viswanadham. 2017-10-23 10:47:16 -07:00
John Zhuge
14b3c2695b HADOOP-14954. MetricsSystemImpl#init should increment refCount when already initialized. Contributed by John Zhuge. 2017-10-20 16:13:41 -07:00
Kai Zheng
ce7cf66e5e HDFS-12448. Make sure user defined erasure coding policy ID will not overflow. Contributed by Huafeng Wang 2017-10-20 09:42:04 +08:00
Wei-Chiu Chuang
97c70c7ac6 HADOOP-14880. [KMS] Document&test missing KMS client side configs. Contributed by Gabor Bota. 2017-10-19 06:02:13 -07:00
Junping Du
b016f08f67 HADOOP-14958. Fix source-level compatibility after HADOOP-11252. Contributed by Junping Du. 2017-10-18 15:06:30 -07:00
Xiao Chen
86ee0c5e4e HADOOP-14944. Add JvmMetrics to KMS. 2017-10-17 19:06:45 -07:00
Lei Xu
31ebccc962 HDFS-12613. Native EC coder should implement release() as idempotent function. (Lei (Eddy) Xu) 2017-10-16 19:44:30 -07:00
Wei-Chiu Chuang
e906108fc9 HADOOP-14948. Document missing config key hadoop.treat.subject.external. Contributed by Ajay Kumar. 2017-10-16 16:42:59 -07:00
Manoj Govindassamy
133d7ca76e HADOOP-13055. Implement linkMergeSlash and linkFallback for ViewFileSystem 2017-10-13 17:43:21 -07:00
Robert Kanter
e163f41850 HADOOP-14938. Configuration.updatingResource map should be initialized lazily (misha@cloudera.com via rkanter) 2017-10-13 13:52:58 -07:00
Anu Engineer
075358eb6f HADOOP-13102. Update GroupsMapping documentation to reflect the new changes. Contributed by Esther Kundin. 2017-10-11 15:58:20 -07:00
Arun Suresh
b6c2c9058e HADOOP-13556. Change Configuration.getPropsWithPrefix to use getProps instead of iterator. (Larry McCay via asuresh) 2017-10-11 15:21:21 -07:00
Steve Loughran
3de574413c HADOOP-14913. Sticky bit implementation for rename() operation in Azure WASB. 
Contributed by Varada Hemeswari.
 2017-10-11 18:06:43 +01:00
Jason Lowe
1123f8f0b6 HADOOP-14912. FairCallQueue may defer servicing calls. Contributed by Daryn Sharp 2017-10-10 14:47:25 -05:00
John Zhuge
9676ed19be HADOOP-14184. Remove service loader config entry for ftp fs. Contributed by Sen Zhao. 2017-10-09 00:33:17 -07:00
Xiaoyu Yao
2b08a1fc64 HADOOP-14920. KMSClientProvider won't work with KMS delegation token retrieved from non-Java client. Contributed by Xiaoyu Yao. 2017-10-06 10:12:24 -07:00
Xiao Chen
25f31d9fc4 HADOOP-14521. KMS client needs retry logic. Contributed by Rushabh S Shah. 2017-10-05 19:46:47 -07:00
Akira Ajisaka
fe84cdc9da
HADOOP-14616. Client fails to read a block with erasure code (XOR, native) when one of the data block is lost. Contributed by Huafeng Wang. 2017-10-05 11:06:09 +09:00
Daniel Templeton
20e9ce3ab3 HADOOP-14459. SerializationFactory shouldn't throw a NullPointerException if the serializations list is not defined 
(Contributed by Nandor Kollar via Daniel Templeton)
 2017-10-04 15:04:02 +02:00
Andrew Wang
b34b3ff94b Add 3.0.0-beta1 CHANGES and RELEASENOTES. 2017-10-03 17:14:40 -07:00
Xiaoyu Yao
107c177782 HDFS-12455. WebHDFS - Adding "snapshot enabled" status to ListStatus query result. Contributed by Ajay Kumar. 2017-10-03 13:02:12 -07:00
Arpit Agarwal
b91305119b HDFS-11968. ViewFS: StoragePolicies commands fail with HDFS federation. Contributed by Mukul Kumar Singh. 2017-10-03 11:23:40 -07:00
Allen Wittenauer
4d5dd75b60
HADOOP-14908. CrossOriginFilter should trigger regex on more input (Johannes Alberti via aw) 2017-10-03 10:58:28 -07:00
Anu Engineer
442ea85545 HADOOP-14752. TestCopyFromLocal#testCopyFromLocalWithThreads is fleaky. Contributed by Andras Bokor. 2017-10-02 13:11:47 -07:00
Chen Liang
563dcdfc1d HADOOP-14915. method name is incorrect in ConfServlet. Contributed by Bharat Viswanadham. 2017-10-02 10:58:19 -07:00
Jason Lowe
6f789fe057 HADOOP-14902. LoadGenerator#genFile write close timing is incorrectly calculated. Contributed by Hanisha Koneru 2017-09-28 16:38:30 -05:00
Akira Ajisaka
d08b8c801a
HADOOP-14893. WritableRpcEngine should use Time.monotonicNow. Contributed by Chetna Chaudhari. 2017-09-26 16:46:03 +09:00
Anu Engineer
0889e5a8b7 HDFS-12529. Get source for config tags from file name. Contributed by Ajay Kumar. 2017-09-25 13:55:09 -07:00
Jason Lowe
ac05a51bbb HADOOP-14881. LoadGenerator should use Time.monotonicNow() to measure durations. Contributed by Bharat Viswanadham 2017-09-25 15:35:44 -05:00
Akira Ajisaka
3a10367a17
HADOOP-14892. MetricsSystemImpl should use Time.monotonicNow for measuring durations. Contributed by Chetna Chaudhari. 2017-09-26 01:47:02 +09:00
Daniel Templeton
0807470808 HADOOP-14897. Loosen compatibility guidelines for native dependencies 2017-09-25 14:12:25 +02:00
Anu Engineer
1c4a798505 HADOOP-14894. ReflectionUtils should use Time.monotonicNow to mesaure duration. Contributed by Bharat Viswanadham. 2017-09-23 11:56:34 -07:00
Anu Engineer
423ebcaf51 HDFS-12320. Add quantiles for transactions batched in Journal sync. Contributed by Hanisha Koneru. 2017-09-23 10:34:13 -07:00