212 Commits

Author SHA1 Message Date
Ahmed Hussein
98fcffe93f HADOOP-17101. Replace Guava Function with Java8+ Function
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
2020-07-15 09:53:18 -05:00
Eric Badger
fbb8775430 Revert "MAPREDUCE-7277. IndexCache totalMemoryUsed differs from cache contents. Contributed by Jon Eagles (jeagles)."
This reverts commit e2322e1117a2a3435aabecd49de0ad3c8d2f64b9.
2020-06-08 20:35:27 +00:00
Eric E Payne
e2322e1117 MAPREDUCE-7277. IndexCache totalMemoryUsed differs from cache contents. Contributed by Jon Eagles (jeagles). 2020-04-27 19:10:00 +00:00
Jason Lowe
c613296dc8 MAPREDUCE-7241. FileInputFormat listStatus with less memory footprint. Contributed by Zhihua Deng 2020-04-01 07:46:33 -05:00
Szilard Nemeth
ac6c4f0b29 MAPREDUCE-7197. Fix order of actual and expected expression in assert statements. Contributed by Adam Antal 2019-08-12 13:54:28 +02:00
Szilard Nemeth
a7371a779c MAPREDUCE-7225: Fix broken current folder expansion during MR job start. Contributed by Peter Bacsko. 2019-08-01 13:01:30 +02:00
Akira Ajisaka
1129288cf5
HADOOP-14178. Move Mockito up to version 2.23.4. Contributed by Akira Ajisaka and Masatake Iwasaki. 2019-01-29 18:29:56 -08:00
Jason Lowe
4d8de7ab69 MAPREDUCE-7164. FileOutputCommitter does not report progress while merging paths. Contributed by Kuhu Shukla 2018-11-28 14:54:59 -06:00
Xiao Chen
25f8fcb064 MAPREDUCE-7132. JobSplitWriter prints unnecessary warnings if EC(RS10,4) is used. Contributed by Peter Bacsko. 2018-10-16 10:23:31 -07:00
Haibo Chen
84973d1049 MAPREDUCE-6861. Add metrics tags for ShuffleClientMetrics. (Contributed by Zoltan Siegl) 2018-08-27 16:53:06 +02:00
Gera Shegalov
04b74eddc6 MAPREDUCE-7108. TestFileOutputCommitter fails on Windows. (Zuoming Zhang via gera) 2018-06-12 11:21:51 -07:00
Inigo Goiri
0afc036deb MAPREDUCE-7103. Fix TestHistoryViewerPrinter on windows due to a mismatch line separator. Contributed by Giovanni Matteo Fumarola. 2018-06-05 12:24:57 -07:00
Takanobu Asanuma
d1e2b80980
MAPREDUCE-7098. Upgrade common-langs version to 3.7 in hadoop-mapreduce-project
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-05-31 18:31:10 +09:00
Bibin A Chundatt
1a95a4524a
MAPREDUCE-7073. Optimize TokenCache#obtainTokensForNamenodesInternal
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-05-02 16:14:28 +09:00
Jason Lowe
68c6ec719d MAPREDUCE-7086. Add config to allow FileInputFormat to ignore directories when recursive=false. Contributed by Sergey Shelukhin 2018-05-01 16:19:53 -05:00
Ewan Higgs
84ecfe3ceb
HADOOP-14188. Remove the usage of org.mockito.internal.util.reflection.Whitebox
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-04-27 19:21:37 +09:00
Robert Kanter
1b9ecc264a MAPREDUCE-7072. mapred job -history prints duplicate counter in human output (wilfreds via rkanter) 2018-04-24 11:30:38 -07:00
Steve Loughran
dde1579096 HADOOP-15146. Remove DataOutputByteBuffer.
Contributed by BELUGA BEHR.
2018-04-02 15:52:40 +01:00
Takanobu Asanuma
86816da5b4
MAPREDUCE-7066. TestQueue fails on Java9
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-03-19 09:39:49 +09:00
Jason Lowe
87e2570a14 MAPREDUCE-7048. Uber AM can crash due to unknown task in statusUpdate. Contributed by Peter Bacsko 2018-02-12 13:21:09 -06:00
Eric Payne
5a725bb886 MAPREDUCE-7033: Map outputs implicitly rely on permissive umask for shuffle. Contributed by Jason Lowe 2018-01-31 15:45:30 -06:00
Jason Lowe
a37e7f0ad8 MAPREDUCE-7022. Fast fail rogue jobs based on task scratch dir size. Contributed by Johan Gustavsson 2018-01-26 14:36:45 -06:00
Jason Lowe
6e42d05829 MAPREDUCE-7029. FileOutputCommitter is slow on filesystems lacking recursive delete. Contributed by Karthik Palaniappan 2018-01-17 08:14:11 -06:00
Takanobu Asanuma
1a9c5d479e
MAPREDUCE-7034. Moving logging APIs over to slf4j the rest of all in hadoop-mapreduce
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-01-15 15:38:03 +09:00
Steve Loughran
de8b6ca5ef HADOOP-13786 Add S3A committer for zero-rename commits to S3 endpoints.
Contributed by Steve Loughran and Ryan Blue.
2017-11-22 15:28:12 +00:00
Chris Douglas
782ba3bf9d MAPREDUCE-7011. TestClientDistributedCacheManager::testDetermineCacheVisibilities assumes all parent dirs set other exec 2017-11-21 20:42:28 -08:00
Akira Ajisaka
178751ed8c
MAPREDUCE-6983. Moving logging APIs over to slf4j in hadoop-mapreduce-client-core. Contributed by Jinjiang Ling. 2017-11-02 17:43:08 +09:00
Chris Trezzo
e46d5bb962 MAPREDUCE-5951. Add support for the YARN Shared Cache. 2017-10-12 10:59:20 -07:00
Robert Kanter
0adc0471d0 MAPREDUCE-6954. Disable erasure coding for files that are uploaded to the MR staging area (pbacsko via rkanter) 2017-09-18 10:40:06 -07:00
Robert Kanter
5f496683fb Revert "YARN-7162. Remove XML excludes file format (rkanter)" - wrong commit message
This reverts commit 3a8d57a0a2e047b34be82f602a2b6cf5593d2125.
2017-09-18 10:32:08 -07:00
Robert Kanter
3a8d57a0a2 YARN-7162. Remove XML excludes file format (rkanter) 2017-09-15 12:00:26 -07:00
Steve Loughran
11390c2d11
MAPREDUCE-6956 FileOutputCommitter to gain abstract superclass PathOutputCommitter.
Contributed by Steve Loughran
2017-09-15 16:59:04 +01:00
Jason Lowe
4d98936eec MAPREDUCE-6957. shuffle hangs after a node manager connection timeout. Contributed by Jooseong Kim 2017-09-13 17:21:13 -05:00
Ray Chiang
556812c179 HADOOP-14799. Update nimbus-jose-jwt to 4.41.1. (rchiang) 2017-09-12 10:19:34 -07:00
Haibo Chen
d04f85f387 MAPREDUCE-6892. Issues with the count of failed/killed tasks in the jhist file. (Peter Bacsko via Haibo Chen) 2017-08-30 10:07:48 -07:00
Mingliang Liu
ae7abad3d9 MAPREDUCE-6945. TestMapFileOutputFormat missing @after annotation. Contributed by Ajay Kumar 2017-08-27 15:19:55 -07:00
Arpit Agarwal
8b7cbe3840 HADOOP-14729. Upgrade JUnit 3 test cases to JUnit 4. Contributed by Ajay Kumar. 2017-08-25 09:55:46 -07:00
Yufei Gu
b8e8241854 MAPREDUCE-6914. Tests use assertTrue(....equals(...)) instead of assertEquals()). (Daniel Templeton via Yufei Gu) 2017-08-03 11:44:34 -07:00
Jason Lowe
1e71fe8c42 MAPREDUCE-6882. Increase MapReduce test timeouts from 1 second to 10 seconds. Contributed by Eric Badger 2017-05-10 09:57:41 -05:00
Daniel Templeton
fc0885da29 MAPREDUCE-6846. Fragments specified for libjar paths are not handled correctly
(Contributed by Chris Trezzo via Daniel Templeton)
2017-04-05 17:25:40 -07:00
Ming Ma
ceab00ac62 MAPREDUCE-6862. Fragments are not handled correctly by resource limit checking. (Chris Trezzo via mingma) 2017-03-29 17:41:58 -07:00
Karthik Kambatla
c65f884fc7 MAPREDUCE-6829. Add peak memory usage counter for each task. (Miklos Szegedi via kasha) 2017-01-26 11:08:13 -08:00
Rohith Sharma K S
683e0c71fe MAPREDUCE-6793. io.sort.factor code default and mapred-default.xml values inconsistent. Contributed by Gera Shegalov. 2016-11-21 22:10:57 +05:30
Mingliang Liu
5af572b644 HADOOP-13427. Eliminate needless uses of FileSystem#{exists(), isFile(), isDirectory()}. Contributed by Steve Loughran and Mingliang Liu 2016-11-15 10:57:00 -08:00
Junping Du
9b7ce948a5 MAPREDUCE-6792. Allow user's full principal name as owner of MapReduce staging directory in JobSubmissionFiles#JobStagingDir(). Contributed by Santhosh G Nayak. 2016-10-25 11:20:23 -07:00
Robert Kanter
d4725bfcb2 MAPREDUCE-6728. Give fetchers hint when ShuffleHandler rejects a shuffling connection (haibochen via rkanter) 2016-10-21 17:46:17 -07:00
Robert Kanter
4fc632ae19 MAPREDUCE-6632. Master.getMasterAddress() should be updated to use YARN-4629 (templedf via rkanter) 2016-09-22 16:12:56 -07:00
Jason Lowe
6fa9bf4407 MAPREDUCE-6761. Regression when handling providers - invalid configuration ServiceConfiguration causes Cluster initialization failure. Contributed by Peter Vary 2016-08-24 14:40:51 +00:00
Varun Saxena
d37b45d613 MAPREDUCE-6762. ControlledJob#toString failed with NPE when job status is not successfully updated (Weiwei Yang via Varun Saxena) 2016-08-21 21:46:17 +05:30
Jason Lowe
f80a729832 MAPREDUCE-6690. Limit the number of resources a single map reduce job can submit for localization. Contributed by Chris Trezzo 2016-08-17 16:22:31 +00:00