big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
22,344 Commits 2 Branches 2 Tags 264 MiB
79f6118dcc
Commit Graph

576 Commits

Author SHA1 Message Date
Masatake Iwasaki
79f6118dcc
MAPREDUCE-6521. MiniMRYarnCluster should not create /tmp/hadoop-yarn/staging on local filesystem in unit test. 
Contributed by Masatake Iwasaki.

Change-Id: Id74b90eb7cfb9d676188fc5aa47249ee8904a3d5
 2019-07-18 13:15:18 +01:00
Adam Antal
856cbf62d3 MAPREDUCE-7190. Add SleepJob additional parameter to make parallel runs distinguishable. Contributed by Adam Antal. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-04-01 10:25:38 -07:00
Steve Loughran
9f1c017f44
HADOOP-16058. S3A tests to include Terasort. 
Contributed by Steve Loughran.

This includes
 - HADOOP-15890. Some S3A committer tests don't match ITest* pattern; don't run in maven
 - MAPREDUCE-7090. BigMapOutput example doesn't work with paths off cluster fs
 - MAPREDUCE-7091. Terasort on S3A to switch to new committers
 - MAPREDUCE-7092. MR examples to work better against cloud stores
 2019-03-21 11:15:37 +00:00
Akira Ajisaka
c991e2ca5a
MAPREDUCE-7177. Disable speculative execution in TestDFSIO. Contributed by Zhaohui Xin. 2019-02-04 14:03:34 +09:00
Akira Ajisaka
1129288cf5
HADOOP-14178. Move Mockito up to version 2.23.4. Contributed by Akira Ajisaka and Masatake Iwasaki. 2019-01-29 18:29:56 -08:00
Haibo Chen
881230da21 YARN-9051. Integrate multiple CustomResourceTypesConfigurationProvider implementations into one. (Contributed by Szilard Nemeth) 2018-12-11 11:41:32 -08:00
Billie Rinaldi
27ffec7ba7 YARN-8778. Add command line interface to invoke interactive docker shell. Contributed by Eric Yang 2018-11-19 17:59:12 -08:00
Arpit Agarwal
1e15c7e855 HADOOP-15936. [JDK 11] MiniDFSClusterManager & MiniHadoopClusterManager compilation fails due to the usage of '_' as identifier. Contributed by Zsolt Venczel. 2018-11-15 12:42:31 -08:00
bibinchundatt
8cf6a9a2bd YARN-7892. Revisit NodeAttribute class structure. Contributed by Naganarasimha G R. 2018-09-12 16:01:00 +05:30
Naganarasimha
5dc7d6e0f3 YARN-8104. Add API to fetch node to attribute mapping. Contributed by Bibin A Chundatt. 2018-09-12 16:01:00 +05:30
Naganarasimha
0a01b1350d YARN-8100. Support API interface to query cluster attributes and attribute to nodes. Contributed by Bibin A Chundatt. 2018-09-12 16:01:00 +05:30
Inigo Goiri
a804b7c9d2 MAPREDUCE-7102. Fix TestJavaSerialization for Windows due a mismatch line separator. Contributed by Giovanni Matteo Fumarola. 2018-06-02 20:25:42 -07:00
Takanobu Asanuma
d1e2b80980
MAPREDUCE-7098. Upgrade common-langs version to 3.7 in hadoop-mapreduce-project 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-05-31 18:31:10 +09:00
Abhishek Modi
e9159db623
MAPREDUCE-7093. Use assertEquals instead of assertTrue(a == b) in TestMapReduceJobControlWithMocks 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-05-07 01:15:14 +09:00
Haibo Chen
bb92bfb4ef YARN-7088. Add application launch time to Resource Manager REST API. (Kanwaljeet Sachdev via Haibo Chen) 2018-04-17 11:15:20 -07:00
Haibo Chen
86dfb70875 Revert "YARN-7088. Add application launch time to Resource Manager REST API. (Kanwaljeet Sachdev via Haibo Chen)" 
This reverts commit 0c14b6f0d8.
 2018-04-17 06:04:38 -07:00
Haibo Chen
0c14b6f0d8 YARN-7088. Add application launch time to Resource Manager REST API. (Kanwaljeet Sachdev via Haibo Chen) 2018-04-17 05:59:03 -07:00
Wangda Tan
995cba65fe MAPREDUCE-7077. Pipe mapreduce job fails with Permission denied for jobTokenPassword. (Akira Ajisaka via wangda) 
Change-Id: Ie8f01425d58409fa3661f768205b7616128c8aa4
(cherry picked from commit 035e0f97ea44b0495707949a781d8792dcf6ea6b)
 2018-04-13 13:52:10 -07:00
Erik Krogen
1a407bc990 HADOOP-14970. MiniHadoopClusterManager doesn't respect lack of format option. Contributed by Erik Krogen 2018-04-12 23:27:51 -07:00
Jason Lowe
4571351ccc MAPREDUCE-7069. Add ability to specify user environment variables individually. Contributed by Jim Brennan 2018-04-12 11:12:46 -05:00
Sean Mackrory
2bc3351eaf HADOOP-6852. apparent bug in concatenated-bzip2 support (decoding). Contributed by Zsolt Venczel. 2018-02-21 12:57:14 -07:00
Jason Lowe
0898ff42e9 HADOOP-15206. BZip2 drops and duplicates records when input split size is small. Contributed by Aki Tanaka 2018-02-16 14:49:00 -06:00
Jason Lowe
a53d62ab26 MAPREDUCE-7052. TestFixedLengthInputFormat#testFormatCompressedIn is flaky. Contributed by Peter Bacsko 2018-02-15 15:12:57 -06:00
Takanobu Asanuma
56feaa40bb
MAPREDUCE-7036. ASF License warning in hadoop-mapreduce-client 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-01-30 00:39:39 +09:00
Jason Lowe
a37e7f0ad8 MAPREDUCE-7022. Fast fail rogue jobs based on task scratch dir size. Contributed by Johan Gustavsson 2018-01-26 14:36:45 -06:00
Takanobu Asanuma
1a9c5d479e
MAPREDUCE-7034. Moving logging APIs over to slf4j the rest of all in hadoop-mapreduce 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-01-15 15:38:03 +09:00
Chen Liang
c7a4dda3c5 HADOOP-15109. TestDFSIO -read -random doesn't work on file sized 4GB. Contributed by Ajay Kumar. 2017-12-18 13:25:47 -08:00
Akira Ajisaka
d4cae977a2
MAPREDUCE-6998. Moving logging APIs over to slf4j in hadoop-mapreduce-client-jobclient. Contributed by Gergely Novák. 2017-12-07 16:21:25 +09:00
Jason Lowe
21d3627355 MAPREDUCE-5124. AM lacks flow control for task events. Contributed by Peter Bacsko 2017-12-01 14:04:25 -06:00
Xiao Chen
e565b5277d HDFS-12682. ECAdmin -listPolicies will always show SystemErasureCodingPolicies state as DISABLED. 2017-11-02 21:27:35 -07:00
Akira Ajisaka
178751ed8c
MAPREDUCE-6983. Moving logging APIs over to slf4j in hadoop-mapreduce-client-core. Contributed by Jinjiang Ling. 2017-11-02 17:43:08 +09:00
Jason Lowe
2fae63aa60 HADOOP-14919. BZip2 drops records when reading data in splits. Contributed by Jason Lowe 2017-10-31 09:30:13 -05:00
Daniel Templeton
9a7e810838 YARN-6927. Add support for individual resource types requests in MapReduce 
(Contributed by Gergo Repas via Daniel Templeton)
 2017-10-30 11:04:22 -07:00
Chris Trezzo
e46d5bb962 MAPREDUCE-5951. Add support for the YARN Shared Cache. 2017-10-12 10:59:20 -07:00
Wangda Tan
df3855541a YARN-6789. Add Client API to get all supported resource types from RM. (Sunil G via wangda) 
Change-Id: I366d8db6f6700acd087db5acb7a1be7e41b2b68d
 2017-09-12 09:19:12 -07:00
Varun Vasudev
c2032e251e YARN-5708. Implement APIs to get resource profiles from the RM. Contributed by Varun Vasudev. 2017-09-12 09:19:10 -07:00
Robert Kanter
a323f73bae MAPREDUCE-6953. Skip the testcase testJobWithChangePriority if FairScheduler is used (pbacsko via rkanter) 2017-09-08 13:14:36 -07:00
Konstantin V Shvachko
3e0e2033cc MAPREDUCE-6931. Remove TestDFSIO "Total Throughput" calculation. Contributed by Dennis Huo. 2017-08-30 14:07:23 -07:00
Varun Saxena
3d00c8f394 Made fixes for whitespace errors and checstyle warnings before merge. 2017-08-30 11:29:54 +05:30
Sangjin Lee
8499299740 MAPREDUCE-6818. Remove direct reference to TimelineClientImpl. Contributed by Li Lu. 2017-08-30 11:29:51 +05:30
Sangjin Lee
092fead5d9 YARN-5792. Adopt the id prefix for YARN, MR, and DS entities. Contributed by Varun Saxena. 2017-08-30 11:29:51 +05:30
Arpit Agarwal
8b7cbe3840 HADOOP-14729. Upgrade JUnit 3 test cases to JUnit 4. Contributed by Ajay Kumar. 2017-08-25 09:55:46 -07:00
Chris Douglas
645a8f2a4d HADOOP-14726. Mark FileStatus::isDir as final 2017-08-14 21:57:20 -07:00
Allen Wittenauer
4222c97108
HADOOP-10392. Use FileSystem#makeQualified(Path) instead of Path#makeQualified(FileSystem) (ajisakaa via aw) 2017-08-11 09:25:56 -07:00
Yufei Gu
b8e8241854 MAPREDUCE-6914. Tests use assertTrue(....equals(...)) instead of assertEquals()). (Daniel Templeton via Yufei Gu) 2017-08-03 11:44:34 -07:00
Akira Ajisaka
ceacadc51e
MAPREDUCE-6921. TestUmbilicalProtocolWithJobToken#testJobTokenRpc fails. 2017-08-01 14:56:42 +09:00
Junping Du
fbb7d6bcbb Revert "MAPREDUCE-5875. Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild. (Gera Shegalov via kasha)" 
This reverts commit e8a31f2e1c.
 2017-07-31 14:09:16 -07:00
Junping Du
713349a9af Revert "MAPREDUCE-6199. AbstractCounters are not reset completely on deserialization (adhoot via rkanter)" 
This reverts commit 390a7c12f5.
 2017-07-28 14:21:04 -07:00
Akira Ajisaka
cca51e916b
HADOOP-14681. Remove MockitoMaker class. Contributed by Andras Bokor. 2017-07-25 15:24:56 +09:00
Sunil G
fb3b5d33ff MAPREDUCE-6889. Add Job#close API to shutdown MR client services. Contributed by Rohith Sharma K S. 2017-07-17 13:35:15 +05:30