Szilard Nemeth
a7371a779c
MAPREDUCE-7225: Fix broken current folder expansion during MR job start. Contributed by Peter Bacsko.
2019-08-01 13:01:30 +02:00
Chen Zhang
8a59cd1b8a
HDFS-12967. NNBench should support multi-cluster access. Contributed by Chen Zhang.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-07-29 05:55:12 -07:00
Mehul Garnara
c0a0c353e8
MAPREDUCE-6973. Fix comments on creating _SUCCESS file.
...
This closes #280
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-07-26 21:21:26 +09:00
pingsutw
ee87e9a42e
MAPREDUCE-7076. TestNNBench#testNNBenchCreateReadAndDelete failing in our internal build
...
This closes #1089
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-07-23 10:34:10 +09:00
Masatake Iwasaki
79f6118dcc
MAPREDUCE-6521. MiniMRYarnCluster should not create /tmp/hadoop-yarn/staging on local filesystem in unit test.
...
Contributed by Masatake Iwasaki.
Change-Id: Id74b90eb7cfb9d676188fc5aa47249ee8904a3d5
2019-07-18 13:15:18 +01:00
Wanqiang Ji
b417a4c854
MAPREDUCE-7214. Remove unused pieces related to mapreduce.job.userlog.retain.hours
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-06-11 18:40:35 +09:00
Akira Ajisaka
3ea4f41d9f
MAPREDUCE-6794. Remove unused properties from TTConfig.java
2019-06-07 10:27:41 +09:00
Wanqiang Ji
e7e30a5f8b
MAPREDUCE-7210. Replace mapreduce.job.counters.limit
with mapreduce.job.counters.max
in mapred-default.xml
...
This closes #878
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-06-05 10:45:23 +09:00
Akira Ajisaka
afd844059c
HADOOP-16331. Fix ASF License check in pom.xml
...
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2019-05-29 17:25:13 +09:00
Akira Ajisaka
9f933e6446
HADOOP-16323. https everywhere in Maven settings.
2019-05-27 15:24:59 +09:00
Akira Ajisaka
5565f2c532
MAPREDUCE-7198. mapreduce.task.timeout=0 configuration used to disable timeout doesn't work.
2019-05-23 10:21:11 +09:00
Wanqiang Ji
67f9a7b165
MAPREDUCE-7205. Treat container scheduler kill exit code as a task attempt killing event.
...
This closes #821
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-05-22 14:23:13 +09:00
Gabor Bota
d7979079ea
HADOOP-16210. Update guava to 27.0-jre in hadoop-project trunk. Contributed by Gabor Bota.
2019-04-03 12:59:39 -06:00
Adam Antal
856cbf62d3
MAPREDUCE-7190. Add SleepJob additional parameter to make parallel runs distinguishable. Contributed by Adam Antal.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-04-01 10:25:38 -07:00
David Mollitor
246ab77f28
HADOOP-16196. Path Parameterize Comparable.
...
Author: David Mollitor <david.mollitor@cloudera.com>
2019-03-22 10:26:24 +00:00
Steve Loughran
9f1c017f44
HADOOP-16058. S3A tests to include Terasort.
...
Contributed by Steve Loughran.
This includes
- HADOOP-15890. Some S3A committer tests don't match ITest* pattern; don't run in maven
- MAPREDUCE-7090. BigMapOutput example doesn't work with paths off cluster fs
- MAPREDUCE-7091. Terasort on S3A to switch to new committers
- MAPREDUCE-7092. MR examples to work better against cloud stores
2019-03-21 11:15:37 +00:00
Weiwei Yang
159a715eef
MAPREDUCE-7192. JobHistoryServer attempts page support jump to containers log page in NM when logAggregation is disable. Contributed by Jiandan Yang.
2019-03-13 17:00:35 +08:00
Weiwei Yang
f0605146b3
MAPREDUCE-7191. JobHistoryServer should log exception when loading/parsing history file failed. Contributed by Jiandan Yang.
2019-03-11 15:02:54 +08:00
Steve Loughran
f365957c63
HADOOP-15229. Add FileSystem builder-based openFile() API to match createFile();
...
S3A to implement S3 Select through this API.
The new openFile() API is asynchronous, and implemented across FileSystem and FileContext.
The MapReduce V2 inputs are moved to this API, and you can actually set must/may
options to pass in.
This is more useful for setting things like s3a seek policy than for S3 select,
as the existing input format/record readers can't handle S3 select output where
the stream is shorter than the file length, and splitting plain text is suboptimal.
Future work is needed there.
In the meantime, any/all filesystem connectors are now free to add their own filesystem-specific
configuration parameters which can be set in jobs and used to set filesystem input stream
options (seek policy, retry, encryption secrets, etc).
Contributed by Steve Loughran
2019-02-05 11:51:02 +00:00
Akira Ajisaka
c991e2ca5a
MAPREDUCE-7177. Disable speculative execution in TestDFSIO. Contributed by Zhaohui Xin.
2019-02-04 14:03:34 +09:00
Akira Ajisaka
1129288cf5
HADOOP-14178. Move Mockito up to version 2.23.4. Contributed by Akira Ajisaka and Masatake Iwasaki.
2019-01-29 18:29:56 -08:00
Eric Yang
1ab69a9543
YARN-9221. Added flag to disable dynamic auxiliary service feature.
...
Contributed by Billie Rinaldi
2019-01-25 19:05:36 -05:00
Eric Yang
a33ef4fd31
YARN-8867. Added resource localization status to YARN service status call.
...
Contributed by Chandni Singh
2019-01-24 18:43:21 -05:00
Eric Yang
2fa9389c2e
YARN-9146. Added REST API to configure auxiliary service.
...
Contributed by Billie Rinaldi
2019-01-22 18:24:43 -05:00
Steve Loughran
6d0bffe17e
HADOOP-14556. S3A to support Delegation Tokens.
...
Contributed by Steve Loughran and Daryn Sharp.
2019-01-14 17:59:27 +00:00
Akira Ajisaka
7f78397036
Revert "HADOOP-14556. S3A to support Delegation Tokens."
...
This reverts commit d7152332b3
.
2019-01-08 14:51:30 +09:00
Steve Loughran
d7152332b3
HADOOP-14556. S3A to support Delegation Tokens.
...
Contributed by Steve Loughran.
2019-01-07 13:18:03 +00:00
Eric Yang
f82922dcfa
YARN-5168. Added exposed port information for Docker container.
...
Contributed by Xun Liu
2018-12-21 19:44:07 -05:00
Eric Yang
7affa3053c
YARN-9131. Updated document usage for dynamic auxiliary service.
...
Contributed by Billie Rinaldi
2018-12-20 18:45:49 -05:00
Akira Ajisaka
499c70eda5
MAPREDUCE-7166. map-only job should ignore node lost event when task is already succeeded. Contributed by Lei Li.
2018-12-20 10:09:50 +09:00
Eric Yang
c7a5a4435e
YARN-9075. Add ability to register/remove auxiliary service without restart node manager.
...
Contributed by Billie Rinaldi
2018-12-18 17:05:51 -05:00
Akira Ajisaka
e0d75088f2
MAPREDUCE-7170. Doc typo in PluggableShuffleAndPluggableSort.md. Contributed by Zhaohui Xin.
2018-12-13 16:26:25 +09:00
Haibo Chen
881230da21
YARN-9051. Integrate multiple CustomResourceTypesConfigurationProvider implementations into one. (Contributed by Szilard Nemeth)
2018-12-11 11:41:32 -08:00
Jason Lowe
9886c27c71
MAPREDUCE-7159. FrameworkUploader: ensure proper permissions of generated framework tar.gz if restrictive umask is used. Contributed by Peter Bacsko
2018-12-06 14:48:17 -06:00
Jason Lowe
4d8de7ab69
MAPREDUCE-7164. FileOutputCommitter does not report progress while merging paths. Contributed by Kuhu Shukla
2018-11-28 14:54:59 -06:00
Akira Ajisaka
13a21f6607
MAPREDUCE-6190. If a task stucks before its first heartbeat, it never timeouts and the MR job becomes stuck. Contributed by Zhaohui Xin.
2018-11-28 17:57:42 +09:00
Akira Ajisaka
1aad99a718
MAPREDUCE-7162. TestEvents#testEvents fails. Contributed by Zhaohui Xin.
2018-11-27 20:01:11 +09:00
Billie Rinaldi
27ffec7ba7
YARN-8778. Add command line interface to invoke interactive docker shell. Contributed by Eric Yang
2018-11-19 17:59:12 -08:00
Arpit Agarwal
1e15c7e855
HADOOP-15936. [JDK 11] MiniDFSClusterManager & MiniHadoopClusterManager compilation fails due to the usage of '_' as identifier. Contributed by Zsolt Venczel.
2018-11-15 12:42:31 -08:00
Wangda Tan
762a56cc64
MAPREDUCE-7158. Inefficient Flush Logic in JobHistory EventWriter. (Zichen Sun via wangda)
...
Change-Id: I99ace87980da03bb35a8012cea7218d602a8817a
2018-11-13 11:25:41 -08:00
Jason Lowe
0b6625a973
MAPREDUCE-7148. Fast fail jobs when exceeds dfs quota limitation. Contributed by Wang Yan
2018-11-07 08:20:49 -06:00
Jason Lowe
ba1f9d66d9
MAPREDUCE-7156. NullPointerException when reaching max shuffle connections. Contributed by Peter Bacsko
2018-11-06 17:55:51 -06:00
Jason Lowe
ab6aa4c726
YARN-8865. RMStateStore contains large number of expired RMDelegationToken. Contributed by Wilfred Spiegelenburg
2018-11-06 08:40:59 -06:00
bibinchundatt
74a5e683fe
MAPREDUCE-7151. RMContainerAllocator#handleJobPriorityChange expects application_priority always. Contributed by Bilwa S T.
2018-10-24 23:22:05 +05:30
Haibo Chen
823bb5dda8
MAPREDUCE-4669. MRAM web UI does not work with HTTPS. (Contributed by Robert Kanter)
2018-10-23 15:28:13 -07:00
Haibo Chen
babd1449bf
MAPREDUCE-7150. Optimize collections used by MR JHS to reduce its memory. (Contributed by Misha Dmitriev)
2018-10-16 13:44:41 -07:00
Xiao Chen
25f8fcb064
MAPREDUCE-7132. JobSplitWriter prints unnecessary warnings if EC(RS10,4) is used. Contributed by Peter Bacsko.
2018-10-16 10:23:31 -07:00
Steve Loughran
735a36afd5
MAPREDUCE-7149. Javadocs for FileInputFormat and OutputFormat to mention DT collection.
...
Contributed by Steve Loughran.
2018-10-11 20:01:42 +01:00
Akira Ajisaka
6fa3feb577
HADOOP-15832. Upgrade BouncyCastle to 1.60. Contributed by Robert Kanter.
2018-10-10 10:16:57 +09:00
Akira Ajisaka
5b1cfcaeff
MAPREDUCE-7035. Skip javadoc build for auto-generated sources in hadoop-mapreduce-client. Contributed by Mukul Kumar Singh.
2018-10-09 21:13:04 +09:00