big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
17,821 Commits 2 Branches 2 Tags 264 MiB
a53d62ab26
Commit Graph

691 Commits

Author SHA1 Message Date
Wangda Tan
60f9e60b3b Preparing for 3.2.0 development 
Change-Id: I6d0e01f3d665d26573ef2b957add1cf0cddf7938
 2018-02-11 11:17:38 +08:00
Arun Suresh
38af237969 YARN-6599. Support anti-affinity constraint via AppPlacementAllocator. (Wangda Tan via asuresh) 2018-01-31 01:30:17 -08:00
Jason Lowe
6eef3d7f1a MAPREDUCE-7020. Task timeout in uber mode can crash AM. Contributed by Peter Bacsko 2018-01-26 15:31:43 -06:00
Jason Lowe
a37e7f0ad8 MAPREDUCE-7022. Fast fail rogue jobs based on task scratch dir size. Contributed by Johan Gustavsson 2018-01-26 14:36:45 -06:00
Haibo Chen
cc10852252 MAPREDUCE-7041. MR should not try to clean up at first job attempt. (Gergo Repas via Haibo Chen) 2018-01-25 16:11:30 -08:00
Haibo Chen
cce71dceef MAPREDUCE-6984. MR AM to clean up temporary files from previous attempt in case of no recovery. (Gergo Repas via Haibo Chen) 2018-01-19 12:56:17 -08:00
Jason Lowe
fe35103591 MAPREDUCE-7028. Concurrent task progress updates causing NPE in Application Master. Contributed by Gergo Repas 2018-01-03 11:01:38 -06:00
Jason Lowe
21d3627355 MAPREDUCE-5124. AM lacks flow control for task events. Contributed by Peter Bacsko 2017-12-01 14:04:25 -06:00
Steve Loughran
de8b6ca5ef HADOOP-13786 Add S3A committer for zero-rename commits to S3 endpoints. 
Contributed by Steve Loughran and Ryan Blue.
 2017-11-22 15:28:12 +00:00
Daniel Templeton
9a7e810838 YARN-6927. Add support for individual resource types requests in MapReduce 
(Contributed by Gergo Repas via Daniel Templeton)
 2017-10-30 11:04:22 -07:00
Chris Trezzo
e46d5bb962 MAPREDUCE-5951. Add support for the YARN Shared Cache. 2017-10-12 10:59:20 -07:00
Akira Ajisaka
453d48bdfb
MAPREDUCE-6971. Moving logging APIs over to slf4j in hadoop-mapreduce-client-app. Contributed by Jinjiang Ling. 2017-10-03 12:14:54 +09:00
Andrew Wang
0d419c984f Preparing for 3.1.0 development 2017-09-01 11:53:48 -07:00
Haibo Chen
d04f85f387 MAPREDUCE-6892. Issues with the count of failed/killed tasks in the jhist file. (Peter Bacsko via Haibo Chen) 2017-08-30 10:07:48 -07:00
Varun Saxena
3d00c8f394 Made fixes for whitespace errors and checstyle warnings before merge. 2017-08-30 11:29:54 +05:30
Jian He
08f40bcc7f MAPREDUCE-6838. [ATSv2 Security] Add timeline delegation token received in allocate response to UGI. Contributed by Varun Saxena 2017-08-30 11:29:54 +05:30
Rohith Sharma K S
7594d1de7b YARN-6130. [ATSv2 Security] Generate a delegation token for AM when app collector is created and pass it to AM via NM and RM. Contributed by Varun Saxena. 2017-08-30 11:29:53 +05:30
Sangjin Lee
684de1a902 YARN-4675. Reorganize TimelineClient and TimelineClientImpl into separate classes for ATSv1.x and ATSv2. Contributed by Naganarasimha G R. 2017-08-30 11:29:52 +05:30
Sangjin Lee
092fead5d9 YARN-5792. Adopt the id prefix for YARN, MR, and DS entities. Contributed by Varun Saxena. 2017-08-30 11:29:51 +05:30
Haibo Chen
a32e0138fb MAPREDUCE-6870. Add configuration for MR job to finish when all reducers are complete. (Peter Bacsko via Haibo Chen) 2017-08-10 15:17:36 -07:00
Jason Lowe
735fce5bec MAPREDUCE-6927. MR job should only set tracking url if history was successfully written. Contributed by Eric Badger 2017-08-08 14:46:47 -05:00
Arun Suresh
35dc782923 YARN-5977. ContainerManagementProtocol changes to support change of container ExecutionType. (Kartheek Muthyala via asuresh) 2017-08-03 21:15:40 -07:00
Yufei Gu
b8e8241854 MAPREDUCE-6914. Tests use assertTrue(....equals(...)) instead of assertEquals()). (Daniel Templeton via Yufei Gu) 2017-08-03 11:44:34 -07:00
Junping Du
fbb7d6bcbb Revert "MAPREDUCE-5875. Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild. (Gera Shegalov via kasha)" 
This reverts commit e8a31f2e1c.
 2017-07-31 14:09:16 -07:00
Akira Ajisaka
38c6fa5c7a
HADOOP-11875. [JDK9] Adding a second copy of Hamlet without _ as a one-character identifier. 2017-07-28 09:10:52 +09:00
Varun Saxena
f66fd11e51 YARN-5548. Use MockRMMemoryStateStore to reduce test failures (Bibin A Chundatt via Varun Saxena) 2017-07-27 23:14:50 +05:30
Akira Ajisaka
cca51e916b
HADOOP-14681. Remove MockitoMaker class. Contributed by Andras Bokor. 2017-07-25 15:24:56 +09:00
Andrew Wang
af2773f609 Updating version for 3.0.0-beta1 development 2017-06-29 17:57:40 -07:00
Akira Ajisaka
a5c0476a99
MAPREDUCE-6697. Concurrent task limits should only be applied when necessary. Contributed by Nathan Roberts. 2017-06-28 10:50:09 +09:00
Ravi Prakash
82bbcbf37f MAPREDUCE-6897. Add Unit Test to ensure Job end notification gets sent even when appMaster stop gets YarnRuntimeException. Contributed by Gergely Novák 2017-06-16 14:25:44 -07:00
Jason Lowe
9ef2743fa1 Revert "MAPREDUCE-6898. TestKill.testKillTask is flaky. Contributed by Peter Bacsko" 
This reverts commit e78542f4be.
 2017-06-16 14:48:51 -05:00
Jason Lowe
e78542f4be MAPREDUCE-6898. TestKill.testKillTask is flaky. Contributed by Peter Bacsko 2017-06-16 14:24:54 -05:00
Junping Du
6ed54f3439 MAPREDUCE-6895. Job end notification not send due to YarnRuntimeException. Contributed by yunjiong zhao. 2017-06-13 15:21:04 -07:00
Akira Ajisaka
d4015f8628
MAPREDUCE-6887. Modifier 'static' is redundant for inner enums. Contributed by ZhangBing Lin. 2017-05-30 14:48:58 +09:00
Andrew Wang
16ad896d5c Update maven version for 3.0.0-alpha4 development 2017-05-26 14:09:44 -07:00
Sunil G
b6f66b0da1 YARN-6584. Correct license headers in hadoop-common, hdfs, yarn and mapreduce. Contributed by Yeliang Cang. 2017-05-22 14:10:06 +05:30
Akira Ajisaka
3ed3062fe3
MAPREDUCE-6881. Fix warnings from Spotbugs in hadoop-mapreduce. Contributed by Weiwei Yang. 2017-04-27 15:45:33 +09:00
Daniel Templeton
dd43b895c2 YARN-6202. Configuration item Dispatcher.DISPATCHER_EXIT_ON_ERROR_KEY is disregarded 
(Contributed by Yufei Gu via Daniel Templeton)
 2017-04-19 11:44:55 -07:00
Akira Ajisaka
845529b3ab MAPREDUCE-6824. TaskAttemptImpl#createCommonContainerLaunchContext is longer than 150 lines. Contributed by Chris Trezzo. 2017-04-03 13:06:54 +09:00
Jason Lowe
4d1fac5df2 YARN-6411. Clean up the overwrite of createDispatcher() in subclass of MockRM. Contributed by Yufei Gu 2017-03-31 10:05:34 -05:00
Jason Lowe
c9b7ce9273 MAPREDUCE-6836. exception thrown when accessing the job configuration web UI. Contributed by Haibo Chen 2017-03-30 12:55:32 -05:00
Robert Kanter
38d75dfd3a MAPREDUCE-6839. TestRecovery.testCrashed failed (pairg via rkanter) 2017-03-07 13:34:46 -08:00
Eric Yang
172b23af33 HADOOP-14077. Add ability to access jmx via proxy. Contributed by Yuanbo Liu. 2017-02-18 18:34:13 -08:00
Sangjin Lee
4fa1afdb88 YARN-4675. Reorganize TimelineClient and TimelineClientImpl into separate classes for ATSv1.x and ATSv2. Contributed by Naganarasimha G R. 2017-02-16 11:41:04 -08:00
Junping Du
7afe5b1a31 MAPREDUCE-6404. Allow AM to specify a port range for starting its webapp. Contributed by Varun Saxena. 2017-02-07 01:40:49 -08:00
Junping Du
d401e63b6c MAPREDUCE-6338. MR AppMaster does not honor ephemeral port range. Contributed by Frank Nguyen. 2017-02-05 19:28:01 -08:00
Andrew Wang
5d8b80ea9b Preparing for 3.0.0-alpha3 development 2017-01-19 15:50:07 -08:00
Jason Lowe
78b487bde1 MAPREDUCE-6831. Flaky test TestJobImpl.testKilledDuringKillAbort. Contributed by Peter Bacsko 2017-01-17 16:01:42 +00:00
Arun Suresh
85826f6ca5 YARN-6066. Opportunistic containers Minor fixes : API annotations, parameter name changes, checkstyles. (asuresh) 
(cherry picked from commit 4985217de453a04ddffd7b52644bdc8d153f753c)
 2017-01-08 09:04:20 -08:00
Junping Du
679478d0c6 MAPREDUCE-6711. JobImpl fails to handle preemption events on state COMMITTING. Contributed by Prabhu Joseph. 2017-01-07 22:04:18 -08:00