big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
15,030 Commits 2 Branches 2 Tags 264 MiB
b3befc021b
Commit Graph

3031 Commits

Author SHA1 Message Date
Daniel Templeton
b3befc021b YARN-4997. Update fair scheduler to use pluggable auth provider (Contributed by Tao Jie via Daniel Templeton) 2016-11-30 09:50:33 -08:00
Kai Zheng
cfd8076f81 Revert due to an error "HDFS-10994. Support an XOR policy XOR-2-1-64k in HDFS. Contributed by Sammi Chen" 
This reverts commit 5614f847b2ef2a5b70bd9a06edc4eba06174c6.
 2016-11-30 15:44:52 +08:00
Daniel Templeton
dd9a96cb17 YARN-5851. TestContainerManagerSecurity testContainerManager[1] failed (Contributed by Haibo Chen via Daniel Templeton) 2016-11-29 14:22:19 -08:00
Daniel Templeton
3b9d3acd20 YARN-5890. FairScheduler should log information about AM-resource-usage and max-AM-share for queues 
(Contributed by Yufei Gu via Daniel Templeton)
 2016-11-29 12:46:05 -08:00
Daniel Templeton
25f9872be6 YARN-5774. MR Job stuck in ACCEPTED status without any progress in Fair Scheduler 
if set yarn.scheduler.minimum-allocation-mb to 0. (Contributed by Yufei Gu via Daniel Templeton)
 2016-11-29 09:40:49 -08:00
Daniel Templeton
62b42ef5dd YARN-5725. Test uncaught exception in TestContainersMonitorResourceChange.testContainersResourceChange 
when setting IP and host (Contributed by Miklos Szegedi via Daniel Templeton)
 2016-11-28 15:19:52 -08:00
Kai Zheng
5d5614f847 HDFS-10994. Support an XOR policy XOR-2-1-64k in HDFS. Contributed by Sammi Chen 2016-11-28 14:34:44 +08:00
Akira Ajisaka
209e805430 HADOOP-13506. Redundant groupid warning in child projects. Contributed by Kai Sasaki. 2016-11-28 14:34:57 +09:00
Naganarasimha
c06114d6a3 YARN-5572. HBaseTimelineWriterImpl appears to reference a bad property name. Contributed by Varun Saxena. 2016-11-27 23:35:53 +05:30
Naganarasimha
e2f77ca77b Revert "YARN-5921. Incorrect synchronization in RMContextImpl#setHAServiceState/getHAServiceState. Contributed by Varun Saxena." 
This reverts commit 9f2960be19.
 2016-11-27 23:32:59 +05:30
Naganarasimha
9f2960be19 YARN-5921. Incorrect synchronization in RMContextImpl#setHAServiceState/getHAServiceState. Contributed by Varun Saxena. 2016-11-27 23:31:27 +05:30
Sunil
eb0a483ed0 YARN-4206. Add Application timeouts in Application report and CLI. Contributed by Rohith Sharma K S. 2016-11-24 18:18:42 +05:30
Rohith Sharma K S
e15c20edba YARN-5920. Fix deadlock in TestRMHA.testTransitionedToStandbyShouldNotHang. Contributed by Varun Saxena. 2016-11-24 12:18:38 +05:30
Daniel Templeton
10468529a9 YARN-4752. Improved preemption in FairScheduler. (kasha) 
Contains:
YARN-5605. Preempt containers (all on one node) to meet the requirement of starved applications
YARN-5821. Drop left-over preemption-related code and clean up method visibilities in the Schedulable hierarchy
YARN-5783. Verify identification of starved applications.
YARN-5819. Verify fairshare and minshare preemption
YARN-5885. Cleanup YARN-4752 branch for merge

Change-Id: Iee0962377d019dd64dc69a020725d2eaf360858c
 2016-11-23 19:48:59 -10:00
Jian He
1f12867a69 YARN-5649. Add REST endpoints for updating application timeouts. Contributed by Rohith Sharma K S 2016-11-23 16:25:39 -08:00
Arun Suresh
005850b28f YARN-5918. Handle Opportunistic scheduling allocate request failure when NM is lost. (Bibin A Chundatt via asuresh) 2016-11-23 09:53:31 -08:00
Naganarasimha
3541ed8068 YARN-4330. MiniYARNCluster is showing multiple Failed to instantiate default resource calculator warning messages. Contributed by Varun Saxena 2016-11-23 14:12:23 +05:30
Naganarasimha
4667564162 YARN-5911. DrainDispatcher does not drain all events on stop even if setDrainEventsOnStop is true. Contributed by Varun Saxena. 2016-11-23 08:49:48 +05:30
Daniel Templeton
83cc7263af YARN-5722. FairScheduler hides group resolution exceptions when assigning queue (Contributed by Wilfred Spiegelenburg via Daniel Templeton) 2016-11-22 11:16:00 -08:00
Rohith Sharma K S
a926f895c1 YARN-5865. Retrospect updateApplicationPriority api to handle state store exception in align with YARN-5611. Contributed by Sunil G. 2016-11-22 14:49:15 +05:30
Akira Ajisaka
6f8074298d YARN-5713. Update jackson from 1.9.13 to 2.x in hadoop-yarn. 2016-11-22 10:20:31 +09:00
Jason Lowe
009452bb6d YARN-5859. TestResourceLocalizationService#testParallelDownloadAttemptsForPublicResource sometimes fails. Contributed by Eric Badger 2016-11-21 16:36:08 +00:00
Wei-Chiu Chuang
09520cb439 YARN-5271. ATS client doesn't work with Jersey 2 on the classpath. Contributed by Weiwei Yang. 2016-11-17 22:17:23 -06:00
Subru Krishnan
140b9939da YARN-5904. Reduce the number of default server threads for AMRMProxyService. 2016-11-17 18:26:25 -08:00
Junping Du
f05a9ceb4a YARN-3538. TimelineWebService doesn't catch runtime exception. Contributed by Steve Loughran and Tsuyoshi Ozawa. 2016-11-17 16:55:39 -08:00
Rohith Sharma K S
ce2847e794 YARN-5870. Expose getApplications API in YarnClient with GetApplicationsRequest parameter. Contributed by Jian He. 2016-11-17 08:48:35 +05:30
Jason Lowe
59bfcbf357 YARN-5836. Malicious AM can kill containers of other apps running in any node its containers are running. Contributed by Botong Huang 2016-11-16 22:21:03 +00:00
Varun Saxena
6a11877ab5 YARN-5891. yarn rmadmin -help contains a misspelled ResourceManager (Grant Sohn via Varun Saxena) 2016-11-17 03:26:13 +05:30
Rohith Sharma K S
d65603517e YARN-5375. invoke MockRM#drainEvents implicitly in MockRM methods to reduce test failures. Contributed by sandflee. 2016-11-16 15:14:00 +05:30
Xiao Chen
f121d0b036 YARN-5875. TestTokenClientRMService#testTokenRenewalWrongUser fails. Contributed by Gergely Novák. 2016-11-15 13:58:11 -08:00
Daniel Templeton
264ddb13ff YARN-5736 Addendum. Fixes segfault due to unterminated string. (Contributed by Miklos Szegedi via Daniel Templeton) 2016-11-15 13:01:57 -08:00
Mingliang Liu
5af572b644 HADOOP-13427. Eliminate needless uses of FileSystem#{exists(), isFile(), isDirectory()}. Contributed by Steve Loughran and Mingliang Liu 2016-11-15 10:57:00 -08:00
Arun Suresh
3219b7b4ac YARN-4597. Introduce ContainerScheduler and a SCHEDULED state to NodeManager container lifecycle. (asuresh) 2016-11-15 07:56:25 -08:00
Naganarasimha
7ffb9943b8 YARN-4355. NPE while processing localizer heartbeat. Contributed by Varun Saxena & Jonathan Hung. 2016-11-15 15:41:56 +05:30
Naganarasimha
43aef303bf Reverted due to issue YARN-5765. Revert "YARN-5287. LinuxContainerExecutor fails to set proper permission. Contributed by Ying Zhang" 
This reverts commit 131d58a24e.

server-nodemanager/src/main/native/container-executor/test/test-
container-executor.c
 2016-11-15 13:27:37 +05:30
Rohith Sharma K S
b7070f3308 YARN-5874. RM -format-state-store and -remove-application-from-state-store commands fail with NPE. Contributed by Varun Saxena. 2016-11-15 10:58:25 +05:30
Rohith Sharma K S
296c5de0cf YARN-5873. RM crashes with NPE if generic application history is enabled. Contributed by Varun Saxena. 2016-11-15 10:56:28 +05:30
Karthik Kambatla
79448d4ab1 YARN-4033. In FairScheduler, parent queues should also display queue status. (Siqi Li via kasha) 2016-11-13 21:38:36 -08:00
Jian He
fad9609d13 YARN-5825. ProportionalPreemptionalPolicy should use readLock over LeafQueue instead of synchronized block. Contributed by Sunil G 2016-11-11 15:16:21 -08:00
Wangda Tan
ede1a473f5 YARN-5552. Add Builder methods for common yarn API records. (Tao Jie via wangda) 2016-11-11 13:34:56 -08:00
Naganarasimha
503e73e849 YARN-5545. Fix issues related to Max App in capacity scheduler. Contributed by Bibin A Chundatt 2016-11-11 20:48:31 +05:30
Eric Payne
93eeb13164 YARN-4218. Metric for resource*time that was preempted. Contributed by Chang Li. 2016-11-10 22:35:12 +00:00
Karthik Kambatla
3a98419532 YARN-5834. TestNodeStatusUpdater.testNMRMConnectionConf compares nodemanager wait time to the incorrect value. (Chang Li via kasha) 2016-11-10 14:08:51 -08:00
Karthik Kambatla
86ac1ad9fd YARN-5453. FairScheduler#update may skip update demand resource of child queue/app if current demand reached maxResource. (sandflee via kasha) 2016-11-09 23:44:02 -08:00
Varun Saxena
c8bc7a8475 YARN-5843. Incorrect documentation for timeline service entityType/events REST end points (Bibin A Chundatt via Varun Saxena) 2016-11-10 12:10:03 +05:30
Varun Saxena
c202a10923 YARN-5862. TestDiskFailures.testLocalDirsFailures failed (Yufei Gu via Varun Saxena) 2016-11-10 11:41:34 +05:30
Naganarasimha
de3a5f8d08 YARN-5856. Unnecessary duplicate start container request sent to NM State store. Contributed by Varun Saxena. 2016-11-10 05:42:30 +05:30
Jian He
bcc15c6290 YARN-5611. Provide an API to update lifetime of an application. Contributed by Rohith Sharma K S 2016-11-09 16:08:05 -08:00
Naganarasimha
edbee9e609 YARN-4498. Application level node labels stats to be available in REST (addendum patch). Contributed by Bibin A Chundatt. 2016-11-10 05:00:05 +05:30
Daniel Templeton
59ee8b7a88 YARN-4329. [YARN-5437] Allow fetching exact reason as to why a submitted app 
is in ACCEPTED state in Fair Scheduler (Contributed by Yufei Gu)
 2016-11-09 13:11:37 -08:00