big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
16,699 Commits 2 Branches 2 Tags 264 MiB
dae65f3bef
Commit Graph

2807 Commits

Author SHA1 Message Date
Carlo Curino
6600abbb5c YARN-6587. Refactor of ResourceManager#startWebApp in a Util class. (Giovanni Matteo Fumarola via curino). 2017-05-12 17:48:55 -07:00
Haibo Chen
6c35001b9f YARN-6580. Incorrect logger for FairSharePolicy. (Vrushali C via Haibo Chen) 2017-05-12 13:05:40 -07:00
Daniel Templeton
90cb5b4635 YARN-6380. FSAppAttempt keeps redundant copy of the queue 2017-05-11 13:47:02 -07:00
Min Shen
2ada100da7 YARN-5543. ResourceManager SchedulingMonitor could potentially terminate the preemption checker thread. Contributed by Min Shen. 2017-05-11 13:25:31 -07:00
Brahma Reddy Battula
4aae2d40a3 HADOOP-14410. Correct spelling of 'beginning' and variants. Contributed By Dongtao Zhang 2017-05-11 23:33:45 +05:30
Carlo Curino
5cb6e3e082 YARN-6473. Create ReservationInvariantChecker to validate ReservationSystem + Scheduler operations. 2017-05-10 14:49:25 -07:00
Daniel Templeton
e7654c4a1f YARN-6571. Fix JavaDoc issues in SchedulingPolicy 
(Contributed by Weiwei Yang via Daniel Templeton)
 2017-05-10 14:34:13 -07:00
Daniel Templeton
74a61438ca YARN-6475. Fix some long function checkstyle issues 
(Contributed by Soumabrata Chakraborty via Daniel Templeton)
 2017-05-10 10:46:50 -07:00
Jason Lowe
6099deebcb YARN-6552. Increase YARN test timeouts from 1 second to 10 seconds. Contributed by Eric Badger 2017-05-10 08:47:48 -05:00
Haibo Chen
ab2bb93a2a YARN-6561. Update exception message during timeline collector aux service initialization. (Vrushali C via Haibo Chen) 2017-05-09 21:37:30 -07:00
Haibo Chen
461ee44d28 YARN-6435. [ATSv2] Can't retrieve more than 1000 versions of metrics in time series. (Rohith Sharma K S via Haibo Chen) 2017-05-09 21:12:57 -07:00
Karthik Kambatla
166be0ee95 YARN-3742. YARN RM will shut down if ZKClient creation times out. (Daniel Templeton via kasha) 2017-05-09 14:44:16 -07:00
Daniel Templeton
a2f680493f YARN-5301. NM mount cpu cgroups failed on some systems 
(Contributed by Miklos Szegedi via Daniel Templeton)
 2017-05-09 12:05:46 -07:00
Vrushali Channapattan
7dd258d8f4 YARN-6563 ConcurrentModificationException in TimelineCollectorManager while stopping RM (Contributed by Haibo Chen via Vrushali C) 2017-05-09 11:18:12 -07:00
Andrew Wang
543aac9f28 HADOOP-14386. Rewind trunk from Guava 21.0 back to Guava 11.0.2. 2017-05-09 09:22:53 -07:00
Subru Krishnan
749e5c09b9 YARN-6281. Cleanup when AMRMProxy fails to initialize a new interceptor chain. (Botong Huang via Subru) 
(cherry picked from commit 57a9afbd45b7ef8e6021cc58f96bd0074bf1389d)
 2017-05-08 16:55:47 -07:00
Subru Krishnan
cd9ff27ffc YARN-6234. Support multiple attempts on the node when AMRMProxy is enabled. (Giovanni Matteo Fumarola via Subru). 2017-05-08 16:41:30 -07:00
Jason Lowe
424887ecb7 YARN-3839. Quit throwing NMNotYetReadyException. Contributed by Manikandan R 2017-05-08 17:14:37 -05:00
Subru Krishnan
d6eed5acca YARN-6565. Fix memory leak and finish app trigger in AMRMProxy. (Botong Huang via Subru). 2017-05-05 16:27:49 -07:00
Chris Douglas
e4f34ecb04 HADOOP-14390. Correct spelling of 'succeed' and variants. Contributed by Dongtao Zhang 2017-05-05 12:10:50 -07:00
Vrushali Channapattan
54e2b9e876 YARN-6375 App level aggregation should not consider metric values reported in the previous aggregation cycle (Varun Saxena via Vrushali C) 2017-05-04 15:25:56 -07:00
Sidharta S
fd5cb2c946 YARN-6374. Improve test coverage and add utility classes for common Docker operations. Contributed by Shane Kumpf 2017-05-03 14:49:30 -07:00
Daniel Templeton
9f0aea0ee2 YARN-6481. Yarn top shows negative container number in FS 
(Contributed by Tao Jie via Daniel Templeton)
 2017-05-02 13:06:47 -07:00
Subru Krishnan
6bf42e48ef YARN-5331. Extend RLESparseResourceAllocation with period for supporting recurring reservations in YARN ReservationSystem. (Sangeetha Abdu Jyothi via Subru). 2017-05-01 18:48:36 -07:00
Subru Krishnan
e514fc432a YARN-6542. Fix the logger in TestAlignedPlanner and TestGreedyReservationAgent. 2017-05-01 18:10:07 -07:00
Subru Krishnan
a3a615eeab YARN-4359. Update LowCost agents logic to take advantage of YARN-4358. (Jonathan Yaniv and Ishai Menache via Subru). 2017-05-01 16:01:07 -07:00
Naganarasimha
30fc580196 YARN-6519. Fix warnings from Spotbugs in hadoop-yarn-server-resourcemanager. Contributed by Weiwei Yang. 2017-05-01 20:15:27 +05:30
Naganarasimha
0f1af3178e YARN-6518. Fix warnings from Spotbugs in hadoop-yarn-server-timelineservice. Contributed by Weiwei Yang. 2017-05-01 16:49:30 +05:30
Daniel Templeton
2e52789edf YARN-2962. ZKRMStateStore: Limit the number of znodes under a znode (Contributed by Varun Sexena via Daniel Templeton) 2017-04-28 13:30:28 -07:00
Robert Kanter
68e45f554b YARN-6472. Improve Java sandbox regex (gphillips via rkanter) 2017-04-28 11:01:50 -07:00
Robert Kanter
371b6467dc YARN-5894. fixed license warning caused by de.ruedigermoeller:fst:jar:2.24 (haibochen via rkanter) 2017-04-27 17:49:38 -07:00
Varun Saxena
793bbf216d YARN-6455. Enhance the timelinewriter.flush() race condition fix (Haibo Chen via Varun Saxena) 2017-04-27 15:01:58 +05:30
Akira Ajisaka
8a99eba96d
Revert "YARN-5007. Remove deprecated constructors of MiniYARNCluster and MiniMRYarnCluster. Contributed by Andras Bokor." 
This reverts commit 34ab8e73d4.
 2017-04-26 15:34:23 +09:00
Wangda Tan
2ba21d6376 YARN-6392. Add submit time to Application Summary log. (Zhihai Xu via wangda) 2017-04-24 13:17:35 -07:00
Haibo Chen
8ac50e1322 YARN-6500. Do not mount inaccessible cgroups directories in CgroupsLCEResourcesHandler. (Miklos Szegedi via Haibo Chen) 2017-04-24 11:37:52 -07:00
Eric Payne
52adf71914 YARN-5617. AMs only intended to run one attempt can be run more than once? Contributed by Jason Lowe. 2017-04-24 12:03:28 -05:00
Daniel Templeton
46940d92e2 YARN-6302. Fail the node if Linux Container Executor is not configured properly 
(Contributed by Miklos Szegedi via Daniel Templeton)
 2017-04-19 12:23:49 -07:00
Daniel Templeton
dd43b895c2 YARN-6202. Configuration item Dispatcher.DISPATCHER_EXIT_ON_ERROR_KEY is disregarded 
(Contributed by Yufei Gu via Daniel Templeton)
 2017-04-19 11:44:55 -07:00
Daniel Templeton
41ac190dba YARN-6438. Code can be improved in ContainersMonitorImpl.java 
(Contributed by Miklos Szegedi via Daniel Templeton)
 2017-04-19 10:53:08 -07:00
Sunil G
9ff25408bc YARN-6164. Expose Queue Configurations per Node Label through YARN client api. Contributed by Benson Qiu. 2017-04-19 20:39:09 +05:30
Chris Douglas
af8e9842d2 YARN-6451. Add RM monitor validating metrics invariants. Contributed by Carlo Curino 2017-04-18 10:29:44 -07:00
Junping Du
6ed9d36224 YARN-6304. Skip rm.transitionToActive call to RM if RM is already active. Contributed by Rohith Sharma K S. 2017-04-17 15:51:10 -07:00
Jason Lowe
4168805502 YARN-6480. Timeout is too aggressive for TestAMRestart.testPreemptedAMRestartOnRMRestart. Contributed by Eric Badger 2017-04-14 17:15:48 -05:00
Karthik Kambatla
8a1d7480f7 YARN-6433. Only accessible cgroup mount directories should be selected for a controller. (Miklos Szegedi via kasha) 2017-04-14 15:07:14 -07:00
Eric Payne
a41f8dd58e YARN-5994. TestCapacityScheduler.testAMLimitUsage fails intermittently. Contributed by Eric Badger 2017-04-14 10:53:09 -05:00
Karthik Kambatla
c3375175d6 YARN-6432. FairScheduler: Reserve preempted resources for corresponding applications. (Miklos Szegedi via kasha) 2017-04-12 14:21:20 -07:00
Eric Payne
a16ab2be91 YARN-6450. TestContainerManagerWithLCE requires override for each new test added to ContainerManagerTest? Contributed by Jason Lowe. 2017-04-12 10:20:08 -05:00
Wangda Tan
4d4ad0ebb7 YARN-6439. Fix ReservationSystem creation of default ReservationQueue. (Carlo Curino via wangda) 2017-04-11 14:56:18 -07:00
Naganarasimha
7d873c465b YARN-6463. Correct spelling mistake in FileSystemRMStateStore. Contributed by Yeliang Cang. 2017-04-11 22:42:08 +05:30
Jason Lowe
0e065f2ede YARN-6195. Export UsedCapacity and AbsoluteUsedCapacity to JMX. Contributed by Benson Qiu 2017-04-11 08:44:18 -05:00
Konstantinos Karanasos
7999318af1 YARN-6344. Add parameter for rack locality delay in CapacityScheduler. (kkaranasos) 2017-04-10 15:34:44 -07:00
Daniel Templeton
e9ac61cc0e YARN-6372. Add default value for NM disk validator (Contributed by Yufei Gu via Daniel Templeton) 2017-04-10 14:56:42 -07:00
Karthik Kambatla
2aa8967809 YARN-6298. Metric preemptCall is not used in new preemption. (Yufei Gu via kasha) 2017-04-07 17:31:59 -07:00
Robert Kanter
63f7322522 YARN-6368. Decommissioning an NM results in a -1 exit code (miklos.szegedi@cloudera.com via rkanter) 2017-04-07 14:28:07 -07:00
Ming Ma
0116c3c957 YARN-5797. Add metrics to the node manager for cleaning the PUBLIC and PRIVATE caches. (Chris Trezzo via mingma) 2017-04-06 16:54:43 -07:00
Jason Lowe
1b081ca27e YARN-6288. Exceptions during aggregated log writes are mishandled. Contributed by Akira Ajisaka 2017-04-06 16:24:36 -05:00
Rohith Sharma K S
1a9439e299 YARN-6424. TimelineCollector is not stopped when an app finishes in RM. Contributed by Varun Saxena. 2017-04-06 10:15:22 +05:30
Daniel Templeton
a2c57bb70d YARN-6381. FSAppAttempt has several variables that should be final 
(Contributed by Ameet Zaveri via Daniel Templeton)
 2017-04-05 16:07:24 -07:00
Andrew Wang
539ef5aa2e HFDS-11596. hadoop-hdfs-client jar is in the wrong directory in release tarball. Contributed by Yuanbo Liu. 2017-04-05 16:04:09 -07:00
Karthik Kambatla
b4c4f36594 YARN-6448. Continuous scheduling thread crashes while sorting nodes. (Yufei Gu via kasha) 2017-04-05 15:43:14 -07:00
Varun Saxena
7d963c477a YARN-6437. TestSignalContainer#testSignalRequestDeliveryToNM fails intermittently (Jason Lowe via Varun Saxena) 2017-04-06 01:20:41 +05:30
Wangda Tan
87e2ef8c98 YARN-6406. Remove SchedulerRequestKeys when no more pending ResourceRequest. (Arun Suresh via wangda) 2017-04-05 11:28:30 -07:00
Jason Lowe
e8071aa249 YARN-6403. Invalid local resource request can raise NPE and make NM exit. Contributed by Tao Yang 2017-04-05 12:30:45 -05:00
Akira Ajisaka
34ab8e73d4
YARN-5007. Remove deprecated constructors of MiniYARNCluster and MiniMRYarnCluster. Contributed by Andras Bokor. 2017-04-05 17:33:07 +09:00
Varun Saxena
19b89c4c7b YARN-6420. RM startup failure due to wrong order in nodelabel editlog (Bibin A Chundatt via Varun Saxena) 2017-04-05 09:38:35 +05:30
Ming Ma
2d5c09b848 YARN-6004. Refactor TestResourceLocalizationService#testDownloadingResourcesOnContainer so that it is less than 150 lines. (Chris Trezzo via mingma) 2017-04-04 17:56:21 -07:00
Karthik Kambatla
9cc04b4708 YARN-6436. TestSchedulingPolicy#testParseSchedulingPolicy timeout is too low. (Eric Badger via kasha) 2017-04-04 17:08:33 -07:00
Wangda Tan
3fdae0a2b6 YARN-6109. Add an ability to convert ChildQueue to ParentQueue. (Xuan Gong via wangda) 2017-04-04 14:39:01 -07:00
Chris Douglas
6eba79232f HADOOP-14271. Correct spelling of 'occurred' and variants. Contributed by Yeliang Cang 2017-04-03 20:13:14 -07:00
Varun Saxena
a4b5aa8493 YARN-6377. NMTimelinePublisher#serviceStop does not stop timeline clients (Haibo Chen via Varun Saxena) 2017-04-02 04:54:12 +05:30
Varun Saxena
b8838578ce YARN-6414. ATSv2 HBase related tests fail due to guava version upgrade (Haibo Chen via Varun Saxena) 2017-04-02 04:37:34 +05:30
Eric Payne
318bfb01bc YARN-6354. LeveldbRMStateStore can parse invalid keys when recovering reservations. Contributed by Jason Lowe 2017-03-31 12:30:35 -05:00
Jason Lowe
4d1fac5df2 YARN-6411. Clean up the overwrite of createDispatcher() in subclass of MockRM. Contributed by Yufei Gu 2017-03-31 10:05:34 -05:00
Varun Saxena
b58777a9c9 YARN-6376. Exceptions caused by synchronous putEntities requests can be swallowed (Haibo Chen via Varun Saxena) 2017-03-31 02:17:20 +05:30
Daniel Templeton
640ba1d23f YARN-5685. RM configuration allows all failover methods to disabled when automatic failover is enabled 2017-03-29 12:38:11 -07:00
Akira Ajisaka
82fb9ce8df
YARN-6329. Remove unnecessary TODO comment from AppLogAggregatorImpl.java. Contributed by victor bertschinger. 2017-03-29 10:36:24 +09:00
Varun Saxena
063b513b1c YARN-6357. Implement putEntitiesAsync API in TimelineCollector (Haibo Chen via Varun Saxena) 2017-03-29 03:48:03 +05:30
Varun Saxena
01aca54a22 YARN-5368. Memory leak in timeline server (Jonathan Eagles via Varun Saxena) 2017-03-29 01:53:20 +05:30
Jason Lowe
fdf8f8ebca YARN-6359. TestRM#testApplicationKillAtAcceptedState fails rarely due to race condition. Contributed by Robert Kanter 2017-03-28 09:33:26 -05:00
Robert Kanter
9bae6720cb YARN-6050. AMs can't be scheduled on racks or nodes (rkanter) 2017-03-27 17:23:13 -07:00
Wangda Tan
cd014d57aa YARN-6339. Improve performance for createAndGetApplicationReport. (Yunjiong Zhao via wangda) 2017-03-27 13:29:09 -07:00
Akira Ajisaka
b63ae854f9
YARN-6379. Remove unused argument in ClientRMService. Contributed by Kai Sasaki. 2017-03-27 11:23:41 +09:00
Sunil G
cffea251be YARN-5956. Refactor ClientRMService for unify error handling across apis. Contributed by Kai Sasaki. 2017-03-25 22:26:19 +05:30
Tsuyoshi Ozawa
84ddedc0b2 HADOOP-10101. Update guava dependency to the latest version. (ozawa) 2017-03-25 10:06:52 +09:00
Ravi Prakash
d4f73e7e27 HDFS-11574. Spelling mistakes in the Java source. Contributed by Hu Xiaodong. 2017-03-24 09:38:17 -07:00
Varun Saxena
f462e1ff68 YARN-5934. Fix TestTimelineWebServices.testPrimaryFilterNumericString (Akira Ajisaka via Varun Saxena) 2017-03-22 04:11:53 +05:30
Daniel Templeton
0a05c5c598 YARN-6284. hasAlreadyRun should be final in ResourceManager.StandByTransitionRunnable 
(Contributed by Laura Adams via Daniel Templeton)
 2017-03-21 14:15:40 -07:00
Varun Saxena
35034653d0 YARN-6353. Clean up OrderingPolicy javadoc (Daniel Templeton via Varun Saxena) 2017-03-20 23:29:09 +05:30
Jason Lowe
4a8e304502 YARN-6217. TestLocalCacheDirectoryManager test timeout is too aggressive. Contributed by Miklos Szegedi 2017-03-17 14:06:35 -05:00
Jason Lowe
7114baddb6 YARN-4051. ContainerKillEvent lost when container is still recovering and application finishes. Contributed by sandflee 2017-03-16 09:30:10 -05:00
Sunil G
2d0e24eb01 YARN-6328. Fix a spelling mistake in CapacityScheduler. Contributed by Jin Yibo. This closes #202 2017-03-15 14:48:05 +05:30
Ray Chiang
4c66a8d19b YARN-6331. Fix flakiness in TestFairScheduler#testDumpState. (Yufei Gu via rchiang) 2017-03-14 15:09:47 -07:00
Naganarasimha
0a3aa40fe7 YARN-6327. Removing queues from CapacitySchedulerQueueManager and ParentQueue should be done with iterator. Contributed by Jonathan Hung. 2017-03-15 01:22:25 +05:30
Junping Du
34424e98a6 YARN-6314. Potential infinite redirection on YARN log redirection web service. Contributed by Xuan Gong. 
(cherry picked from commit 5a9dda796f0e73060ada794ad5752cc6a237ab2e)
 2017-03-14 02:58:07 -07:00
Ray Chiang
4db9cc70d0 YARN-6042. Dump scheduler and queue state information into FairScheduler DEBUG log. (Yufei Gu via rchiang) 2017-03-10 16:13:31 -08:00
Varun Saxena
229c7c9f89 YARN-6069. CORS support in timeline v2 (Rohith Sharma K S via Varun Saxena) 2017-03-11 04:09:01 +05:30
Jason Lowe
9649c27864 YARN-6321. TestResources test timeouts are too aggressive. Contributed by Eric Badger 2017-03-10 13:06:54 -06:00
Karthik Kambatla
a96afae125 YARN-6264. AM not launched when a single vcore is available on the cluster. (Yufei Gu via kasha) 2017-03-09 23:11:54 -08:00
Daniel Templeton
846a0cd678 YARN-1047. Expose # of pre-emptions as a queue counter (Contributed by Karthik Kambatla via Daniel Templeton) 2017-03-09 17:51:47 -08:00
Sidharta S
e96a0b8c92 YARN-5669. Add support for docker pull command (Contribtued by luhuichun) 2017-03-09 16:22:19 -08:00
Daniel Templeton
822a74f2ae YARN-6300. NULL_UPDATE_REQUESTS is redundant in TestFairScheduler (Contributed by Yuanbo Liu via Daniel Templeton) 2017-03-09 12:14:33 -08:00
Jason Lowe
d7762a5511 YARN-6165. Intra-queue preemption occurs even when preemption is turned off for a specific queue. Contributed by Eric Payne 2017-03-08 16:46:09 -06:00
Robert Kanter
287ba4ffa6 YARN-6297. TestAppLogAggregatorImp.verifyFilesUploaded() should check # of filed uploaded with that of files expected (haibochen via rkanter) 2017-03-08 10:45:33 -08:00
Sunil G
1eb8186703 YARN-6207. Move application across queues should handle delayed event processing. Contributed by Bibin A Chundatt. 2017-03-08 12:04:30 +05:30
Daniel Templeton
e0c239cdbd YARN-6287. RMCriticalThreadUncaughtExceptionHandler.rmContext should be final (Contributed by Corey Barker via Daniel Templeton) 2017-03-07 11:58:48 -08:00
Sunil G
c1386a90d7 YARN-6248. user is not removed from UsersManager’s when app is killed with pending container requests. Contributed by Eric Payne. 2017-03-04 13:46:35 +05:30
Junping Du
8db7a8c3ae YARN-6271. yarn rmadin -getGroups returns information from standby RM. Contributed by Jian He. 2017-03-03 14:39:31 -08:00
Ray Chiang
2148b83993 YARN-6218. Fix TestAMRMClient when using FairScheduler. (Miklos Szegedi via rchiang) 2017-03-03 13:04:51 -08:00
Jason Lowe
e58fc76030 YARN-6263. NMTokenSecretManagerInRM.createAndGetNMToken is not thread safe. Contributed by Haibo Chen 2017-03-03 08:23:08 -06:00
Robert Kanter
6f6dfe0202 YARN-5280. Allow YARN containers to run with Java Security Manager (gphillips via rkanter) 2017-03-01 14:53:47 -08:00
Jian He
235203dffd YARN-6153. KeepContainer does not work when AM retry window is set. Contributed by kyungwan nam 2017-02-28 13:23:36 -08:00
Xuan
e0bb867c3f YARN-6189: Improve application status log message when RM restarted when 
app is in NEW state. Contributed by Junping Du
 2017-02-28 11:04:56 -08:00
Wangda Tan
eac6b4c35c YARN-6216. Unify Container Resizing code paths with Container Updates making it scheduler agnostic. (Arun Suresh via wangda) 2017-02-28 10:35:50 -08:00
Yiqun Lin
0f35443bf2 YARN-6239. Fix javadoc warnings in YARN that caused by deprecated FileSystem APIs. Contributed by Yiqun Lin. 2017-02-28 18:51:56 +08:00
Karthik Kambatla
f187d63816 YARN-6231. FairSchedulerTestBase helper methods should call scheduler.update to avoid flakiness. (kasha) 2017-02-27 22:10:57 -08:00
Naganarasimha
5f5b031d1f YARN-5703. ReservationAgents are not correctly configured. Contributed by Manikandan R. 2017-02-27 20:38:29 +05:30
Karthik Kambatla
fbfe86deea YARN-6172. FSLeafQueue demand update needs to be atomic. (Miklos Szegedi via kasha) 2017-02-26 20:36:33 -08:00
Karthik Kambatla
815d53506f YARN-6215. FairScheduler preemption and update should not run concurrently. (Tao Jie via kasha) 2017-02-26 20:16:36 -08:00
Xuan
53d372a255 YARN-6228: EntityGroupFSTimelineStore should allow configurable cache 
stores. Contributed by Li Lu
 2017-02-24 10:59:35 -08:00
Karthik Kambatla
694e680d20 YARN-6222. TestFairScheduler.testReservationMetrics is flaky. (Yufei Gu via kasha) 2017-02-23 15:21:52 -08:00
Sunil G
a207aa9930 YARN-6211. Synchronization improvement for moveApplicationAcrossQueues and updateApplicationPriority. Contributed by Bibin A Chundatt. 2017-02-23 14:19:07 +05:30
Karthik Kambatla
b10e962224 YARN-6194. Cluster capacity in SchedulingPolicy is updated only on allocation file reload. (Yufei Gu via kasha) 2017-02-22 15:58:49 -08:00
Karthik Kambatla
718ad9f6ee YARN-6210. FairScheduler: Node reservations can interfere with preemption. (kasha) 2017-02-22 15:46:07 -08:00
Wangda Tan
1a6ca75f38 YARN-6143. Fix incompatible issue caused by YARN-3583. (Sunil G via wangda) 2017-02-22 11:17:09 -08:00
Eric Yang
172b23af33 HADOOP-14077. Add ability to access jmx via proxy. Contributed by Yuanbo Liu. 2017-02-18 18:34:13 -08:00
Karthik Kambatla
dbbfcf74ab YARN-6193. FairScheduler might not trigger preemption when using DRF. (kasha) 2017-02-17 14:07:31 -08:00
Daniel Templeton
c7a36e6130 YARN-6125. The application attempt's diagnostic message should have a maximum size 
(Contributed by Andras Piros via Daniel Templeton)
 2017-02-17 13:40:58 -08:00
Daniel Templeton
9a92837786 YARN-6188. Fix OOM issue with decommissioningNodesWatcher in the case of clusters with 
large number of nodes (Contributed by Ajay Jadhav via Daniel Templeton)
 2017-02-17 13:13:46 -08:00
Karthik Kambatla
a77f432449 YARN-6171. ConcurrentModificationException on FSAppAttempt.containersToPreempt. (Miklos Szegedi via kasha) 2017-02-16 14:54:58 -08:00
Sangjin Lee
4fa1afdb88 YARN-4675. Reorganize TimelineClient and TimelineClientImpl into separate classes for ATSv1.x and ATSv2. Contributed by Naganarasimha G R. 2017-02-16 11:41:04 -08:00
Rohith Sharma K S
e63a7814d2 YARN-6200. addendum to fix compilation error caused by reverting YARN-5068. 2017-02-16 18:19:57 +05:30
Rohith Sharma K S
a393e84c6f YARN-6200. Reverting since the same functionality achieved by YARN-1623. 
Revert "YARN-5068. Expose scheduler queue to application master. (Harish Jaiprakash via rohithsharmaks)"

This reverts commit b7ac85259c.
 2017-02-16 18:00:25 +05:30
Karthik Kambatla
74dd142250 YARN-5798. Set UncaughtExceptionHandler for all FairScheduler threads. (Yufei Gu via kasha) 2017-02-16 00:03:09 -08:00
Karthik Kambatla
11be3f70e0 YARN-4212. FairScheduler: Can't create a DRF queue under a FAIR policy queue. (Yufei Gu via kasha) 2017-02-15 23:51:22 -08:00
Karthik Kambatla
6c25dbcdc0 YARN-6163. FS Preemption is a trickle for severely starved applications. (kasha) 2017-02-15 23:16:12 -08:00
Junping Du
ce2d5bfa5f YARN-6174. Log files pattern should be same for both running and finished container. Contributed by Xuan Gong. 2017-02-15 09:05:14 -08:00
Varun Saxena
b7613e0f40 YARN-6156. AM blacklisting to consider node label partition (Bibin A Chundatt via Varun Saxena) 2017-02-15 14:48:17 +05:30
Karthik Kambatla
353a9b2d91 YARN-6061. Addendum. Remove extraneous change. 2017-02-14 15:19:52 -08:00
Karthik Kambatla
652679aa8a YARN-6061. Add an UncaughtExceptionHandler for critical threads in RM. (Yufei Gu via kasha) 2017-02-14 13:39:41 -08:00
Arun Suresh
aaf106fde3 YARN-5966. AMRMClient changes to support ExecutionType update. (asuresh) 2017-02-14 06:09:10 -08:00
Karthik Kambatla
646c6d6509 YARN-3933. FairScheduler: Multiple calls to completedContainer are not safe. (Shiwei Guo and Miklos Szegedi via kasha) 2017-02-13 11:26:30 -08:00
Junping Du
464ff479ce YARN-6113. Re-direct NM Web Service to get container logs for finished applications. Contributed by Xuan Gong. 2017-02-13 06:12:54 -08:00
Varun Saxena
649deb72fb YARN-6170. TimelineReaderServer should wait to join with HttpServer2 (Sangjin Lee via Varun Saxena) 2017-02-11 19:21:45 +05:30
Karthik Kambatla
08f93978f3 YARN-6144. FairScheduler: preempted resources can become negative. (Miklos Szegedi via kasha) 2017-02-09 14:45:04 -08:00
Karthik Kambatla
9b85053583 YARN-6112. UpdateCallDuration is calculated only when debug logging is enabled. (Yufei Gu via kasha) 2017-02-09 13:09:32 -08:00
Wangda Tan
5fb723bb77 YARN-5889. Improve and refactor user-limit calculation in Capacity Scheduler. (Sunil G via wangda) 2017-02-09 10:23:50 -08:00
Junping Du
9dbfab1284 YARN-6108. Improve AHS webservice to accept NM address as a parameter to get container logs. Contributed by Xuan Gong. 2017-02-07 02:13:14 -08:00
Junping Du
327c9980aa YARN-6100. Improve YARN webservice to output aggregated container logs. Contributed by Xuan Gong. 2017-02-02 00:41:18 -08:00
Daniel Templeton
3e06475307 YARN-4658. Typo in o.a.h.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler comment (Contributed by Udai Potluri via Daniel Templeton) 2017-01-31 16:25:33 -08:00
Daniel Templeton
87852b6ef4 YARN-6103. Log updates for ZKRMStateStore (Contributed by Daniel Sturman via Daniel Templeton) 2017-01-30 21:44:48 -08:00
Karthik Kambatla
11e44bdda7 YARN-6131. FairScheduler: Lower update interval for faster tests. (kasha) 2017-01-30 13:35:23 -08:00
Sunil G
165f07f51a YARN-6123. [YARN-5864] Add a test to make sure queues of orderingPolicy will be updated when childQueues is added or removed. Contributed by Wangda Tan. 2017-01-27 18:37:51 +05:30
Daniel Templeton
f85b74ccf9 YARN-4975. Fair Scheduler: exception thrown when a parent queue marked 'parent' has configured child queues 
(Contributed by Yufei Gu via Daniel Templeton)
 2017-01-26 10:31:09 -08:00
Jason Lowe
9e19f758c1 YARN-5641. Localizer leaves behind tarballs after container is complete. Contributed by Eric Badger 2017-01-25 21:41:43 +00:00
Karthik Kambatla
abedb8a9d8 YARN-5830. FairScheduler: Avoid preempting AM containers. (Yufei Gu via kasha) 2017-01-25 12:17:28 -08:00
Junping Du
8528d85a68 YARN-6099. Improve webservice to list aggregated log files. Contributed by Xuan Gong. 2017-01-24 15:26:53 -08:00
Jason Lowe
a33ce45e35 YARN-5547. NMLeveldbStateStore should be more tolerant of unknown keys. Contributed by Ajith S 2017-01-24 16:17:36 +00:00
Rohith Sharma K S
0101267d9d YARN-6082. Invalid REST api response for getApps since queueUsagePercentage is coming as INF. Contributed by Sunil G. 2017-01-24 10:59:55 +05:30
Wangda Tan
ce832059db YARN-5864. Capacity Scheduler - Queue Priorities. (wangda) 2017-01-23 10:52:14 -08:00
Jason Lowe
69fa81679f YARN-5910. Support for multi-cluster delegation tokens. Contributed by Jian He 2017-01-23 17:12:51 +00:00
Sunil G
3fa0d540df YARN-6031. Application recovery has failed when node label feature is turned off during RM recovery. Contributed by Ying Zhang. 2017-01-23 12:19:35 +05:30
Sangjin Lee
b01514f65b YARN-5928. Move ATSv2 HBase backend code into a new module that is only dependent at runtime by yarn servers. Contributed by Haibo Chen. 2017-01-19 20:52:55 -08:00
Andrew Wang
5d8b80ea9b Preparing for 3.0.0-alpha3 development 2017-01-19 15:50:07 -08:00
Wangda Tan
72054a817d YARN-5556. CapacityScheduler: Support deleting queues without requiring a RM restart. (Naganarasimha G R via wangda) 2017-01-18 13:31:33 -08:00
Karthik Kambatla
e224c96234 YARN-5831. FairScheduler: Propagate allowPreemptionFrom flag all the way down to the app. (Yufei Gu via kasha) 2017-01-17 17:01:31 -08:00
Subru Krishnan
4d1f3d9020 YARN-6016. Fix minor bugs in handling of local AMRMToken in AMRMProxy. (Botong Huang via Subru). 2017-01-17 14:48:03 -08:00
Daniel Templeton
695f110f93 YARN-6071. Fix incompatible API change on AM-RM protocol due to YARN-3866 (trunk only) 
(Contributed by Wangda Tan via Daniel Templeton)
 2017-01-17 12:55:47 -08:00
Junping Du
cf695577aa YARN-6011. Add a new web service to list the files on a container in AHSWebService. Contributed by Xuan Gong. 2017-01-16 16:20:24 -08:00
Sunil G
d3170f9eba YARN-6081. LeafQueue#getTotalPendingResourcesConsideringUserLimit should deduct reserved from pending to avoid unnecessary preemption of reserved container. Contributed by Wangda Tan. 2017-01-13 18:22:29 +05:30
Naganarasimha
a6b06f7179 YARN-6072. RM unable to start in secure mode. Contributed by Ajith S. 2017-01-12 06:40:41 +05:30
Daniel Templeton
e6f13fe5d1 YARN-5849. Automatically create YARN control group for pre-mounted cgroups (Contributed by Miklos Szegedi via Daniel Templeton) 2017-01-11 16:00:01 -08:00
Daniel Templeton
7979939428 YARN-5554. MoveApplicationAcrossQueues does not check user permission on the target queue 
(Contributed by Wilfred Spiegelenburg via Daniel Templeton)
 2017-01-11 14:34:10 -08:00
Jason Lowe
357eab9566 YARN-5416. TestRMRestart#testRMRestartWaitForPreviousAMToFinish failed intermittently due to not wait SchedulerApplicationAttempt to be stopped. Contributed by Junping Du 2017-01-11 17:13:37 +00:00
Junping Du
4db119b7b5 YARN-6079. Fix simple spelling errors in yarn test code. Contributed by vijay. 2017-01-10 18:20:27 -08:00
Naganarasimha
4c431a6940 YARN-6054. TimelineServer fails to start when some LevelDb state files are missing. Contributed by Ravi Prakash. 2017-01-10 15:54:16 +05:30
Sunil G
9594c35dcb YARN-5899. Debug log in AbstractCSQueue#canAssignToThisQueue needs improvement. Contributed by Ying Zhang. 2017-01-10 14:08:01 +05:30
Junping Du
945db55f2e YARN-4148. When killing app, RM releases app's resource before they are released by NM. Contributed by Jason Lowe. 2017-01-09 18:14:46 -08:00
Wangda Tan
287d3d6804 YARN-3955. Support for application priority ACLs in queues of CapacityScheduler. (Sunil G via wangda) 2017-01-09 08:40:39 -08:00
Varun Saxena
f59e36b4ce YARN-6068. Log aggregation get failed when NM restart even with recovery (Junping Du via Varun Saxena) 2017-01-09 11:17:19 +05:30
Daniel Templeton
ac16400e1f YARN-6022. Revert changes of AbstractResourceRequest (Contributed by Wangda Tan via Daniel Templeton) 2017-01-07 08:24:30 -08:00
Naganarasimha
a59df15757 YARN-6015. AsyncDispatcher thread name can be set to improved debugging. Contributed by Ajith S. 2017-01-07 08:11:58 +05:30
Jian He
020316458d YARN-6009. Skip validating app timeout value on recovery. Contributed by Rohith Sharma K S 2017-01-06 18:19:09 -08:00
Arun Suresh
2977bc6a14 YARN-6040. Introduce api independent PendingAsk to replace usage of ResourceRequest within Scheduler classes. (Wangda Tan via asuresh) 2017-01-06 09:59:57 -08:00
Wangda Tan
0a55bd841e YARN-5959. RM changes to support change of container ExecutionType. (Arun Suresh via wangda) 2017-01-05 10:31:05 -08:00
Rohith Sharma K S
e49e0a6e37 YARN-5988. RM unable to start in secure setup. Contributed by Ajith S. 2017-01-04 12:09:29 +05:30
Junping Du
8fadd69047 YARN-5923. Unable to access logs for a running application if YARN_ACL_ENABLE is enabled. Contributed by Xuan Gong. 2017-01-03 15:03:38 -08:00
Wangda Tan
f69a107aec YARN-6025. Fix synchronization issues of AbstractYarnScheduler#nodeUpdate and its implementations. (Naganarasimha G R via wangda) 
(cherry picked from commit e0f2379312)
 2017-01-03 14:53:36 -08:00
Wangda Tan
ab1faa4ba8 Revert "Fix synchronization issues of AbstractYarnScheduler#nodeUpdate and its implementations. (Naganarasimha G R via wangda)" to add correct JIRA number 
This reverts commit e0f2379312.
 2017-01-03 14:53:13 -08:00
Wangda Tan
e0f2379312 Fix synchronization issues of AbstractYarnScheduler#nodeUpdate and its implementations. (Naganarasimha G R via wangda) 2017-01-03 14:46:41 -08:00
Daniel Templeton
165d01a73e YARN-5931. Document timeout interfaces CLI and REST APIs (Contributed by Rohith Sharma K S via Daniel Templeton) 2016-12-30 08:41:52 -08:00
Robert Kanter
f216276d21 YARN-4882. Change the log level to DEBUG for recovering completed applications (templedf via rkanter) 2016-12-28 15:21:52 -08:00
Sunil G
9ca54f4810 YARN-5906. Update AppSchedulingInfo to use SchedulingPlacementSet. Contributed by Wangda Tan. 2016-12-28 22:48:19 +05:30
Varun Vasudev
972da46cb4 YARN-5719. Enforce a C standard for native container-executor. Contributed by Chris Douglas. 2016-12-28 14:59:57 +05:30
Sunil G
f6715b26b6 YARN-6024. Capacity Scheduler 'continuous reservation looking' doesn't work when sum of queue's used and reserved resources is equal to max. Contributed by Wangda Tan. 2016-12-28 12:33:02 +05:30
Wangda Tan
0840b4329b YARN-5756. Add state-machine implementation for scheduler queues. (Xuan Gong via wangda) 2016-12-27 21:18:33 -08:00
Robert Kanter
9262797e86 YARN-5257. Fix unreleased resources and null dereferences (yufeigu via rkanter) 2016-12-27 14:14:08 -08:00
Robert Kanter
1bbd023275 YARN-5962. Spelling errors in logging and exceptions for resource manager code (gsohn via rkanter) 2016-12-27 13:35:34 -08:00
Daniel Templeton
e216e8e233 HADOOP-13932. Fix indefinite article in comments (Contributed by LiXin Ge via Daniel Templeton) 2016-12-27 13:03:16 -08:00
Arun Suresh
ac1e5d4f77 YARN-5938. Refactoring OpportunisticContainerAllocator to use SchedulerRequestKey instead of Priority and other misc fixes (asuresh) 2016-12-27 12:40:44 -08:00
Karthik Kambatla
c3973e7080 YARN-5969. FairShareComparator: Cache value of getResourceUsage for better performance. (zhangshilong via kasha) 2016-12-27 12:30:35 -08:00
Karthik Kambatla
4d3f73acc0 YARN-6000. Make AllocationFileLoaderService.Listener public. (Tao Jie via kasha) 2016-12-23 11:40:56 -08:00
Akira Ajisaka
ae401539ea YARN-4994. Use MiniYARNCluster with try-with-resources in tests. Contributed by Andras Bokor. 2016-12-22 14:32:24 +09:00
Junping Du
736f54b727 YARN-4990. Re-direction of a particular log file within in a container in NM UI does not redirect properly. Contributed by Xuan Gong. 2016-12-21 14:14:42 -08:00
Sangjin Lee
8b042bc1e6 YARN-5976. Update hbase version to 1.2. Contributed by Vrushali C. 2016-12-21 09:43:17 -08:00
Sunil G
575773a357 YARN-5877. Allow all env's from yarn.nodemanager.env-whitelist to get overridden during launch. Contributed by Bibin A Chundatt. 2016-12-20 11:39:06 +05:30
Sunil G
ef2dd7b78c YARN-5650. Render Application Timeout value in web UI. Contributed by Akhil PB. 2016-12-19 11:44:03 +05:30
Arun Suresh
2273a74c1f YARN-5646. Add documentation and update config parameter names for scheduling of OPPORTUNISTIC containers. (Konstantinos Karanasos via asuresh) 2016-12-16 08:14:34 -08:00
Jian He
ada876cd1d Revert YARN-4126. RM should not issue delegation tokens in unsecure mode. 2016-12-13 20:49:54 -08:00
Robert Kanter
4b149a1e77 YARN-4457. Cleanup unchecked types for EventHandler (templedf via rkanter) 2016-12-09 17:28:25 -08:00
Jian He
a6410a542e YARN-5709. Cleanup leader election configs and pluggability. Contribtued by Karthik Kambatla 2016-12-09 17:00:37 -08:00
Arun Suresh
b0aace21b1 YARN-5982. Simplify opportunistic container parameters and metrics. (Konstantinos Karanasos via asuresh) 2016-12-09 16:41:25 -08:00
Sangjin Lee
55f5886ea2 YARN-5925. Extract hbase-backend-exclusive utility methods from TimelineStorageUtil. Contributed by Haibo Chen. 2016-12-09 16:17:24 -08:00
Sangjin Lee
a5a55a54ab YARN-5922. Remove direct references of HBaseTimelineWriter/Reader in core ATS classes. Contributed by Haibo Chen. 2016-12-08 12:31:12 -08:00
Sunil G
74d0066d33 YARN-5970. Validate application update timeout request parameters. Contributed by Rohith Sharma K S. 2016-12-08 15:53:56 +05:30
Robert Kanter
72fe546841 YARN-5963. Spelling errors in logging and exceptions for node manager, client, web-proxy, common, and app history code (gsohn via rkanter) 2016-12-07 15:07:25 -08:00
Daniel Templeton
9f5d2c4fff YARN-5136. Error in handling event type APP_ATTEMPT_REMOVED to the scheduler 
(Contributed by Wilfred Spiegelenburg via Daniel Templeton)
 2016-12-07 11:12:14 -08:00
Sunil
ab923a53fc YARN-5965. Retrospect ApplicationReport#getApplicationTimeouts. Contributed by Rohith Sharma K S 2016-12-08 00:27:25 +05:30
Rohith Sharma K S
563480dccd YARN-5932. Retrospect moveApplicationToQueue in align with YARN-5611. Contributed by Sunil G. 2016-12-07 10:39:14 +05:30
Naganarasimha
f3b8ff54ab YARN-5921. Incorrect synchronization in RMContextImpl#setHAServiceState/getHAServiceState. Contributed by Varun Saxena 2016-12-06 06:53:38 +05:30
Jian He
43ebff2e35 YARN-5559. Analyse 2.8.0/3.0.0 jdiff reports and fix any issues. Contributed by Akira Ajisaka & Wangda Tan 2016-12-05 11:39:34 -08:00
Jian He
f885160f4a YARN-5746. The state of the parentQueue and its childQueues should be synchronized. Contributed by Xuan Gong 2016-12-02 16:17:31 -08:00
Daniel Templeton
5bd18c49bd YARN-5929. Missing scheduling policy in the FS queue metric. (Contributed by Yufei Gu via Daniel Templeton) 2016-12-02 13:55:42 -08:00
Li Lu
69fb70c31a YARN-5761. Separate QueueManager from Scheduler. (Xuan Gong via gtcarrera9) 2016-11-30 13:38:42 -08:00
Daniel Templeton
b3befc021b YARN-4997. Update fair scheduler to use pluggable auth provider (Contributed by Tao Jie via Daniel Templeton) 2016-11-30 09:50:33 -08:00
Kai Zheng
cfd8076f81 Revert due to an error "HDFS-10994. Support an XOR policy XOR-2-1-64k in HDFS. Contributed by Sammi Chen" 
This reverts commit 5614f847b2ef2a5b70bd9a06edc4eba06174c6.
 2016-11-30 15:44:52 +08:00
Daniel Templeton
dd9a96cb17 YARN-5851. TestContainerManagerSecurity testContainerManager[1] failed (Contributed by Haibo Chen via Daniel Templeton) 2016-11-29 14:22:19 -08:00
Daniel Templeton
3b9d3acd20 YARN-5890. FairScheduler should log information about AM-resource-usage and max-AM-share for queues 
(Contributed by Yufei Gu via Daniel Templeton)
 2016-11-29 12:46:05 -08:00
Daniel Templeton
25f9872be6 YARN-5774. MR Job stuck in ACCEPTED status without any progress in Fair Scheduler 
if set yarn.scheduler.minimum-allocation-mb to 0. (Contributed by Yufei Gu via Daniel Templeton)
 2016-11-29 09:40:49 -08:00
Daniel Templeton
62b42ef5dd YARN-5725. Test uncaught exception in TestContainersMonitorResourceChange.testContainersResourceChange 
when setting IP and host (Contributed by Miklos Szegedi via Daniel Templeton)
 2016-11-28 15:19:52 -08:00
Kai Zheng
5d5614f847 HDFS-10994. Support an XOR policy XOR-2-1-64k in HDFS. Contributed by Sammi Chen 2016-11-28 14:34:44 +08:00
Akira Ajisaka
209e805430 HADOOP-13506. Redundant groupid warning in child projects. Contributed by Kai Sasaki. 2016-11-28 14:34:57 +09:00
Naganarasimha
c06114d6a3 YARN-5572. HBaseTimelineWriterImpl appears to reference a bad property name. Contributed by Varun Saxena. 2016-11-27 23:35:53 +05:30
Naganarasimha
e2f77ca77b Revert "YARN-5921. Incorrect synchronization in RMContextImpl#setHAServiceState/getHAServiceState. Contributed by Varun Saxena." 
This reverts commit 9f2960be19.
 2016-11-27 23:32:59 +05:30
Naganarasimha
9f2960be19 YARN-5921. Incorrect synchronization in RMContextImpl#setHAServiceState/getHAServiceState. Contributed by Varun Saxena. 2016-11-27 23:31:27 +05:30
Sunil
eb0a483ed0 YARN-4206. Add Application timeouts in Application report and CLI. Contributed by Rohith Sharma K S. 2016-11-24 18:18:42 +05:30
Rohith Sharma K S
e15c20edba YARN-5920. Fix deadlock in TestRMHA.testTransitionedToStandbyShouldNotHang. Contributed by Varun Saxena. 2016-11-24 12:18:38 +05:30
Daniel Templeton
10468529a9 YARN-4752. Improved preemption in FairScheduler. (kasha) 
Contains:
YARN-5605. Preempt containers (all on one node) to meet the requirement of starved applications
YARN-5821. Drop left-over preemption-related code and clean up method visibilities in the Schedulable hierarchy
YARN-5783. Verify identification of starved applications.
YARN-5819. Verify fairshare and minshare preemption
YARN-5885. Cleanup YARN-4752 branch for merge

Change-Id: Iee0962377d019dd64dc69a020725d2eaf360858c
 2016-11-23 19:48:59 -10:00
Jian He
1f12867a69 YARN-5649. Add REST endpoints for updating application timeouts. Contributed by Rohith Sharma K S 2016-11-23 16:25:39 -08:00
Arun Suresh
005850b28f YARN-5918. Handle Opportunistic scheduling allocate request failure when NM is lost. (Bibin A Chundatt via asuresh) 2016-11-23 09:53:31 -08:00
Naganarasimha
3541ed8068 YARN-4330. MiniYARNCluster is showing multiple Failed to instantiate default resource calculator warning messages. Contributed by Varun Saxena 2016-11-23 14:12:23 +05:30
Daniel Templeton
83cc7263af YARN-5722. FairScheduler hides group resolution exceptions when assigning queue (Contributed by Wilfred Spiegelenburg via Daniel Templeton) 2016-11-22 11:16:00 -08:00
Rohith Sharma K S
a926f895c1 YARN-5865. Retrospect updateApplicationPriority api to handle state store exception in align with YARN-5611. Contributed by Sunil G. 2016-11-22 14:49:15 +05:30
Akira Ajisaka
6f8074298d YARN-5713. Update jackson from 1.9.13 to 2.x in hadoop-yarn. 2016-11-22 10:20:31 +09:00
Jason Lowe
009452bb6d YARN-5859. TestResourceLocalizationService#testParallelDownloadAttemptsForPublicResource sometimes fails. Contributed by Eric Badger 2016-11-21 16:36:08 +00:00
Junping Du
f05a9ceb4a YARN-3538. TimelineWebService doesn't catch runtime exception. Contributed by Steve Loughran and Tsuyoshi Ozawa. 2016-11-17 16:55:39 -08:00
Jason Lowe
59bfcbf357 YARN-5836. Malicious AM can kill containers of other apps running in any node its containers are running. Contributed by Botong Huang 2016-11-16 22:21:03 +00:00
Rohith Sharma K S
d65603517e YARN-5375. invoke MockRM#drainEvents implicitly in MockRM methods to reduce test failures. Contributed by sandflee. 2016-11-16 15:14:00 +05:30
Xiao Chen
f121d0b036 YARN-5875. TestTokenClientRMService#testTokenRenewalWrongUser fails. Contributed by Gergely Novák. 2016-11-15 13:58:11 -08:00
Daniel Templeton
264ddb13ff YARN-5736 Addendum. Fixes segfault due to unterminated string. (Contributed by Miklos Szegedi via Daniel Templeton) 2016-11-15 13:01:57 -08:00
Mingliang Liu
5af572b644 HADOOP-13427. Eliminate needless uses of FileSystem#{exists(), isFile(), isDirectory()}. Contributed by Steve Loughran and Mingliang Liu 2016-11-15 10:57:00 -08:00
Arun Suresh
3219b7b4ac YARN-4597. Introduce ContainerScheduler and a SCHEDULED state to NodeManager container lifecycle. (asuresh) 2016-11-15 07:56:25 -08:00
Naganarasimha
7ffb9943b8 YARN-4355. NPE while processing localizer heartbeat. Contributed by Varun Saxena & Jonathan Hung. 2016-11-15 15:41:56 +05:30
Naganarasimha
43aef303bf Reverted due to issue YARN-5765. Revert "YARN-5287. LinuxContainerExecutor fails to set proper permission. Contributed by Ying Zhang" 
This reverts commit 131d58a24e.

server-nodemanager/src/main/native/container-executor/test/test-
container-executor.c
 2016-11-15 13:27:37 +05:30
Rohith Sharma K S
b7070f3308 YARN-5874. RM -format-state-store and -remove-application-from-state-store commands fail with NPE. Contributed by Varun Saxena. 2016-11-15 10:58:25 +05:30
Rohith Sharma K S
296c5de0cf YARN-5873. RM crashes with NPE if generic application history is enabled. Contributed by Varun Saxena. 2016-11-15 10:56:28 +05:30
Karthik Kambatla
79448d4ab1 YARN-4033. In FairScheduler, parent queues should also display queue status. (Siqi Li via kasha) 2016-11-13 21:38:36 -08:00
Jian He
fad9609d13 YARN-5825. ProportionalPreemptionalPolicy should use readLock over LeafQueue instead of synchronized block. Contributed by Sunil G 2016-11-11 15:16:21 -08:00
Wangda Tan
ede1a473f5 YARN-5552. Add Builder methods for common yarn API records. (Tao Jie via wangda) 2016-11-11 13:34:56 -08:00
Naganarasimha
503e73e849 YARN-5545. Fix issues related to Max App in capacity scheduler. Contributed by Bibin A Chundatt 2016-11-11 20:48:31 +05:30
Eric Payne
93eeb13164 YARN-4218. Metric for resource*time that was preempted. Contributed by Chang Li. 2016-11-10 22:35:12 +00:00
Karthik Kambatla
3a98419532 YARN-5834. TestNodeStatusUpdater.testNMRMConnectionConf compares nodemanager wait time to the incorrect value. (Chang Li via kasha) 2016-11-10 14:08:51 -08:00
Karthik Kambatla
86ac1ad9fd YARN-5453. FairScheduler#update may skip update demand resource of child queue/app if current demand reached maxResource. (sandflee via kasha) 2016-11-09 23:44:02 -08:00
Varun Saxena
c202a10923 YARN-5862. TestDiskFailures.testLocalDirsFailures failed (Yufei Gu via Varun Saxena) 2016-11-10 11:41:34 +05:30
Naganarasimha
de3a5f8d08 YARN-5856. Unnecessary duplicate start container request sent to NM State store. Contributed by Varun Saxena. 2016-11-10 05:42:30 +05:30
Jian He
bcc15c6290 YARN-5611. Provide an API to update lifetime of an application. Contributed by Rohith Sharma K S 2016-11-09 16:08:05 -08:00
Naganarasimha
edbee9e609 YARN-4498. Application level node labels stats to be available in REST (addendum patch). Contributed by Bibin A Chundatt. 2016-11-10 05:00:05 +05:30
Daniel Templeton
59ee8b7a88 YARN-4329. [YARN-5437] Allow fetching exact reason as to why a submitted app 
is in ACCEPTED state in Fair Scheduler (Contributed by Yufei Gu)
 2016-11-09 13:11:37 -08:00
Robert Kanter
09f43fa9c0 YARN-5736. YARN container executor config does not handle white space (miklos.szegedi@cloudera.com via rkanter) 2016-11-09 13:34:40 +01:00
Arun Suresh
283fa33feb YARN-5823. Update NMTokens in case of requests with only opportunistic containers. (Konstantinos Karanasos via asuresh) 2016-11-09 00:11:25 -08:00
Andrew Wang
e1c6ef2efa HADOOP-13789. Hadoop Common includes generated test protos in both jar and test-jar. Contributed by Sean Busbey. 2016-11-08 16:33:55 -08:00
Jason Lowe
3f93ac0733 YARN-5356. NodeManager should communicate physical resource capability to ResourceManager. Contributed by Inigo Goiri 2016-11-08 22:01:26 +00:00
Arun Suresh
f38a6d03a1 YARN-5377. Fix TestQueuingContainerManager.testKillMultipleOpportunisticContainers. (Konstantinos Karanasos via asuresh) 2016-11-07 22:10:03 -08:00
Jian He
de3b4aac56 YARN-5716. Add global scheduler interface definition and update CapacityScheduler to use it. Contributed by Wangda Tan 2016-11-07 10:14:39 -08:00
Wangda Tan
c00b5d1e51 YARN-5804. New UI2 is not able to launch with jetty 9 upgrade post HADOOP-10075. (Sunil G via wangda) 
(cherry picked from commit ddebaa6b7ae1031899139d153d57df5387d60542)
 2016-11-06 13:13:31 -08:00
Wangda Tan
3de0da2a76 YARN-5698. [YARN-3368] Launch new YARN UI under hadoop web app port. (Sunil G via wangda) 2016-11-06 13:13:31 -08:00
Wangda Tan
817d5f4c96 YARN-4849. Addendum patch to fix javadocs. (Sunil G via wangda) 2016-11-06 13:13:31 -08:00
Wangda Tan
c85cc3b56e YARN-4515. [YARN-3368] Support hosting web UI framework inside YARN RM. (Sunil G via wangda) 
YARN-5000. [YARN-3368] App attempt page is not loading when timeline server is not started (Sunil G via wangda)
YARN-5038. [YARN-3368] Application and Container pages shows wrong values when RM is stopped. (Sunil G via wangda)
 2016-11-06 13:13:31 -08:00
Jason Lowe
6bb741ff0e YARN-5837. NPE when getting node status of a decommissioned node after an RM restart. Contributed by Robert Kanter 2016-11-04 22:20:21 +00:00
Arun Suresh
0aafc122d4 YARN-2995. Enhance UI to show cluster resource utilization of various container Execution types. (Konstantinos Karanasos via asuresh) 2016-11-04 07:31:54 -07:00
Sunil
19b3779ae7 YARN-5802. updateApplicationPriority api in scheduler should ensure to re-insert app to correct ordering policy. Contributred by Bibin A Chundatt 2016-11-04 16:07:28 +05:30
Varun Vasudev
9ee0e3172e YARN-5822. Log ContainerRuntime initialization error in LinuxContainerExecutor. Contributed by Sidharta Seethana. 2016-11-03 22:33:43 +05:30
Jason Lowe
352cbaa7a5 YARN-4862. Handle duplicate completed containers in RMNodeImpl. Contributed by Rohith Sharma K S 2016-11-03 13:54:31 +00:00
Varun Saxena
377919010b YARN-5815. Random failure of TestApplicationPriority.testOrderOfActivatingThePriorityApplicationOnRMRestart (Bibin A Chundatt via Varun Saxena) 2016-11-03 00:37:09 +05:30
Jason Lowe
76893a4100 YARN-5001. Aggregated Logs root directory is created with wrong group if nonexistent. Contributed by Haibo Chen 2016-11-01 20:23:24 +00:00
Rohith Sharma K S
9d0c2ad5a5 YARN-3732. Change NodeHeartbeatResponse.java and RegisterNodeManagerResponse.java as abstract classes. Contributed by Devaraj K. 2016-11-01 16:25:45 +05:30
Varun Saxena
7d2d8d25ba YARN-5788. Apps not activiated and AM limit resource in UI and REST not updated after -replaceLabelsOnNode (Bibin A Chundatt via Varun Saxena) 2016-11-01 15:32:04 +05:30
Varun Saxena
310aa4686e YARN-5805. Add isDebugEnabled check for debug logs in nodemanager (Bibin A Chundatt via Varun Saxena) 2016-11-01 14:51:31 +05:30
Karthik Kambatla
733aa99313 YARN-4998. Minor cleanup to UGI use in AdminService. (Daniel Templeton via kasha) 2016-10-31 16:26:01 -07:00
Wangda Tan
90dd3a8148 YARN-2009. CapacityScheduler: Add intra-queue preemption for app priority support. (Sunil G via wangda) 2016-10-31 15:18:31 -07:00
Daniel Templeton
f3eb4c3c73 YARN-5793. Trim configuration values in DockerLinuxContainerRuntime (Contributed by Tianyin Xu via Daniel Templeton) 2016-10-31 13:51:07 -07:00
Daniel Templeton
2528bea67f YARN-4396. Log the trace information on FSAppAttempt#assignContainer (Contributed by Yiqun Li via Daniel Templeton) 2016-10-31 13:34:25 -07:00
Daniel Templeton
cc2c993a8a YARN-4907. Make all MockRM#waitForState consistent. (Contributed by Yufei Gu via Daniel Templeton) 2016-10-31 13:20:56 -07:00
Naganarasimha
e0bebbbcdd YARN-4498. Application level node labels stats to be available in REST. Contributed by Bibin A Chundatt 2016-10-31 04:38:20 +05:30
Arun Suresh
aa3cab1eb2 YARN-5799. Fix Opportunistic Allocation to set the correct value of Node Http Address. (asuresh) 2016-10-29 02:03:57 -07:00
Varun Saxena
1c8ab41e8b YARN-5773. RM recovery too slow due to LeafQueue#activateApplications (Bibin A Chundatt via Varun Saxena) 2016-10-29 13:47:39 +05:30
Jason Lowe
1eae719bce YARN-4963. capacity scheduler: Make number of OFF_SWITCH assignments per heartbeat configurable. Contributed by Nathan Roberts 2016-10-28 17:30:15 +00:00
Jason Lowe
1b79c417dc YARN-5767. Fix the order that resources are cleaned up from the local Public/Private caches. Contributed by Chris Trezzo 2016-10-28 15:58:04 +00:00
Jason Lowe
7146359bfd YARN-5027. NM should clean up app log dirs after NM restart. Contributed by sandflee 2016-10-28 15:48:58 +00:00
Jason Lowe
c017171da0 HADOOP-13770. Shell.checkIsBashSupported swallowed an interrupted exception. Contributed by Wei-Chiu Chuang 2016-10-28 14:57:50 +00:00
Karthik Kambatla
4df8ed63ed YARN-4743. FairSharePolicy breaks TimSort assumption. (Zephyr Guo and Yufei Gu via kasha) 2016-10-27 17:45:48 -07:00
Subru Krishnan
b2c4f24c31 YARN-2306. Add test for leakage of reservation metrics in fair scheduler. (Hong Zhiguo and Yufei Gu via subru). 2016-10-27 17:43:13 -07:00
Robert Kanter
5877f20f9c HADOOP-10075. Update jetty dependency to version 9 (rkanter) 2016-10-27 16:09:00 -07:00
Vrushali Channapattan
022bf783aa YARN-4765 Split TestHBaseTimelineStorage into multiple test classes (Varun Saxena via Vrushali C) 
(cherry picked from commit 513dcf6817dd76fde8096ff04cd888d7c908461d)
 2016-10-27 15:37:36 -07:00
Carlo Curino
892a8348fc YARN-3432. Cluster metrics have wrong Total Memory when there is reserved memory on CS. (Brahma Reddy Battula via curino) 2016-10-27 15:15:49 -07:00
Subru Krishnan
79ae78dcbe YARN-3568. TestAMRMTokens should use some random port. (Takashi Ohnishi via Subru). 2016-10-27 15:11:12 -07:00
Daniel Templeton
b98fc8249f YARN-4710. Reduce logging application reserved debug info in FSAppAttempt#assignContainer (Contributed by Yiqun Lin via Daniel Templeton) 2016-10-27 14:42:19 -07:00
Robert Kanter
9449519a25 YARN-5776. Checkstyle: MonitoringThread.Run method length is too long (miklos.szegedi@cloudera.com via rkanter) 2016-10-27 14:36:38 -07:00
Jason Lowe
7e3c327d31 YARN-4831. Recovered containers will be killed after NM stateful restart. Contributed by Siqi Li 2016-10-27 20:42:52 +00:00
Robert Kanter
6fbfb501f2 YARN-4456. Clean up Lint warnings in nodemanager (templedf via rkanter) 2016-10-27 12:37:01 -07:00
Varun Saxena
79aeddc88f YARN-5308. FairScheduler: Move continuous scheduling related tests to TestContinuousScheduling (Kai Sasaki via Varun Saxena) 2016-10-28 00:34:50 +05:30
Naganarasimha
b3c15e4ef7 YARN-5420. Delete org.apache.hadoop.yarn.server.resourcemanager.resource.Priority as its not necessary. Contributed by Sunil G. 2016-10-27 18:22:07 +05:30
Naganarasimha
6c8830992c YARN-3848. TestNodeLabelContainerAllocation is timing out. Contributed by Varun Saxena 2016-10-27 17:10:02 +05:30
Varun Saxena
7b4e9ec3b0 YARN-5686. DefaultContainerExecutor random working dir algorigthm skews results (Vrushali C via Varun Saxena) 2016-10-27 14:02:29 +05:30
Varun Saxena
5c39a419a6 YARN-5752. TestLocalResourcesTrackerImpl#testLocalResourceCache times out (Eric Badger via Varun Saxena) 2016-10-27 13:41:49 +05:30
Rohith Sharma K S
e29cba61a0 YARN-4363. In TestFairScheduler, testcase should not create FairScheduler redundantly. Conntributed by Tao Jie. 2016-10-27 11:57:17 +05:30
Rohith Sharma K S
b110c4b5e8 YARN-4555. TestDefaultContainerExecutor#testContainerLaunchError fails on non-english locale environment. Contributed by Takashi Ohnishi. 2016-10-27 11:34:07 +05:30
Sangjin Lee
f511cc89b6 YARN-5433. Audit dependencies for Category-X. Contributed by Sangjin Lee. 2016-10-26 11:31:00 -07:00
Akira Ajisaka
d3bb69a667 YARN-5575. Many classes use bare yarn. properties instead of the defined constants. Contributed by Daniel Templeton. 2016-10-26 15:32:07 +09:00
Karthik Kambatla
de6faae97c YARN-5388. Deprecate and remove DockerContainerExecutor. (Daniel Templeton via kasha) 2016-10-25 13:35:47 -07:00
Karthik Kambatla
a71fc81655 YARN-5754. Null check missing for earliest in FifoPolicy. (Yufei Gu via kasha) 2016-10-24 22:19:23 -07:00
Varun Saxena
44eb2bd7ae YARN-5747. Application timeline metric aggregation in timeline v2 will lose last round aggregation when an application finishes (Li Lu via Varun Saxena) 2016-10-22 01:14:49 +05:30
Karthik Kambatla
754cb4e30f YARN-5047. Refactor nodeUpdate across schedulers. (Ray Chiang via kasha) 2016-10-20 21:17:48 -07:00
Karthik Kambatla
a064865abf YARN-4911. Bad placement policy in FairScheduler causes the RM to crash 2016-10-20 20:57:04 -07:00
Sangjin Lee
e9c4616b5e YARN-5561. [Atsv2] : Support for ability to retrieve apps/app-attempt/containers and entities via REST. Contributed by Rohith Sharma K S. 2016-10-19 09:45:23 -07:00
Xuan
b733a6f862 YARN-5718. TimelineClient (and other places in YARN) shouldn't over-write HDFS client retry settings which could cause unexpected behavior. Contributed by Junping Du. 2016-10-18 11:06:47 -07:00
Varun Saxena
b154d3edce YARN-5743. [Atsv2] Publish queue name and RMAppMetrics to ATS (Rohith Sharma K S via Varun Saxena) 2016-10-18 23:32:52 +05:30
Robert Kanter
f5d9235914 YARN-5466. DefaultContainerExecutor needs JavaDocs (templedf via rkanter) 2016-10-17 14:29:09 -07:00
Sangjin Lee
1f304b0c7f YARN-5699. Retrospect yarn entity fields which are publishing in events info fields. Contributed by Rohith Sharma K S. 2016-10-15 13:54:40 -07:00
Chris Douglas
cf3f43e95b YARN-5717. Add tests for container-executor is_feature_enabled. Contributed by Sidharta Seethana 2016-10-13 20:49:07 -07:00
Karthik Kambatla
6476934ae5 YARN-5677. RM should transition to standby when connection is lost for an extended period. (Daniel Templeton via kasha) 2016-10-11 22:07:10 -07:00
Karthik Kambatla
6378845f9e YARN-4464. Lower the default max applications stored in the RM and store. (Daniel Templeton via kasha) 2016-10-11 21:42:08 -07:00
Naganarasimha
0773ffd0f8 YARN-5057. Resourcemanager.security.TestDelegationTokenRenewer fails in trunk. Contributed by Jason Lowe. 2016-10-10 18:04:47 -04:00
Rohith Sharma K S
4d2f380d78 YARN-5101. YARN_APPLICATION_UPDATED event is parsed in ApplicationHistoryManagerOnTimelineStore#convertToApplicationReport with reversed order. Contributed by Sunil G. 2016-10-06 20:42:36 +05:30
Akira Ajisaka
272a21747e MAPREDUCE-6789. Fix TestAMWebApp failure. Contributed by Daniel Templeton. 2016-10-06 15:57:15 +09:00
Varun Vasudev
0992708d79 YARN-5704. Provide config knobs to control enabling/disabling new/work in progress features in container-executor. Contributed by Sidharta Seethana. 2016-10-05 15:31:38 +05:30
Jian He
31f8da22d0 YARN-3139. Improve locks in AbstractYarnScheduler/CapacityScheduler/FairScheduler. Contributed by Wangda Tan 2016-10-04 17:23:13 -07:00
Karthik Kambatla
736d33cddd YARN-4767. Network issues can cause persistent RM UI outage. (Daniel Templeton via kasha) 2016-10-03 14:35:57 -07:00
Karthik Kambatla
0da54e8848 YARN-5672. FairScheduler: Wrong queue name in log when adding application. (Wilfred Spiegelenburg via kasha) 2016-10-03 06:03:46 -07:00
Naganarasimha
6e130c308c YARN-4855. Should check if node exists when replace nodelabels. Contributeed by Tao Jie 2016-10-03 02:02:26 -04:00
Karthik Kambatla
82857037b6 YARN-5678. Log demand as demand in FSLeafQueue and FSParentQueue. (Yufei Gu via kasha) 2016-10-02 22:09:43 -07:00
Subru Krishnan
3a3697deab YARN-5384. Expose priority in ReservationSystem submission APIs. (Sean Po via Subru). 2016-09-30 19:41:43 -07:00
Arun Suresh
10be45986c YARN-5486. Update OpportunisticContainerAllocatorAMService::allocate method to handle OPPORTUNISTIC container requests. (Konstantinos Karanasos via asuresh) 2016-09-29 15:11:41 -07:00
Jian He
2ae5a3a5bf YARN-4205. Add a service for monitoring application life time out. Contributed by Rohith Sharma K S 2016-09-29 22:00:31 +08:00
Robert Kanter
bcb2528a51 YARN-5400. Light cleanup in ZKRMStateStore (templedf via rkanter) 2016-09-28 14:56:41 -07:00
Varun Saxena
9b0fd01d2e YARN-5599. Publish AM launch command to ATS (Rohith Sharma K S via Varun Saxena) 2016-09-28 16:10:10 +05:30
Varun Vasudev
bc2656f09f YARN-5662. Provide an option to enable ContainerMonitor. Contributed by Jian He. 2016-09-28 15:18:18 +05:30
Wangda Tan
1831be8e73 YARN-3142. Improve locks in AppSchedulingInfo. (Varun Saxena via wangda) 2016-09-27 11:54:55 -07:00
Naganarasimha
ebf528c443 YARN-5660. Wrong audit constants are used in Get/Put of priority in RMWebService. Contributed by Rohith Sharma K S 2016-09-27 16:37:59 +05:30
Arun Suresh
4815d024c5 YARN-5609. Expose upgrade and restart API in ContainerManagementProtocol. Contributed by Arun Suresh 2016-09-26 08:46:54 -07:00
Arun Suresh
2f163cd5cf Revert "YARN-5609. Expose upgrade and restart API in ContainerManagementProtocol. Contributed by Arun Suresh" 
This reverts commit fe644bafe7.
 2016-09-26 08:36:59 -07:00