big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
15,946 Commits 2 Branches 2 Tags 264 MiB
6672810eea
Commit Graph

3560 Commits

Author SHA1 Message Date
Eric Payne
52adf71914 YARN-5617. AMs only intended to run one attempt can be run more than once? Contributed by Jason Lowe. 2017-04-24 12:03:28 -05:00
Sunil G
561718e05d YARN-6291. Introduce query parameters (sort, filter, etc.) for tables to keep state on refresh/navigation in new YARN UI. Contributed by Gergely Novák. 2017-04-24 13:28:57 +05:30
Daniel Templeton
f356f0f4cf YARN-6453. fairscheduler-statedump.log gets generated regardless of service 
(Contributed by Yufei Gu via Daniel Templeton)
 2017-04-20 14:13:07 -07:00
Sunil G
c0cf11e107 YARN-6402. Move 'Long Running Services' to an independent tab at top level for new Yarn UI. Contributed by Akhil PB. 2017-04-20 12:33:53 +05:30
Daniel Templeton
46940d92e2 YARN-6302. Fail the node if Linux Container Executor is not configured properly 
(Contributed by Miklos Szegedi via Daniel Templeton)
 2017-04-19 12:23:49 -07:00
Daniel Templeton
dd43b895c2 YARN-6202. Configuration item Dispatcher.DISPATCHER_EXIT_ON_ERROR_KEY is disregarded 
(Contributed by Yufei Gu via Daniel Templeton)
 2017-04-19 11:44:55 -07:00
Daniel Templeton
41ac190dba YARN-6438. Code can be improved in ContainersMonitorImpl.java 
(Contributed by Miklos Szegedi via Daniel Templeton)
 2017-04-19 10:53:08 -07:00
Daniel Templeton
15acdbdba2 YARN-3427. Remove deprecated methods from ResourceCalculatorProcessTree 
(Contributed by Miklos Szegedi via Daniel Templeton)
 2017-04-19 10:31:14 -07:00
Sunil G
9ff25408bc YARN-6164. Expose Queue Configurations per Node Label through YARN client api. Contributed by Benson Qiu. 2017-04-19 20:39:09 +05:30
Chris Douglas
af8e9842d2 YARN-6451. Add RM monitor validating metrics invariants. Contributed by Carlo Curino 2017-04-18 10:29:44 -07:00
Junping Du
6ed9d36224 YARN-6304. Skip rm.transitionToActive call to RM if RM is already active. Contributed by Rohith Sharma K S. 2017-04-17 15:51:10 -07:00
Jason Lowe
4168805502 YARN-6480. Timeout is too aggressive for TestAMRestart.testPreemptedAMRestartOnRMRestart. Contributed by Eric Badger 2017-04-14 17:15:48 -05:00
Karthik Kambatla
8a1d7480f7 YARN-6433. Only accessible cgroup mount directories should be selected for a controller. (Miklos Szegedi via kasha) 2017-04-14 15:07:14 -07:00
Eric Payne
a41f8dd58e YARN-5994. TestCapacityScheduler.testAMLimitUsage fails intermittently. Contributed by Eric Badger 2017-04-14 10:53:09 -05:00
Karthik Kambatla
c3375175d6 YARN-6432. FairScheduler: Reserve preempted resources for corresponding applications. (Miklos Szegedi via kasha) 2017-04-12 14:21:20 -07:00
Junping Du
9d9087a67a YARN-3760. FSDataOutputStream leak in AggregatedLogFormat.LogWriter.close(). Contributed by Haibo Chen. 
(cherry picked from commit c26ccf1adb3a72df3f68e1150b86b813c691203a)
 2017-04-12 13:43:18 -07:00
Eric Payne
a16ab2be91 YARN-6450. TestContainerManagerWithLCE requires override for each new test added to ContainerManagerTest? Contributed by Jason Lowe. 2017-04-12 10:20:08 -05:00
Sunil G
f48e968bec YARN-6421. Upgrade frontend-maven-plugin to 1.1 to fix new YARN UI build error in ppc64le. Contributed by Sonia Garudi. 2017-04-12 12:01:47 +05:30
Wangda Tan
4d4ad0ebb7 YARN-6439. Fix ReservationSystem creation of default ReservationQueue. (Carlo Curino via wangda) 2017-04-11 14:56:18 -07:00
Naganarasimha
7d873c465b YARN-6463. Correct spelling mistake in FileSystemRMStateStore. Contributed by Yeliang Cang. 2017-04-11 22:42:08 +05:30
Jason Lowe
90d97372ed YARN-6461. TestRMAdminCLI has very low test timeouts. Contributed by Eric Badger 2017-04-11 09:16:11 -05:00
Jason Lowe
0e065f2ede YARN-6195. Export UsedCapacity and AbsoluteUsedCapacity to JMX. Contributed by Benson Qiu 2017-04-11 08:44:18 -05:00
Konstantinos Karanasos
7999318af1 YARN-6344. Add parameter for rack locality delay in CapacityScheduler. (kkaranasos) 2017-04-10 15:34:44 -07:00
Daniel Templeton
e9ac61cc0e YARN-6372. Add default value for NM disk validator (Contributed by Yufei Gu via Daniel Templeton) 2017-04-10 14:56:42 -07:00
Daniel Templeton
14a39907c6 YARN-6343. Docker docs MR example is broken (Contributed by Prashant Jha via Daniel Templeton) 2017-04-10 09:59:53 -07:00
Sunil G
443aa51bc1 YARN-5153. Add a toggle button to switch between timeline view / table view for containers and application-attempts in new YARN UI. Contributed by Akhil PB. 2017-04-10 13:35:08 +05:30
Karthik Kambatla
2aa8967809 YARN-6298. Metric preemptCall is not used in new preemption. (Yufei Gu via kasha) 2017-04-07 17:31:59 -07:00
Robert Kanter
63f7322522 YARN-6368. Decommissioning an NM results in a -1 exit code (miklos.szegedi@cloudera.com via rkanter) 2017-04-07 14:28:07 -07:00
Allen Wittenauer
96cbb4fce2 HADOOP-14202. fix jsvc/secure user var inconsistencies 
Signed-off-by: John Zhuge <jzhuge@apache.org>
 2017-04-07 08:59:21 -07:00
Sunil G
ad24464be8 YARN-6258. localBaseAddress for CORS proxy configuration is not working when suffixed with forward slash in new YARN UI. Contributed by Gergely Novák. 2017-04-07 11:42:50 +05:30
Ming Ma
0116c3c957 YARN-5797. Add metrics to the node manager for cleaning the PUBLIC and PRIVATE caches. (Chris Trezzo via mingma) 2017-04-06 16:54:43 -07:00
Jason Lowe
1b081ca27e YARN-6288. Exceptions during aggregated log writes are mishandled. Contributed by Akira Ajisaka 2017-04-06 16:24:36 -05:00
Rohith Sharma K S
1a9439e299 YARN-6424. TimelineCollector is not stopped when an app finishes in RM. Contributed by Varun Saxena. 2017-04-06 10:15:22 +05:30
Daniel Templeton
a2c57bb70d YARN-6381. FSAppAttempt has several variables that should be final 
(Contributed by Ameet Zaveri via Daniel Templeton)
 2017-04-05 16:07:24 -07:00
Andrew Wang
539ef5aa2e HFDS-11596. hadoop-hdfs-client jar is in the wrong directory in release tarball. Contributed by Yuanbo Liu. 2017-04-05 16:04:09 -07:00
Karthik Kambatla
b4c4f36594 YARN-6448. Continuous scheduling thread crashes while sorting nodes. (Yufei Gu via kasha) 2017-04-05 15:43:14 -07:00
Varun Saxena
7d963c477a YARN-6437. TestSignalContainer#testSignalRequestDeliveryToNM fails intermittently (Jason Lowe via Varun Saxena) 2017-04-06 01:20:41 +05:30
Wangda Tan
87e2ef8c98 YARN-6406. Remove SchedulerRequestKeys when no more pending ResourceRequest. (Arun Suresh via wangda) 2017-04-05 11:28:30 -07:00
Jason Lowe
e8071aa249 YARN-6403. Invalid local resource request can raise NPE and make NM exit. Contributed by Tao Yang 2017-04-05 12:30:45 -05:00
Akira Ajisaka
34ab8e73d4
YARN-5007. Remove deprecated constructors of MiniYARNCluster and MiniMRYarnCluster. Contributed by Andras Bokor. 2017-04-05 17:33:07 +09:00
Varun Saxena
19b89c4c7b YARN-6420. RM startup failure due to wrong order in nodelabel editlog (Bibin A Chundatt via Varun Saxena) 2017-04-05 09:38:35 +05:30
Ming Ma
2d5c09b848 YARN-6004. Refactor TestResourceLocalizationService#testDownloadingResourcesOnContainer so that it is less than 150 lines. (Chris Trezzo via mingma) 2017-04-04 17:56:21 -07:00
Karthik Kambatla
9cc04b4708 YARN-6436. TestSchedulingPolicy#testParseSchedulingPolicy timeout is too low. (Eric Badger via kasha) 2017-04-04 17:08:33 -07:00
Wangda Tan
3fdae0a2b6 YARN-6109. Add an ability to convert ChildQueue to ParentQueue. (Xuan Gong via wangda) 2017-04-04 14:39:01 -07:00
Chris Douglas
6eba79232f HADOOP-14271. Correct spelling of 'occurred' and variants. Contributed by Yeliang Cang 2017-04-03 20:13:14 -07:00
Varun Saxena
a4b5aa8493 YARN-6377. NMTimelinePublisher#serviceStop does not stop timeline clients (Haibo Chen via Varun Saxena) 2017-04-02 04:54:12 +05:30
Varun Saxena
b8838578ce YARN-6414. ATSv2 HBase related tests fail due to guava version upgrade (Haibo Chen via Varun Saxena) 2017-04-02 04:37:34 +05:30
Eric Payne
318bfb01bc YARN-6354. LeveldbRMStateStore can parse invalid keys when recovering reservations. Contributed by Jason Lowe 2017-03-31 12:30:35 -05:00
Jason Lowe
4d1fac5df2 YARN-6411. Clean up the overwrite of createDispatcher() in subclass of MockRM. Contributed by Yufei Gu 2017-03-31 10:05:34 -05:00
Varun Saxena
b58777a9c9 YARN-6376. Exceptions caused by synchronous putEntities requests can be swallowed (Haibo Chen via Varun Saxena) 2017-03-31 02:17:20 +05:30
Varun Saxena
7c2bc444b3 YARN-6342. Make TimelineV2Client's drain timeout after stop configurable (Haibo Chen via Varun Saxena) 2017-03-31 02:02:57 +05:30
Chris Douglas
7297e82cb4 HADOOP-14250. Correct spelling of 'separate' and variants. Contributed by Doris Gu 2017-03-30 10:17:11 -07:00
Daniel Templeton
640ba1d23f YARN-5685. RM configuration allows all failover methods to disabled when automatic failover is enabled 2017-03-29 12:38:11 -07:00
Akira Ajisaka
82fb9ce8df
YARN-6329. Remove unnecessary TODO comment from AppLogAggregatorImpl.java. Contributed by victor bertschinger. 2017-03-29 10:36:24 +09:00
Varun Saxena
063b513b1c YARN-6357. Implement putEntitiesAsync API in TimelineCollector (Haibo Chen via Varun Saxena) 2017-03-29 03:48:03 +05:30
Varun Saxena
01aca54a22 YARN-5368. Memory leak in timeline server (Jonathan Eagles via Varun Saxena) 2017-03-29 01:53:20 +05:30
Jason Lowe
fdf8f8ebca YARN-6359. TestRM#testApplicationKillAtAcceptedState fails rarely due to race condition. Contributed by Robert Kanter 2017-03-28 09:33:26 -05:00
Robert Kanter
9bae6720cb YARN-6050. AMs can't be scheduled on racks or nodes (rkanter) 2017-03-27 17:23:13 -07:00
Wangda Tan
cd014d57aa YARN-6339. Improve performance for createAndGetApplicationReport. (Yunjiong Zhao via wangda) 2017-03-27 13:29:09 -07:00
Akira Ajisaka
b63ae854f9
YARN-6379. Remove unused argument in ClientRMService. Contributed by Kai Sasaki. 2017-03-27 11:23:41 +09:00
Junping Du
6df029db36 Add COMMON/MAPREDUCE/YARN jdiff of 2.8.0 release. 2017-03-25 14:28:48 -07:00
Sunil G
cffea251be YARN-5956. Refactor ClientRMService for unify error handling across apis. Contributed by Kai Sasaki. 2017-03-25 22:26:19 +05:30
Tsuyoshi Ozawa
84ddedc0b2 HADOOP-10101. Update guava dependency to the latest version. (ozawa) 2017-03-25 10:06:52 +09:00
Daniel Templeton
33815af424 YARN-6334. TestRMFailover#testAutomaticFailover always passes even when it should fail 
(Contributed by Yufei Gu via Daniel Templeton)
 2017-03-24 11:42:54 -07:00
Ravi Prakash
d4f73e7e27 HDFS-11574. Spelling mistakes in the Java source. Contributed by Hu Xiaodong. 2017-03-24 09:38:17 -07:00
Varun Saxena
f462e1ff68 YARN-5934. Fix TestTimelineWebServices.testPrimaryFilterNumericString (Akira Ajisaka via Varun Saxena) 2017-03-22 04:11:53 +05:30
Daniel Templeton
0a05c5c598 YARN-6284. hasAlreadyRun should be final in ResourceManager.StandByTransitionRunnable 
(Contributed by Laura Adams via Daniel Templeton)
 2017-03-21 14:15:40 -07:00
Junping Du
c01d15ab27 YARN-6367. YARN logs CLI needs alway check containerLogsInfo/containerLogInfo before parse the JSON object from NMWebService. Contributed by Xuan Gong. 2017-03-21 10:53:27 -07:00
Sunil G
1a98a279f4 YARN-6362. Use frontend-maven-plugin 0.0.22 version for new yarn ui. Contributed by Kai Sasaki. 2017-03-21 13:46:31 +05:30
Daniel Templeton
49efd5d204 YARN-6309. Fair scheduler docs should have the queue and queuePlacementPolicy 
elements listed in bold so that they're easier to see
(Contributed by Esmaeil Mirzaee via Daniel Templeton)
 2017-03-20 14:25:00 -07:00
Varun Saxena
35034653d0 YARN-6353. Clean up OrderingPolicy javadoc (Daniel Templeton via Varun Saxena) 2017-03-20 23:29:09 +05:30
Jason Lowe
4a8e304502 YARN-6217. TestLocalCacheDirectoryManager test timeout is too aggressive. Contributed by Miklos Szegedi 2017-03-17 14:06:35 -05:00
Jason Lowe
7114baddb6 YARN-4051. ContainerKillEvent lost when container is still recovering and application finishes. Contributed by sandflee 2017-03-16 09:30:10 -05:00
Jian He
6d95866dcf YARN-6332. Make RegistrySecurity use short user names for ZK ACLs. Contributed by Billie Rinaldi 2017-03-16 12:59:55 +08:00
Junping Du
221783b35b YARN-6294. ATS client should better handle Socket closed case. Contributed by Li Lu. 2017-03-15 11:48:17 -07:00
Rohith Sharma K S
bb6a214cd3 YARN-6336. Jenkins report YARN new UI build failure. Contributed by Sunil G. 2017-03-15 15:54:09 +05:30
Sunil G
2d0e24eb01 YARN-6328. Fix a spelling mistake in CapacityScheduler. Contributed by Jin Yibo. This closes #202 2017-03-15 14:48:05 +05:30
Ray Chiang
4c66a8d19b YARN-6331. Fix flakiness in TestFairScheduler#testDumpState. (Yufei Gu via rchiang) 2017-03-14 15:09:47 -07:00
Junping Du
871dc420f8 YARN-6313. YARN logs cli should provide logs for a completed container even when application is still running. Contributed by Xuan Gong. 
(cherry picked from commit b88f5e0f7858d1d89b79dfd325b767c34416052d)
 2017-03-14 12:58:12 -07:00
Naganarasimha
0a3aa40fe7 YARN-6327. Removing queues from CapacitySchedulerQueueManager and ParentQueue should be done with iterator. Contributed by Jonathan Hung. 2017-03-15 01:22:25 +05:30
Junping Du
34424e98a6 YARN-6314. Potential infinite redirection on YARN log redirection web service. Contributed by Xuan Gong. 
(cherry picked from commit 5a9dda796f0e73060ada794ad5752cc6a237ab2e)
 2017-03-14 02:58:07 -07:00
Sunil G
e5f2eedcbf YARN-5496. Make Node Heatmap Chart categories clickable in new YARN UI. Contributed by Gergely Novák. 2017-03-14 11:47:11 +05:30
Ray Chiang
4db9cc70d0 YARN-6042. Dump scheduler and queue state information into FairScheduler DEBUG log. (Yufei Gu via rchiang) 2017-03-10 16:13:31 -08:00
Varun Saxena
229c7c9f89 YARN-6069. CORS support in timeline v2 (Rohith Sharma K S via Varun Saxena) 2017-03-11 04:09:01 +05:30
Jason Lowe
9649c27864 YARN-6321. TestResources test timeouts are too aggressive. Contributed by Eric Badger 2017-03-10 13:06:54 -06:00
Jian He
4478273e5f HADOOP-14062. ApplicationMasterProtocolPBClientImpl.allocate fails with EOFException when RPC privacy is enabled. Contributed by Steven Rand 2017-03-10 09:25:58 -08:00
Jason Lowe
deb9f56946 YARN-6310. OutputStreams in AggregatedLogFormat.LogWriter can be left open upon exceptions. Contributed by Haibo Chen 2017-03-10 11:08:33 -06:00
Sunil G
e06ff18ab6 YARN-6196. Improve Resource Donut chart with better label in Node page of new YARN UI. Contributed by Akhil PB. 2017-03-10 16:17:48 +05:30
Karthik Kambatla
a96afae125 YARN-6264. AM not launched when a single vcore is available on the cluster. (Yufei Gu via kasha) 2017-03-09 23:11:54 -08:00
Daniel Templeton
846a0cd678 YARN-1047. Expose # of pre-emptions as a queue counter (Contributed by Karthik Kambatla via Daniel Templeton) 2017-03-09 17:51:47 -08:00
Sidharta S
e96a0b8c92 YARN-5669. Add support for docker pull command (Contribtued by luhuichun) 2017-03-09 16:22:19 -08:00
Daniel Templeton
822a74f2ae YARN-6300. NULL_UPDATE_REQUESTS is redundant in TestFairScheduler (Contributed by Yuanbo Liu via Daniel Templeton) 2017-03-09 12:14:33 -08:00
Jason Lowe
d7762a5511 YARN-6165. Intra-queue preemption occurs even when preemption is turned off for a specific queue. Contributed by Eric Payne 2017-03-08 16:46:09 -06:00
Jian He
2be8947d12 Revert "HADOOP-14062. ApplicationMasterProtocolPBClientImpl.allocate fails with EOFException when RPC privacy is enabled. Contributed by Steven Rand" 
This reverts commit 241c1cc05b.
 2017-03-08 13:20:01 -08:00
Jian He
241c1cc05b HADOOP-14062. ApplicationMasterProtocolPBClientImpl.allocate fails with EOFException when RPC privacy is enabled. Contributed by Steven Rand 2017-03-08 10:48:27 -08:00
Robert Kanter
287ba4ffa6 YARN-6297. TestAppLogAggregatorImp.verifyFilesUploaded() should check # of filed uploaded with that of files expected (haibochen via rkanter) 2017-03-08 10:45:33 -08:00
Sunil G
1eb8186703 YARN-6207. Move application across queues should handle delayed event processing. Contributed by Bibin A Chundatt. 2017-03-08 12:04:30 +05:30
Daniel Templeton
e0c239cdbd YARN-6287. RMCriticalThreadUncaughtExceptionHandler.rmContext should be final (Contributed by Corey Barker via Daniel Templeton) 2017-03-07 11:58:48 -08:00
Sunil G
c1386a90d7 YARN-6248. user is not removed from UsersManager’s when app is killed with pending container requests. Contributed by Eric Payne. 2017-03-04 13:46:35 +05:30
Wangda Tan
c61bc2d843 YARN-6278. Enforce to use correct node and npm version in new YARN-UI build. (Sunil G via wangda) 2017-03-03 23:43:25 -08:00