big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
22,358 Commits 2 Branches 2 Tags 264 MiB
e60f5e2572
Commit Graph

5346 Commits

Author SHA1 Message Date
Sunil G
649666e118 YARN-9573. DistributedShell cannot specify LogAggregationContext. Contributed by Adam Antal. 2019-06-06 14:51:55 +05:30
Eric Yang
294695dd57 HADOOP-16314. Make sure all web end points are covered by the same authentication filter. 
Contributed by Prabhu Joseph
 2019-06-05 18:55:13 -04:00
Weiwei Yang
433e97cd34 YARN-9600. Support self-adaption width for columns of containers table on app attempt page. Contributed by Tao Yang. 2019-06-05 13:55:30 +08:00
Eric Yang
d45669cd3c YARN-7537. Add ability to load hbase config from distributed file system. 
Contributed by Prabhu Joseph
 2019-06-04 19:26:06 -04:00
Zhankun Tang
606061aa14 YARN-9595. FPGA plugin: NullPointerException in FpgaNodeResourceUpdateHandler.updateConfiguredResource(). Contributed by Peter Bacsko. 2019-06-04 09:56:59 +08:00
Weiwei Yang
bd2590d71b YARN-9580. Fulfilled reservation information in assignment is lost when transferring in ParentQueue#assignContainers. Contributed by Tao Yang. 2019-06-03 22:59:02 +08:00
Sunil G
59719dc560 YARN-8906. [UI2] NM hostnames not displayed correctly in Node Heatmap Chart. Contributed by Akhil PB. 2019-06-03 15:53:23 +05:30
Sunil G
7f46dda513 YARN-8947. [UI2] Active User info missing from UI2. Contributed by Akhil PB. 2019-06-03 12:24:35 +05:30
Weiwei Yang
4530f4500d YARN-9507. Fix NPE in NodeManager#serviceStop on startup failure. Contributed by Bilwa S T. 2019-06-03 14:09:37 +08:00
Giovanni Matteo Fumarola
2210897609 YARN-9592. Use Logger format in ContainersMonitorImpl. Contributed by Inigo Goiri. 2019-05-31 17:35:49 -07:00
Eric Yang
4cb559ea7b YARN-9027. Fixed LevelDBCacheTimelineStore initialization. 
Contributed by Prabhu Joseph
 2019-05-31 14:31:44 -04:00
Sunil G
52128e352a YARN-9543. [UI2] Handle ATSv2 server down or failures cases gracefully in YARN UI v2. Contributed by Zoltan Siegl and Akhil P B. 2019-05-31 12:29:44 +05:30
Sunil G
e49162f4b3 YARN-9545. Create healthcheck REST endpoint for ATSv2. Contributed by Zoltan Siegl. 2019-05-31 10:28:09 +05:30
Sunil G
7861a5eb1a YARN-9033. ResourceHandlerChain#bootstrap is invoked twice during NM start if LinuxContainerExecutor enabled. Contributed by Zhankun Tang. 2019-05-31 10:22:26 +05:30
Giovanni Matteo Fumarola
f1552f6edb YARN-9553. Fix NPE in EntityGroupFSTimelineStore#getEntityTimelines. Contributed by Prabhu Joseph. 2019-05-30 11:42:27 -07:00
Sunil G
30c6dd92e1 YARN-9452. Fix TestDistributedShell and TestTimelineAuthFilterForV2 failures. Contributed by Prabhu Joseph. 2019-05-30 22:32:41 +05:30
Ahmed Hussein
abf76ac371 YARN-9563. Resource report REST API could return NaN or Inf (Ahmed Hussein via jeagles) 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
 2019-05-29 11:24:08 -05:00
Eric E Payne
3c63551101 YARN-8625. Aggregate Resource Allocation for each job is not present in ATS. Contributed by Prabhu Joseph. 2019-05-29 16:05:39 +00:00
Weiwei Yang
544876fe12 YARN-8693. Add signalToContainer REST API for RMWebServices. Contributed by Tao Yang. 2019-05-29 16:34:48 +08:00
Akira Ajisaka
afd844059c HADOOP-16331. Fix ASF License check in pom.xml 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2019-05-29 17:25:13 +09:00
Akira Ajisaka
4a692bc3be
YARN-9500. Fix typos in ResourceModel.md. Contributed by leiqiang. 2019-05-28 16:53:43 +09:00
Akira Ajisaka
9078e28a24
YARN-9503. Fix JavaDoc error in TestSchedulerOvercommit. Contributed by Wanqiang Ji. 2019-05-28 15:52:39 +09:00
Akira Ajisaka
9f933e6446
HADOOP-16323. https everywhere in Maven settings. 2019-05-27 15:24:59 +09:00
Weiwei Yang
9f056d905f YARN-9497. Support grouping by diagnostics for query results of scheduler and app activities. Contributed by Tao Yang. 2019-05-26 09:56:36 -04:00
Eric Yang
460ba7fb14 YARN-9558. Fixed LogAggregation test cases. 
Contributed by Prabhu Joseph
 2019-05-23 18:38:47 -04:00
Giovanni Matteo Fumarola
6a0e7dd454 YARN-9482. DistributedShell job with localization fails in unsecure cluster. Contributed by Prabhu Joseph. 2019-05-23 13:27:52 -07:00
Igor Rudenko
e2b79912c3 YARN-9469. Fix typo in YarnConfiguration. 2019-05-23 19:00:38 +02:00
Eric Yang
7b03072fd4 YARN-9080. Added clean up of bucket directories. 
Contributed by Prabhu Joseph, Peter Bacsko, Szilard Nemeth
 2019-05-23 12:08:44 -04:00
Giovanni Matteo Fumarola
a85451c8d2 YARN-9575. Fix TestYarnConfigurationFields testcase failing. Contributed by Prabhu Joseph. 2019-05-21 11:25:30 -07:00
Sunil G
24c53e057a YARN-9546. Add configuration option for YARN Native services AM classpath. Contributed by Gergely Pollak. 2019-05-20 10:53:01 -04:00
Giovanni Matteo Fumarola
12c81610e0 YARN-9505. Add container allocation latency for Opportunistic Scheduler. Contributed by Abhishek Modi. 2019-05-17 12:03:21 -07:00
Eric Yang
fab5b80a36 YARN-9554. Fixed TimelineEntity DAO serialization handling. 
Contributed by Prabhu Joseph
 2019-05-16 16:39:50 -04:00
Giovanni Matteo Fumarola
55bd35921c YARN-9552. FairScheduler: NODE_UPDATE can cause NoSuchElementException. Contributed by Peter Bacsko. 2019-05-15 11:50:46 -07:00
bibinchundatt
570fa2da20 YARN-9508. YarnConfiguration areNodeLabel enabled is costly in allocation flow. Contributed by Bilwa S T. 2019-05-15 13:30:09 +05:30
bibinchundatt
2de1e30658 YARN-9547. ContainerStatusPBImpl default execution type is not returned. Contributed by Bilwa S T. 2019-05-15 13:21:39 +05:30
Sunil G
7d831eca64 YARN-9519. TFile log aggregation file format is not working for yarn.log-aggregation.TFile.remote-app-log-dir config. Contributed by Adam Antal. 2019-05-14 10:48:08 -07:00
Giovanni Matteo Fumarola
206e633946 YARN-9453. Clean up code long if-else chain in ApplicationCLI#run. Contributed by Wanqiang Ji. 2019-05-13 11:11:20 -07:00
Giovanni Matteo Fumarola
29ff7fb140 YARN-9493. Scheduler Page does not display the right page by query string. Contributed by Wanqiang Ji. 2019-05-13 10:57:12 -07:00
Weiwei Yang
1a47c2b7ae YARN-9539.Improve cleanup process of app activities and make some conditions configurable. Contributed by Tao Yang. 2019-05-12 22:31:39 -07:00
Sunil G
64c7f36ab1 YARN-9504. [UI2] Fair scheduler queue view page does not show actual capacity. Contributed by Zoltan Siegl. 2019-05-10 14:27:23 +05:30
Giovanni Matteo Fumarola
1b48100a5e YARN-9522. AppBlock ignores full qualified class name of PseudoAuthenticationHandler. Contributed by Prabhu Joseph. 2019-05-09 14:02:58 -07:00
Giovanni Matteo Fumarola
ec36126346 YARN-9483. DistributedShell does not release container when failed to localize at launch. Contributed by Prabhu Joseph. 2019-05-09 10:59:31 -07:00
Weiwei Yang
90add05caa YARN-9489. Support filtering by request-priorities and allocation-request-ids for query results of app activities. Contributed by Tao Yang. 2019-05-09 21:54:09 +08:00
Eric Yang
0c5fa2e7d9 YARN-9535. Fixed spell check for container-executor in Docker documentation. 
Contributed by Charan Hebri
 2019-05-08 16:56:26 -04:00
Akira Ajisaka
3172f6cbf9
YARN-9513. Addendum patch: Fix ASF License warnings. Contributed by Giovanni Matteo Fumarola. 2019-05-08 14:56:23 +09:00
Weiwei Yang
c336af3847 YARN-9432. Reserved containers leak after its request has been cancelled or satisfied when multi-nodes enabled. Contributed by Tao Yang. 2019-05-08 09:54:16 +08:00
Akira Ajisaka
66c2a4ef89
HADOOP-16270. [JDK 11] Remove unintentional override of the version of Maven Dependency Plugin. Contributed by Xieming Li. 2019-05-08 10:50:07 +09:00
Giovanni Matteo Fumarola
8ecbf61cca YARN-9513. [JDK11] Fix TestMetricsInvariantChecker#testManyRuns in case of JDK greater than 8. Contributed by Adam Antal. 2019-05-07 10:59:02 -07:00
Eric Yang
49e1292ea3 YARN-9524. Fixed TestAHSWebService and TestLogsCLI unit tests. 
Contributed by Prabhu Joseph
 2019-05-06 19:48:45 -04:00
Haibo Chen
597fa47ad1 YARN-9529. Log correct cpu controller path on error while initializing CGroups. (Contributed by Jonathan Hung) 2019-05-06 11:56:22 -07:00
Weiwei Yang
12b7059ddc YARN-9440. Improve diagnostics for scheduler and app activities. Contributed by Tao Yang. 2019-05-06 20:00:15 +08:00
Giovanni Matteo Fumarola
d331a2a0c2 YARN-9528. Federation RMs starting up at the same time can give duplicate application IDs. Contributed by Young Chen. 2019-05-03 15:05:17 -07:00
Eric E Payne
b094b94d43 YARN-9285: RM UI progress column is of wrong type. Contributed by Ahmed Hussein. 2019-05-02 19:39:26 +00:00
Eric Yang
accb811e57 YARN-6929. Improved partition algorithm for yarn remote-app-log-dir. 
Contributed by Prabhu Joseph
 2019-04-30 17:04:59 -04:00
Zhankun Tang
7fbaa7d66f YARN-9476. [YARN-9473] Create unit tests for VE plugin. Contributed by Peter Bacsko. 2019-04-30 11:06:44 +08:00
Eric Badger
79d3d35398 YARN-9486. Docker container exited with failure does not get clean up correctly. Contributed by Eric Yang 2019-04-26 01:21:28 +00:00
Sean Mackrory
a703dae25e HADOOP-16222. Fix new deprecations after guava 27.0 update in trunk. Contributed by Gabor Bota. 2019-04-24 10:39:00 -06:00
Giovanni Matteo Fumarola
3f2f4186f6 YARN-9424. Change getDeclaredMethods to getMethods in FederationClientInterceptor#invokeConcurrent. Contributed by Shen Yinjie. 2019-04-23 19:58:41 -07:00
Akira Ajisaka
9d400627c2 YARN-9081. Update jackson from 1.9.13 to 2.x in hadoop-yarn-services-core. 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2019-04-24 09:50:25 +09:00
Giovanni Matteo Fumarola
fec9bf4b0b YARN-9501. TestCapacitySchedulerOvercommit#testReducePreemptAndCancel fails intermittent. Contributed by Prabhu Joseph. 2019-04-23 15:42:56 -07:00
Giovanni Matteo Fumarola
4a0ba24959 YARN-9491. TestApplicationMasterServiceFair#ApplicationMasterServiceTestBase.testUpdateTrackingUrl fails intermittent. Contributed by Prabhu Joseph. 2019-04-23 15:27:04 -07:00
Inigo Goiri
c504eee0c2 YARN-9339. Apps pending metric incorrect after moving app to a new queue. Contributed by Abhishek Modi. 2019-04-23 12:40:44 -07:00
Zhankun Tang
8a95ea61e1 YARN-9475. [YARN-9473] Create basic VE plugin. Contributed by Peter Bacsko. 2019-04-23 17:33:58 +08:00
Weiwei Yang
1c8046d67e YARN-9325. TestQueueManagementDynamicEditPolicy fails intermittent. Contributed by Prabhu Joseph. 2019-04-23 14:21:13 +08:00
Eric Yang
a54c1e3ace YARN-8551. Project setup for MaWo application. 
Contributed by Yesha Vora
 2019-04-22 16:56:26 -04:00
Inigo Goiri
96e3027e46 YARN-2889. Limit the number of opportunistic container allocated per AM heartbeat. Contributed by Abhishek Modi. 2019-04-22 09:49:03 -07:00
Weiwei Yang
adefd37105 YARN-9495. Fix findbugs warnings in hadoop-yarn-server-resourcemanager module. Contributed by Tao Yang. 2019-04-20 21:51:00 +08:00
Billie Rinaldi
317fcbabe3 YARN-9254. Add support for storing application catalog data to HDFS. Contributed by Eric Yang 2019-04-19 10:29:57 -07:00
Inigo Goiri
aeadb9432f YARN-9448. Fix Opportunistic Scheduling for node local allocations. Contributed by Abhishek Modi. 2019-04-19 09:41:06 -07:00
Eric Yang
ef97a20831 YARN-8622. Fixed container-executor compilation on MacOSX. 
Contributed by Siyao Meng
 2019-04-18 18:59:21 -04:00
Eric Yang
df76cdc895 YARN-6695. Fixed NPE in publishing appFinished events to ATSv2. 
Contributed by Prabhu Joseph
 2019-04-18 12:29:37 -04:00
Prabhu Joseph
aa4c744aef
YARN-9470. Fix order of actual and expected expression in assert statements 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2019-04-18 15:40:37 +09:00
Siyao Meng
6e4399ea61 YARN-9487. NodeManager native build shouldn't link against librt on macOS. Contributed by Siyao Meng. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-04-17 22:56:57 -07:00
Eric Yang
9cf7401794 YARN-9349. Improved log level practices for InvalidStateTransitionException. 
Contributed by Anuhan Torgonshar

(cherry picked from commit fe2370e039e1ee980d74769ae85d67434e0993cf)
 2019-04-16 19:53:45 -04:00
Szilard Nemeth
b8086aed86 YARN-9123. Clean up and split testcases in TestNMWebServices for GPU support. Contributed by Szilard Nemeth. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-04-16 11:06:25 -07:00
Billie Rinaldi
2364c7d0bf YARN-9466. Fixed application catalog navigation bar height in Safari. Contributed by Eric Yang 2019-04-16 10:04:27 -07:00
Billie Rinaldi
ad865888a6 YARN-8530. Add SPNEGO filter to application catalog. Contributed by Eric Yang 2019-04-16 09:52:14 -07:00
Akira Ajisaka
c65218ee8e YARN-8943. Upgrade JUnit from 4 to 5 in hadoop-yarn-api. 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2019-04-16 12:13:49 +09:00
Eric Badger
5583e1b6fc YARN-7848 Force removal of docker containers that do not get removed on first try. Contributed by Eric Yang 2019-04-15 20:47:09 +00:00
Eric Badger
254efc9358 YARN-9379. Can't specify docker runtime through environment. Contributed by caozhiqiang 2019-04-15 18:24:37 +00:00
Weiwei Yang
7fa73fac26 YARN-9439. Support asynchronized scheduling mode and multi-node lookup mechanism for app activities. Contributed by Tao Yang. 2019-04-16 00:12:43 +08:00
Akira Ajisaka
c4c16cae7d
YARN-9481. [JDK 11] Build fails due to hard-coded target version in hadoop-yarn-applications-catalog-webapp. Contributed by Kei Kori. 2019-04-15 13:08:28 +09:00
Inigo Goiri
7a68e7abd5 YARN-9474. Remove hard coded sleep from Opportunistic Scheduler tests. Contributed by Abhishek Modi. 2019-04-14 20:11:20 -07:00
Billie Rinaldi
b2cdf809bc YARN-9281. Add express upgrade button to Appcatalog UI. Contributed by Eric Yang 2019-04-13 18:55:11 +03:00
Gabor Bota
1943db5571
HADOOP-16237. Fix new findbugs issues after updating guava to 27.0-jre. 
Author:    Gabor Bota <gabor.bota@cloudera.com>
 2019-04-12 18:28:38 -07:00
Giovanni Matteo Fumarola
ed3747c1cc YARN-9435. Add Opportunistic Scheduler metrics in ResourceManager. Contributed by Abhishek Modi. 2019-04-11 11:49:19 -07:00
Weiwei Yang
8c1bba375b YARN-9463. Add queueName info when failing with queue capacity sanity check. Contributed by Aihua Xu. 2019-04-10 22:51:28 +08:00
Igor Rudenko
32722d2661
YARN-9433. Remove unused constants in YARN resource manager 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2019-04-10 18:37:27 +09:00
Giovanni Matteo Fumarola
cfec455c45 YARN-999. In case of long running tasks, reduce node resource should balloon out resource quickly by calling preemption API and suspending running task. Contributed by Inigo Goiri. 2019-04-09 10:59:43 -07:00
Weiwei Yang
fc05b0e70e YARN-9313. Support asynchronized scheduling mode and multi-node lookup mechanism for scheduler activities. Contributed by Tao Yang. 2019-04-08 13:40:53 +08:00
Weiwei Yang
ec143cbf67 YARN-9413. Queue resource leak after app fail for CapacityScheduler. Contributed by Tao Yang. 2019-04-06 19:59:36 +08:00
Vrushali C
22362c876d YARN-9335 [atsv2] Restrict the number of elements held in timeline collector when backend is unreachable for async calls. Contributed by Abhishesk Modi. 2019-04-05 12:06:51 -07:00
Vrushali C
27039a29ae YARN-9382 Publish container killed, paused and resumed events to ATSv2. Contributed by Abhishesk Modi. 2019-04-05 12:02:43 -07:00
Eric Yang
2bf82aee82 YARN-9441. Updated YARN app catalog name for consistency. 
Contributed by Weiwei Yang
 2019-04-04 13:21:30 -04:00
Eric Yang
8d150067e2 YARN-9396. Fixed duplicated RM Container created event to ATS. 
Contributed by Prabhu Joseph
 2019-04-04 13:01:56 -04:00
Weiwei Yang
945e8c6064 YARN-9394. Use new API of RackResolver to get better performance. Contributed by Lantao Jin. 2019-04-04 18:05:01 +08:00
Vrushali C
eb03f7c419 YARN-9303 Username splits won't help timelineservice.app_flow table. Contributed by Prabhu Joseph. 2019-04-03 22:53:05 -07:00
Gabor Bota
d7979079ea HADOOP-16210. Update guava to 27.0-jre in hadoop-project trunk. Contributed by Gabor Bota. 2019-04-03 12:59:39 -06:00
Sunil G
002dcc4ebf YARN-4901. QueueMetrics needs to be cleared before MockRM is initialized. Contributed by Peter Bacsko. 2019-04-03 18:57:28 +05:30
Yufei Gu
2f752830ba YARN-9214. Add AbstractYarnScheduler#getValidQueues method to remove duplication. Contributed by Wanqiang Ji. 2019-04-01 20:05:15 -07:00