big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
24,051 Commits 2 Branches 2 Tags 264 MiB
e277d338da
Commit Graph

4114 Commits

Author SHA1 Message Date
Eric Badger
ac5f21dbef YARN-4771. Some containers can be skipped during log aggregation after NM 
restart. Contributed by Jason Lowe and Jim Brennan.
 2020-07-24 22:35:16 +00:00
Prabhu Joseph
247eb0979b YARN-10319. Record Last N Scheduler Activities from ActivitiesManager 
Reviewed by Tao Yang and Adam Antal.
 2020-07-24 11:43:31 +05:30
bibinchundatt
bfcd775381 YARN-10315. Avoid sending RMNodeResourceupdate event if resource is same. Contributed by Sushil Ks. 2020-07-23 16:41:35 +05:30
Masatake Iwasaki
1b29c9bfee
HADOOP-17138. Fix spotbugs warnings surfaced after upgrade to 4.0.6. (#2155) 2020-07-22 13:40:20 +09:00
Eric Badger
736bed6d6d [YARN-10353] Log vcores used and cumulative cpu in containers monitor. 
Contributed by Jim Brennan
 2020-07-20 18:08:27 +00:00
Ayush Saxena
6bcb24d269 HADOOP-17100. Replace Guava Supplier with Java8+ Supplier in Hadoop. Contributed by Ahmed Hussein. 2020-07-18 14:33:43 +05:30
Prabhu Joseph
cc71d50b21 YARN-10339. Fix TimelineClient in NodeManager failing when Simple Http Auth used in Secure Cluster 
Contributed by Tarun Parimi.
 2020-07-16 23:36:49 +05:30
Ahmed Hussein
1f71c4ae71 HADOOP-17099. Replace Guava Predicate with Java8+ Predicate 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
 2020-07-15 11:39:06 -05:00
Bilwa S T
8980c23fc2
YARN-10350. TestUserGroupMappingPlacementRule fails 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2020-07-15 14:31:34 +09:00
Eric Badger
48f90115b5 YARN-10348. Allow RM to always cancel tokens after app completes. Contributed by 
Jim Brennan
 2020-07-13 23:10:39 +00:00
Eric E Payne
0427100b75 YARN-10297. TestContinuousScheduling#testFairSchedulerContinuousSchedulingInitTime fails intermittently. Contributed by Jim Brennan (Jim_Brennan) 2020-07-13 18:57:50 +00:00
Xiaoyu Yao
f91a8ad88b
HADOOP-17079. Optimize UGI#getGroups by adding UGI#getGroupsSet. (#2085) 2020-07-09 11:33:37 -07:00
Prabhu Joseph
3a4d05b850 YARN-8047. RMWebApp make external class pluggable. 
Contributed by Bilwa S T.
 2020-07-08 12:43:20 +05:30
Prabhu Joseph
2bbd00dff4 YARN-10337. Fix failing testcase TestRMHATimelineCollectors. 
Contributed by Bilwa S T.
 2020-07-07 17:10:59 +05:30
Akira Ajisaka
639acb6d89
HADOOP-17111. Replace Guava Optional with Java8+ Optional. Contributed by Ahmed Hussein. 2020-07-06 16:08:36 +09:00
Szilard Nemeth
04abd0eb17 YARN-10330. Add missing test scenarios to TestUserGroupMappingPlacementRule and TestAppNameMappingPlacementRule. Contributed by Peter Bacsko 2020-07-01 14:10:55 +02:00
Eric Yang
e8dc862d38 YARN-9809. Added node manager health status to resource manager registration call. 
Contributed by Eric Badger via eyang
 2020-06-30 11:39:16 -07:00
Eric E Payne
74fc13cf91 YARN-9903: Support reservations continue looking for Node Labels. Contributed by Jim Brennan (Jim_Brennan). 2020-06-29 18:39:53 +00:00
Brahma Reddy Battula
2c03524fa4 YARN-6526. Refactoring SQLFederationStateStore by avoiding to recreate a connection at every call. COntributed by Bilwa S T. 2020-06-26 20:43:27 +05:30
Szilard Nemeth
6459cc73e6 YARN-10318. ApplicationHistory Web UI incorrect column indexing. Contributed by Andras Gyori 2020-06-26 13:15:14 +02:00
Szilard Nemeth
33d3df7ee2 YARN-10277. CapacityScheduler test TestUserGroupMappingPlacementRule should build proper hierarchy. Contributed by Szilard Nemeth 2020-06-26 11:14:24 +02:00
Szilard Nemeth
6a8fd73b27 YARN-10279. Avoid unnecessary QueueMappingEntity creations. Contributed by Marton Hudaky 2020-06-25 17:27:58 +02:00
Szilard Nemeth
03f855e3e7 YARN-10316. FS-CS converter: convert maxAppsDefault, maxRunningApps settings. Contributed by Peter Bacsko 2020-06-23 12:13:04 +02:00
Prabhu Joseph
ce1008fe61 YARN-10321. Break down TestUserGroupMappingPlacementRule#testMapping into test scenarios 
Contributed by Szilard Nemeth. Reviewed by Gergely Pollak.
 2020-06-22 12:49:14 +05:30
Surendra Singh Lilhore
b2facc84a1 YARN-9460. QueueACLsManager and ReservationsACLManager should not use instanceof checks. Contributed by Bilwa S T. 2020-06-20 20:09:31 +05:30
Szilard Nemeth
469841446f YARN-9930. Support max running app logic for CapacityScheduler. Contributed by Peter Bacsko 2020-06-19 14:50:24 +02:00
Szilard Nemeth
5b1a56f9f1 YARN-10281. Redundant QueuePath usage in UserGroupMappingPlacementRule and AppNameMappingPlacementRule. Contributed by Gergely Pollak 2020-06-17 14:34:40 +02:00
Vinayakumar B
e154084770
HADOOP-17046. Support downstreams' existing Hadoop-rpc implementations using non-shaded protobuf classes (#2026) 2020-06-12 23:16:33 +05:30
Tao Yang
7c4de59fc1 YARN-10293. Reserved Containers not allocated from available space of other nodes in CandidateNodeSet for MultiNodePlacement. Contributed by Prabhu Joseph. 2020-06-12 22:10:25 +08:00
Eric E Payne
56247db302 YARN-10300: appMasterHost not set in RM ApplicationSummary when AM fails before first heartbeat. Contributed by Eric Badger (ebadger). 2020-06-09 18:43:16 +00:00
Szilard Nemeth
8abff5151a YARN-10292. FS-CS converter: add an option to enable asynchronous scheduling in CapacityScheduler. Contributed by Benjamin Teke 2020-06-05 15:00:39 +02:00
Szilard Nemeth
8b146c17b3 YARN-10274. Merge QueueMapping and QueueMappingEntity. Contributed by Gergely Pollak 2020-06-05 11:38:32 +02:00
Szilard Nemeth
b5efdea4fd YARN-10254. CapacityScheduler incorrect User Group Mapping after leaf queue change. Contributed by Gergely Pollak 2020-06-02 18:29:15 +02:00
Szilard Nemeth
aa6d13455b YARN-10284. Add lazy initialization of LogAggregationFileControllerFactory in LogServlet. Contributed by Adam Antal 2020-06-02 15:53:48 +02:00
Szilard Nemeth
e0a0741ac8 YARN-10286. PendingContainers bugs in the scheduler outputs. Contributed by Andras Gyori 2020-06-02 15:42:39 +02:00
Szilard Nemeth
593af878c0 YARN-10108. FS-CS converter: nestedUserQueue with default rule results in invalid queue mapping. Contributed by Gergely Pollak 2020-05-27 13:05:52 +02:00
Szilard Nemeth
fb030f3239 Revert "FS-CS converter: nestedUserQueue with default rule results in invalid queue mapping. Contributed by Gergely Pollak" 
This reverts commit ac4540dd8e.
Reverting because of missing Jira-ID in the commit message.
 2020-05-27 13:04:48 +02:00
Jonathan Hung
c30c23cb66 YARN-6492. Generate queue metrics for each partition. Contributed by Manikandan R 2020-05-26 16:29:42 -07:00
Szilard Nemeth
ac4540dd8e FS-CS converter: nestedUserQueue with default rule results in invalid queue mapping. Contributed by Gergely Pollak 2020-05-21 13:11:44 +02:00
Prabhu Joseph
cef0756929 YARN-9606. Set sslfactory for AuthenticatedURL() while creating LogsCLI#webServiceClient. 
Contributed by Bilwa S T. Reviewed by Akira Ajisaka.
 2020-05-20 11:36:52 +05:30
Prabhu Joseph
6ce295b787 YARN-10259. Fix reservation logic in Multi Node Placement. 
Reviewed by Wangda Tan.
 2020-05-14 16:52:11 +05:30
Inigo Goiri
108ecf992f YARN-8942. PriorityBasedRouterPolicy throws exception if all sub-cluster weights have negative value. Contributed by Bilwa S T. 2020-05-13 10:04:12 -07:00
Prabhu Joseph
450e5aa9dd YARN-10154. Addendum Patch which fixes below bugs 
1. RM fails to start when LeafQueueTemplate max capacity is not specified.
2. Job stuck in ACCEPTED state with DominantResourceCalculator as Queue
   Capacity is set to NaN during RM startup with clusterResource is zero.

Reviewed by Sunil G and Manikandan R.
 2020-05-13 14:35:37 +05:30
Jonathan Hung
fff1d2c122 YARN-10260. Allow transitioning queue from DRAINING to RUNNING state. Contributed by Bilwa S T 2020-05-12 10:48:54 -07:00
Inigo Goiri
35010120fb YARN-9017. PlacementRule order is not maintained in CS. Contributed by Bilwa S T. 2020-05-06 13:22:54 -07:00
Ahmed Hussein
92e3ebb401 YARN-8959. TestContainerResizing fails randomly (Ahmed Hussein via jeagles) 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
 2020-05-06 12:27:17 -05:00
Szilard Nemeth
cb6399c109 YARN-10257. FS-CS converter: skip increment properties for mem/vcores and fix DRF check. Contributed by Peter Bacsko 2020-05-05 16:00:43 +02:00
Szilard Nemeth
0debe55d6c YARN-10160. Add auto queue creation related configs to RMWebService#CapacitySchedulerQueueInfo. Contributed by Prabhu Joseph 2020-05-05 13:11:28 +02:00
Ahmed Hussein
f5081a9a5d YARN-10256. Refactor TestContainerSchedulerQueuing.testContainerUpdateExecTypeGuaranteedToOpportunistic (Ahmed Hussein via jeagles) 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
 2020-05-04 10:13:51 -05:00
Surendra Singh Lilhore
257b04ce4e YARN-6553. Replace MockResourceManagerFacade with MockRM for AMRMProxy/Router tests. Contributed by Bilwa S T. 2020-05-02 12:43:20 +05:30