big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
24,475 Commits 2 Branches 2 Tags 264 MiB
9b2956e254
Commit Graph

6049 Commits

Author SHA1 Message Date
Jonathan Hung
c30c23cb66 YARN-6492. Generate queue metrics for each partition. Contributed by Manikandan R 2020-05-26 16:29:42 -07:00
Szilard Nemeth
ac4540dd8e FS-CS converter: nestedUserQueue with default rule results in invalid queue mapping. Contributed by Gergely Pollak 2020-05-21 13:11:44 +02:00
Eric Yang
726b8e324b YARN-10228. Relax restriction of file path character in yarn.service.am.java.opts. 
Contributed by Bilwa S T via eyang
 2020-05-20 09:23:56 -07:00
Prabhu Joseph
cef0756929 YARN-9606. Set sslfactory for AuthenticatedURL() while creating LogsCLI#webServiceClient. 
Contributed by Bilwa S T. Reviewed by Akira Ajisaka.
 2020-05-20 11:36:52 +05:30
Akira Ajisaka
b65815d691
Revert "YARN-9606. Set sslfactory for AuthenticatedURL() while creating LogsCLI#webServiceClient." 
This reverts commit 7836bc4c35.
 2020-05-18 16:29:07 +09:00
Prabhu Joseph
7836bc4c35 YARN-9606. Set sslfactory for AuthenticatedURL() while creating LogsCLI#webServiceClient. 
Contributed by Bilwa S T. Reviewed by Brahma Reddy Battula.
 2020-05-14 19:40:42 +05:30
Prabhu Joseph
6ce295b787 YARN-10259. Fix reservation logic in Multi Node Placement. 
Reviewed by Wangda Tan.
 2020-05-14 16:52:11 +05:30
Surendra Singh Lilhore
1958cb7c2b YARN-10265. Upgrade Netty-all dependency to latest version 4.1.50 to fix ARM support issue. Contributed by liusheng 2020-05-14 15:09:34 +05:30
Ayush Saxena
0918433b4d YARN-9898. Dependency netty-all-4.1.27.Final doesn't support ARM platform. Contributed by liusheng. 2020-05-14 00:36:20 +05:30
Inigo Goiri
108ecf992f YARN-8942. PriorityBasedRouterPolicy throws exception if all sub-cluster weights have negative value. Contributed by Bilwa S T. 2020-05-13 10:04:12 -07:00
Prabhu Joseph
450e5aa9dd YARN-10154. Addendum Patch which fixes below bugs 
1. RM fails to start when LeafQueueTemplate max capacity is not specified.
2. Job stuck in ACCEPTED state with DominantResourceCalculator as Queue
   Capacity is set to NaN during RM startup with clusterResource is zero.

Reviewed by Sunil G and Manikandan R.
 2020-05-13 14:35:37 +05:30
Jonathan Hung
fff1d2c122 YARN-10260. Allow transitioning queue from DRAINING to RUNNING state. Contributed by Bilwa S T 2020-05-12 10:48:54 -07:00
Elixir Kook
a3f945fb84
HADOOP-17035. fixed typos (timeout, interruped) (#2007) 
Co-authored-by: Sungpeo Kook <elixir.kook@kakaocorp.com>
 2020-05-12 10:50:04 -05:00
Inigo Goiri
35010120fb YARN-9017. PlacementRule order is not maintained in CS. Contributed by Bilwa S T. 2020-05-06 13:22:54 -07:00
Ahmed Hussein
92e3ebb401 YARN-8959. TestContainerResizing fails randomly (Ahmed Hussein via jeagles) 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
 2020-05-06 12:27:17 -05:00
Szilard Nemeth
cb6399c109 YARN-10257. FS-CS converter: skip increment properties for mem/vcores and fix DRF check. Contributed by Peter Bacsko 2020-05-05 16:00:43 +02:00
Szilard Nemeth
0debe55d6c YARN-10160. Add auto queue creation related configs to RMWebService#CapacitySchedulerQueueInfo. Contributed by Prabhu Joseph 2020-05-05 13:11:28 +02:00
Ahmed Hussein
f5081a9a5d YARN-10256. Refactor TestContainerSchedulerQueuing.testContainerUpdateExecTypeGuaranteedToOpportunistic (Ahmed Hussein via jeagles) 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
 2020-05-04 10:13:51 -05:00
Surendra Singh Lilhore
257b04ce4e YARN-6553. Replace MockResourceManagerFacade with MockRM for AMRMProxy/Router tests. Contributed by Bilwa S T. 2020-05-02 12:43:20 +05:30
Inigo Goiri
d125d39108 YARN-6973. Adding RM Cluster Id in ApplicationReport. Contributed by Bilwa S T. 2020-04-29 11:23:58 -07:00
Szilard Nemeth
410c605aec YARN-10247. Application priority queue ACLs are not respected. Contributed by Sunil G 2020-04-29 15:53:30 +02:00
Szilard Nemeth
ab36429559 YARN-10215. Endpoint for obtaining direct URL for the logs. Contributed by Andras Gyori 2020-04-28 18:53:28 +02:00
Akira Ajisaka
6ef01646ba
YARN-9848. Revert YARN-4946. Contributed by Steven Rand. 2020-04-28 05:35:36 +09:00
Prabhu Joseph
9224568b0e YARN-10156. Destroy Jersey Client in TimelineConnector. 
Contributed by Tanu Ajmera.
 2020-04-27 18:49:15 +05:30
Szilard Nemeth
f91e21ac10 YARN-10194. YARN RMWebServices /scheduler-conf/validate leaks ZK Connections. Contributed by Prabhu Joseph 2020-04-27 08:26:11 +02:00
Szilard Nemeth
d4874585f4 YARN-10237. Add isAbsoluteResource config for queue in scheduler response. Contributed by Prabhu Joseph 2020-04-24 17:04:47 +02:00
Akira Ajisaka
9827ff2961
YARN-10223. Remove jersey-test-framework-core dependency from yarn-server-common. (#1939) 2020-04-24 09:32:25 +09:00
Prabhu Joseph
60fa15366e YARN-10240. Prevent Fatal CancelledException in TimelineV2Client when stopping 
Contributed by Tarun Parimi.
 2020-04-21 12:01:33 +05:30
Szilard Nemeth
92b28e38b7 YARN-9996. Code cleanup in QueueAdminConfigurationMutationACLPolicy. Contributed by Siddharth Ahuja 2020-04-18 10:13:37 +02:00
Szilard Nemeth
76900b4f5b YARN-10189. Code cleanup in LeveldbRMStateStore. Contributed by Benjamin Teke 2020-04-18 09:57:27 +02:00
Jonathan Hung
3601054822 YARN-9954. Configurable max application tags and max tag length. Contributed by Bilwa S T 2020-04-17 10:34:44 -07:00
Sunil G
2fe122e322 YARN-10154. CS Dynamic Queues cannot be configured with absolute resources. Contributed by Manikandan R. 2020-04-17 18:51:19 +05:30
Szilard Nemeth
4db598e0e6 YARN-9999. TestFSSchedulerConfigurationStore: Extend from ConfigurationStoreBaseTest, general code cleanup. Contributed by Benjamin Teke 2020-04-15 13:18:44 +02:00
Prabhu Joseph
94f7470c71 YARN-10233. Fix YARN UI2 Daemon Logs 
Contributed by Akhil PB.
 2020-04-15 13:39:37 +05:30
Szilard Nemeth
55fcbcb5c2 YARN-10234. FS-CS converter: don't enable auto-create queue property for root. Contributed by Peter Bacsko 2020-04-15 07:24:04 +02:00
Prabhu Joseph
c791b0e90e YARN-10219. Fix YARN Native Service Placement Constraints with Node Attributes. 
Contributed by Eric Yang.
 2020-04-14 12:34:53 +05:30
Jonathan Hung
23481ad378 YARN-10212. Create separate configuration for max global AM attempts. Contributed by Bilwa S T 2020-04-09 11:06:11 -07:00
Szilard Nemeth
c04a484aed YARN-9995. Code cleanup in TestSchedConfCLI. Contributed by Bilwa S T. 2020-04-09 15:46:25 +02:00
Sunil G
6fb29d0f17 YARN-10226. NPE in Capacity Scheduler while using %primary_group queue mapping. Contributed by Peter Bacsko. 2020-04-09 18:51:42 +05:30
Szilard Nemeth
8e4517f251 YARN-5277. When localizers fail due to resource timestamps being out, provide more diagnostics. Contributed by Siddharth Ahuja 2020-04-09 12:31:01 +02:00
Akira Ajisaka
e53d472bb0
HADOOP-14836. Upgrade maven-clean-plugin to 3.1.0 (#1933) 2020-04-09 01:47:31 +09:00
Wilfred Spiegelenburg
2214005c0f
YARN-10063. Add container-executor arguments --http/--https to usage. Contributed by Siddharth Ahuja 2020-04-08 11:30:03 +10:00
Szilard Nemeth
bffb43b00e YARN-10207. CLOSE_WAIT socket connection leaks during rendering of (corrupted) aggregated logs on the JobHistoryServer Web UI. Contributed by Siddharth Ahuja 2020-04-07 17:03:17 +02:00
Szilard Nemeth
45362a9f4c YARN-10001. Add explanation of unimplemented methods in InMemoryConfigurationStore. Contributed by Siddharth Ahuja 2020-04-07 15:51:55 +02:00
Prabhu Joseph
7a39da3537 YARN-10120. Amendment fix for Java Doc. 
Contributed by Bilwa S T.
 2020-04-03 13:07:41 +05:30
Prabhu Joseph
25361b077b YARN-10120. Https Support in Router WebServiceClient. 
Contributed by Bilwa S T.
 2020-04-02 18:11:43 +05:30
Akira Ajisaka
c162648aff
YARN-10202. Fix documentation about NodeAttributes. Contributed by Sen Zhao. 2020-04-01 16:05:47 +09:00
Brahma Reddy Battula
8914cf9167 Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30
RogPodge
50f7f6dfd1
YARN-10210. Add a RMFailoverProxyProvider that does DNS resolution on failover. 2020-03-26 08:14:30 -07:00
Szilard Nemeth
3d5ade1839 YARN-10043. FairOrderingPolicy Improvements. Contributed by Manikandan R 2020-03-26 13:23:43 +01:00
Szilard Nemeth
9a297ff31d YARN-10197. FS-CS converter: fix emitted ordering policy string and max-am-resource percent value. Contributed by Peter Bacsko 2020-03-26 12:35:10 +01:00
Szilard Nemeth
348685dcb9 YARN-9998. Code cleanup in LeveldbConfigurationStore. Contributed by Benjamin Teke 2020-03-26 12:18:15 +01:00
Jonathan Hung
6ce189c621 YARN-10200. Add number of containers to RMAppManager summary 2020-03-25 10:41:59 -07:00
RogPodge
2d294bd575
HADOOP-16938. Make non-HA proxy providers pluggable 2020-03-25 08:06:58 -07:00
Sunil G
cdb2107066 YARN-9879. Allow multiple leaf queues with the same name in CapacityScheduler. Contributed by Gergely Pollak. 2020-03-25 16:50:19 +05:30
Prabhu Joseph
d312991ead YARN-10198. Remove Validation of Queue Exist under Managed Parent Queue 
Contributed by Peter Bacsko. Reviewed by Manikandan R.
 2020-03-23 18:01:14 +05:30
Prabhu Joseph
f2d3ac2a3f YARN-10034. Remove Allocation Tags from released container from Decommission node 
Contributed by Kyungwan Nam. Reviewed by Adam Antal.
 2020-03-19 12:55:14 +05:30
Eric Badger
1975479285 YARN-2710. RM HA tests failed intermittently on trunk. Contributed by Ahmed 
Hussein.
 2020-03-16 22:24:02 +00:00
Weiwei Yang
b9d825f178 YARN-9567. Add diagnostics for outstanding resource requests on app attempts page. Contributed by Tao Yang. 2020-03-13 14:39:36 -07:00
Weiwei Yang
d273f6a2b4 YARN-9538. Document scheduler/app activities and REST APIs. Contributed by Tao Yang. 2020-03-13 14:36:52 -07:00
Bilwa
c83644deac YARN-10110. Adding RouterPolicyProvider for RM Federation. 
Contributed by Bilwa S T.
 2020-03-13 16:56:44 +05:30
Szilard Nemeth
5ead9c15ca YARN-9997. Code cleanup in ZKConfigurationStore. Contributed by Andras Gyori 2020-03-12 12:29:03 +01:00
Szilard Nemeth
38d87883b6 YARN-10193. FS-CS converter: fix incorrect capacity conversion. Contributed by Peter Bacsko 2020-03-12 12:18:37 +01:00
Sunil G
0fd8bf5f6b YARN-10191. FS-CS converter: call System.exit function call for every code path in main method. Contributed by Peter Bacsko. 2020-03-12 15:03:12 +05:30
Szilard Nemeth
cf9cf83a43 YARN-9354. Resources should be created with ResourceTypesTestHelper instead of TestUtils. Contributed by Andras Gyori 2020-03-10 16:44:48 +01:00
Szilard Nemeth
61f4cf3055 YARN-10002. Code cleanup and improvements in ConfigurationStoreBaseTest. Contributed by Benjamin Teke 2020-03-10 16:35:04 +01:00
Szilard Nemeth
9314ef947f YARN-10168. FS-CS Converter: tool doesn't handle min/max resource conversion correctly. Contributed by Peter Bacsko 2020-03-10 16:07:46 +01:00
Eric E Payne
ede05b19d1 YARN-942. TestContainerSchedulerQueuing.testKillOnlyRequiredOpportunisticContainers fails sporadically Contributed by Ahmed Hussein (ahussein) 2020-03-10 14:17:45 +00:00
Szilard Nemeth
44afe1154d YARN-9419. Log a warning if GPU isolation is enabled but LinuxContainerExecutor is disabled. Contribued by Andras Gyori 2020-03-09 16:08:24 +01:00
bibinchundatt
3859fa76d0 YARN-6924. Metrics for Federation AMRMProxy. Contributed by Young Chen 2020-03-07 09:34:42 +05:30
Szilard Nemeth
ea0444851d YARN.10003. YarnConfigurationStore#checkVersion throws exception that belongs to RMStateStore. Contributed by Benjamin Teke 2020-03-06 17:49:45 +01:00
Szilard Nemeth
004e955348 YARN-10167. FS-CS Converter: Need to validate c-s.xml after converting. Contributed by Peter Bacsko 2020-03-05 09:56:42 +01:00
Eric Badger
2649f8b327 YARN-10173. Make pid file generation timeout configurable in case of reacquired 
container. Contributed by Adam Antal.
 2020-03-04 23:31:57 +00:00
Szilard Nemeth
c0d084247c YARN-10175. FS-CS converter: only convert placement rules if a cmd line switch is defined. Contributed by Peter Bacsko 2020-03-03 09:46:32 +01:00
Szilard Nemeth
e9eecedf69 YARN-10148. addendum: Fix method call parameter order of setAdminAndSubmitACL in TestCapacitySchedulerQueueACLs. Contributed by Kinga Marton 2020-03-02 16:25:14 +01:00
Inigo Goiri
a43510e21d YARN-10161. TestRouterWebServicesREST is corrupting STDOUT. Contributed by Jim Brennan. 2020-02-27 13:18:30 -08:00
Inigo Goiri
b420ddeada YARN-10155. TestDelegationTokenRenewer.testTokenThreadTimeout fails in trunk. Contributed by Manikandan R. 2020-02-27 12:53:20 -08:00
Szilard Nemeth
10461e0193 YARN-10148. Add Unit test for queue ACL for both FS and CS. Contributed by Kinga Marton 2020-02-27 20:10:32 +01:00
Akira Ajisaka
7dfa37e8f0
YARN-10152. Fix findbugs warnings in hadoop-yarn-applications-mawo-core module (#1852) 2020-02-27 10:36:49 +09:00
Brahma Reddy Battula
3a9ccf7f6d YARN-10141.Interceptor in FederationInterceptorREST doesnt update on RM switchover. Contributed by D M Murali Krishna Reddy. 2020-02-26 23:24:00 +05:30
tanu.ajmera
b5df1da3a1 YARN-9593. Support Comma in the value of Scheduler Configuration Mutation API. 2020-02-26 15:04:29 +05:30
Elixir Kook
d608e94f92
YARN-10156. Fix typo 'complaint' which means quite different in Federation.md (#1856) 2020-02-26 17:29:26 +09:00
Szilard Nemeth
d68616b045 YARN-10130. FS-CS converter: Do not allow output dir to be the same as input dir. Contributed by Adam Antal 2020-02-25 21:30:04 +01:00
Wei-Chiu Chuang
dda00d3ff5
YARN-10074. Update netty to 4.1.42Final in yarn-csi. Contributed by Wei-Chiu Chuang. (#1807) 2020-02-25 13:47:52 +09:00
Szilard Nemeth
72fa9c7f1b YARN-10135. FS-CS converter tool: issue warning on dynamic auto-create mapping rules. Contributed by Peter Bacsko 2020-02-24 21:54:07 +01:00
Szilard Nemeth
34cf63c7d0 YARN-10157. FS-CS converter: initPropertyActions() is not called without rules file. Contributed by Peter Bacsko 2020-02-24 21:45:49 +01:00
Prabhu Joseph
181e6d033e YARN-10119. Option to reset AM failure count for YARN Service 
Contributed by kyungwan nam.
 2020-02-20 20:34:06 +05:30
Szilard Nemeth
4af2556b48 YARN-10143. YARN-10101 broke Yarn logs CLI. Contributed by Adam Antal 2020-02-20 15:27:15 +01:00
Szilard Nemeth
3f1aad05f0 YARN-10147. FPGA plugin can't find the localized aocx file. Contributed by Peter Bacsko 2020-02-19 15:54:25 +01:00
Sunil G
6526f95bd2 YARN-10139. ValidateAndGetSchedulerConfiguration API fails when cluster max allocation > default 8GB. Contributed by Prabhu Joseph. 2020-02-19 11:17:22 +05:30
Akira Ajisaka
d6d7f8d8c5
YARN-8374. Upgrade objenesis to 2.6 (#1798) 2020-02-19 09:50:37 +09:00
Brahma Reddy Battula
20add89718 YARN-10136. [Router] : Application metrics are hardcode as N/A in UI. Contributed by Bilwa S T. 2020-02-14 16:50:29 +05:30
Szilard Nemeth
da99ac7e93 YARN-10137. UIv2 build is broken in trunk. Contributed by Adam Antal 2020-02-13 16:31:35 +01:00
Szilard Nemeth
f1b1b332f5 YARN-10029. Add option to UIv2 to get container logs from the new JHS API. Contributed by Adam Antal 2020-02-13 12:08:54 +01:00
Prabhu Joseph
fe7d67a8a2 YARN-9521. Handle FileSystem close in ApiServiceClient 
Contributed by kyungwan nam. Reviewed by Eric Yang.
 2020-02-13 09:39:13 +05:30
Prabhu Joseph
e637797211 YARN-10127. Remove setting App Ordering Policy to ParentQueue in FSQueueConverter 
Contributed by Peter Bacsko.
 2020-02-11 22:01:58 +05:30
Sunil G
28f730b317 YARN-10109. Allow stop and convert from leaf to parent queue in a single Mutation API call. Contributed by Prabhu Joseph 2020-02-09 21:14:53 +05:30
Ayush Saxena
3f0a7cd17a YARN-9624. Use switch case for ProtoUtils#convertFromProtoFormat containerState. Contributed by Bilwa S T 2020-02-09 19:14:18 +05:30
Vinayakumar B
7dac7e1d13
HADOOP-16596. [pb-upgrade] Use shaded protobuf classes from hadoop-thirdparty dependency (#1635). Contributed by Vinayakumar B. 2020-02-07 14:51:24 +05:30
Szilard Nemeth
71b2c2ffe9 YARN-10101. Support listing of aggregated logs for containers belonging to an application attempt. Contributed by Adam Antal 2020-02-06 12:25:06 +01:00
Jonathan Hung
314e2f9d2e YARN-10116. Expose diagnostics in RMAppManager summary 2020-02-04 17:44:05 -08:00
Giovanni Matteo Fumarola
bf8686f43f YARN-8982. [Router] Add locality policy. Contributed by Young Chen. 2020-01-30 16:59:36 -08:00
Szilard Nemeth
a7d72c523a YARN-10099. FS-CS converter: handle allow-undeclared-pools and user-as-default-queue properly and fix misc issues. Contributed by Peter Bacsko 2020-01-30 16:03:38 +01:00
Akira Ajisaka
a5ef08b619
YARN-9743. [JDK11] TestTimelineWebServices.testContextFactory fails. (#1824) Contributed by Akira Ajisaka and Kinga Marton. 2020-01-30 14:10:31 +09:00
Prabhu Joseph
825db8fe2a YARN-10107. Fix GpuResourcePlugin#getNMResourceInfo to honor Auto Discovery Enabled 
Contributed by Szilard Nemeth.
 2020-01-29 13:30:00 +05:30
Eric Badger
e578e52aae YARN-10084. Allow inheritance of max app lifetime / default app lifetime. Contributed by Eric Payne. 2020-01-29 03:54:43 +00:00
Inigo Goiri
5abd0148eb YARN-9768. RM Renew Delegation token thread should timeout and retry. Contributed by Manikandan R. 2020-01-28 10:06:37 -08:00
Prabhu Joseph
1ab9c692fa YARN-10022. RM Rest API to validate the CapacityScheduler Configuration change 
Contributed by Kinga Marton.
 2020-01-28 23:16:04 +05:30
Szilard Nemeth
94f060242e YARN-10104. FS-CS converter: dry run should work without output defined. Contributed by Peter Bacsko 2020-01-28 15:36:04 +01:00
Szilard Nemeth
ca29768035 YARN-10085. FS-CS converter: remove mixed ordering policy check. Contributed by Peter Bacsko 2020-01-28 15:22:48 +01:00
Szilard Nemeth
9520b2ad79 YARN-10083. Provide utility to ask whether an application is in final status. Contributed by Adam Antal 2020-01-22 16:25:07 +01:00
Inigo Goiri
b4870bce3a Revert "YARN-9768. RM Renew Delegation token thread should timeout and retry. Contributed by Manikandan R." 
This reverts commit 0696828a09.
 2020-01-21 17:45:17 -08:00
Inigo Goiri
0696828a09 YARN-9768. RM Renew Delegation token thread should timeout and retry. Contributed by Manikandan R. 2020-01-21 13:41:01 -08:00
Akira Ajisaka
1defe3a65a
HADOOP-16753. Refactor HAAdmin. Contributed by Xieming Li. 2020-01-21 10:58:32 +09:00
Szilard Nemeth
581072a8f0 YARN-7913. Improve error handling when application recovery fails with exception. Contributed by Wilfred Spiegelenburg 2020-01-20 13:10:32 +01:00
Szilard Nemeth
6d52bbbfcf YARN-9525. IFile format is not working against s3a remote folder. Contributed by Adam Antal 2020-01-20 12:36:55 +01:00
Szilard Nemeth
8b3ee2f7e9 YARN-9462. TestResourceTrackerService.testNodeRemovalGracefully fails sporadically. Contributed by Prabhu Joseph 2020-01-20 09:54:22 +01:00
Szilard Nemeth
57aad0f43a YARN-10081. Exception message from ClientRMProxy#getRMAddress is misleading. Contributed by Ravuri Sushma Sree 2020-01-20 09:41:06 +01:00
Szilard Nemeth
14d0f9a775 YARN-8148. Update decimal values for queue capacities shown on queue status CLI. Contributed by Prabhu Joseph 2020-01-20 09:28:23 +01:00
Prabhu Joseph
a0ff42d761 YARN-10070. Fix NPE if no queue mapping defined for proxy user when using application tag based placement. 
Contributed by Kinga Marton. Reviewed by Adam Antal.
 2020-01-16 12:55:27 +05:30
Akira Ajisaka
14c2c3d69d
YARN-9512. [JDK11] TestAuxServices#testCustomizedAuxServiceClassPath fails because of ClassCastException. (#1806) 2020-01-16 12:21:50 +09:00
Szilard Nemeth
2aa065d98f YARN-10082. FS-CS converter: disable terminal placement rule checking. Contributed by Peter Bacsko 2020-01-15 12:42:13 +01:00
Szilard Nemeth
7c5cecc3b3 YARN-9970. Refactor TestUserGroupMappingPlacementRule#verifyQueueMapping. Contributed by Manikandan R 2020-01-15 09:55:41 +01:00
Szilard Nemeth
1c51f36be7 YARN-9788. Queue Management API does not support parallel updates. Contributed by Prabhu Joseph 2020-01-14 12:26:03 +01:00
Szilard Nemeth
13cea0412c YARN-10028. Integrate the new abstract log servlet to the JobHistory server. Contributed by Adam Antal 2020-01-14 11:00:08 +01:00
Szilard Nemeth
621c5eac38 YARN-9912. Capacity scheduler: support u:user2:%secondary_group queue mapping. Contributed by Manikandan R 2020-01-13 17:23:00 +01:00
Szilard Nemeth
b7ef8a333f YARN-9868. Validate %primary_group queue in CS queue manager. Contributed by Manikandan R 2020-01-13 17:15:09 +01:00
Szilard Nemeth
2576c31644 YARN-9989. Typo in CapacityScheduler documentation: Runtime Configuration. Contributed by Kevin Su 2020-01-13 16:50:07 +01:00
Szilard Nemeth
d842dfffa5 YARN-9866. u:user2:%primary_group is not working as expected. Contributed by Manikandan R 2020-01-12 14:04:15 +01:00
Szilard Nemeth
24e6a9e43a YARN-10067. Add dry-run feature to FS-CS converter tool. Contributed by Peter Bacsko 2020-01-12 13:48:39 +01:00
Eric Yang
2301b25899 YARN-10019. Improved container-executor exec() calls. 
Contributed by Peter Bacsko
 2020-01-10 19:04:04 -05:00
Adam Antal
20a90c0b5b
YARN-10071. Sync Mockito version with other modules 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2020-01-10 17:41:04 +09:00
Eric E Payne
93233a7d6e YARN-9018. Add functionality to AuxiliaryLocalPathHandler to return all locations to read for a given path. Contributed by Kuhu Shukla (kshukla) 2020-01-09 17:18:44 +00:00
Eric E Payne
b1e07d27cc YARN-7387: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestIncreaseAllocationExpirer fails intermittently. Contributed by Jim Brennan (Jim_Brennan) 2020-01-08 19:26:01 +00:00
Eric E Payne
6899be5a17 YARN-10072: TestCSAllocateCustomResource failures. Contributed by Jim Brennan (Jim_Brennan) 2020-01-08 17:29:56 +00:00
Prabhu Joseph
571795cd18 YARN-10068. Fix TimelineV2Client leaking File Descriptors. 
Contributed by Anand Srinivasan. Reviewed by Adam Antal.
 2020-01-08 12:01:30 +05:30
Eric Yang
d81d45ff2f YARN-9956. Improved connection error message for YARN ApiServerClient. 
Contributed by Prabhu Joseph
 2020-01-06 13:24:16 -05:00
Szilard Nemeth
dd2607e3ec YARN-10026. Pull out common code pieces from ATS v1.5 and v2. Contributed by Adam Antal 2020-01-06 17:16:11 +01:00
Szilard Nemeth
768ee22e9e YARN-10035. Add ability to filter the Cluster Applications API request by name. Contributed by Adam Antal 2020-01-06 16:26:33 +01:00
Prabhu Joseph
eca7e14c2f YARN-10053. Use Shared Group Mapping Service in Placement Rules. 
Contributed by Wilfred Spiegelenburg.
 2020-01-02 14:13:57 +05:30
Prabhu Joseph
21ada4d1b0 Revert "YARN-10053. Use Shared Group Mapping Service in Placement Rules." 
This reverts commit 217b56ffdd.
 2020-01-02 14:12:43 +05:30
Prabhu Joseph
217b56ffdd YARN-10053. Use Shared Group Mapping Service in Placement Rules. 
Contributed by Wilfred Spiegelenburg.
 2020-01-02 14:07:49 +05:30
Liu sheng
0fed874adf YARN-10041. Should not use AbstractPath to create unix domain socket (#1771) 2019-12-27 16:50:15 +05:30
Akira Ajisaka
40887c9b12
YARN-10057. Upgrade the dependencies managed by yarnpkg. (#1780) 2019-12-25 14:10:44 +09:00
Akira Ajisaka
34ff7dbaf5
YARN-10055. bower install fails. (#1778) 2019-12-23 19:45:47 +09:00
Eric Badger
412035b47a YARN-10009. In Capacity Scheduler, DRC can treat minimum user limit percent as a max when custom resource is defined. Contributed by Eric Payne. 2019-12-20 18:07:00 +00:00
Zhankun Tang
12722ab0c7 YARN-10042. Upgrade grpc-xxx depdencies to 1.26.0. Contributed by Sheng Liu. 2019-12-20 11:10:27 +08:00
Giovanni Matteo Fumarola
ef59ffd362 YARN-10038. [UI] Finish Time is not correctly parsed in the RM Apps page. Contributed by Inigo Goiri. 2019-12-19 11:37:17 -08:00
Masatake Iwasaki
7868da894a YARN-10037. Upgrade build tools for YARN Web UI v2. 2019-12-19 19:27:31 +09:00
Jonathan Hung
7b93575b92 YARN-9894. CapacitySchedulerPerf test for measuring hundreds of apps in a large number of queues. Contributed by Eric Payne 2019-12-18 13:18:11 -08:00
Jonathan Hung
fddc3d55c3 YARN-10039. Allow disabling app submission from REST endpoints 2019-12-18 10:48:05 -08:00
Masatake Iwasaki
92c8962f98
YARN-10020. Fix build instruction of hadoop-yarn-ui. (#1750) 2019-12-18 13:34:12 +09:00
Eric Badger
f47dcf2d4c YARN-10033. TestProportionalCapacityPreemptionPolicy not initializing vcores for effective max resources. Contributed by Eric Payne. 2019-12-17 17:11:23 +00:00
Szilard Nemeth
631dbbc6f2 YARN-9923. Introduce HealthReporter interface to support multiple health checker files. Contributed by Adam Antal 2019-12-15 17:28:04 +01:00
prabhujoseph
d4bde134e3 YARN-7721. Fix TestContinuousScheduling fails sporadically with NPE. 
Contributed by Wilfred Spiegelenburg.
 2019-12-10 15:00:23 +05:30
Akira Ajisaka
dc66de7448
YARN-9985. Unsupported transitionToObserver option displaying for rmadmin command. Contributed by Ayush Saxena. 2019-12-09 18:37:34 +09:00
dependabot[bot]
e9c5bb8078 Bump bower from 1.7.7 to 1.8.8 (#1683) 
Bumps [bower](https://github.com/bower/bower) from 1.7.7 to 1.8.8.
- [Release notes](https://github.com/bower/bower/releases)
- [Changelog](https://github.com/bower/bower/blob/master/CHANGELOG.md)
- [Commits](https://github.com/bower/bower/compare/v1.7.7...v1.8.8)

Signed-off-by: dependabot[bot] <support@github.com>
 2019-12-09 15:46:08 +09:00
Eric Badger
8dffd8dc89 YARN-9561. Add C changes for the new RuncContainerRuntime. Contributed by Eric Badger 2019-12-09 01:25:10 +00:00
Eric Badger
ab2cc5ac92 Revert "YARN-9561. Add C changes for the new RuncContainerRuntime. Contributed by Eric Badger" 
This reverts commit 76bb297db9.
 2019-12-09 01:25:10 +00:00
Jonathan Hung
6d970d8100 YARN-10012. Guaranteed and max capacity queue metrics for custom resources. Contributed by Manikandan R 2019-12-08 16:35:02 -08:00
Jonathan Hung
cd4779f317 Revert "YARN-10012. Guaranteed and max capacity queue metrics for custom resources" 
This reverts commit 92bce918dc.
 2019-12-08 16:34:46 -08:00
Jonathan Hung
92bce918dc YARN-10012. Guaranteed and max capacity queue metrics for custom resources 2019-12-08 15:52:17 -08:00
Eric Badger
76bb297db9 YARN-9561. Add C changes for the new RuncContainerRuntime. Contributed by Eric Badger 2019-12-06 18:33:58 +00:00
prabhujoseph
e575df5ca6 YARN-10006. IOException used in place of YARNException in CapaitySheduler 
Contributed by Adam Antal. Reviewed by Szilard Nemeth.
 2019-12-06 12:51:28 +05:30
Szilard Nemeth
4f758dd4c6 YARN-9607. Auto-configuring rollover-size of IFile format for non-appendable filesystems. Contributed by Adam Antal 2019-12-05 21:49:00 +01:00
Szilard Nemeth
cf68857631 YARN-9938. Validate Parent Queue for QueueMapping contains dynamic group as parent queue. Contributed by Manikandan R 2019-12-05 21:04:42 +01:00
Szilard Nemeth
0ab9c0398d YARN-9969. Improve yarn.scheduler.capacity.queue-mappings documentation. Contributed by Manikandan R 2019-12-05 20:53:43 +01:00
Szilard Nemeth
5cc6f945da YARN-9789. Disable Option for Write Ahead Logs of LogMutation. Contributed by Prabhu Joseph 2019-12-05 20:43:26 +01:00
Szilard Nemeth
4627dd6708 YARN-9780. SchedulerConf Mutation API does not Allow Stop and Remove Queue in a single call. Contributed by Prabhu Joseph 2019-12-05 20:38:37 +01:00
Szilard Nemeth
c71befaf8f YARN-9781. SchedConfCli to get current stored scheduler configuration. Contributed by Prabhu Joseph 2019-12-05 20:31:24 +01:00
Szilard Nemeth
520fe2c99b YARN-5106. Provide a builder interface for FairScheduler allocations for use in tests. Contributed by Adam Antal 2019-12-05 17:37:40 +01:00
Sunil G
682e6fdeda YARN-9052. Replace all MockRM submit method definitions with a builder. Contributed by Szilard Nemeth. 2019-12-05 13:26:23 +05:30
Wanqiang Ji
c48de9aa2d YARN-9958. Remove the invalid lock in ContainerExecutor (#1704) 2019-12-04 14:35:41 +09:00
Abhishek Modi
a2dadac790 YARN-9990. Testcase fails with Insufficient configured threads: required=16 < max=10. Contributed by Prabhu Joseph. 2019-11-29 11:54:33 +05:30
Szilard Nemeth
ef950b0863 YARN-9290. Invalid SchedulingRequest not rejected in Scheduler PlacementConstraintsHandler. Contributed by Prabhu Joseph 2019-11-26 22:04:07 +01:00
Szilard Nemeth
828ab400ee YARN-9362. Code cleanup in TestNMLeveldbStateStoreService. Contributed by Denes Gerencser 2019-11-26 21:45:12 +01:00
prabhujoseph
aa7ab2719f YARN-9991. Fix Application Tag prefix to userid. Contributed by Szilard Nemeth. 2019-11-27 01:59:12 +05:30
Szilard Nemeth
8c9018d5c7 YARN-9899. Migration tool that help to generate CS config based on FS config [Phase 2]. Contributed by Peter Bacsko 2019-11-26 21:22:35 +01:00
Szilard Nemeth
52e9ee39a1 YARN-9444. YARN API ResourceUtils's getRequestedResourcesFromConfig doesn't recognize yarn.io/gpu as a valid resource. Contributed by Gergely Pollak 2019-11-26 16:42:59 +01:00
Szilard Nemeth
448ffb12ec YARN-9937. addendum: Add missing queue configs in RMWebService#CapacitySchedulerQueueInfo. Contributed by Prabhu Joseph 2019-11-26 13:41:41 +01:00
Eric Badger
6c20512b68 Revert "YARN-9561. Add C changes for the new RuncContainerRuntime. Contributed by Eric Badger" 
This reverts commit 289bbca870.
 2019-11-25 17:46:42 +00:00
HUAN-PING SU
f8e36e03b4 YARN-9966. Code duplication in UserGroupMappingPlacementRule (#1709) 2019-11-25 15:28:53 +09:00
Szilard Nemeth
4c1a1287bc YARN-9968. Public Localizer is exiting in NodeManager due to NullPointerException. Contributed by Tarun Parimi 2019-11-22 12:58:10 +01:00
Tao Yang
8afabe41e4 YARN-9838. Fix resource inconsistency for queues when moving app with reserved container to another queue. Contributed by jiulongzhu. 2019-11-22 16:06:14 +08:00
Szilard Nemeth
dfdc6d6dd9 YARN-9886. Queue mapping based on userid passed through application tag. Contributed by Julia Kinga Marton 2019-11-19 21:14:15 +01:00
Eric Yang
ea8ffac121 YARN-9983. Fixed typo in YARN Service overview. 
Contributed by Denes Gerencser
 2019-11-19 14:16:51 -05:00
Prabhu Joseph
30b2096933 YARN-9986. Allow app owner to signal container via RMWebServices. Contributed by kyungwan nam. 2019-11-19 11:35:00 +00:00
Sunil G
ea68756c0c YARN-8373. RM Received RMFatalEvent of type CRITICAL_THREAD_CRASH. Contributed by Wilfred Spiegelenburg. 2019-11-19 14:10:41 +05:30
Sunil G
215f2052fc YARN-9984. FSPreemptionThread can cause NullPointerException while app is unregistered with containers running on a node. Contributed by Wilfred Spiegelenburg. 2019-11-19 14:03:02 +05:30
Abhishek Modi
dc3f4fc2f4 YARN-9965. Fix NodeManager failing to start on subsequent times when Hdfs Auxillary Jar is set (addendum). Contributed by Prabhu Joseph. 2019-11-19 08:08:40 +05:30
Eric Badger
0e22e9ab83 YARN-9562. Add Java changes for the new RuncContainerRuntime. Contributed by Eric Badger 2019-11-18 21:56:31 +00:00
Eric Badger
289bbca870 YARN-9561. Add C changes for the new RuncContainerRuntime. Contributed by Eric Badger 2019-11-18 21:54:16 +00:00
Szilard Nemeth
27642367ef YARN-9011. Race condition during decommissioning. Contributed by Peter Bacsko 2019-11-18 16:29:39 +01:00
prabhujoseph
bd454348b0 YARN-9982. Fix Container API example link in NodeManager REST API doc. Contributed by Charan Hebri. 2019-11-18 15:19:47 +05:30
Yufei Gu
b83b9ab418 YARN-9537. Add configuration to disable AM preemption. Contributed by ZhouKang 2019-11-12 09:40:16 -08:00
Abhishek Modi
fb512f5087 YARN-9697. Efficient allocation of Opportunistic containers. Contributed by Abhishek Modi. 2019-11-12 16:34:04 +05:30
Szilard Nemeth
30b93f914b YARN-9865. Capacity scheduler: add support for combined %user + %secondary_group mapping. Contributed by Manikandan R 2019-11-11 13:28:02 +01:00
Abhishek Modi
516377bfa6 YARN-9965. Fix NodeManager failing to start when Hdfs Auxillary Jar is set. Contributed by Prabhu Joseph. 2019-11-11 16:01:44 +05:30
Szilard Nemeth
ceb9c6175e YARN-9890. [UI2] Add Application tag to the app table and app detail page. Contributed by Julia Kinga Marton 2019-11-09 14:12:04 +01:00
Szilard Nemeth
31f172fd96 YARN-9677. Make FpgaDevice and GpuDevice classes more similar to each other. Contributed by kevin su 2019-11-09 14:04:14 +01:00