big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
26,054 Commits 2 Branches 2 Tags 264 MiB
d383cc4525
Commit Graph

4526 Commits

Author SHA1 Message Date
slfan1989
d383cc4525
YARN-11236. Implement FederationReservationHomeSubClusterStore With MemoryStore. (#4711) 2022-08-13 10:37:20 -07:00
zhengchenyu
9f6bbc90a8
YARN-11148. In federation and security mode, nm recover may fail. (#4308) 2022-08-13 10:33:16 -07:00
slfan1989
6ca2d3f848
YARN-6539. Create SecureLogin inside Router. (#4712) 2022-08-11 13:25:51 -07:00
slfan1989
133e8aabf0
YARN-11227. [Federation] Add getAppTimeout, getAppTimeouts, updateApplicationTimeout REST APIs for Router. (#4715) 2022-08-10 14:53:46 -07:00
slfan1989
ffa9ed93a4
YARN-6572. Refactoring Router services to use common util classes for pipeline creations. (#4594) 2022-08-09 14:44:29 -07:00
Ashutosh Gupta
1cda2dcb6e
YARN-10793. Upgrade Junit from 4 to 5 in hadoop-yarn-server-applicationhistoryservice (#4603) 
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2022-08-07 16:15:47 +09:00
slfan1989
52c2d99889
YARN-11228. [Federation] Add getAppAttempts, getAppAttempt REST APIs for Router. (#4695) 2022-08-06 09:36:26 -07:00
slfan1989
6f7c4c74ea
YARN-11235. Refactor Policy Code and Define getReservationHomeSubcluster (#4656) 2022-08-04 10:16:08 -07:00
slfan1989
c5ec727435
YARN-11230. [Federation] Add getContainer, signalToContainer REST APIs for Router. (#4689) 2022-08-03 11:21:48 -07:00
slfan1989
6463f86f83
YARN-11029. Refactor AMRMProxy Service code and Added Some Metrics. (#4650) 2022-08-03 09:38:00 -07:00
slfan1989
c5eba323bc
YARN-6972. Adding RM ClusterId in AppInfo. (#4673) 2022-08-03 09:35:40 -07:00
slfan1989
1f0a71a92b
YARN-8973. [Router] Add missing methods in RMWebProtocol. (#4664) 2022-08-02 14:07:09 -07:00
slfan1989
57da4bb0a1
YARN-11220. [Federation] Add getLabelsToNodes, getClusterNodeLabels, getLabelsOnNode REST APIs for Router (#4657) 2022-08-02 12:09:55 -07:00
slfan1989
2680f17eb4
YARN-11180. Refactor some code of getNewApplication, submitApplication etc. (#4618) 2022-07-29 08:23:11 -07:00
slfan1989
e994635a95
YARN-11212. [Federation] Add getNodeToLabels REST APIs for Router. (#4614) 2022-07-28 11:53:04 -07:00
9uapaw
bf570bd4ac YARN-11063. Support auto queue creation template wildcards for arbitrary queue depths. Contributed by Bence Kosztolnik. 2022-07-28 17:32:20 +02:00
slfan1989
bf8782d0ac
YARN-10883. [Router] Router Audit Log Add Client IP Address. (#4426) 2022-07-25 11:55:40 -07:00
slfan1989
edeb99548a
YARN-11161. Support getAttributesToNodes, getClusterNodeAttributes, getNodesToAttributes API's for Federation (#4610) 2022-07-25 10:05:45 -07:00
slfan1989
63db1a85e3
YARN-11203. Fix typo in hadoop-yarn-server-router module. (#4510). Contributed by fanshilun. 
Reviewed-by: Fei Hui <feihui.ustc@gmail.com>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2022-07-23 20:28:45 +05:30
slfan1989
5c84cb81ba
YARN-8900. [Router] Federation: routing getContainers REST invocations transparently to multiple RMs (#4543) 2022-07-22 17:06:38 -07:00
slfan1989
838020ce3b
YARN-11160. Support getResourceProfiles, getResourceProfile API's for Federation (#4540) 2022-07-21 11:57:24 -07:00
Szilard Nemeth
f4b635c4dc YARN-11211. QueueMetrics leaks Configuration objects when validation API is called multiple times. Contributed by Andras Gyori 2022-07-21 14:20:34 +02:00
PJ Fanning
34e548cb62
HADOOP-18332: remove rs-api dependency as it conflicts with jsr311-api (#4547) 
This downgrades jackson from the version switched to in
    HADOOP-18033 (2.13.0), to Jackson 2.12.7.
    This removes the dependency on javax.ws.rs-api,
    so avoiding runtime problems with applications using
    jersey-core v1 and/or jsr311-api.
    
    The 2.12.7 release still contains the fix for CVE-2020-36518.
    
    Contributed by PJ Fanning
 2022-07-17 21:37:54 +05:30
Samrat
84ce592a85
YARN-11198. clean up numa resources from statestore (#4546) 
* YARN-11198. clean up numa resources from levelDB

Co-authored-by: Deb <dbsamrat@3c22fba1b03f.ant.amazon.com>
 2022-07-14 11:07:48 +05:30
slfan1989
161b1fac2e
YARN-11169. Support moveApplicationAcrossQueues, getQueueInfo API's for Federation. (#4464) 2022-07-05 11:24:29 -07:00
Ashutosh Gupta
151bb31c47
YARN-9403.GET /apps/{appid}/entities/YARN_APPLICATION accesses application table instead of entity table (#4516) 
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
 2022-07-02 21:59:28 +05:30
Ashutosh Gupta
57cbde9abf
YARN-10287.Update scheduler-conf corrupts the CS configuration when removing queue which is referred in queue mapping (#4515) 
* YARN-10287.Update scheduler-conf corrupts the CS configuration when removing queue which is referred in queue mapping

Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
 2022-07-02 21:58:56 +05:30
Tamas Domok
3cad632709 YARN-11202. Optimize ClientRMService.getApplications. Contributed by Tamas Domok. 
Change-Id: I55ddb46fd0e4cdb644747d6d43083215f10861b5
 2022-07-01 10:50:48 +02:00
slfan1989
073b8ea1d5
HADOOP-18284. Remove Unnecessary semicolon ';' (#4422). Contributed by fanshilun. 2022-06-29 15:20:41 +05:30
Ashutosh Gupta
a177232ebc
YARN-9822.TimelineCollectorWebService#putEntities blocked when ATSV2 HBase is down (#4492) 
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
 2022-06-28 09:32:07 +05:30
Ashutosh Gupta
734b6f19ad
YARN-9874.Remove unnecessary LevelDb write call in LeveldbConfigurationStore#confirmMutation (#4487) 
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
 2022-06-23 21:59:27 +05:30
slfan1989
0af4bb3b42
YARN-11192. TestRouterWebServicesREST failing after YARN-9827. (#4484). Contributed by fanshilun. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2022-06-23 13:21:36 +05:30
Szilard Nemeth
3a66348fda YARN-11185. Pending app metrics are increased doubly when a queue reaches its max-parallel-apps limit. Contributed by Andras Gyori 2022-06-20 15:03:58 +02:00
slfan1989
62e4476102
YARN-10122. Support signalToContainer API for Federation. (#4421) 2022-06-17 16:38:36 -07:00
zhengchenyu
80446dcd08
YARN-11172. Fix TestClientRMTokens#testDelegationToken introduced by HDFS-16563. (#4408) 
Regression caused by HDFS-16563; the hdfs exception text was changed, but because it was
a YARN test doing the check, Yetus didn't notice.

Contributed by zhengchenyu
 2022-06-17 19:49:36 +01:00
Benjamin Teke
020201cb65 Queue filter in CS UI v1 does not work as expected. Contributed by Chengbing Liu. 2022-06-17 19:28:32 +02:00
slfan1989
98ca6fa10a
YARN-11159. Support failApplicationAttempt, updateApplicationPriority, updateApplicationTimeouts API's for Federation (#4396) 2022-06-08 17:34:43 -07:00
slfan1989
1cc2416f68
YARN-10487. Support getQueueUserAcls, listReservations, getApplicationAttempts, getContainerReport, getContainers, getResourceTypeInfo API's for Federation (#4341) 2022-06-02 12:54:29 -07:00
Ashutosh Gupta
7c66266b4e
YARN-11142.Remove unused Imports in Hadoop YARN project (#4381). Contributed by Ashutosh Gupta. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2022-05-31 23:17:11 +05:30
Ashutosh Gupta
e3e9369c1d
YARN-11128. Fix comments in TestProportionalCapacityPreemptionPolicy* (#4271) 
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Reviewed-by: Hemanth Boyina <hemanthboyina@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2022-05-26 15:51:55 +09:00
slfan1989
678768035d
YARN-11137. Improve log message in FederationClientInterceptor (#4336) 2022-05-24 17:21:04 -07:00
Owen O'Malley
f390edaec4
YARN-11162. Set the zk acl for nodes created by ZKConfigurationStore. (#4350) 2022-05-24 14:07:19 +09:00
slfan1989
8dd3ef1f08
YARN-10465. Support getNodeToLabels, getLabelsToNodes, getClusterNodeLabels API's for Federation (#4317) 2022-05-20 13:19:42 -07:00
Viraj Jasani
0733e968ab
HADOOP-18224. Upgrade maven compiler plugin to 3.10.1 (#4267) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2022-05-21 03:21:13 +09:00
9uapaw
0e6a6d1880 YARN-11152. QueueMetrics is leaking memory when creating a new queue during reinitialisation 2022-05-18 18:22:42 +02:00
9uapaw
54cd0174c0 YARN-11147. ResourceUsage and QueueCapacities classes provide node label iterators that are not thread safe 2022-05-18 18:16:34 +02:00
slfan1989
6896c35a8d
YARN-11122. Support getClusterNodes API in FederationClientInterceptor (#4274) 2022-05-15 09:16:06 -07:00
Ashutosh Gupta
6985f9aabe
YARN-10080. Support show app id on localizer thread pool (#4283) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2022-05-14 01:41:06 +09:00
Jian Chen
d2c9eb653a
YARN-11073. Avoid unnecessary preemption for tiny queues under certain corner cases (#4110) 
Co-authored-by: Jian Chen <jian.chen@airbnb.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2022-05-14 01:11:42 +09:00
Szilard Nemeth
eced5bec68 YARN-11123. ResourceManager webapps test failures due to org.apache.hadoop.metrics2.MetricsException and subsequent java.net.BindException: Address already in use. Contributed by Szilard Nemeth 2022-05-13 13:16:12 +02:00