big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
27,425 Commits 2 Branches 2 Tags 264 MiB
b078f86d69
Commit Graph

469 Commits

Author SHA1 Message Date
slfan1989
7cb22eb72d
YARN-11371. [Federation] Refactor FederationInterceptorREST#createNewApplication\submitApplication Use FederationActionRetry. (#5130) 2022-11-22 14:38:24 -08:00
slfan1989
eccd2d0492
YARN-11359. [Federation] Routing admin invocations transparently to multiple RMs. (#5057) 2022-11-16 18:00:38 -08:00
slfan1989
b90dfdff3f
YARN-11366. Improve equals, hashCode(), toString() methods of the Federation Base Object. (#5096) 2022-11-03 21:33:53 -07:00
slfan1989
9adf0ca089
YARN-11342. [Federation] Refactor getNewApplication, submitApplication Use FederationActionRetry. (#5005) 2022-10-20 09:22:24 -07:00
slfan1989
48b6f9f335
YARN-11328. Refactoring part of the code of SQLFederationStateStore. (#4976) 2022-10-19 16:11:28 -07:00
slfan1989
1962851356
YARN-11294. [Federation] Router Support DelegationToken store/update/remove Token With MemoryStateStore. (#4915) 2022-10-13 16:52:22 -07:00
slfan1989
1ff7e84caf
YARN-11334. Improve SubClusterState#fromString parameter and LogMessage. (#4988). Contributed by fanshilun. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2022-10-13 12:06:24 +05:30
slfan1989
d78b0b39a6
YARN-11323. [Federation] Improve ResourceManager Handler FinishApps. (#4954) 2022-10-11 14:53:02 -07:00
slfan1989
a708ff96f1
YARN-11324. Fix some PBImpl classes to avoid NPE. (#4961) 2022-10-04 14:30:20 -04:00
slfan1989
22bd5e3b53
YARN-11238. Optimizing FederationClientInterceptor Call with Parallelism. (#4904) 2022-10-04 09:17:00 -07:00
slfan1989
bfd6415827
YARN-11290. Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster. (#4846) 2022-09-27 13:28:52 -07:00
slfan1989
aeba204fa2
YARN-11306. [Federation] Refactor NM#FederationInterceptor#recover Code. (#4897) 2022-09-26 15:46:06 -07:00
slfan1989
e526f48fa4
YARN-11296. [Federation] Fix SQLFederationStateStore#Sql script bug. (#4858) 2022-09-22 16:40:47 -07:00
slfan1989
342c4856b8
YARN-11293. [Federation] StoreNewMasterKey/removeStoredMasterKey With MemoryStateStore. (#4852) 2022-09-19 13:14:55 -07:00
slfan1989
1965708d49
YARN-11273. Federation StateStore: Support storage/retrieval of Reservations With SQL. (#4817) 2022-09-02 10:39:58 -07:00
slfan1989
b266f852d7
YARN-11284. [Federation] Improve UnmanagedAMPoolManager WithoutBlock ServiceStop (#4814) 2022-09-02 10:28:38 -07:00
slfan1989
33edbed54c
YARN-11272. Federation StateStore: Support storage/retrieval of Reservations With Zk. (#4781) 2022-08-31 10:15:15 -07:00
slfan1989
c60a900583
YARN-11275. [Federation] Add batchFinishApplicationMaster in UAMPoolManager. (#4792) 2022-08-27 10:17:00 -07:00
slfan1989
eda4bb5dcd
YARN-11250. Capture the Performance Metrics of ZookeeperFederationStateStore. (#4738) 2022-08-22 14:09:20 -07:00
slfan1989
f75c58a1ca
YARN-11252. Yarn Federation Router Supports Update / Delete Reservation in MemoryStore. (#4741) 2022-08-18 21:13:43 -07:00
slfan1989
eff3b8c59a
YARN-10885. Make FederationStateStoreFacade#getApplicationHomeSubCluster use JCache. (#4701) 2022-08-15 13:46:40 -07:00
slfan1989
d383cc4525
YARN-11236. Implement FederationReservationHomeSubClusterStore With MemoryStore. (#4711) 2022-08-13 10:37:20 -07:00
slfan1989
6ca2d3f848
YARN-6539. Create SecureLogin inside Router. (#4712) 2022-08-11 13:25:51 -07:00
slfan1989
ffa9ed93a4
YARN-6572. Refactoring Router services to use common util classes for pipeline creations. (#4594) 2022-08-09 14:44:29 -07:00
slfan1989
6f7c4c74ea
YARN-11235. Refactor Policy Code and Define getReservationHomeSubcluster (#4656) 2022-08-04 10:16:08 -07:00
slfan1989
5c84cb81ba
YARN-8900. [Router] Federation: routing getContainers REST invocations transparently to multiple RMs (#4543) 2022-07-22 17:06:38 -07:00
Viraj Jasani
0733e968ab
HADOOP-18224. Upgrade maven compiler plugin to 3.10.1 (#4267) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2022-05-21 03:21:13 +09:00
Ashutosh Gupta
ae47846a5b
HADOOP-16515. Update the link to compatibility guide (#4226) 
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2022-05-08 07:38:32 +09:00
minni31
4b26635a34
YARN-11034. Add enhanced headroom in AllocateResponse (#3766) 2022-01-25 09:25:18 -08:00
Andrew Chung
d3f0b7eab7
YARN-10760. Number of allocated OPPORTUNISTIC containers can dip below 0 (#3642) 2021-11-23 13:21:51 -08:00
Viraj Jasani
c0bdba8fac
HADOOP-18017. unguava: remove Preconditions from hadoop-yarn-project modules (#3687) 2021-11-23 13:36:22 +09:00
Viraj Jasani
d336227e5c
HADOOP-17963. Replace Guava VisibleForTesting by Hadoop's own annotation in hadoop-yarn-project modules (#3541) 2021-10-14 18:03:01 +09:00
Viraj Jasani
1151edf12e
HADOOP-17956. Replace all default Charset usage with UTF-8 (#3529) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-10-14 13:07:24 +09:00
Jack Jiang
d8ec8ab965
HADOOP-17599. Remove NULL checks before instanceof (#2804) 2021-03-23 08:46:11 -07:00
adamantal
3234e5eaf3 YARN-10031. Create a general purpose log request with additional query parameters. Contributed by Andras Gyori 2020-12-12 09:42:22 +01:00
Ayush Saxena
1e3a6efcef
HADOOP-17288. Use shaded guava from thirdparty. (#2342). Contributed by Ayush Saxena. 2020-10-17 12:01:18 +05:30
Masatake Iwasaki
5e12dc51f0
YARN-10419. Javadoc error in hadoop-yarn-server-common module. (#2271) 2020-09-03 16:17:52 +09:00
Szilard Nemeth
4ffe26f9b8 [UI1] Provide a way to hide Tools section in Web UIv1. Contributed by Andras Gyori 2020-08-31 16:00:39 +02:00
Adam Antal
82a7505646 YARN-10304. Create an endpoint for remote application log directory path query. Contributed by Andras Gyori 2020-08-25 13:29:12 +02:00
hemanthboyina
86bbd38c8d YARN-10336. RM page should throw exception when command injected in RM REST API to get applications. Contributed by Bilwa S T. 2020-08-13 23:44:42 +05:30
Jonathan Hung
64753addba YARN-10251. Show extended resources on legacy RM UI. Contributed by Eric Payne 2020-08-07 17:43:01 -07:00
Prabhu Joseph
cc71d50b21 YARN-10339. Fix TimelineClient in NodeManager failing when Simple Http Auth used in Secure Cluster 
Contributed by Tarun Parimi.
 2020-07-16 23:36:49 +05:30
Eric Yang
e8dc862d38 YARN-9809. Added node manager health status to resource manager registration call. 
Contributed by Eric Badger via eyang
 2020-06-30 11:39:16 -07:00
Brahma Reddy Battula
2c03524fa4 YARN-6526. Refactoring SQLFederationStateStore by avoiding to recreate a connection at every call. COntributed by Bilwa S T. 2020-06-26 20:43:27 +05:30
Szilard Nemeth
6459cc73e6 YARN-10318. ApplicationHistory Web UI incorrect column indexing. Contributed by Andras Gyori 2020-06-26 13:15:14 +02:00
Vinayakumar B
e154084770
HADOOP-17046. Support downstreams' existing Hadoop-rpc implementations using non-shaded protobuf classes (#2026) 2020-06-12 23:16:33 +05:30
Szilard Nemeth
aa6d13455b YARN-10284. Add lazy initialization of LogAggregationFileControllerFactory in LogServlet. Contributed by Adam Antal 2020-06-02 15:53:48 +02:00
Prabhu Joseph
cef0756929 YARN-9606. Set sslfactory for AuthenticatedURL() while creating LogsCLI#webServiceClient. 
Contributed by Bilwa S T. Reviewed by Akira Ajisaka.
 2020-05-20 11:36:52 +05:30
Inigo Goiri
108ecf992f YARN-8942. PriorityBasedRouterPolicy throws exception if all sub-cluster weights have negative value. Contributed by Bilwa S T. 2020-05-13 10:04:12 -07:00
Szilard Nemeth
ab36429559 YARN-10215. Endpoint for obtaining direct URL for the logs. Contributed by Andras Gyori 2020-04-28 18:53:28 +02:00
Prabhu Joseph
7a39da3537 YARN-10120. Amendment fix for Java Doc. 
Contributed by Bilwa S T.
 2020-04-03 13:07:41 +05:30
Prabhu Joseph
25361b077b YARN-10120. Https Support in Router WebServiceClient. 
Contributed by Bilwa S T.
 2020-04-02 18:11:43 +05:30
Vinayakumar B
7dac7e1d13
HADOOP-16596. [pb-upgrade] Use shaded protobuf classes from hadoop-thirdparty dependency (#1635). Contributed by Vinayakumar B. 2020-02-07 14:51:24 +05:30
Szilard Nemeth
71b2c2ffe9 YARN-10101. Support listing of aggregated logs for containers belonging to an application attempt. Contributed by Adam Antal 2020-02-06 12:25:06 +01:00
Giovanni Matteo Fumarola
bf8686f43f YARN-8982. [Router] Add locality policy. Contributed by Young Chen. 2020-01-30 16:59:36 -08:00
Szilard Nemeth
9520b2ad79 YARN-10083. Provide utility to ask whether an application is in final status. Contributed by Adam Antal 2020-01-22 16:25:07 +01:00
Szilard Nemeth
13cea0412c YARN-10028. Integrate the new abstract log servlet to the JobHistory server. Contributed by Adam Antal 2020-01-14 11:00:08 +01:00
Szilard Nemeth
dd2607e3ec YARN-10026. Pull out common code pieces from ATS v1.5 and v2. Contributed by Adam Antal 2020-01-06 17:16:11 +01:00
Szilard Nemeth
768ee22e9e YARN-10035. Add ability to filter the Cluster Applications API request by name. Contributed by Adam Antal 2020-01-06 16:26:33 +01:00
Giovanni Matteo Fumarola
ef59ffd362 YARN-10038. [UI] Finish Time is not correctly parsed in the RM Apps page. Contributed by Inigo Goiri. 2019-12-19 11:37:17 -08:00
Abhishek Modi
fb512f5087 YARN-9697. Efficient allocation of Opportunistic containers. Contributed by Abhishek Modi. 2019-11-12 16:34:04 +05:30
Abhishek Modi
4d3c580b03 YARN-9859. Refactoring of OpportunisticContainerAllocator. Contributed by Abhishek Modi. 2019-09-30 23:40:15 +05:30
Duo Zhang
bfe1dac29d
HADOOP-16560. [YARN] use protobuf-maven-plugin to generate protobuf classes (#1496) 
Signed-off-by:  Vinayakumar B <vinayakumarb@apache.org>
 2019-09-24 08:50:36 +08:00
hunshenshi
22d7d1f8bf YARN-9601.Potential NPE in ZookeeperFederationStateStore#getPoliciesConfigurations (#908) Contributed by hunshenshi. 2019-08-07 21:26:14 -07:00
hunshenshi
b1dafc3506 YARN-9661:Fix typo in LocalityMulticastAMRMProxyPolicy.java and AbstractConfigurableFederationPolicy.java (#1042) 2019-07-01 10:46:33 -07:00
Eric Yang
294695dd57 HADOOP-16314. Make sure all web end points are covered by the same authentication filter. 
Contributed by Prabhu Joseph
 2019-06-05 18:55:13 -04:00
Weiwei Yang
433e97cd34 YARN-9600. Support self-adaption width for columns of containers table on app attempt page. Contributed by Tao Yang. 2019-06-05 13:55:30 +08:00
Eric E Payne
3c63551101 YARN-8625. Aggregate Resource Allocation for each job is not present in ATS. Contributed by Prabhu Joseph. 2019-05-29 16:05:39 +00:00
Giovanni Matteo Fumarola
12c81610e0 YARN-9505. Add container allocation latency for Opportunistic Scheduler. Contributed by Abhishek Modi. 2019-05-17 12:03:21 -07:00
Giovanni Matteo Fumarola
1b48100a5e YARN-9522. AppBlock ignores full qualified class name of PseudoAuthenticationHandler. Contributed by Prabhu Joseph. 2019-05-09 14:02:58 -07:00
Eric E Payne
b094b94d43 YARN-9285: RM UI progress column is of wrong type. Contributed by Ahmed Hussein. 2019-05-02 19:39:26 +00:00
Inigo Goiri
96e3027e46 YARN-2889. Limit the number of opportunistic container allocated per AM heartbeat. Contributed by Abhishek Modi. 2019-04-22 09:49:03 -07:00
Inigo Goiri
aeadb9432f YARN-9448. Fix Opportunistic Scheduling for node local allocations. Contributed by Abhishek Modi. 2019-04-19 09:41:06 -07:00
Giovanni Matteo Fumarola
ed3747c1cc YARN-9435. Add Opportunistic Scheduler metrics in ResourceManager. Contributed by Abhishek Modi. 2019-04-11 11:49:19 -07:00
Eric Yang
5f6e225166 YARN-9363. Replaced debug logging with SLF4J parameterized log message. 
Contributed by Prabhu Joseph
 2019-03-18 13:57:18 -04:00
Shweta Yakkali
0e7e9013d4 YARN-9340. [Clean-up] Remove NULL check before instanceof in ResourceRequestSetKey 
(Contributed by Shweta Yakkali via Daniel Templeton)

Change-Id: I932e29b36f086f7b7c76a250e33b473617ddbda1
 2019-03-18 15:08:37 +01:00
Eric Yang
2064ca015d YARN-9349. Changed logging to use slf4j api. 
Contributed by Prabhu Joseph
 2019-03-15 19:20:59 -04:00
Giovanni Matteo Fumarola
95372657fc YARN-9287. Consecutive StringBuilder append should be reuse. Contributed by Ayush Saxena. 2019-02-25 11:45:37 -08:00
bibinchundatt
b8de78c570 YARN-9286. [Timeline Server] Sorting based on FinalStatus shows pop-up message. Contributed by Bilwa S T. 2019-02-20 01:19:20 +05:30
Sunil G
ba56bc24f5 YARN-9213. RM Web UI v1 does not show custom resource allocations for containers page. Contributed by Szilard Nemeth. 2019-02-17 20:08:20 +05:30
Eric E Payne
d1ca9432dd YARN-7171: RM UI should sort memory / cores numerically. Contributed by Ahmed Hussein 2019-02-07 16:38:11 +00:00
Eric E Payne
04105bbfdb YARN-6616: YARN AHS shows submitTime for jobs same as startTime. Contributed by Prabhu Joseph 2019-01-29 17:22:24 +00:00
Jason Lowe
6a923464af YARN-6523. Optimize system credentials sent in node heartbeat responses. Contributed by Manikandan R 2019-01-08 16:54:05 -06:00
Eric Yang
f82922dcfa YARN-5168. Added exposed port information for Docker container. 
Contributed by Xun Liu
 2018-12-21 19:44:07 -05:00
Weiwei Yang
f659485ee8 YARN-8925. Updating distributed node attributes only when necessary. Contributed by Tao Yang. 2018-12-21 10:56:42 +08:00
Giovanni Matteo Fumarola
0d8406135f YARN-9054. Fix FederationStateStoreFacade#buildGetSubClustersCacheRequest. Contributed by Bibin A Chundatt. 2018-11-26 10:57:38 -08:00
Sunil G
5fb14e0635 YARN-8953. [CSI] CSI driver adaptor module support in NodeManager. Contributed by Weiwei Yang. 2018-11-20 08:02:11 +05:30
Rohith Sharma K S
ee3355be3c YARN-8303. YarnClient should contact TimelineReader for application/attempt/container report. 2018-11-16 17:58:19 +05:30
Sunil G
4e7284443e YARN-8902. [CSI] Add volume manager that manages CSI volume lifecycle. Contributed by Weiwei Yang. 2018-11-12 11:57:02 +05:30
Botong Huang
b5ec85d966 YARN-8933. [AMRMProxy] Fix potential empty fields in allocation response, move SubClusterTimeout to FederationInterceptor. Contributed by Botong Huang. 2018-11-11 11:12:53 -08:00
Giovanni Matteo Fumarola
989715ec50 YARN-8893. [AMRMProxy] Fix thread leak in AMRMClientRelayer and UAM client. Contributed by Botong Huang. 2018-11-02 15:30:08 -07:00
Giovanni Matteo Fumarola
aed836efbf YARN-8897. LoadBasedRouterPolicy throws NPE in case of sub cluster unavailability. Contributed by Bilwa S T. 2018-11-02 11:27:11 -07:00
Haibo Chen
c2288ac45b YARN-8448. AM HTTPS Support for AM communication with RMWeb proxy. (Contributed by Robert Kanter) 2018-10-16 13:36:26 -07:00
Vrushali C
8d1981806f YARN-5742 Serve aggregated logs of historical apps from timeline service. Contributed by Rohith Sharma KS 2018-10-11 16:26:07 -07:00
Inigo Goiri
c3d22d3b45 YARN-7652. Handle AM register requests asynchronously in FederationInterceptor. Contributed by Botong Huang. 2018-10-09 10:29:40 -07:00
Giovanni Matteo Fumarola
59d5af21b7 YARN-8760. [AMRMProxy] Fix concurrent re-register due to YarnRM failover in AMRMClientRelayer. Contributed by Botong Huang. 2018-10-01 13:12:38 -07:00
Giovanni Matteo Fumarola
3090922805 YARN-8696. [AMRMProxy] FederationInterceptor upgrade: home sub-cluster heartbeat async. Contributed by Botong Huang. 2018-09-24 11:37:05 -07:00
Giovanni Matteo Fumarola
02b9bfdf9e YARN-8658. [AMRMProxy] Metrics for AMRMClientRelayer inside FederationInterceptor. Contributed by Young Chen. 2018-09-12 11:46:35 -07:00
Naganarasimha
eb08543c7a YARN-8103. Add CLI interface to query node attributes. Contributed by Bibin A Chundatt. 2018-09-12 16:01:01 +05:30
Weiwei Yang
d9d93e3925 YARN-7842. PB changes to carry node-attributes in NM heartbeat. Contributed by Weiwei Yang. 2018-09-12 16:00:59 +05:30