big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
26,067 Commits 2 Branches 2 Tags 264 MiB
5cc8c574d1
Commit Graph

6491 Commits

Author SHA1 Message Date
Ashutosh Gupta
f02ff1afe2
YARN-11248. Add unit test for FINISHED_CONTAINERS_PULLED_BY_AM event on DECOMMISSIONING (#4721) 
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2022-08-16 19:07:05 +09:00
slfan1989
eff3b8c59a
YARN-10885. Make FederationStateStoreFacade#getApplicationHomeSubCluster use JCache. (#4701) 2022-08-15 13:46:40 -07:00
slfan1989
ab88e4b65d
YARN-11223. [Federation] Add getAppPriority, updateApplicationPriority REST APIs for Router. (#4733) 2022-08-14 19:22:16 -07:00
slfan1989
d383cc4525
YARN-11236. Implement FederationReservationHomeSubClusterStore With MemoryStore. (#4711) 2022-08-13 10:37:20 -07:00
zhengchenyu
9f6bbc90a8
YARN-11148. In federation and security mode, nm recover may fail. (#4308) 2022-08-13 10:33:16 -07:00
slfan1989
6ca2d3f848
YARN-6539. Create SecureLogin inside Router. (#4712) 2022-08-11 13:25:51 -07:00
slfan1989
133e8aabf0
YARN-11227. [Federation] Add getAppTimeout, getAppTimeouts, updateApplicationTimeout REST APIs for Router. (#4715) 2022-08-10 14:53:46 -07:00
slfan1989
ffa9ed93a4
YARN-6572. Refactoring Router services to use common util classes for pipeline creations. (#4594) 2022-08-09 14:44:29 -07:00
Ashutosh Gupta
92abd99450
YARN-11237. Fix Bug while disabling proxy failover with Federation (#4658) 2022-08-08 13:29:27 -07:00
slfan1989
d8d3325d2f
HADOOP-18387. Fix incorrect placeholder in hadoop-common (#4679). Contributed by fanshilun. 2022-08-08 02:35:39 +05:30
Ashutosh Gupta
1cda2dcb6e
YARN-10793. Upgrade Junit from 4 to 5 in hadoop-yarn-server-applicationhistoryservice (#4603) 
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2022-08-07 16:15:47 +09:00
slfan1989
52c2d99889
YARN-11228. [Federation] Add getAppAttempts, getAppAttempt REST APIs for Router. (#4695) 2022-08-06 09:36:26 -07:00
Steve Loughran
62dbefd8f2
HADOOP-18305. Release Hadoop 3.3.4: upstream changelog and jdiff files 
Add the r3.3.4 changelog, release notes and jdiff xml files.
 2022-08-05 14:06:22 +01:00
slfan1989
6f7c4c74ea
YARN-11235. Refactor Policy Code and Define getReservationHomeSubcluster (#4656) 2022-08-04 10:16:08 -07:00
slfan1989
c5ec727435
YARN-11230. [Federation] Add getContainer, signalToContainer REST APIs for Router. (#4689) 2022-08-03 11:21:48 -07:00
slfan1989
6463f86f83
YARN-11029. Refactor AMRMProxy Service code and Added Some Metrics. (#4650) 2022-08-03 09:38:00 -07:00
slfan1989
c5eba323bc
YARN-6972. Adding RM ClusterId in AppInfo. (#4673) 2022-08-03 09:35:40 -07:00
slfan1989
1f0a71a92b
YARN-8973. [Router] Add missing methods in RMWebProtocol. (#4664) 2022-08-02 14:07:09 -07:00
slfan1989
57da4bb0a1
YARN-11220. [Federation] Add getLabelsToNodes, getClusterNodeLabels, getLabelsOnNode REST APIs for Router (#4657) 2022-08-02 12:09:55 -07:00
slfan1989
2680f17eb4
YARN-11180. Refactor some code of getNewApplication, submitApplication etc. (#4618) 2022-07-29 08:23:11 -07:00
slfan1989
e994635a95
YARN-11212. [Federation] Add getNodeToLabels REST APIs for Router. (#4614) 2022-07-28 11:53:04 -07:00
9uapaw
bf570bd4ac YARN-11063. Support auto queue creation template wildcards for arbitrary queue depths. Contributed by Bence Kosztolnik. 2022-07-28 17:32:20 +02:00
KevinWikant
213ea03758
YARN-11210. Fix YARN RMAdminCLI retry logic for non-retryable kerbero… (#4563) 
Co-authored-by: Kevin Wikant <wikak@amazon.com>
 2022-07-26 09:21:37 +05:30
slfan1989
bf8782d0ac
YARN-10883. [Router] Router Audit Log Add Client IP Address. (#4426) 2022-07-25 11:55:40 -07:00
slfan1989
edeb99548a
YARN-11161. Support getAttributesToNodes, getClusterNodeAttributes, getNodesToAttributes API's for Federation (#4610) 2022-07-25 10:05:45 -07:00
slfan1989
63db1a85e3
YARN-11203. Fix typo in hadoop-yarn-server-router module. (#4510). Contributed by fanshilun. 
Reviewed-by: Fei Hui <feihui.ustc@gmail.com>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2022-07-23 20:28:45 +05:30
slfan1989
5c84cb81ba
YARN-8900. [Router] Federation: routing getContainers REST invocations transparently to multiple RMs (#4543) 2022-07-22 17:06:38 -07:00
Masatake Iwasaki
3cce41a1f6 Make upstream aware of 3.2.4 release. 
(cherry picked from commit e1637a57dfd41385dbce5de90620c48a45abb263)
 2022-07-22 02:27:19 +00:00
slfan1989
838020ce3b
YARN-11160. Support getResourceProfiles, getResourceProfile API's for Federation (#4540) 2022-07-21 11:57:24 -07:00
Szilard Nemeth
f4b635c4dc YARN-11211. QueueMetrics leaks Configuration objects when validation API is called multiple times. Contributed by Andras Gyori 2022-07-21 14:20:34 +02:00
PJ Fanning
34e548cb62
HADOOP-18332: remove rs-api dependency as it conflicts with jsr311-api (#4547) 
This downgrades jackson from the version switched to in
    HADOOP-18033 (2.13.0), to Jackson 2.12.7.
    This removes the dependency on javax.ws.rs-api,
    so avoiding runtime problems with applications using
    jersey-core v1 and/or jsr311-api.
    
    The 2.12.7 release still contains the fix for CVE-2020-36518.
    
    Contributed by PJ Fanning
 2022-07-17 21:37:54 +05:30
Samrat
84ce592a85
YARN-11198. clean up numa resources from statestore (#4546) 
* YARN-11198. clean up numa resources from levelDB

Co-authored-by: Deb <dbsamrat@3c22fba1b03f.ant.amazon.com>
 2022-07-14 11:07:48 +05:30
slfan1989
161b1fac2e
YARN-11169. Support moveApplicationAcrossQueues, getQueueInfo API's for Federation. (#4464) 2022-07-05 11:24:29 -07:00
Ashutosh Gupta
151bb31c47
YARN-9403.GET /apps/{appid}/entities/YARN_APPLICATION accesses application table instead of entity table (#4516) 
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
 2022-07-02 21:59:28 +05:30
Ashutosh Gupta
57cbde9abf
YARN-10287.Update scheduler-conf corrupts the CS configuration when removing queue which is referred in queue mapping (#4515) 
* YARN-10287.Update scheduler-conf corrupts the CS configuration when removing queue which is referred in queue mapping

Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
 2022-07-02 21:58:56 +05:30
Tamas Domok
3cad632709 YARN-11202. Optimize ClientRMService.getApplications. Contributed by Tamas Domok. 
Change-Id: I55ddb46fd0e4cdb644747d6d43083215f10861b5
 2022-07-01 10:50:48 +02:00
9uapaw
2d133a54ac YARN-11204. Various MapReduce tests fail with NPE in AggregatedLogDeletionService.stopRMClient. Contributed by Szilard Nemeth. 2022-06-29 15:12:04 +02:00
slfan1989
073b8ea1d5
HADOOP-18284. Remove Unnecessary semicolon ';' (#4422). Contributed by fanshilun. 2022-06-29 15:20:41 +05:30
Samrat
7eefdf8642
YARN-11195. Adding document to enable numa (#4501) 
Contributed by Samrat Deb.
 2022-06-28 17:46:43 +05:30
Ashutosh Gupta
a177232ebc
YARN-9822.TimelineCollectorWebService#putEntities blocked when ATSV2 HBase is down (#4492) 
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
 2022-06-28 09:32:07 +05:30
Ashutosh Gupta
734b6f19ad
YARN-9874.Remove unnecessary LevelDb write call in LeveldbConfigurationStore#confirmMutation (#4487) 
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
 2022-06-23 21:59:27 +05:30
Ashutosh Gupta
4abb2ba58c
YARN-10320.Replace FSDataInputStream#read with readFully in Log Aggregation (#4486) 
* YARN-10320.Replace FSDataInputStream#read with readFully in Log Aggregation

Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
 2022-06-23 21:58:32 +05:30
slfan1989
0af4bb3b42
YARN-11192. TestRouterWebServicesREST failing after YARN-9827. (#4484). Contributed by fanshilun. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2022-06-23 13:21:36 +05:30
9uapaw
e6ecc4f3e4 YARN-11188. Only files belong to the first file controller are removed even if multiple log aggregation file controllers are configured. Contributed by Szilard Nemeth. 2022-06-22 14:40:20 +02:00
Ashutosh Gupta
cbdabe9ec8
YARN-9971.YARN Native Service HttpProbe logs THIS_HOST in error messages (#4436) 
* YARN-9971.YARN Native Service HttpProbe logs THIS_HOST in error messages

Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
 2022-06-22 09:38:19 +05:30
Szilard Nemeth
3a66348fda YARN-11185. Pending app metrics are increased doubly when a queue reaches its max-parallel-apps limit. Contributed by Andras Gyori 2022-06-20 15:03:58 +02:00
9uapaw
5d08ffa769 YARN-11182. Refactor TestAggregatedLogDeletionService: 2nd phase. Contributed by Szilard Nemeth. 2022-06-20 14:12:51 +02:00
Ashutosh Gupta
4f425b641c
YARN-9827.Fix Http Response code in GenericExceptionHandler (#4393) 
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>

Reviewed by Akira Ajisaka.
 2022-06-20 10:44:47 +05:30
slfan1989
62e4476102
YARN-10122. Support signalToContainer API for Federation. (#4421) 2022-06-17 16:38:36 -07:00
zhengchenyu
80446dcd08
YARN-11172. Fix TestClientRMTokens#testDelegationToken introduced by HDFS-16563. (#4408) 
Regression caused by HDFS-16563; the hdfs exception text was changed, but because it was
a YARN test doing the check, Yetus didn't notice.

Contributed by zhengchenyu
 2022-06-17 19:49:36 +01:00