big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
26,666 Commits 2 Branches 2 Tags 264 MiB
9f1e23cc67
Commit Graph

6681 Commits

Author SHA1 Message Date
slfan1989
86c250a54a
YARN-7720. Race condition between second app attempt and UAM timeout when first attempt node is down. (#5672) 2023-05-29 10:37:08 -07:00
Xianming Lei
97afb33c73
YARN-11276. Add LRU cache for RMWebServices.getApps. (#4793) 
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
 2023-05-26 20:46:00 +08:00
slfan1989
b977065cc4
YARN-11478. [Federation] SQLFederationStateStore Support Store ApplicationSubmitData. (#5663) 2023-05-24 11:43:20 -07:00
Tamas Domok
aeb3f6f1a8
YARN-11490. Reverting YARN-11211 and eliminating the use of DefaultMetricsSystem during configuration validation (#5644) 2023-05-23 10:36:37 +02:00
Xianming Lei
0110e24ed8
YARN-11496. Improve TimelineService log format. (#5677). Contributed by Xianming Lei. 
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-05-20 14:57:45 +05:30
NishthaShah
5272ed8670
HADOOP-17518. Update the regex to A-Z (#5669). Contributed by Nishtha Shah. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-05-20 06:21:13 +05:30
slfan1989
bba663038d
YARN-8898. Fix FederationInterceptor#allocate to set application priority in allocateResponse. (#5645) 2023-05-18 11:57:38 -07:00
Peter Szucs
ff8eac517a
YARN-11463. Node Labels root directory creation doesn't have a retry logic - 2nd addendum (#5670) 2023-05-18 14:48:43 +02:00
slfan1989
5ddaf2e133
YARN-11493. [Federation] ConfiguredRMFailoverProxyProvider Supports Randomly Select an Router. (#5651) 2023-05-17 11:09:10 -07:00
Chun Chen
11af08d67a
YARN-11489. Fix memory leak of DelegationTokenRenewer futures in DelegationTokenRenewerPoolTracker. (#5629). Contributed by Chun Chen. 
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2023-05-14 21:38:04 +08:00
slfan1989
e0938b4c2a
YARN-11495. Fix typos in hadoop-yarn-server-web-proxy. (#5652). Contributed by Shilun Fan. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-05-13 11:41:38 +05:30
slfan1989
2f87f716fa
YARN-3660. BackPort [GPG] Federation Global Policy Generator (service hook only). (#5625) 2023-05-12 18:12:05 -07:00
susheel-gupta
0f3406ac34
YARN-11312: [UI2] Refresh buttons don't work after EmberJS upgrade (#5649) 2023-05-12 14:20:26 +02:00
Steve Loughran
e76c09ac3b
HADOOP-18724. Open file fails with NumberFormatException for S3AFileSystem (#5611) 
This:

1. Adds optLong, optDouble, mustLong and mustDouble
   methods to the FSBuilder interface to let callers explicitly
   passin long and double arguments.
2. The opt() and must() builder calls which take float/double values
   now only set long values instead, so as to avoid problems
   related to overloaded methods resulting in a ".0" being appended
   to a long value.
3. All of the relevant opt/must calls in the hadoop codebase move to
   the new methods
4. And the s3a code is resilient to parse errors in is numeric options
   -it will downgrade to the default.

This is nominally incompatible, but the floating-point builder methods
were never used: nothing currently expects floating point numbers.

For anyone who wants to safely set numeric builder options across all compatible
releases, convert the number to a string and then use the opt(String, String)
and must(String, String) methods.

Contributed by Steve Loughran
 2023-05-11 17:57:25 +01:00
slfan1989
690db3c34b
YARN-11479. [Federation] ZookeeperFederationStateStore Support Store ApplicationSubmitData. (#5631) 2023-05-10 09:37:47 -07:00
cxzl25
be50d221f5
YARN-11467. RM failover may fail when the nodes.exclude-path file does not exist (#5565) 2023-05-10 15:16:33 +08:00
slfan1989
d95b5c679d
YARN-11424. [Federation] Router Supports DeregisterSubCluster. (#5363) 2023-05-09 16:17:23 -07:00
slfan1989
bdeca45294
YARN-11340. [Federation] Improve SQLFederationStateStore DataSource Config. (#5403) 2023-05-08 10:13:09 -07:00
slfan1989
cda9863d54
YARN-11477. [Federation] MemoryFederationStateStore Support Store ApplicationSubmitData. (#5616) 2023-05-05 13:43:40 -07:00
slfan1989
eab4c33d09
YARN-11470. FederationStateStoreFacade Cache Support Guava Cache. (#5609) 2023-05-05 13:41:38 -07:00
Szilard Nemeth
c7699d3dcd YARN-11079. Make an AbstractParentQueue to store common ParentQueue and ManagedParentQueue functionality. Contributed by Susheel Gupta 2023-05-04 22:16:18 -04:00
Peter Szucs
bd607951c0
YARN-11463. Node Labels root directory creation doesn't have a retry logic - addendum (#5614) 2023-05-04 12:27:25 +02:00
slfan1989
476f60a806
YARN-10144. Federation: Add missing FederationClientInterceptor APIs. (#5587) 2023-05-03 09:21:56 -07:00
slfan1989
c1d10f3872
YARN-9049. Add application submit data to state store. (#5606) 2023-05-03 09:19:54 -07:00
slfan1989
668c0a0930
YARN-11379. [Federation] Support mapAttributesToNodes, getGroupsForUser API's for Federation. (#5596) 2023-05-02 13:46:06 -07:00
slfan1989
87e17b2713
YARN-11437. [hotfix][Federation] SQLFederationStateStore Support Version. (#5598) 2023-05-01 16:01:50 -07:00
Hexiaoqiao
70c0aa342e
YARN-11482. Fix bug of DRF comparision DominantResourceFairnessComparator2 in fair scheduler. (#5607). Contributed by Xiaoqiao He. 
Reviewed-by: Shilun Fan <slfan1989@apache.org>
 2023-04-29 11:18:42 +08:00
slfan1989
5ed7e912dc
YARN-11469. Refactor FederationStateStoreFacade Cache Code. (#5570) 
Co-authored-by: slfan1989 <louj1988@@>
 2023-04-28 14:11:13 -07:00
Riya Khandelwal
60a7e8acaa
YARN-11459 Changed label called "max resource" on UIv1 and UIv2 (#5527) 2023-04-27 15:25:25 +02:00
yl09099
245fde17d7
YARN-11474.The yarn queue list is displayed on the CLI (#5577) 
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
 2023-04-27 18:05:48 +08:00
slfan1989
55eebcf277
YARN-11378. [Federation] Support checkForDecommissioningNodes、refreshClusterMaxPriority API's for Federation. (#5551) 2023-04-25 14:12:38 -07:00
slfan1989
a716459cdf
YARN-11437. [Federation] SQLFederationStateStore Support Version. (#5589) 2023-04-25 14:09:45 -07:00
Ashutosh Gupta
964c1902c8
YARN-11463. Node Labels root directory creation doesn't have a retry logic (#5562) 
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
 2023-04-21 14:03:22 +02:00
rdingankar
5119d0c72f
HDFS-16982 Use the right Quantiles Array for Inverse Quantiles snapshot (#5556) 2023-04-18 10:47:37 -07:00
slfan1989
a258f1f235
YARN-11326. [Federation] Add RM FederationStateStoreService Metrics. (#4963) 2023-04-18 09:13:08 -07:00
slfan1989
635521db4c
YARN-11438. [Federation] ZookeeperFederationStateStore Support Version. (#5537) 2023-04-18 09:05:52 -07:00
Steve Loughran
405ed1dde6
HADOOP-18470. Hadoop 3.3.5 release wrap-up (#5558) 
Post-release updates of the branches

* Add jdiff xml files from 3.3.5 release.
* Declare 3.3.5 as the latest stable release.
* Copy release notes.
 2023-04-18 10:12:07 +01:00
yl09099
2c4d6bf33d
YARN-11465. Improved YarnClient Log Format (#5550) 
Co-authored-by: yl09099 <shaq376260428@163.com>
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
 2023-04-17 09:27:52 +08:00
slfan1989
0bcdea7912
YARN-11239. Optimize FederationClientInterceptor audit log. (#5127) 2023-04-14 13:09:18 -07:00
slfan1989
06f9bdffa6
YARN-10846. Add dispatcher metrics to NM. (#4687) 2023-04-12 09:53:20 -07:00
slfan1989
dd6d0ac510
YARN-11462. Fix Typo of hadoop-yarn-common. (#5539) 
Co-authored-by: Shilun Fan <slfan1989@apache.org>
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
 2023-04-12 11:08:23 +08:00
slfan1989
bffa49a64f
YARN-11377. [Federation] Support addToClusterNodeLabels、removeFromClusterNodeLabels、replaceLabelsOnNode API's for Federation. (#5525) 2023-04-11 09:47:58 -07:00
slfan1989
69b90b5698
YARN-11436. [Federation] MemoryFederationStateStore Support Version. (#5518) 2023-04-05 10:35:24 -07:00
slfan1989
eb1d3ebe2f
YARN-11442. Refactor FederationInterceptorREST Code. (#5420) 2023-03-31 15:29:18 -07:00
slfan1989
5bc8f25327
YARN-11446. [Federation] Add updateSchedulerConfiguration, getSchedulerConfiguration REST APIs for Router. (#5476) 2023-03-28 09:33:19 -07:00
slfan1989
aa602381c5
YARN-11426. Improve YARN NodeLabel Memory Display. (#5335) 
YARN-11426. Improve YARN NodeLabel Memory Display.
Co-authored-by: slfan1989 <louj1988@@>
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: Chris Nauroth <cnauroth@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
 2023-03-28 22:48:46 +08:00
slfan1989
926993cb73
YARN-11376. [Federation] Support updateNodeResource、refreshNodesResources API's for Federation. (#5496) 2023-03-27 09:27:21 -07:00
Tamas Domok
69748aae32
YARN-11461. fix NPE in determineMissingParents (auto queue creation / CS). (#5506) 
Change-Id: Iaaaf43a545588eaff8a0a20f6f3c27258a45f390
 2023-03-24 09:38:53 +01:00
slfan1989
fa723ae839
YARN-11445. [Federation] Add getClusterInfo, getClusterUserInfo REST APIs for Router. (#5472) 2023-03-17 11:59:45 -07:00
Viraj Jasani
cf4a678ce9
HADOOP-18649. CLA and CRLA appenders to be replaced with RFA (#5448) 2023-03-16 00:46:17 +08:00