big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
27,432 Commits 2 Branches 2 Tags 264 MiB
01401d71ef
Commit Graph

27432 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
slfan1989
845cf8bc28
YARN-11368. [Federation] Improve Yarn Router's Federation Page style. (#5105) 2022-11-07 10:13:23 -08:00
Simbarashe Dzinamarira
44b8bb7224
HDFS-16821: Fixes regression in HDFS-13522 that enables observer reads by default (#5078) 2022-11-07 08:43:04 -08:00
Takanobu Asanuma
660530205e
HDFS-16833. NameNode should log internal EC blocks instead of the EC block group when it receives block reports. (#5106) 
Reviewed-by: Tao Li <tomscut@apache.org>
 2022-11-07 10:49:48 +09:00
slfan1989
5d6ab15860
YARN-11354. [Federation] Add Yarn Router's NodeLabel Web Page. (#5073) 2022-11-04 14:39:57 -07:00
Steve Vaughan
2ba982a061
MAPREDUCE-7386. Maven parallel builds (skipping tests) fail (#4415) 
Contributed by Steve Vaughan Jr
 2022-11-04 11:50:43 +00:00
huhaiyang
e9319e696c
HDFS-16811. Support DecommissionBackoffMonitor Parameters reconfigurablereconfigurable (#5068) 
Signed-off-by: Tao Li <tomscut@apache.org>
 2022-11-04 14:18:59 +08:00
slfan1989
b90dfdff3f
YARN-11366. Improve equals, hashCode(), toString() methods of the Federation Base Object. (#5096) 2022-11-03 21:33:53 -07:00
Ashutosh Gupta
e62ba16a02
HADOOP-18484. Upgrade hsqldb to v2.7.1 to mitigate CVE-2022-41853 (#4991) 2022-11-02 08:41:27 +01:00
Ashutosh Gupta
83acb55981
YARN-11364. Docker Container to accept docker Image name with sha256 digest (#5092) 
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Reviewed-by: slfan1989 <55643692+slfan1989@users.noreply.github.com>
Signed-off-by: Chris Nauroth <cnauroth@apache.org>
 2022-11-01 14:44:35 -07:00
Ashutosh Gupta
69225ae5b9
YARN-11363. Remove unused TimelineVersionWatcher and TimelineVersion from hadoop-yarn-server-tests (#5091) 
Reviewed-by: slfan1989 <55643692+slfan1989@users.noreply.github.com>
Signed-off-by: Chris Nauroth <cnauroth@apache.org>
 2022-11-01 14:02:06 -07:00
wangteng13
388f2f182f
document fix for MAPREDUCE-7425 (#5090) 
Reviewed-by: Ashutosh Gupta <ashutosh.gupta@st.niituniversity.in>
Signed-off-by: Chris Nauroth <cnauroth@apache.org>
 2022-11-01 13:34:59 -07:00
PJ Fanning
7ba304d1c6
HADOOP-18512: upgrade woodstox-core to 5.4.0 for security fix (#5087). Contributed by PJ Fanning. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2022-11-02 00:11:41 +05:30
Daniel Carl Jones
0b577992ef
HADOOP-18482. ITestS3APrefetchingInputStream to skip if CSV test file unavailable (#4983) 
Contributed by Danny Jones
 2022-10-31 21:19:34 +00:00
Steve Loughran
3b10cb5a3b
HADOOP-18507. VectorIO FileRange type to support a "reference" field (#5076) 
Contributed by Steve Loughran
 2022-10-31 21:12:13 +00:00
Gautham B A
b1f418f802
YARN-11365. Fix NM class not found on Windows (#5093) 2022-10-31 09:19:07 -07:00
sabertiger
af7dd660e0
HADOOP-18233. Possible race condition with TemporaryAWSCredentialsProvider (#5024) 
This fixes a race condition with the TemporaryAWSCredentialProvider
one which has existed for a long time but which only surfaced
(usually in Spark) when the bucket existence probe was disabled
by setting fs.s3a.bucket.probe to 0 -a performance speedup
which was made the default in HADOOP-17454.

Contributed by Jimmy Wong.
 2022-10-31 12:43:30 +00:00
Ashutosh Gupta
cbe02c2e77
YARN-11264. Upgrade JUnit from 4 to 5 in hadoop-yarn-server-tests (#4776) 
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2022-10-30 04:33:57 +09:00
Samrat
e04c9e810b
MAPREDUCE-7426. Fix typo in StartEndTimeBase (#4894) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2022-10-30 04:23:30 +09:00
Ashutosh Gupta
c096803387
YARN-11339. Upgrade Junit 4 to 5 in hadoop-yarn-services-api (#4995) 
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2022-10-30 03:07:16 +09:00
PJ Fanning
d6a65a4180
HADOOP-18472. Upgrade to snakeyaml 1.33 (#4958) 
Reviewed-by: Dinesh Chitlangia <dineshc@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2022-10-30 02:30:41 +09:00
Ashutosh Gupta
2aae7ffe08
YARN-11337. Upgrade Junit 4 to 5 in hadoop-yarn-applications-mawo (#4993) 
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2022-10-30 02:25:53 +09:00
slfan1989
070a2d4880
YARN-11332. [Federation] Improve FederationClientInterceptor#ThreadPool thread pool configuration. (#4982) 2022-10-28 15:39:04 -07:00
slfan1989
b1cd88c598
YARN-11229. [Federation] Add checkUserAccessToQueue REST APIs for Router. (#4929) 2022-10-28 15:37:35 -07:00
Chris Nauroth
bfb84cd7f6
YARN-11360: Add number of decommissioning/shutdown nodes to YARN cluster metrics. (#5060) 2022-10-28 11:07:01 -07:00
jianghuazhu
88f7f5bc01
HDFS-16802.Print options when accessing ClientProtocol#rename2(). (#5013) 
Reviewed-by: Tao Li <tomscut@apache.org>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: Zander Xu <zanderxu@apache.org>
 2022-10-28 16:31:20 +08:00
M1eyu2018
8396caa484
HDFS-16716. Improve appendToFile command: support appending on file with new block (#4697) 
Reviewed-by: xuzq <15040255127@163.com>
Signed-off-by: Tao Li <tomscut@apache.org>
 2022-10-27 19:03:15 +08:00
huhaiyang
d26c35b228
HDFS-16817. Remove useless DataNode lock related configuration (#5072) 
Reviewed-by: litao <tomleescut@gmail.com>
Signed-off-by: ZanderXu <zanderxu@apache.org>
 2022-10-27 17:28:19 +08:00
Takanobu Asanuma
545a556883
HDFS-16822. HostRestrictingAuthorizationFilter should pass through requests if they don't access WebHDFS API. (#5079) 
Reviewed-by: Ashutosh Gupta <ashugpt@amazon.com>
Reviewed-by: Tao Li <tomscut@apache.org>
 2022-10-27 14:39:01 +09:00
slfan1989
ba77530ff4
YARN-11357. Fix FederationClientInterceptor#submitApplication Can't Update SubClusterId (#5055) 2022-10-26 16:42:22 -07:00
Bence Kosztolnik
562b693374 YARN-11356. Upgrade DataTables to 1.11.5 to fix CVEs. Contributed by Bence Kosztolnik. 2022-10-26 22:29:01 +02:00
Mehakmeet Singh
fba46aa5bb
HADOOP-18499. S3A to support HTTPS web proxies (#5051) 
The option "fs.s3a.proxy.ssl.enabled" controls
whether the s3a connects to a proxy over HTTP (default) or HTTPS.
Set to "true" to use HTTPS.

Contributed by Mehakmeet Singh
 2022-10-26 11:45:20 +01:00
Wang Yu
37bff63c0f
Refactor CallerContext's constructor to eliminate duplicate code (#5070) 
Reviewed-by: Tao Li <tomscut@apache.org>
Reviewed-by: Zander Xu <zanderxu@apache.org>
 2022-10-26 06:40:31 +08:00
FuzzingTeam
f140506d67
HADOOP-18504. Fixed an unhandled NullPointerException in class KeyProvider (#5064) 
Contributed by FuzzingTeam
 2022-10-25 18:07:49 +01:00
Ashutosh Gupta
0a26d84df1
HADOOP-9946. NumAllSinks metrics shows lower value than NumActiveSinks (#5002) 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
 2022-10-25 17:22:25 +08:00
Ashutosh Gupta
e6edbf1b4b
YARN-11338. Upgrade Junit 4 to 5 in hadoop-yarn-applications-unmanaged-am-launcher (#4994) 
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2022-10-25 17:21:12 +09:00
Ashutosh Gupta
21b7790866
YARN-11336. Upgrade Junit 4 to 5 in hadoop-yarn-applications-catalog-webapp (#4992) 
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2022-10-25 16:52:31 +09:00
slfan1989
454157a384
YARN-11345. [Federation] Refactoring Yarn Router's Application Web Page. (#5030) 2022-10-24 09:35:16 -07:00
Gautham B A
833750f72a
HADOOP-18506. Update build instructions for Windows using VS2019 (#5066) 2022-10-24 09:28:29 -07:00
PJ Fanning
aac87ffe76
MAPREDUCE-7411: use secure XML parsers in mapreduce modules (#4980) 
Lockdown of parsers in hadoop-mapreduce.

Follow-on to HADOOP-18469. Add secure XML parser factories to XMLUtils

Contributed by P J Fanning
 2022-10-21 14:02:11 +01:00
slfan1989
d93e6f0cbb
YARN-11295. [Federation] Router Support DelegationToken in MemoryStore mode. (#5032) 2022-10-20 13:14:38 -07:00
Willi Raschkowski
c4aa41aa80
HADOOP-18500. Upgrade maven-shade-plugin to 3.3.0 (#5045) 
Contributed by Willi Raschkowski
 2022-10-20 18:47:33 +01:00
FuzzingTeam
7f69e09290
HADOOP-18471. Fixed ArrayIndexOutOfBoundsException in DefaultStringifier (#4957) 
Contributed by FuzzingTeam
 2022-10-20 18:12:17 +01:00
Sneha Vijayarajan
a996d889ec
HADOOP-17767. ABFS: Update test scripts (#3124) 
Contributed by Sneha Vijayarajan
 2022-10-20 18:07:04 +01:00
slfan1989
9adf0ca089
YARN-11342. [Federation] Refactor getNewApplication, submitApplication Use FederationActionRetry. (#5005) 2022-10-20 09:22:24 -07:00
jianghuazhu
c5c00f3d2c
HDFS-16803.Improve some annotations in hdfs module. (#5031) 2022-10-20 10:58:23 +08:00
slfan1989
48b6f9f335
YARN-11328. Refactoring part of the code of SQLFederationStateStore. (#4976) 2022-10-19 16:11:28 -07:00
Viraj Jasani
8aa04b0b24
HADOOP-18189 S3APrefetchingInputStream to support status probes when closed (#5036) 
Contributed by Viraj Jasani
 2022-10-19 14:38:11 +01:00
Daniel Carl Jones
6207ac47e0
HADOOP-18304. Improve user-facing S3A committers documentation (#4478) 
Contributed by: Daniel Carl Jones
 2022-10-19 12:56:47 +05:30
Steve Loughran
d80db6c9e5
HADOOP-18476. Abfs and S3A FileContext bindings to close wrapped filesystems in finalizer (#4966) 
This is to try and close the underlying filesystems when the FileContext APIs are used.
Without this, threads may be leaked
 2022-10-18 14:53:02 +01:00
slfan1989
ee886cacd7
YARN-11247. Remove unused classes introduced by YARN-9615. (#4720) 2022-10-18 09:40:39 -04:00