Steve Loughran
dcd9dc6983
HADOOP-18641. Cloud connector dependency and LICENSE fixup. ( #5429 )
...
POM and LICENSE fixup of transient dependencies
* Exclude hadoop-cloud-storage imports which come in with hadoop-common
* Add explicit import of hadoop's org.codehaus.jettison declaration
to hadoop-aliyun
* Tune aliyun jars imports
* Update LICENSE-binary for the current set of libraries.
Contributed by Steve Loughran
2023-02-28 10:48:54 +00:00
rdingankar
0ca5686034
HDFS-16917 Add transfer rate quantile metrics for DataNode reads ( #5397 )
...
Co-authored-by: Ravindra Dingankar <rdingankar@linkedin.com>
2023-02-27 18:26:32 +00:00
Simbarashe Dzinamarira
61f369c43e
HDFS-16890: RBF: Ensures router periodically refreshes its record of a namespace's state. ( #5298 )
2023-02-27 17:56:24 +00:00
slfan1989
8798b94ee1
YARN-11221. [Federation] Add replaceLabelsOnNodes, replaceLabelsOnNode REST APIs for Router. ( #5302 )
2023-02-27 09:34:39 -08:00
Viraj Jasani
a90238c0b8
HADOOP-18631. Migrate Async appenders to log4j properties ( #5418 )
2023-02-26 01:47:44 +08:00
slfan1989
25ebd0b8b1
YARN-11222. [Federation] Add addToClusterNodeLabels, removeFromClusterNodeLabels REST APIs for Router. ( #5328 )
2023-02-24 10:52:57 -08:00
slfan1989
27a54955f9
YARN-5604. [Federation] Add versioning for FederationStateStore. ( #5394 )
2023-02-24 10:51:19 -08:00
Steve Loughran
4067facae6
HADOOP-18470. Remove HDFS RBF text in the 3.3.5 index.md file
...
+ add a link to mukund's apachecon talk
Change-Id: I3d04b385ff1312aabf2a81d034f54f124d544a54
2023-02-23 13:23:35 +00:00
Steve Loughran
e2d7919dc1
Revert "HADOOP-18590. Publish SBOM artifacts ( #5281 )"
...
Causes HADOOP-18641. cyclonedx maven plugin breaks on recent maven releases
This reverts commit 6f99558c2e
2023-02-23 11:23:53 +00:00
Owen O'Malley
8025a60ae7
HDFS-16901: Minor fix for unit test.
...
Signed-off-by: Owen O'Malley <oomalley@linkedin.com>
2023-02-22 16:37:49 -08:00
Simbarashe Dzinamarira
4cc33e5e37
HDFS-16901: RBF: Propagates real user's username via the caller context, when a proxy user is being used. ( #5346 )
2023-02-22 21:58:44 +00:00
slfan1989
2e997d818d
YARN-11370. [Federation] Refactor MemoryFederationStateStore code. ( #5126 )
2023-02-22 12:37:35 -08:00
Ayush Saxena
e8a6b2c2c4
HADOOP-18582. Addendum: Skip unnecessary cleanup logic in DistCp. ( #5409 )
...
Followup to the original HADOOP-18582.
Temporary path cleanup is re-enabled for -append jobs
as these will create temporary files when creating or overwriting files.
Contributed by Ayush Saxena
2023-02-22 19:29:41 +00:00
hchaverr
fb31393b65
HADOOP-18535. Implement token storage solution based on MySQL
...
Fixes #1240
Signed-off-by: Owen O'Malley <oomalley@linkedin.com>
2023-02-22 10:38:50 -08:00
Steve Loughran
11a220c6e7
HADOOP-18636 LocalDirAllocator cannot recover from directory tree deletion ( #5412 )
...
Even though DiskChecker.mkdirsWithExistsCheck() will create the directory tree,
it is only called *after* the enumeration of directories with available
space has completed.
Directories which don't exist are reported as having 0 space, therefore
the mkdirs code is never reached.
Adding a simple mkdirs() -without bothering to check the outcome-
ensures that if a dir has been deleted then it will be reconstructed
if possible. If it can't it will still have 0 bytes of space
reported and so be excluded from the allocation.
Contributed by Steve Loughran
2023-02-22 11:48:12 +00:00
susheel-gupta
49b8ac19f2
YARN-11408. Add a check of autoQueueCreation is disabled for emitDefaultUserLimitFactor method ( #5278 )
...
* added a check of autoQueueCreationV2Disabled
Change-Id: If1e36c5969d270c1b81a4bbd2e883fa819c81f20
* added check of AutoCreateChildQueueDisabled
Change-Id: Ia011b4393ccd8d4d419a2e46b06a5237d050851c
* removed auto-create-child-queue-enabled check and emit
Change-Id: I7a154124519ecbd81379b46a238707c16db1e82a
2023-02-22 09:46:42 +01:00
Ayush Saxena
fe5bb49ad9
Revert "YARN-11404. Add junit5 dependency to hadoop-mapreduce-client-app to fix few unit test failure. Contributed by Susheel Gupta"
...
This reverts commit 8eda456d37
2023-02-22 07:28:13 +05:30
slfan1989
4e6e2f318c
YARN-11394. Fix hadoop-yarn-server-resourcemanager module Java Doc Errors. ( #5288 )
...
Contributed by Shilun Fan
2023-02-21 14:39:32 +00:00
nao
acf82d4d55
HADOOP-18622. Upgrade ant to 1.10.13 ( #5360 ). Contributed by Aleksandr Nikolaev.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-02-21 16:48:49 +05:30
Viraj Jasani
88914cada0
HDFS-16925. Namenode audit log to only include IP address of client ( #5407 )
...
Reviewed-by: Takanobu Asanuma <tasanuma@apache.org>
Signed-off-by: Tao Li <tomscut@apache.org>
2023-02-21 08:26:32 +08:00
zhtttylz
a3b500d046
HDFS-16916. Improve the use of JUnit Test in DFSClient ( #5404 )
...
Co-authored-by: Zhtttylz <hualong.z@hotmail.com>
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
2023-02-18 13:17:37 +08:00
slfan1989
7e486038ea
YARN-11439. Fix Typo of hadoop-yarn-ui README.md. ( #5405 )
...
Co-authored-by: Shilun Fan <slfan1989@apache.org>
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
2023-02-18 13:14:01 +08:00
Arnout Engelen
02fd87a4d8
HADOOP-18627. Add stronger wording in 'secure mode' introduction ( #5406 )
...
Make it more clear that when deploying Hadoop 'secure mode' is generally not optional.
Contributed by Arnout Engelen
2023-02-17 16:30:41 +00:00
Steve Loughran
10e7ca481c
YARN-11441. Revert YARN-10495.
...
This reverts commit 7d3c8ef606
2023-02-17 15:05:06 +00:00
Bryan Beaudreault
7e19bc31b6
HADOOP-18215. Enhance WritableName to be able to return aliases for classes that use serializers ( #4215 )
2023-02-16 18:13:25 +00:00
Mehakmeet Singh
7a0903b743
HADOOP-18633. fix test AbstractContractDistCpTest#testDistCpUpdateCheckFileSkip ( #5401 )
...
Contributed by: Mehakmeet Singh
2023-02-16 10:09:06 +05:30
hfutatzhanghb
723535b788
HDFS-16914. Add some logs for updateBlockForPipeline RPC. ( #5381 )
...
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Tao Li <tomscut@apache.org>
2023-02-16 09:25:50 +08:00
slfan1989
a5f48eacca
YARN-11425. [Federation] Router Supports SubClusterCleaner. ( #5326 )
2023-02-15 14:40:34 -08:00
slfan1989
c3706597a3
YARN-11349. [Federation] Router Support DelegationToken With SQL. ( #5244 )
2023-02-15 14:38:41 -08:00
Ankit Saurabh
f4f2793f3b
HADOOP-18351. Reduce excess logging of errors during S3A prefetching reads ( #5274 )
...
Contributed by Ankit Saurabh
2023-02-15 18:28:42 +00:00
Zita Dombi
4cbe19f3a2
HDFS-16761. Namenode UI for Datanodes page not loading if any data node is down ( #5390 )
2023-02-15 16:16:04 +00:00
Steve Loughran
d56977e909
HADOOP-18470. More in the 3.3.5 index.html about security ( #5383 )
...
Expands on the comments in cluster config to tell people
they shouldn't be running a cluster without a private VLAN
in cloud, that Knox is good here, and unsecured clusters
without a VLAN are just computation-as-a-service to crypto miners
Contributed by Steve Loughran
2023-02-14 17:22:59 +00:00
SimhadriGovindappa
e2ab35084a
HADOOP-18630. Add gh-pages in asf.yaml to deploy the current trunk doc ( #5393 ). Contributed by Simhadri Govindappa.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-02-14 18:13:29 +05:30
Viraj Jasani
021fcc6c5e
HADOOP-18628. IPC Server Connection should log host name before returning VersionMismatch error ( #5385 )
...
Contributed by Viraj Jasani
2023-02-14 11:48:48 +00:00
GuoPhilipse
fe0541b58d
HDFS-16913. Fix flaky some unit tests since they offen timeout ( #5377 )
...
Co-authored-by: gf13871 <gf13871@ly.com>
Reviewed-by: Tao Li <tomscut@apache.org>
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
2023-02-14 15:29:38 +08:00
Ayush Saxena
1def35d802
HADOOP-18524. Addendum: Deploy Hadoop trunk version website. ( #5389 ). Contributed by Ayush Saxena.
...
Reviewed-by: Vinayakumar B <vinayakumarb@apache.org>
2023-02-14 11:05:41 +05:30
Viraj Jasani
90de1ff151
HADOOP-18206 Cleanup the commons-logging references and restrict its usage in future ( #5315 )
2023-02-14 03:24:06 +08:00
Ayush Saxena
30f560554d
HADOOP-18524. Deploy Hadoop trunk version website. ( #5386 ). Contributed by Ayush Saxena.
...
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
2023-02-14 00:03:02 +05:30
Tamas Domok
e4b5314991
MAPREDUCE-7433. Remove unused mapred/LoggingHttpResponseEncoder.java. ( #5388 )
2023-02-13 16:21:27 +01:00
Steve Vaughan
f42c89dffb
HDFS-16904. Close webhdfs during TestSymlinkHdfs teardown ( #5372 )
...
This is a followup to the original patch, 08f58ecf07
2023-02-13 14:31:32 +00:00
hfutatzhanghb
f3c4277576
HDFS-16882. RBF: Add cache hit rate metric in MountTableResolver#getDestinationForPath ( #5276 )
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Tao Li <tomscut@apache.org>
2023-02-11 08:00:43 +08:00
Owen O'Malley
26fba8701c
HDFS-18324. Fix race condition in closing IPC connections. ( #5371 )
2023-02-10 17:51:03 +00:00
Tamas Domok
151b71d7af
MAPREDUCE-7431. ShuffleHandler refactor and fix after Netty4 upgrade. ( #5311 )
2023-02-10 17:40:21 +01:00
Viraj Jasani
17c8cdf63c
HDFS-16907. ADDENDUM: Remove unused variables from testDataNodeMXBeanLastHeartbeats. ( #5373 )
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-02-10 00:33:20 +05:30
Mehakmeet Singh
9e4f50d8a0
HADOOP-18596. Distcp -update to use modification time while checking for file skip. ( #5308 )
...
Adding toggleable support for modification time during distcp -update between two stores with incompatible checksum comparison.
Contributed by: Mehakmeet Singh <mehakmeet.singh.behl@gmail.com>
2023-02-09 21:31:09 +05:30
huhaiyang
113a9e40cb
HADOOP-18625. Fix method name of RPC.Builder#setnumReaders ( #5301 )
...
Changes method name of RPC.Builder#setnumReaders to setNumReaders()
The original method is still there, just marked deprecated.
It is the one which should be used when working with older branches.
Contributed by Haiyang Hu
2023-02-09 13:28:34 +00:00
huhaiyang
d5c046518e
HDFS-16910. Fix incorrectly initializing RandomAccessFile caused flush performance decreased for JN ( #5359 )
2023-02-09 10:47:57 +08:00
Viraj Jasani
4fcceff535
HADOOP-18620 Avoid using grizzly-http-* APIs ( #5356 )
2023-02-09 10:45:07 +08:00
slfan1989
af20841fb1
YARN-11217. [Federation] Add dumpSchedulerLogs REST APIs for Router. ( #5272 )
2023-02-08 11:48:38 -08:00
Steve Vaughan
08f58ecf07
HDFS-16904. Close webhdfs during TestSymlinkHdfs teardown ( #5342 )
...
Contributed by Steve Vaughan Jr
2023-02-08 17:15:42 +00:00
