Tamas Domok
69748aae32
YARN-11461. fix NPE in determineMissingParents (auto queue creation / CS). ( #5506 )
...
Change-Id: Iaaaf43a545588eaff8a0a20f6f3c27258a45f390
2023-03-24 09:38:53 +01:00
Kidd5368
5cf62d1498
HDFS-16948. Update log of BlockManager#chooseExcessRedundancyStriped when EC internal block is moved by balancer. ( #5474 ). Contributed by Kidd53685368.
...
Reviewed-by: zhangshuyan <zqingchai@gmail.com>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-03-23 17:00:23 +08:00
zhaixiaojuan@loongson.cn
028cde0006
HADOOP-18644. Add bswap support for LoongArch64. ( #5453 ). Contributed by zhaixiaojuan.
...
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-03-23 11:08:59 +08:00
Ayush Saxena
e3cb9573e1
HADOOP-18662. ListFiles with recursive fails with FNF. ( #5477 ). Contributed by Ayush Saxena.
...
Reviewed-by: Steve Loughran <stevel@apache.org
2023-03-23 08:30:08 +05:30
Yubi Lee
67e02a92e0
HADOOP-18666. A whitelist of endpoints to skip Kerberos authentication doesn't work for ResourceManager and Job History Server ( #5480 )
2023-03-22 10:54:41 +09:00
Viraj Jasani
0dbe1d3284
HADOOP-18668. Path capability probe for truncate is only honored by RawLocalFileSystem ( #5492 )
2023-03-21 10:23:16 +08:00
Viraj Jasani
9a8287c36f
HADOOP-18669. Remove Log4Json Layout ( #5493 )
2023-03-21 10:07:06 +08:00
Viraj Jasani
f8d0949f7d
HDFS-16953. RBF: Mount table store APIs should update cache only if state store record is successfully updated ( #5482 )
2023-03-18 14:43:25 -07:00
Viraj Jasani
b6a9d7b442
HADOOP-18631. (ADDENDUM) Use LogCapturer to match audit log pattern and remove hdfs async audit log configs ( #5451 )
2023-03-18 06:33:50 +08:00
slfan1989
fa723ae839
YARN-11445. [Federation] Add getClusterInfo, getClusterUserInfo REST APIs for Router. ( #5472 )
2023-03-17 11:59:45 -07:00
Pranav Saxena
759ddebb13
HADOOP-18647. x-ms-client-request-id to identify the retry of an API. ( #5437 )
...
The x-ms-client-request-id now includes a field to indicate a call is a retry of a previous
operation
Contributed by Pranav Saxena
2023-03-15 20:03:22 +00:00
Masatake Iwasaki
7c42d0f7da
HADOOP-17746. Compatibility table in directory_markers.md doesn't render right. ( #3116 )
...
Contributed by Masatake Iwasaki
2023-03-15 17:10:42 +00:00
Viraj Jasani
15935fa865
HDFS-16947. RBF NamenodeHeartbeatService to report error for not being able to register namenode in state store ( #5470 )
2023-03-16 00:59:55 +08:00
Viraj Jasani
cf4a678ce9
HADOOP-18649. CLA and CRLA appenders to be replaced with RFA ( #5448 )
2023-03-16 00:46:17 +08:00
Viraj Jasani
405bfa2800
HADOOP-18654. Remove unused custom appender TaskLogAppender ( #5457 )
2023-03-16 00:45:37 +08:00
Stephen O'Donnell
eee2ea075d
HDFS-16942. Addendum. Send error to datanode if FBR is rejected due to bad lease ( #5478 ). Contributed by Stephen O'Donnell/
2023-03-15 10:03:00 +05:30
Viraj Jasani
aff840c59c
HADOOP-18653. LogLevel servlet to determine log impl before using setLevel ( #5456 )
...
The log level can only be set on Log4J log implementations;
probes are used to downgrade to a warning when other
logging back ends are used
Contributed by Viraj Jasani
2023-03-13 12:30:12 +00:00
Steve Loughran
09469bf47d
HADOOP-18661. Fix bin/hadoop usage script terminology. ( #5473 )
...
Followup to HADOOP-13209: s/slaves/r/workers in
the usage message you get when you type "bin/hadoop"
Contributed by Steve Loughran
2023-03-13 12:24:36 +00:00
PJ Fanning
476340c699
HADOOP-18658. snakeyaml dependency: upgrade to v2.0 ( #5467 ). Contributed by PJ Fanning.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-03-13 10:08:04 +05:30
Stephen O'Donnell
ca6f5afb6d
HDFS-16942. Send error to datanode if FBR is rejected due to bad lease ( #5460 )
2023-03-11 16:40:07 +00:00
nao
734f7abfb8
HADOOP-18646. Upgrade Netty to 4.1.89.Final to fix CVE-2022-41881 ( #5435 )
...
This fixes CVE-2022-41881.
This also upgrades io.opencensus dependencies to 0.12.3
Contributed by Aleksandr Nikolaev
2023-03-10 15:27:22 +00:00
slfan1989
b406060c6b
YARN-8972. [Router] Add support to prevent DoS attack over ApplicationSubmissionContext size. ( #5382 )
2023-03-08 13:29:30 -08:00
rohit-kb
487368c4b9
HADOOP-18655. Upgrade kerby to 2.0.3 due to CVE-2023-25613 ( #5458 )
...
Upgrade kerby to 2.0.3 due to the CVE https://nvd.nist.gov/vuln/detail/CVE-2023-25613
Contributed by Rohit Kumar Badeau
2023-03-08 15:31:03 +00:00
Pranav Saxena
358bf80c94
HADOOP-18606. ABFS: Add reason in x-ms-client-request-id on a retried API call. ( #5299 )
...
Contributed by Pranav Saxena
2023-03-07 17:02:13 +00:00
slfan1989
927401886a
HDFS-16934. TestDFSAdmin.testAllDatanodesReconfig regression ( #5434 )
...
Contributed by Shilun Fan
2023-03-06 15:26:53 +00:00
zhangshuyan
2cb0c35fc1
HDFS-16939. Fix the thread safety bug in LowRedundancyBlocks. ( #5450 ). Contributed by Shuyan Zhang.
...
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-03-06 20:10:31 +08:00
Varun Saxena
2a0dc2ab2f
YARN-11383. Workflow priority mappings is case sensitive ( #5171 )
...
Contributed by Aparajita Choudhary
2023-03-05 21:25:16 +05:30
ZanderXu
6bd2444815
HDFS-16923. [SBN read] getlisting RPC to observer will throw NPE if path does not exist ( #5400 )
...
Signed-off-by: Erik Krogen <xkrogen@apache.org>
2023-03-01 16:18:38 -08:00
Viraj Jasani
e1ca466bdb
HADOOP-18648. Avoid loading kms log4j properties dynamically by KMSWebServer ( #5441 )
2023-03-02 08:02:07 +08:00
Tom
162288bc0a
HDFS-16896 clear ignoredNodes list when we clear deadnode list on ref… ( #5322 )
...
HDFS-16896 clear ignoredNodes list when we clear deadnode list on refetchLocations.
ignoredNodes list is only used on hedged read codepath
Co-authored-by: Tom McCormick <tmccormi@linkedin.com>
2023-03-01 19:47:04 +00:00
Viraj Jasani
2ab7eb4caa
HDFS-16935. Fix TestFsDatasetImpl#testReportBadBlocks ( #5432 )
...
Contributed by Viraj Jasani
2023-03-01 18:53:10 +00:00
Szilard Nemeth
8f6be3678d
MAPREDUCE-7434. Fix ShuffleHandler tests. Contributed by Tamas Domok
2023-03-01 16:10:05 +01:00
Viraj Jasani
28d2753d2f
HADOOP-18645. Provide keytab file key name with ServiceStateException ( #5433 )
...
Signed-off-by: Tao Li <tomscut@apache.org>
2023-03-01 09:34:12 +08:00
slfan1989
bcc51ce2c5
YARN-11375. [Federation] Support refreshAdminAcls、refreshServiceAcls API's for Federation. ( #5312 )
2023-02-28 14:44:00 -08:00
Steve Loughran
dcd9dc6983
HADOOP-18641. Cloud connector dependency and LICENSE fixup. ( #5429 )
...
POM and LICENSE fixup of transient dependencies
* Exclude hadoop-cloud-storage imports which come in with hadoop-common
* Add explicit import of hadoop's org.codehaus.jettison declaration
to hadoop-aliyun
* Tune aliyun jars imports
* Update LICENSE-binary for the current set of libraries.
Contributed by Steve Loughran
2023-02-28 10:48:54 +00:00
rdingankar
0ca5686034
HDFS-16917 Add transfer rate quantile metrics for DataNode reads ( #5397 )
...
Co-authored-by: Ravindra Dingankar <rdingankar@linkedin.com>
2023-02-27 18:26:32 +00:00
Simbarashe Dzinamarira
61f369c43e
HDFS-16890: RBF: Ensures router periodically refreshes its record of a namespace's state. ( #5298 )
2023-02-27 17:56:24 +00:00
slfan1989
8798b94ee1
YARN-11221. [Federation] Add replaceLabelsOnNodes, replaceLabelsOnNode REST APIs for Router. ( #5302 )
2023-02-27 09:34:39 -08:00
Viraj Jasani
a90238c0b8
HADOOP-18631. Migrate Async appenders to log4j properties ( #5418 )
2023-02-26 01:47:44 +08:00
slfan1989
25ebd0b8b1
YARN-11222. [Federation] Add addToClusterNodeLabels, removeFromClusterNodeLabels REST APIs for Router. ( #5328 )
2023-02-24 10:52:57 -08:00
slfan1989
27a54955f9
YARN-5604. [Federation] Add versioning for FederationStateStore. ( #5394 )
2023-02-24 10:51:19 -08:00
Steve Loughran
4067facae6
HADOOP-18470. Remove HDFS RBF text in the 3.3.5 index.md file
...
+ add a link to mukund's apachecon talk
Change-Id: I3d04b385ff1312aabf2a81d034f54f124d544a54
2023-02-23 13:23:35 +00:00
Steve Loughran
e2d7919dc1
Revert "HADOOP-18590. Publish SBOM artifacts ( #5281 )"
...
Causes HADOOP-18641. cyclonedx maven plugin breaks on recent maven releases
This reverts commit 6f99558c2e
2023-02-23 11:23:53 +00:00
Owen O'Malley
8025a60ae7
HDFS-16901: Minor fix for unit test.
...
Signed-off-by: Owen O'Malley <oomalley@linkedin.com>
2023-02-22 16:37:49 -08:00
Simbarashe Dzinamarira
4cc33e5e37
HDFS-16901: RBF: Propagates real user's username via the caller context, when a proxy user is being used. ( #5346 )
2023-02-22 21:58:44 +00:00
slfan1989
2e997d818d
YARN-11370. [Federation] Refactor MemoryFederationStateStore code. ( #5126 )
2023-02-22 12:37:35 -08:00
Ayush Saxena
e8a6b2c2c4
HADOOP-18582. Addendum: Skip unnecessary cleanup logic in DistCp. ( #5409 )
...
Followup to the original HADOOP-18582.
Temporary path cleanup is re-enabled for -append jobs
as these will create temporary files when creating or overwriting files.
Contributed by Ayush Saxena
2023-02-22 19:29:41 +00:00
hchaverr
fb31393b65
HADOOP-18535. Implement token storage solution based on MySQL
...
Fixes #1240
Signed-off-by: Owen O'Malley <oomalley@linkedin.com>
2023-02-22 10:38:50 -08:00
Steve Loughran
11a220c6e7
HADOOP-18636 LocalDirAllocator cannot recover from directory tree deletion ( #5412 )
...
Even though DiskChecker.mkdirsWithExistsCheck() will create the directory tree,
it is only called *after* the enumeration of directories with available
space has completed.
Directories which don't exist are reported as having 0 space, therefore
the mkdirs code is never reached.
Adding a simple mkdirs() -without bothering to check the outcome-
ensures that if a dir has been deleted then it will be reconstructed
if possible. If it can't it will still have 0 bytes of space
reported and so be excluded from the allocation.
Contributed by Steve Loughran
2023-02-22 11:48:12 +00:00
susheel-gupta
49b8ac19f2
YARN-11408. Add a check of autoQueueCreation is disabled for emitDefaultUserLimitFactor method ( #5278 )
...
* added a check of autoQueueCreationV2Disabled
Change-Id: If1e36c5969d270c1b81a4bbd2e883fa819c81f20
* added check of AutoCreateChildQueueDisabled
Change-Id: Ia011b4393ccd8d4d419a2e46b06a5237d050851c
* removed auto-create-child-queue-enabled check and emit
Change-Id: I7a154124519ecbd81379b46a238707c16db1e82a
2023-02-22 09:46:42 +01:00
