Viraj Jasani
3b65b5d68f
HDFS-17020. RBF: mount table addAll should print failed records in std error ( #5674 )
2023-05-22 18:39:52 -07:00
Gautham B A
afe850ca2c
HADOOP-18746. Install Python 3 for Windows 10 docker image ( #5679 )
...
* This PR installs Python 3.10.11 for
Windows 10 Docker image to fix
the issue with building mvnsite.
* After installing Python 3.10.11, it
creates the hardlink python -> python3
as required by the script.
2023-05-21 21:10:04 +05:30
hfutatzhanghb
5b22dc6ace
HDFS-16909. Improve ReplicaMap#mergeAll method. ( #5353 ). Contributed by ZhangHB.
...
Reviewed-by: zhangshuyan <zqingchai@gmail.com>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-05-21 19:08:58 +08:00
huhaiyang
af933f3a4f
HDFS-17017. Fix the issue of arguments number limit in report command in DFSAdmin ( #5667 ). Contributed by Haiyang Hu.
...
Reviewed-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-21 09:03:34 +05:30
Xianming Lei
0110e24ed8
YARN-11496. Improve TimelineService log format. ( #5677 ). Contributed by Xianming Lei.
...
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-20 14:57:45 +05:30
NishthaShah
9a524ede87
HDFS-17022. Fix the exception message to print the Identifier pattern ( #5678 ). Contributed by Nishtha Shah.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-20 14:40:23 +05:30
zhtttylz
408dbf318e
HDFS-17014. HttpFS Add Support getStatus API ( #5660 ). Contributed by Hualong Zhang.
...
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-20 14:37:38 +05:30
NishthaShah
5272ed8670
HADOOP-17518. Update the regex to A-Z ( #5669 ). Contributed by Nishtha Shah.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-20 06:21:13 +05:30
Keyao Li
0914b3e792
HDFS-16697. Add logs if resources are not available in NameNodeResourcePolicy. ( #5569 ). Contributed by ECFuzz.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-19 18:38:01 +05:30
Keyao Li
339bc7b3a6
HDFS-16653. Improve error messages in ShortCircuitCache. ( #5568 ). Contributed by ECFuzz.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-19 07:43:18 +05:30
Xianming Lei
441fb23293
HDFS-17018. Improve dfsclient log format. ( #5668 ). Contributed by Xianming Lei.
...
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-19 06:25:43 +05:30
Patrick GRANDJEAN
4627242c44
HADOOP-18652. Path.suffix raises NullPointerException ( #5653 ). Contributed by Patrick Grandjean.
...
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-19 05:16:55 +05:30
LiuGuH
f6770dee47
HDFS-16979. RBF: Add proxyuser port in hdfsauditlog ( #5552 ). Contributed by liuguanghua.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: Simbarashe Dzinamarira <sdzinamarira@linkedin.com>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-19 05:02:16 +05:30
slfan1989
bba663038d
YARN-8898. Fix FederationInterceptor#allocate to set application priority in allocateResponse. ( #5645 )
2023-05-18 11:57:38 -07:00
Peter Szucs
ff8eac517a
YARN-11463. Node Labels root directory creation doesn't have a retry logic - 2nd addendum ( #5670 )
2023-05-18 14:48:43 +02:00
jianghuazhu
78cc528739
HDFS-17012.Remove unused DFSConfigKeys#DFS_DATANODE_PMEM_CACHE_DIRS_DEFAULT. ( #5659 ). Contributed by JiangHua Zhu.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-18 14:27:19 +05:30
Tsz-Wo Nicholas Sze
350dcaf616
HDFS-17010. Add a subtree test to TestSnapshotDiffReport. ( #5656 )
2023-05-18 15:53:26 +08:00
wangzhaohui
03163f9de2
HDFS-17011. Fix the metric of "HttpPort" at DataNodeInfo ( #5657 ). Contributed by Zhaohui Wang.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-18 12:12:11 +05:30
slfan1989
5ddaf2e133
YARN-11493. [Federation] ConfiguredRMFailoverProxyProvider Supports Randomly Select an Router. ( #5651 )
2023-05-17 11:09:10 -07:00
Viraj Jasani
8e17385141
HDFS-17009. RBF: state store putAll should also return failed records ( #5664 )
2023-05-17 09:33:34 -07:00
liang3zy22
482897a0f6
Fix typos in HDFS documents. ( #5665 )
2023-05-17 09:28:01 -07:00
Steve Loughran
a90c722143
HADOOP-18724. [FOLLOW-UP] cherrypick changes from branch-3.3 backport ( #5662 )
...
* move FileContext.copy() onto optLong()
* move FileUtil onto optLong()
This brings trunk into sync with the branch-3.3 changes
2023-05-16 18:16:24 +01:00
Viraj Jasani
bef40e9427
HADOOP-18688. S3A audit header to include count of items in delete ops ( #5621 )
...
The auditor-generated http referrer URL now includes the count of keys
to delete in the "ks" query parameter
Contributed by Viraj Jasani
2023-05-16 10:40:16 +01:00
Chun Chen
11af08d67a
YARN-11489. Fix memory leak of DelegationTokenRenewer futures in DelegationTokenRenewerPoolTracker. ( #5629 ). Contributed by Chun Chen.
...
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-05-14 21:38:04 +08:00
smarthan
251439d769
HDFS-16985. Fix data missing issue when delete local block file. ( #5564 ). Contributed by Chengwei Wang.
...
Reviewed-by: Shuyan Zhang <zqingchai@gmail.com>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-05-14 21:33:38 +08:00
slfan1989
e0938b4c2a
YARN-11495. Fix typos in hadoop-yarn-server-web-proxy. ( #5652 ). Contributed by Shilun Fan.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-13 11:41:38 +05:30
slfan1989
2f87f716fa
YARN-3660. BackPort [GPG] Federation Global Policy Generator (service hook only). ( #5625 )
2023-05-12 18:12:05 -07:00
Viraj Jasani
5d0cc455f5
HDFS-17008. Fix RBF JDK 11 javadoc warnings ( #5648 )
2023-05-12 17:27:13 -07:00
zhtttylz
0c77629849
HDFS-17001. Support getStatus API in WebHDFS ( #5628 ). Contributed by Hualong Zhang.
...
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-13 03:49:13 +05:30
Steve Loughran
ad1e3a0f5b
HADOOP-18724. (followup) remove deprecation on optLong/optDouble methods ( #5650 )
...
Somehow @Deprecated crept in to the declaration of the
new FSBuilder optLong/optDouble methods.
2023-05-12 15:22:37 +01:00
susheel-gupta
0f3406ac34
YARN-11312: [UI2] Refresh buttons don't work after EmberJS upgrade ( #5649 )
2023-05-12 14:20:26 +02:00
WangYuanben
905bfa84a8
HDFS-16965. Add switch to decide whether to enable native codec. ( #5520 ). Contributed by WangYuanben.
...
Reviewed-by: Tao Li <tomscut@apache.org>
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-12 04:12:02 +05:30
Steve Loughran
e76c09ac3b
HADOOP-18724. Open file fails with NumberFormatException for S3AFileSystem ( #5611 )
...
This:
1. Adds optLong, optDouble, mustLong and mustDouble
methods to the FSBuilder interface to let callers explicitly
passin long and double arguments.
2. The opt() and must() builder calls which take float/double values
now only set long values instead, so as to avoid problems
related to overloaded methods resulting in a ".0" being appended
to a long value.
3. All of the relevant opt/must calls in the hadoop codebase move to
the new methods
4. And the s3a code is resilient to parse errors in is numeric options
-it will downgrade to the default.
This is nominally incompatible, but the floating-point builder methods
were never used: nothing currently expects floating point numbers.
For anyone who wants to safely set numeric builder options across all compatible
releases, convert the number to a string and then use the opt(String, String)
and must(String, String) methods.
Contributed by Steve Loughran
2023-05-11 17:57:25 +01:00
Viraj Jasani
fe61d8f073
HDFS-16978. RBF: Admin command to support bulk add of mount points ( #5554 ). Contributed by Viraj Jasani.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: Simbarashe Dzinamarira <sdzinamarira@linkedin.com>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-11 08:45:34 +05:30
zhtttylz
5084e881ef
HDFS-16990. HttpFS Add Support getFileLinkStatus API ( #5602 ). Contributed by Hualong Zhang.
...
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-10 23:10:19 +05:30
slfan1989
690db3c34b
YARN-11479. [Federation] ZookeeperFederationStateStore Support Store ApplicationSubmitData. ( #5631 )
2023-05-10 09:37:47 -07:00
cxzl25
be50d221f5
YARN-11467. RM failover may fail when the nodes.exclude-path file does not exist ( #5565 )
2023-05-10 15:16:33 +08:00
slfan1989
d95b5c679d
YARN-11424. [Federation] Router Supports DeregisterSubCluster. ( #5363 )
2023-05-09 16:17:23 -07:00
slfan1989
a2dda0ce03
HADOOP-18359. Update commons-cli from 1.2 to 1.5. ( #5095 ). Contributed by Shilun Fan.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-10 01:42:12 +05:30
zhangshuyan
03bf8f982a
HDFS-16999. Fix wrong use of processFirstBlockReport(). ( #5622 ). Contributed by Shuyan Zhang.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-05-09 10:05:28 +08:00
slfan1989
bdeca45294
YARN-11340. [Federation] Improve SQLFederationStateStore DataSource Config. ( #5403 )
2023-05-08 10:13:09 -07:00
Gautham B A
a80e3dba3b
HADOOP-18734. Create qbt.sh symlink on Windows ( #5626 )
2023-05-08 09:55:15 -07:00
ZanderXu
4ee92efb73
HDFS-16865. The source path is always / after RBF proxied the complete, addBlock and getAdditionalDatanode RPC. ( #5200 ). Contributed by ZanderXu.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-06 08:50:21 +05:30
slfan1989
cda9863d54
YARN-11477. [Federation] MemoryFederationStateStore Support Store ApplicationSubmitData. ( #5616 )
2023-05-05 13:43:40 -07:00
slfan1989
eab4c33d09
YARN-11470. FederationStateStoreFacade Cache Support Guava Cache. ( #5609 )
2023-05-05 13:41:38 -07:00
Gautham B A
c974710d8e
HADOOP-18729. Fix mvnsite on Windows 10 ( #5618 )
2023-05-05 13:08:58 -07:00
Chris
372631c566
HADOOP-18706. Improve S3ABlockOutputStream recovery ( #5563 )
...
Contributed by Chris Bevard
2023-05-05 11:57:42 +01:00
Szilard Nemeth
c7699d3dcd
YARN-11079. Make an AbstractParentQueue to store common ParentQueue and ManagedParentQueue functionality. Contributed by Susheel Gupta
2023-05-04 22:16:18 -04:00
Viraj Jasani
ceb8878d4f
HDFS-16998. RBF: Add ops metrics for getSlowDatanodeReport in RouterClientActivity ( #5615 )
2023-05-04 09:45:40 -07:00
Dongjoon Hyun
27776ac45e
HADOOP-18727. Fix WriteOperations.listMultipartUploads function description ( #5613 )
...
Contributed by Dongjoon Hyun
2023-05-04 13:03:48 +01:00