Mukund Thakur
0507c4160f
HDFS-15626. TestWebHDFS.testLargeDirectory failing ( #2380 )
...
Fixes the regression caused by HADOOP-17281, where the WebHDFS client
listStatusIterator (correctly) throws NoSuchElementException when next()
runs out of values.
Contributed by Mukund Thakur.
Change-Id: I6cec41c20467920cf21f169653553535414b2680
2020-10-13 13:31:00 +01:00
Steve Loughran
d606ddc12c
Revert "HADOOP-17303. TestWebHDFS.testLargeDirectory failing ( #2380 )"
...
(reverting to change the JIRA number)
This reverts commit 3e1b1dfd77
.
Change-Id: I8261357f38268b5d7c1c13f6ef366d037be972dd
2020-10-13 13:30:38 +01:00
Mukund Thakur
3e1b1dfd77
HADOOP-17303. TestWebHDFS.testLargeDirectory failing ( #2380 )
...
Fixes the regression caused by HADOOP-17281, where the WebHDFS client
listStatusIterator (correctly) throws NoSuchElementException when next()
runs out of values.
Contributed by Mukund Thakur.
2020-10-13 13:27:55 +01:00
Pranav Bheda
be3edd0532
HADOOP-17223 update org.apache.httpcomponents:httpclient to 4.5.13 and httpcore to 4.4.13 ( #2242 )
...
* update org.apache.httpcomponents:httpclient from 4.5.6 to 4.5.13
* update org.apache.httpcomponents:httpcore from 4.4.10 to 4.4.13
2020-10-13 17:54:41 +09:00
Szilard Nemeth
42358169f6
YARN-10454: Add applicationName policy. Contributed by Peter Bacsko
2020-10-13 10:11:10 +02:00
Konstantin V Shvachko
b3786d6c3c
HDFS-15567. [SBN Read] HDFS should expose msync() API to allow downstream applications call it explicitly. Contributed by Konstantin V Shvachko.
2020-10-12 17:26:49 -07:00
Uma Maheswara Rao G
b76b36ebbc
HDFS-15625: Namenode trashEmptier should not init ViewFs on startup ( #2378 ). Contributed by Uma Maheswara Rao G.
2020-10-12 14:22:44 -07:00
Swaroopa Kadam
2e46ef9417
MAPREDUCE-7301: Expose Mini MR Cluster attribute for testing
...
Signed-off-by: Mingliang Liu <liuml07@apache.org>
2020-10-12 10:57:02 -07:00
Dongjoon Hyun
b92f72758b
HADOOP-17258. Magic S3Guard Committer to overwrite existing pendingSet file on task commit ( #2371 )
...
Contributed by Dongjoon Hyun and Steve Loughran
Change-Id: Ibaf8082e60eff5298ff4e6513edc386c5bae0274
2020-10-12 13:39:15 +01:00
Akira Ajisaka
69ef9b1ee8
HDFS-15620. RBF: Fix test failures after HADOOP-17281 ( #2375 )
2020-10-12 09:52:12 +09:00
LeonGao91
de055ec8aa
HDFS-15456. TestExternalStoragePolicySatisfier fails intermittently ( #2299 )
2020-10-10 22:08:36 +05:30
Akira Ajisaka
2de7240d4f
HADOOP-17269. [JDK 11] Upgrade SpotBugs to 4.1.3 to fix false-positive warnings. ( #2374 )
2020-10-10 00:28:05 +09:00
Hui Fei
518a212cff
HDFS-13293. RBF: The RouterRPCServer should transfer client IP via CallerContext to NamenodeRpcServer ( #2363 )
2020-10-09 16:12:22 +08:00
Jim Brennan
deb35a32ba
YARN-10455. TestNMProxy.testNMProxyRPCRetry is not consistent. Contributed by Ahmed Hussein
2020-10-08 18:46:47 +00:00
Steve Loughran
f83e07a20f
HADOOP-17293. S3A to always probe S3 in S3A getFileStatus on non-auth paths
...
This reverts changes in HADOOP-13230 to use S3Guard TTL in choosing when
to issue a HEAD request; fixing tests to compensate.
New org.apache.hadoop.fs.s3a.performance.OperationCost cost,
S3GUARD_NONAUTH_FILE_STATUS_PROBE for use in cost tests.
Contributed by Steve Loughran.
Change-Id: I418d55d2d2562a48b2a14ec7dee369db49b4e29e
2020-10-08 15:35:57 +01:00
Jinglun
52db86b0bb
HADOOP-17021. Add concat fs command ( #1993 )
...
Contributed by Jinglun
2020-10-08 10:36:07 +01:00
Karthik Palanisamy
735e85a1d2
HDFS-15610 Reduced datanode upgrade/hardlink thread from 12 to 6 ( #2365 )
2020-10-08 12:51:04 +05:30
Karthik Palanisamy
df4006eb81
HDFS-15253 Default checkpoint transfer speed, 50mb per second ( #2366 )
2020-10-07 22:09:04 +05:30
Mukund Thakur
82522d60fb
HADOOP-17281 Implement FileSystem.listStatusIterator() in S3AFileSystem ( #2354 )
...
Contains HADOOP-17300: FileSystem.DirListingIterator.next() call should
return NoSuchElementException
Contributed by Mukund Thakur
2020-10-07 13:59:06 +01:00
bshashikant
16aea11c94
HDFS-15611. Add list Snapshot command in WebHDFS. ( #2355 )
2020-10-07 10:34:32 +05:30
Akira Ajisaka
074f0d46af
HDFS-15613. RBF: Router FSCK fails after HDFS-14442. ( #2360 )
...
* Support getHAServiceState in DFSRouter
2020-10-07 13:37:45 +09:00
hemanthboyina
921ca1f554
HDFS-15543. RBF: Write Should allow, when a subcluster is unavailable for RANDOM mount points with fault Tolerance enabled. Contributed by Hemanth Boyina.
2020-10-07 09:58:53 +05:30
Doroszlai, Attila
1cfe5916e2
HADOOP-16990. Update Mockserver. Contributed by Attila Doroszlai.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2020-10-06 14:58:42 -07:00
Jim Brennan
b361f29dda
YARN-10451. RM (v1) UI NodesPage can NPE when yarn.io/gpu resource type is defined. Contributed by Eric Payne
2020-10-06 18:18:08 +00:00
Liang-Chi Hsieh
c9ea344f98
HADOOP-17125. Use snappy-java in SnappyCodec ( #2297 )
...
This switches the SnappyCodec to use the java-snappy codec, rather than the native one.
To use the codec, snappy-java.jar (from org.xerial.snappy) needs to be on the classpath.
This comesin as an avro dependency, so it is already on the hadoop-common classpath,
as well as in hadoop-common/lib.
The version used is now managed in the hadoop-project POM; initially 1.1.7.7
Contributed by DB Tsai and Liang-Chi Hsieh
2020-10-06 17:07:54 +01:00
Ikko Ashimine
4347a5c955
HADOOP-17294. Fix typos existance to existence ( #2357 )
2020-10-06 10:10:44 +09:00
Fei Hui
6ece6407e1
HDFS-15458. TestNameNodeRetryCacheMetrics fails intermittently ( #2356 ). Contributed by Hui Fei.
2020-10-05 22:11:24 +05:30
Siyao Meng
43b0c0b054
HDFS-15607. Create trash dir when allowing snapshottable dir ( #2352 )
2020-10-05 05:02:00 -07:00
Akira Ajisaka
7d4bcb312b
HADOOP-17297. Use Yetus before YETUS-994 to enable adding comments to GitHub ( #2359 )
2020-10-05 19:16:37 +09:00
Adam Antal
a1f7e760df
YARN-10393. MR job live lock caused by completed state container leak in heartbeat between node manager and RM. Contributed by zhenzhao wang and Jim Brennan
2020-10-05 10:09:14 +02:00
Fei Hui
d0d10f7e8f
HADOOP-17276. Extend CallerContext to make it include many items ( #2327 )
2020-10-04 02:02:28 +09:00
Arpit Agarwal
18fa4397e6
MAPREDUCE-7298. Distcp doesn't close the job after the job is completed. Contributed by Aasha Medhi.
...
Change-Id: I63d249bbb18ccedaeee9f10123a78e32f9e54ed2
2020-10-02 08:29:55 -07:00
bilaharith
51598d8b1b
HADOOP-17183. ABFS: Enabling checkaccess on ABFS ( #2331 )
...
Contributed by Bilahari TH
2020-10-01 21:29:05 +01:00
Adam Antal
bb8446e80c
YARN-10447. TestLeafQueue: ActivitiesManager thread might interfere with ongoing stubbing. Contributed by Peter Bacsko
2020-10-01 11:42:04 +02:00
maobaolong
d68d2a5c1e
HADOOP-17287. Support new Instance by non default constructor by ReflectionUtils ( #2341 )
2020-09-30 14:22:55 -07:00
He Xiaoqiao
a490d87eb7
HADOOP-17280. Service-user cost shouldn't be accumulated to totalDecayedCallCost and totalRawCallCost. Contributed by Jinglun.
2020-09-30 12:12:52 +08:00
Xiaoyu Yao
4c5ad57818
HADOOP-17284. Support BCFKS keystores for Hadoop Credential Provider. ( #2334 )
2020-09-29 20:15:27 -07:00
huangtianhua
a89ca56a1b
HDFS-15600 Fix TestRouterQuota#testStorageTypeQuota ( #2347 )
2020-09-29 19:00:55 -07:00
Szilard Nemeth
a7a1f1541a
YARN-10413. Change fs2cs to generate mapping rules in the new format. Contributed by Peter Bacsko
2020-09-29 20:18:26 +02:00
Karen Coppage
75d10f8499
HADOOP-17267. Add debug-level logs in Filesystem.close() ( #2321 )
...
When a filesystem is closed, the FileSystem log will, at debug level,
log the method calling close/closeAll.
At trace level: the full calling stack.
Contributed by Karen Coppage.
2020-09-29 16:06:50 +01:00
Wanqiang Ji
143bdd4188
MAPREDUCE-7289. Fix wrong comment in LongLong.java ( #2338 )
2020-09-29 23:06:25 +09:00
Ayush Saxena
d686a655bc
HDFS-15603. RBF: Fix getLocationsForPath twice in create operation. Contributed by wangzhaohui.
2020-09-29 00:03:11 +05:30
Ayush Saxena
fc8a6dd8a9
HDFS-15591. RBF: Fix webHdfs file display error. Contributed by wangzhaohui.
2020-09-28 23:55:31 +05:30
Fei Hui
984875c624
HDFS-15604. Fix Typo for HdfsDataNodeAdminGuide doc ( #2345 )
2020-09-28 09:54:01 -07:00
Inigo Goiri
eae0035a2d
HDFS-15530. RBF: Fix typo in DFS_ROUTER_QUOTA_CACHE_UPDATE_INTERVAL var definition. Contributed by Sha Fanghao.
2020-09-28 09:40:39 -07:00
Takanobu Asanuma
d89af79507
HDFS-15577. Refactor TestTracing. ( #2302 )
...
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
2020-09-28 18:27:41 +09:00
Vinayakumar B
82b86e3754
HDFS-15098. Add SM4 encryption method for HDFS. Contributed by liusheng
2020-09-27 19:27:13 +05:30
Aryan Gupta
bbbfa7d415
HADOOP-17286. Updated jQuery in yarn-common. ( #2340 )
2020-09-27 03:18:01 +09:00
Uma Maheswara Rao G
899dea2a21
HDFS-15598: ViewHDFS#canonicalizeUri should not be restricted to DFS only API. ( #2339 ). Contributed by Uma Maheswara Rao G.
...
Co-authored-by: Uma Maheswara Rao G <umagangumalla@cloudera.com>
2020-09-25 21:21:01 -07:00
Ye Ni
00c4de63cf
HDFS-15594. Lazy calculate live datanodes in safe mode tip ( #2332 )
2020-09-25 09:47:54 -07:00