Karthik Palanisamy
5250cd6db3
HDFS-15201 SnapshotCounter hits MaxSnapshotID limit ( #1870 )
2020-03-24 15:15:46 +05:30
He Xiaoqiao
5eddc82fb8
HDFS-15088. RBF: Correct annotation typo of RouterPermissionChecker#checkPermission. Contributed by Xiaoqiao He.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2020-03-23 22:41:31 -07:00
Akira Ajisaka
f59f6891c8
HDFS-15232. Fix libhdfspp test failures with GCC 7. ( #1906 )
2020-03-24 11:36:14 +09:00
He Xiaoqiao
af64ce2f4a
HDFS-15113. Addendum: Missing IBR when NameNode restart if open processCommand async feature. Contributed by Xiaoqiao He.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2020-03-23 12:43:51 -07:00
Mehakmeet Singh
e2c7ac71b5
ABFS Streams to update FileSystem.Statistics counters on IO.
...
Contributed by Mehakmeet Singh
2020-03-23 13:50:18 +00:00
Prabhu Joseph
d312991ead
YARN-10198. Remove Validation of Queue Exist under Managed Parent Queue
...
Contributed by Peter Bacsko. Reviewed by Manikandan R.
2020-03-23 18:01:14 +05:30
Ayush Saxena
62c8015683
HDFS-15227. NPE if the last block changes from COMMITTED to COMPLETE during FSCK. Contributed by Ayush Saxena.
2020-03-22 11:44:18 +05:30
Takanobu Asanuma
b74e47e31a
HDFS-15214. WebHDFS: Add snapshot counts to Content Summary. Contributed by hemanthboyina.
2020-03-22 01:44:55 +09:00
ishaniahuja
3612317038
HADOOP-16818. ABFS: Combine append+flush calls for blockblob & appendblob
...
Contributed by Ishani Ahuja.
2020-03-20 10:27:41 +00:00
Vinayakumar B
f02d5abacd
HADOOP-16927. Update hadoop-thirdparty dependency version to 1.0.0 ( #1900 ). Contributed by Vinayakumar B.
2020-03-20 15:13:07 +05:30
Ayush Saxena
f9bb2a8cc5
HDFS-15200. Delete Corrupt Replica Immediately Irrespective of Replicas On Stale Storage. Contributed by Ayush Saxena.
2020-03-19 22:26:34 +05:30
Ayush Saxena
bb41ddaf1e
HDFS-15223. FSCK fails if one namenode is not available. Contributed by Ayush Saxena.
2020-03-19 21:23:13 +05:30
Prabhu Joseph
f2d3ac2a3f
YARN-10034. Remove Allocation Tags from released container from Decommission node
...
Contributed by Kyungwan Nam. Reviewed by Adam Antal.
2020-03-19 12:55:14 +05:30
Ayush Saxena
654db35fa2
HDFS-14919. Provide Non DFS Used per DataNode in DataNode UI. Contributed by Lisheng Sun.
2020-03-18 21:00:45 +05:30
bilaharith
6ce5f8734f
HADOOP-16920 ABFS: Make list page size configurable.
...
Contributed by Bilahari T H.
The page limit is set in "fs.azure.list.max.results"; default value is 500.
There's currently a limit of 5000 in the store -there are no range checks
in the client code so that limit can be changed on the server without
any need to update the abfs connector.
2020-03-18 14:14:18 +00:00
Akira Ajisaka
367833cf41
HADOOP-16054. Update Dockerfile to use Bionic.
...
Contributed by Akira Ajisaka.
2020-03-18 13:44:44 +00:00
Wei-Chiu Chuang
096533c2dc
HDFS-15208. Suppress bogus AbstractWadlGeneratorGrammarGenerator in KMS stderr in hdfs.
...
Contributed by Wei-Chiu Chuang.
Change-Id: I7abeaf4b65e2a4758356aa69ab8930f7e993077d
2020-03-18 13:27:13 +00:00
Gabor Bota
c91ff8c18f
HADOOP-16858. S3Guard fsck: Add option to remove orphaned entries ( #1851 ). Contributed by Gabor Bota.
...
Adding a new feature to S3GuardTool's fsck: -fix.
Change-Id: I2cdb6601fea1d859b54370046b827ef06eb1107d
2020-03-18 12:48:52 +01:00
Steve Loughran
8d6373483e
HADOOP-16319. S3A Etag tests fail with default encryption enabled on bucket.
...
Contributed by Ben Roling.
ETag values are unpredictable with some S3 encryption algorithms.
Skip ITestS3AMiscOperations tests which make assertions about etags
when default encryption on a bucket is enabled.
When testing with an AWS an account which lacks the privilege
for a call to getBucketEncryption(), we don't skip the tests.
In the event of failure, developers get to expand the
permissions of the account or relax default encryption settings.
2020-03-17 13:31:48 +00:00
Eric Badger
1975479285
YARN-2710. RM HA tests failed intermittently on trunk. Contributed by Ahmed
...
Hussein.
2020-03-16 22:24:02 +00:00
Wei-Chiu Chuang
f197f05cff
HADOOP-16661. Support TLS 1.3 ( #1880 )
2020-03-16 10:56:30 -07:00
Wanqiang Ji
ea688631b0
MAPREDUCE-7237. Supports config the shuffle's path cache related parameters ( #1397 )
2020-03-16 11:28:36 +09:00
Surendra Singh Lilhore
1d772dc542
HDFS-15211. EC: File write hangs during close in case of Exception during updatePipeline. Contributed by Ayush Saxena.
2020-03-15 20:44:32 +05:30
Ayush Saxena
74780c22eb
HDFS-15197. [SBN read] Change ObserverRetryOnActiveException log to debug. Contributed by Chen Liang.
2020-03-15 16:30:39 +05:30
Ayush Saxena
8a7c54995a
HDFS-15159. Prevent adding same DN multiple times in PendingReconstructionBlocks. Contributed by hemanthboyina.
2020-03-15 16:16:27 +05:30
Wei-Chiu Chuang
7b637a3341
HDFS-14820. The default 8KB buffer of BlockReaderRemote#newBlockReader#BufferedOutputStream is too big. Contributed by Lisheng Sun.
2020-03-13 19:01:23 -07:00
He Xiaoqiao
e9955bb8ff
HDFS-15113. Missing IBR when NameNode restart if open processCommand async feature. Contributed by Xiaoqiao He.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Brahma Reddy Battula <brahma@apache.org>
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
2020-03-13 18:55:06 -07:00
Weiwei Yang
b9d825f178
YARN-9567. Add diagnostics for outstanding resource requests on app attempts page. Contributed by Tao Yang.
2020-03-13 14:39:36 -07:00
Weiwei Yang
d273f6a2b4
YARN-9538. Document scheduler/app activities and REST APIs. Contributed by Tao Yang.
2020-03-13 14:36:52 -07:00
Wei-Chiu Chuang
4b95c242ec
HDFS-14743. Enhance INodeAttributeProvider/ AccessControlEnforcer Interface in HDFS to support Authorization of mkdir, rm, rmdir, copy, move etc... ( #1829 )
...
Reviewed-by: Xiaoyu Yao <xyao@apache.org>
2020-03-13 11:29:44 -07:00
Wei-Chiu Chuang
cd91ef649e
YARN-10195. Dependency divergence building Timeline Service on HBase 2.2.0 and above. ( #1895 )
2020-03-13 11:20:28 -07:00
Ayush Saxena
fb1d7635ae
HDFS-15155. writeIoRate of DataNodeVolumeMetrics is never used. Contributed by Haibin Huang.
2020-03-13 21:10:03 +05:30
S O'Donnell
25497799cb
HDFS-15216. Wrong Use Case of -showprogress in fsck. Contributed by Ravuri Sushma sree.
2020-03-13 15:24:19 +00:00
Bilwa
c83644deac
YARN-10110. Adding RouterPolicyProvider for RM Federation.
...
Contributed by Bilwa S T.
2020-03-13 16:56:44 +05:30
Wei-Chiu Chuang
20903f72b4
HDFS-15039. Cache meta file length of FinalizedReplica to reduce call File.length(). Contributed by Yang Yun.
2020-03-12 19:23:12 -07:00
Inigo Goiri
0695f7a538
HDFS-14612. SlowDiskReport won't update when SlowDisks is always empty in heartbeat. Contributed by Haibin Huang.
2020-03-12 17:22:11 -07:00
Steve Loughran
0a9b3c98b1
HADOOP-15430. hadoop fs -mkdir -p path-ending-with-slash/ fails with s3guard ( #1646 )
...
Contributed by Steve Loughran
* move qualify logic to S3AFileSystem.makeQualified()
* make S3AFileSystem.qualify() a private redirect to that
* ITestS3GuardFsShell turned off
2020-03-12 14:13:55 +00:00
Surendra Singh Lilhore
f736408a83
HDFS-14442. Disagreement between HAUtil.getAddressOfActive and RpcInvocationHandler.getConnectionId. Contributed by Ravuri Sushma sree.
2020-03-12 18:59:17 +05:30
Szilard Nemeth
5ead9c15ca
YARN-9997. Code cleanup in ZKConfigurationStore. Contributed by Andras Gyori
2020-03-12 12:29:03 +01:00
Szilard Nemeth
38d87883b6
YARN-10193. FS-CS converter: fix incorrect capacity conversion. Contributed by Peter Bacsko
2020-03-12 12:18:37 +01:00
Sunil G
0fd8bf5f6b
YARN-10191. FS-CS converter: call System.exit function call for every code path in main method. Contributed by Peter Bacsko.
2020-03-12 15:03:12 +05:30
bilaharith
0b931f36ec
Hadoop 16890. Change in expiry calculation for MSI token provider.
...
Contributed by Bilahari T H
2020-03-11 20:39:10 +00:00
Szilard Nemeth
cf9cf83a43
YARN-9354. Resources should be created with ResourceTypesTestHelper instead of TestUtils. Contributed by Andras Gyori
2020-03-10 16:44:48 +01:00
Szilard Nemeth
61f4cf3055
YARN-10002. Code cleanup and improvements in ConfigurationStoreBaseTest. Contributed by Benjamin Teke
2020-03-10 16:35:04 +01:00
Szilard Nemeth
9314ef947f
YARN-10168. FS-CS Converter: tool doesn't handle min/max resource conversion correctly. Contributed by Peter Bacsko
2020-03-10 16:07:46 +01:00
Eric E Payne
ede05b19d1
YARN-942. TestContainerSchedulerQueuing.testKillOnlyRequiredOpportunisticContainers fails sporadically Contributed by Ahmed Hussein (ahussein)
2020-03-10 14:17:45 +00:00
Szilard Nemeth
44afe1154d
YARN-9419. Log a warning if GPU isolation is enabled but LinuxContainerExecutor is disabled. Contribued by Andras Gyori
2020-03-09 16:08:24 +01:00
Steve Loughran
c734d69a55
HADOOP-16898. Batch listing of multiple directories via an (unstable) interface
...
Contributed by Steve Loughran.
This moves the new API of HDFS-13616 into a interface which is implemented by
HDFS RPC filesystem client (not WebHDFS or any other connector)
This new interface, BatchListingOperations, is in hadoop-common,
so applications do not need to be compiled with HDFS on the classpath.
They must cast the FS into the interface.
instanceof can probe the client for having the new interface -the patch
also adds a new path capability to probe for this.
The FileSystem implementation is cut; tests updated as appropriate.
All new interfaces/classes/constants are marked as @unstable.
Change-Id: I5623c51f2c75804f58f915dd7e60cb2cffdac681
2020-03-09 14:51:16 +00:00
Steve Loughran
d4d4c37810
HADOOP-14630 Contract Tests to verify create, mkdirs and rename under a file is forbidden
...
Contributed by Steve Loughran.
Not all stores do complete validation here; in particular the S3A
Connector does not: checking up the entire directory tree to see if a path matches
is a file significantly slows things down.
This check does take place in S3A mkdirs(), which walks backwards up the list of
parent paths until it finds a directory (success) or a file (failure).
In practice production applications invariably create destination directories
before writing 1+ file into them -restricting check purely to the mkdirs()
call deliver significant speed up while implicitly including the checks.
Change-Id: I2c9df748e92b5655232e7d888d896f1868806eb0
2020-03-09 14:44:28 +00:00
Sebastian Nagel
18050bc583
HADOOP-16909 Typo in distcp counters.
...
Contributed by Sebastian Nagel.
2020-03-09 14:37:08 +00:00