hadoop

Author	SHA1	Message	Date
Yizhong Zhang	82de07c794	HADOOP-17142. Fix outdated properties of JournalNode when performing rollback. Contributed by Deegue.	2020-11-10 09:11:31 -08:00
maobaolong	95c96605b3	HDFS-15667. Audit log record the unexpected allowed result when delete (#2437 )	2020-11-10 13:01:10 +08:00
LeonGao	9a9ab5b48e	HDFS-15548. Allow configuring DISK/ARCHIVE storage types on same device mount (#2288 ). Contributed by Leon Gao.	2020-11-09 15:06:16 -08:00
He Xiaoqiao	3067a25fa1	HDFS-15651. Client could not obtain block when DN CommandProcessingThread exit. Contributed by Aiphago. Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org> Reviewed-by: Yiqun Lin <yqlin@apache.org>	2020-11-04 13:53:46 +08:00
Ayush Saxena	7ac7840fc3	HDFS-15643. EC: Fix checksum computation in case of native encoders. (#2424 ). Contributed by Ayush Saxena.	2020-11-04 09:28:44 +05:30
Konstantin V Shvachko	d07dc7afb4	HDFS-15665. Balancer logging improvements. Contributed by Konstantin V Shvachko.	2020-11-03 12:02:40 -08:00
Ahmed Hussein	324879127a	HDFS-15654. TestBPOfferService#testMissBlocksWhenReregister fails intermittently (#2419 )	2020-10-28 16:24:34 -07:00
Ahmed Hussein	d0c786db4d	HDFS-15460. TestFileCreation#testServerDefaultsWithMinimalCaching fails intermittently. (#2406 )	2020-10-27 18:56:40 -07:00
Ahmed Hussein	98097b8f19	HDFS-15457. TestFsDatasetImpl fails intermittently (#2407 )	2020-10-27 18:52:56 -07:00
Hui Fei	8a6d5b9151	HDFS-15652. Make block size from NNThroughputBenchmark configurable (#2416 )	2020-10-28 09:13:25 +08:00
Ahmed Hussein	ae74407ac4	HDFS-9776. testMultipleAppendsDuringCatchupTailing is flaky (#2410 ) Co-authored-by: ahussein <ahmed.hussein@verizonmedia.com>	2020-10-27 20:45:00 +09:00
Akira Ajisaka	3848b44b37	HDFS-15580. [JDK 12] DFSTestUtil#addDataNodeLayoutVersion fails (#2309 )	2020-10-27 19:18:08 +09:00
Ahmed Hussein	afaab3d332	HDFS-15461. TestDFSClientRetries testGetFileChecksum fails (#2404 ) Co-authored-by: ahussein <ahmed.hussein@verizonmedia.com>	2020-10-27 16:56:30 +09:00
Hui Fei	12c908c827	HDFS-15641. DataNode could meet deadlock if invoke refreshNameNode. Contributed by Hongbing Wang.	2020-10-26 22:12:27 +08:00
Wei-Chiu Chuang	74634eb002	HDFS-15644. Failed volumes can cause DNs to stop block reporting. Contributed by Ahmed Hussein.	2020-10-23 12:05:53 -07:00
Inigo Goiri	e7aa4da0a2	HDFS-15459. TestBlockTokenWithDFSStriped fails intermittently. Contributed by Ahmed Hussein.	2020-10-23 09:47:10 -07:00
Kihwal Lee	da1b6e3cc2	HDFS-15622. Deleted blocks linger in the replications queue. Contributed by Ahmed Hussein.	2020-10-22 16:59:49 -05:00
Kihwal Lee	88a9f42f32	HDFS-15618. Improve datanode shutdown latency. Contributed by Ahmed Hussein.	2020-10-21 00:59:35 -05:00
Ayush Saxena	2e8cafac3b	HDFS-14383. Compute datanode load based on StoragePolicy. Contributed by Ayush Saxena.	2020-10-19 10:48:47 +05:30
Ayush Saxena	173310e2f5	HDFS-15629. Add seqno when warning slow mirror/disk in BlockReceiver. Contributed by Haibin Huang.	2020-10-19 10:34:17 +05:30
Ayush Saxena	1e3a6efcef	HADOOP-17288. Use shaded guava from thirdparty. (#2342 ). Contributed by Ayush Saxena.	2020-10-17 12:01:18 +05:30
Kihwal Lee	740a2c4635	HDFS-15627. Log delete audits before collecting blocks.	2020-10-16 11:13:41 -05:00
Siyao Meng	a308a1ec22	HDFS-15614. Initialize snapshot trash root during NameNode startup if enabled (#2370 )	2020-10-13 10:59:42 -07:00
Gabor Bota	59f01a548e	HADOOP-16878. FileUtil.copy() to throw IOException if the source and destination are the same Contributed by Gabor Bota.	2020-10-13 16:17:44 +01:00
Mukund Thakur	0507c4160f	HDFS-15626. TestWebHDFS.testLargeDirectory failing (#2380 ) Fixes the regression caused by HADOOP-17281, where the WebHDFS client listStatusIterator (correctly) throws NoSuchElementException when next() runs out of values. Contributed by Mukund Thakur. Change-Id: I6cec41c20467920cf21f169653553535414b2680	2020-10-13 13:31:00 +01:00
Steve Loughran	d606ddc12c	Revert "HADOOP-17303. TestWebHDFS.testLargeDirectory failing (#2380 )" (reverting to change the JIRA number) This reverts commit `3e1b1dfd77`. Change-Id: I8261357f38268b5d7c1c13f6ef366d037be972dd	2020-10-13 13:30:38 +01:00
Mukund Thakur	3e1b1dfd77	HADOOP-17303. TestWebHDFS.testLargeDirectory failing (#2380 ) Fixes the regression caused by HADOOP-17281, where the WebHDFS client listStatusIterator (correctly) throws NoSuchElementException when next() runs out of values. Contributed by Mukund Thakur.	2020-10-13 13:27:55 +01:00
Konstantin V Shvachko	b3786d6c3c	HDFS-15567. [SBN Read] HDFS should expose msync() API to allow downstream applications call it explicitly. Contributed by Konstantin V Shvachko.	2020-10-12 17:26:49 -07:00
Uma Maheswara Rao G	b76b36ebbc	HDFS-15625: Namenode trashEmptier should not init ViewFs on startup (#2378 ). Contributed by Uma Maheswara Rao G.	2020-10-12 14:22:44 -07:00
LeonGao91	de055ec8aa	HDFS-15456. TestExternalStoragePolicySatisfier fails intermittently (#2299 )	2020-10-10 22:08:36 +05:30
Karthik Palanisamy	735e85a1d2	HDFS-15610 Reduced datanode upgrade/hardlink thread from 12 to 6 (#2365 )	2020-10-08 12:51:04 +05:30
Karthik Palanisamy	df4006eb81	HDFS-15253 Default checkpoint transfer speed, 50mb per second (#2366 )	2020-10-07 22:09:04 +05:30
bshashikant	16aea11c94	HDFS-15611. Add list Snapshot command in WebHDFS. (#2355 )	2020-10-07 10:34:32 +05:30
Fei Hui	6ece6407e1	HDFS-15458. TestNameNodeRetryCacheMetrics fails intermittently (#2356 ). Contributed by Hui Fei.	2020-10-05 22:11:24 +05:30
Siyao Meng	43b0c0b054	HDFS-15607. Create trash dir when allowing snapshottable dir (#2352 )	2020-10-05 05:02:00 -07:00
Fei Hui	984875c624	HDFS-15604. Fix Typo for HdfsDataNodeAdminGuide doc (#2345 )	2020-09-28 09:54:01 -07:00
Takanobu Asanuma	d89af79507	HDFS-15577. Refactor TestTracing. (#2302 ) Reviewed-by: Akira Ajisaka <aajisaka@apache.org>	2020-09-28 18:27:41 +09:00
Vinayakumar B	82b86e3754	HDFS-15098. Add SM4 encryption method for HDFS. Contributed by liusheng	2020-09-27 19:27:13 +05:30
Ye Ni	00c4de63cf	HDFS-15594. Lazy calculate live datanodes in safe mode tip (#2332 )	2020-09-25 09:47:54 -07:00
Aryan Gupta	486ddb73f9	HADOOP-17283. Hadoop - Upgrade to jQuery 3.5.1 (#2330 ) Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>	2020-09-24 21:15:55 +09:00
huangtianhua	ff59fbb8b0	HDFS-15025. Applying NVDIMM storage media to HDFS (#2189 ) * HDFS-15025. Applying NVDIMM storage media to HDFS Co-authored-by: YaYun-Wang <yywangyayun@163.com> Co-authored-by: YaYun-Wang <34060507+YaYun-Wang@users.noreply.github.com>	2020-09-24 14:27:04 +05:30
bshashikant	368f2f637e	HDFS-15590. namenode fails to start when ordered snapshot deletion feature is disabled (#2326 )	2020-09-24 14:00:41 +05:30
Ye Ni	ae089f2db7	HDFS-15557. Log the reason why a storage log file can't be deleted (#2274 )	2020-09-22 13:23:20 -07:00
Hui Fei	474fa80bfb	HADOOP-17277. Correct spelling errors for separator (#2322 ) Contributed by Hui Fei.	2020-09-22 17:10:33 +01:00
Ayush Saxena	2377df38ad	HDFS-15438. Setting dfs.disk.balancer.max.disk.errors = 0 will fail the block copy. Contributed by AMC-team.	2020-09-18 17:28:11 +05:30
Uma Maheswara Rao G	7bba4c609c	HDFS-15585: ViewDFS#getDelegationToken should not throw UnsupportedOperationException. (#2312 ). Contributed by Uma Maheswara Rao G.	2020-09-18 15:18:10 +05:30
S O'Donnell	20a0e6278d	HDFS-15415. Reduce locking in Datanode DirectoryScanner. Contributed by Stephen O'Donnell	2020-09-17 14:11:42 +01:00
bshashikant	425f48799c	HDFS-15568. namenode start failed to start when dfs.namenode.max.snapshot.limit set. (#2296 )	2020-09-17 14:50:08 +05:30
Uma Maheswara Rao G	e4cb0d3514	HDFS-15578: Fix the rename issues with fallback fs enabled (#2305 ). Contributed by Uma Maheswara Rao G. Co-authored-by: Uma Maheswara Rao G <umagangumalla@cloudera.com>	2020-09-16 22:43:00 -07:00
Takanobu Asanuma	5c5b2ed7c7	HDFS-15576. Erasure Coding: Add rs and rs-legacy codec test for addPolicies. Contributed by Fei Hui.	2020-09-16 01:36:46 +09:00
hemanthboyina	f4ed9f3f91	HDFS-15574. Remove unnecessary sort of block list in DirectoryScanner. Contributed by Stephen O'Donnell.	2020-09-15 10:10:21 +05:30
He Xiaoqiao	924959088e	HDFS-15559. Complement initialize member variables in TestHdfsConfigFields#initializeMemberVariables. Contributed by Lisheng Sun.	2020-09-15 12:05:04 +08:00
Siyao Meng	6d3312de47	HDFS-15539. When disallowing snapshot on a dir, throw exception if its trash root is not empty (#2258 )	2020-09-14 13:31:34 -07:00
Uma Maheswara Rao G	d2779de3f5	HDFS-15532: listFiles on root/InternalDir will fail if fallback root has file. (#2298 ). Contributed by Uma Maheswara Rao G.	2020-09-12 17:06:39 -07:00
Mingliang Liu	f59f7f2175	HDFS-15573. Only log warning if considerLoad and considerStorageType are both true. Contributed by Stephen O'Donnell	2020-09-12 01:41:38 -07:00
hemanthboyina	2a6fdcedeb	HDFS-15564. Add Test annotation for TestPersistBlocks#testRestartDfsWithSync. Contributed by Fei Hui.	2020-09-11 18:21:25 +05:30
1996fanrui	56ebabd426	HADOOP-17222. Create socket address leveraging URI cache (#2241 ) Contributed by fanrui. Signed-off-by: Mingliang Liu <liuml07@apache.org> Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>	2020-09-10 22:30:52 -07:00
zz	12a316cdf9	HADOOP-15891. provide Regex Based Mount Point In Inode Tree (#2185 ). Contributed by Zhenzhao Wang. Co-authored-by: Zhenzhao Wang <zhenzhaowang@gmail.com>	2020-09-10 21:20:32 -07:00
Siyao Meng	43c52d6495	HDFS-15563. Incorrect getTrashRoot return value when a non-snapshottable dir prefix matches the path of a snapshottable dir (#2295 ) Change-Id: Iefcbc14ca939b61f0c1befc9a94d686be1684ddc	2020-09-10 15:55:23 +05:30
He Xiaoqiao	1d6d0d8207	HDFS-14694. Call recoverLease on DFSOutputStream close exception. Contributed by Lisheng Sun. Co-authored-by: Chen Zhang <chzhang1987@gmail.com> Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org> Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>	2020-09-09 20:09:13 +08:00
Uma Maheswara Rao G	ac7d4623ae	HDFS-15558: ViewDistributedFileSystem#recoverLease should call super.recoverLease when there are no mounts configured (#2275 ) Contributed by Uma Maheswara Rao G.	2020-09-07 11:36:13 -07:00
Uma Maheswara Rao G	b3660d0147	HDFS-15529: getChildFilesystems should include fallback fs as well (#2234 ). Contributed by Uma Maheswara Rao G.	2020-09-03 11:06:20 -07:00
He Xiaoqiao	60de592a88	HDFS-15550. Remove unused imports from TestFileTruncate.java. Contributed by Ravuri Sushma sree.	2020-08-31 14:49:12 +08:00
bshashikant	2d03209a06	HDFS-15542. Add identified snapshot corruption tests for ordered snapshot deletion (#2251 )	2020-08-31 11:29:48 +05:30
Steve Loughran	c392d9022a	HDFS-15471. TestHDFSContractMultipartUploader failing (#2252 ) Contributed by Steve Loughran (Was: broken by Steve Loughran)	2020-08-28 15:39:09 +01:00
Vivek Ratnavel Subramanian	06793da100	HDFS-15531. Namenode UI: List snapshots in separate table for each snapshottable directory (#2230 )	2020-08-27 11:36:30 -07:00
Tsz-Wo Nicholas Sze	41182a9b6d	HDFS-15500. In-order deletion of snapshots: Diff lists must be update only in the last snapshot. (#2233 )	2020-08-27 14:54:52 +05:30
Stephen O'Donnell	2ffe00fc46	HDFS-15540. Directories protected from delete can still be moved to the trash. Contributed by Stephen O'Donnell. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2020-08-26 23:05:32 -07:00
S O'Donnell	6a49bf9bff	HDFS-14852. Removing from LowRedundancyBlocks does not remove the block from all queues. Contributed by Fei Hui.	2020-08-25 15:18:36 +01:00
He Xiaoqiao	a7830423c5	HDFS-15448. Remove duplicate BlockPoolManager starting when run DataNode. Contriubted by jianghua zhu.	2020-08-24 21:16:00 +08:00
hemanthboyina	83fd15b412	HDFS-14504. Rename with Snapshots does not honor quota limit.	2020-08-23 16:10:47 +05:30
Chen Liang	f734455e5d	HDFS-15290. NPE in HttpServer during NameNode startup. Contributed by Simbarashe Dzinamarira.	2020-08-20 15:25:10 -07:00
Uma Maheswara Rao G	dd013f2fdf	HDFS-15533: Provide DFS API compatible class, but use ViewFileSystemOverloadScheme inside. (#2229 ). Contributed by Uma Maheswara Rao G.	2020-08-19 09:30:41 -07:00
Siyao Meng	b65e43fe38	HDFS-15525. Make trash root inside each snapshottable directory for WebHDFS (#2220 )	2020-08-18 19:28:19 +09:00
bshashikant	092bfe7c8e	HDFS-15483. Ordered snapshot deletion: Disallow rename between two snapshottable directories. (#2172 ) * HDFS-15483. Ordered snapshot deletion: Disallow rename between two snapshottable directories. * Addressed review comments. * Rebased to latest trunk and added snapshotTrashRoot config check. * Addressed review comments. * Removede empty line added in SnapshotManager.Java. * Addressed whitespace issues.	2020-08-17 12:56:13 +05:30
Ayush Saxena	e3d1966f58	HDFS-15439. Setting dfs.mover.retry.max.attempts to negative value will retry forever. Contributed by AMC-team.	2020-08-15 15:07:58 +05:30
Tsz-Wo Nicholas Sze	b93dd7c281	HDFS-15519. Check inaccessible INodes in FsImageValidation. (#2224 )	2020-08-14 10:10:01 -07:00
bshashikant	15a76e8d65	HDFS-15524. Add edit log entry for Snapshot deletion GC thread snapshot deletion. (#2219 )	2020-08-14 20:46:48 +05:30
Vivek Ratnavel Subramanian	cb50e3fcf7	HDFS-15496. Add UI for deleted snapshots (#2212 )	2020-08-13 22:36:15 +05:30
Aryan Gupta	e592ec5f8b	HDFS-15518. Wrong operation name in FsNamesystem for listSnapshots. (#2217 ) * HDFS-15518. Fixed String operationName = ListSnapshot. * HDFS-15518. Changed ListSnapshot to operationName. Co-authored-by: Aryan Gupta <aryan.gupta@cloudera.com>	2020-08-13 00:09:38 +05:30
S O'Donnell	10716040a8	HDFS-15493. Update block map and name cache in parallel while loading fsimage. Contributed by Chengwei Wang	2020-08-12 09:02:47 +01:00
Tsz-Wo Nicholas Sze	11cec9ab94	HDFS-15523. Fix findbugs warnings from HDFS-15520. (#2218 )	2020-08-11 21:41:28 -07:00
Siyao Meng	3fd3aeb621	HDFS-15492. Make trash root inside each snapshottable directory (#2176 )	2020-08-11 08:52:16 -07:00
Uma Maheswara Rao G	8955a6ceb7	HDFS-15515: mkdirs on fallback should throw IOE out instead of suppressing and returning false (#2205 ) * HDFS-15515: mkdirs on fallback should throw IOE out instead of suppressing and returning false * Used LambdaTestUtils#intercept in test	2020-08-11 00:01:58 -07:00
Tsz-Wo Nicholas Sze	592127bdc2	HDFS-15520 Use visitor pattern to visit namespace tree (#2203 )	2020-08-11 12:30:30 +05:30
Akira Ajisaka	7938ebfb9d	HDFS-15506. [JDK 11] Fix javadoc errors in hadoop-hdfs module. Contributed by Xieming Li.	2020-08-11 11:14:58 +09:00
Ayush Saxena	c2a17659d1	HDFS-15443. Setting dfs.datanode.max.transfer.threads to a very small value can cause strange failure. Contributed by AMC-team.	2020-08-08 14:33:53 +05:30
Ayush Saxena	975b6024dd	HDFS-15514. Remove useless dfs.webhdfs.enabled. Contributed by Fei Hui.	2020-08-07 22:19:17 +05:30
bshashikant	e072d33327	HDFS-15497. Make snapshot limit on global as well per snapshot root directory configurable (#2175 )	2020-08-04 14:10:29 +05:30
hemanthboyina	82f3ffcd64	HDFS-15503. File and directory permissions are not able to be modified from WebUI.	2020-08-03 23:21:01 +05:30
bshashikant	d8a2df25ad	HDFS-15498. Show snapshots deletion status in snapList cmd. (#2181 )	2020-08-02 21:59:56 +05:30
hemanthboyina	528a799a78	HDFS-15229. Truncate info should be logged at INFO level. Contributed by Ravuri Sushma sree.	2020-08-02 12:02:37 +05:30
Tsz-Wo Nicholas Sze	05b3337a46	HDFS-15481. Ordered snapshot deletion: garbage collect deleted snapshots (#2165 )	2020-07-30 23:06:51 +05:30
bshashikant	68287371cc	HDFS-15488. Add a command to list all snapshots for a snaphottable root with snapshot Ids. (#2166 )	2020-07-29 21:33:25 +05:30
touchida	026dce5334	HDFS-15465. Support WebHDFS accesses to the data stored in secure Datanode through insecure Namenode. (#2135 )	2020-07-27 09:55:11 -07:00
bshashikant	2d12496643	HDFS-15480. Ordered snapshot deletion: record snapshot deletion in XAttr (#2163 )	2020-07-22 16:16:27 -07:00
Uma Maheswara Rao G	ac9a07b51a	HDFS-15478: When Empty mount points, we are assigning fallback link to self. But it should not use full URI for target fs. (#2160 ). Contributed by Uma Maheswara Rao G.	2020-07-21 23:29:10 -07:00
Masatake Iwasaki	1b29c9bfee	HADOOP-17138. Fix spotbugs warnings surfaced after upgrade to 4.0.6. (#2155 )	2020-07-22 13:40:20 +09:00
Inigo Goiri	8b7695bb26	HDFS-15246. ArrayIndexOfboundsException in BlockManager CreateLocatedBlock. Contributed by Hemanth Boyina.	2020-07-21 09:00:07 -07:00
Tsz-Wo Nicholas Sze	d57462f2da	HDFS-15479. Ordered snapshot deletion: make it a configurable feature (#2156 )	2020-07-21 11:36:24 +05:30
Shashikant Banerjee	d9441f95c3	HDFS-15470. Added more unit tests to validate rename behaviour across snapshots. Contributed by Shashikant Banerjee.	2020-07-21 11:13:05 +05:30
Chen Liang	3833c616e0	HDFS-15404. ShellCommandFencer should expose info about source. Contributed by Chen Liang.	2020-07-20 12:49:58 -07:00
Ayush Saxena	6cbd8854ee	HDFS-15381. Fix typos corrputBlocksFiles to corruptBlocksFiles. Contributed by bianqi.	2020-07-20 21:49:05 +05:30
Tsz-Wo Nicholas Sze	2cec50cf16	HDFS-15463. Add a tool to validate FsImage. (#2140 )	2020-07-20 11:44:30 +05:30
Ayush Saxena	6bcb24d269	HADOOP-17100. Replace Guava Supplier with Java8+ Supplier in Hadoop. Contributed by Ahmed Hussein.	2020-07-18 14:33:43 +05:30
Ahmed Hussein	1f71c4ae71	HADOOP-17099. Replace Guava Predicate with Java8+ Predicate Signed-off-by: Jonathan Eagles <jeagles@gmail.com>	2020-07-15 11:39:06 -05:00
Ahmed Hussein	98fcffe93f	HADOOP-17101. Replace Guava Function with Java8+ Function Signed-off-by: Jonathan Eagles <jeagles@gmail.com>	2020-07-15 09:53:18 -05:00
Shashikant Banerjee	85d4718ed7	HDFS-15319. Fix INode#isInLatestSnapshot() API. Contributed by Shashikant Banerjee.	2020-07-15 10:32:25 +05:30
zhaoyim	bdce75d737	HDFS-15371. Nonstandard characters exist in NameNode.java (#2032 ) Contributed by zhaoyim	2020-07-14 14:42:12 +01:00
He Xiaoqiao	b97fea65e7	HDFS-14498 LeaseManager can loop forever on the file for which create has failed. Contributed by Stephen O'Donnell.	2020-07-13 14:12:48 +08:00
Uma Maheswara Rao G	3e70006639	HDFS-15464: ViewFsOverloadScheme should work when -fs option pointing to remote cluster without mount links (#2132 ). Contributed by Uma Maheswara Rao G.	2020-07-11 23:50:04 -07:00
Xiaoyu Yao	f91a8ad88b	HADOOP-17079. Optimize UGI#getGroups by adding UGI#getGroupsSet. (#2085 )	2020-07-09 11:33:37 -07:00
Ye Ni	f77bbc2123	HDFS-15312. Apply umask when creating directory by WebHDFS (#2096 )	2020-07-06 19:01:46 -07:00
Uma Maheswara Rao G	dc0626b5f2	HDFS-15449. Optionally ignore port number in mount-table name when picking from initialized uri. Contributed by Uma Maheswara Rao G.	2020-07-06 18:50:03 -07:00
Shanyu Zhao	834372f404	HDFS-15451. Do not discard non-initial block report for provided storage. (#2119 ). Contributed by Shanyu Zhao. Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>	2020-07-06 23:43:34 +08:00
Uma Maheswara Rao G	55a2ae80dc	HDFS-15450. Fix NN trash emptier to work if ViewFSOveroadScheme enabled. Contributed by Uma Maheswara Rao G.	2020-07-04 13:45:49 -07:00
Ayush Saxena	f86f15cf20	HDFS-15446. CreateSnapshotOp fails during edit log loading for /.reserved/raw/path with error java.io.FileNotFoundException: Directory does not exist: /.reserved/raw/path. Contributed by Stephen O'Donnell.	2020-07-04 13:00:43 +05:30
Uma Maheswara Rao G	1f2a80b5e5	HDFS-15430. create should work when parent dir is internalDir and fallback configured. Contributed by Uma Maheswara Rao G.	2020-07-04 00:12:10 -07:00
He Xiaoqiao	9ac498e300	HDFS-15416. Improve DataStorage#addStorageLocations() for empty locations. Contibuted by jianghua zhu.	2020-07-01 12:38:07 +08:00
Stephen O'Donnell	2a67e2b1a0	HDFS-15160. ReplicaMap, Disk Balancer, Directory Scanner and various FsDatasetImpl methods should use datanode readlock. Contributed by Stephen O'Donnell. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2020-06-30 07:10:17 -07:00
Akira Ajisaka	c71ce7ac33	HDFS-15421. IBR leak causes standby NN to be stuck in safe mode.	2020-06-28 16:02:47 +09:00
Ayush Saxena	8db38c98a6	HDFS-15378. TestReconstructStripedFile#testErasureCodingWorkerXmitsWeight is failing on trunk. Contributed by hemanthboyina.	2020-06-27 19:13:01 +05:30
Virajith Jalaparti	bed0a3a374	HDFS-15436. Default mount table name used by ViewFileSystem should be configurable (#2100 ) * HDFS-15436. Default mount table name used by ViewFileSystem should be configurable * Replace Constants.CONFIG_VIEWFS_DEFAULT_MOUNT_TABLE use in tests * Address Uma's comments on PR#2100 * Sort lists in test to match without concern to order * Address comments, fix checkstyle and fix failing tests * Fix checkstyle	2020-06-26 13:19:16 -07:00
Uma Maheswara Rao G	d5e1bb6155	HDFS-15429. mkdirs should work when parent dir is an internalDir and fallback configured. Contributed by Uma Maheswara Rao G.	2020-06-26 01:29:38 -07:00
Uma Maheswara Rao G	7c02d1889b	HDFS-15427. Merged ListStatus with Fallback target filesystem and InternalDirViewFS. Contributed by Uma Maheswara Rao G.	2020-06-23 01:42:25 -07:00
Ayush Saxena	17ffcab5f6	HDFS-14546. Document block placement policies. Contributed by Amithsha.	2020-06-21 17:27:28 +05:30
Uma Maheswara Rao G	b27810aa60	HDFS-15418. ViewFileSystemOverloadScheme should represent mount links as non symlinks. Contributed by Uma Maheswara Rao G.	2020-06-20 00:32:02 -07:00
Stephen O'Donnell	d50e93ce7b	HDFS-15372. Files in snapshots no longer see attribute provider permissions. Contributed by Stephen O'Donnell. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2020-06-18 06:45:15 -07:00
Wei-Chiu Chuang	edf716a5c3	Revert "HDFS-15372. Files in snapshots no longer see attribute provider permissions. Contributed by Stephen O'Donnell." This reverts commit `730a39d138`.	2020-06-18 06:45:15 -07:00
S O'Donnell	123777823e	HDFS-15406. Improve the speed of Datanode Block Scan. Contributed by hemanthboyina	2020-06-18 12:26:22 +01:00
Ayush Saxena	c8ed33cd2a	HADOOP-9851. dfs -chown does not like "+" plus sign in user name. Contributed by Andras Bokor.	2020-06-17 13:55:40 +05:30
Stephen O'Donnell	730a39d138	HDFS-15372. Files in snapshots no longer see attribute provider permissions. Contributed by Stephen O'Donnell. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2020-06-16 15:59:13 -07:00
Takanobu Asanuma	f41a144077	HDFS-15403. NPE in FileIoProvider#transferToSocketFully. Contributed by hemanthboyina.	2020-06-15 09:15:53 +09:00
Inigo Goiri	719b53a79d	HDFS-15351. Blocks scheduled count was wrong on truncate. Contributed by hemanthboyina.	2020-06-13 09:35:05 -07:00
Uma Maheswara Rao G	785b1def95	HDFS-15387. FSUsage#DF should consider ViewFSOverloadScheme in processPath. Contributed by Uma Maheswara Rao G.	2020-06-12 14:32:19 -07:00
Vinayakumar B	e154084770	HADOOP-17046. Support downstreams' existing Hadoop-rpc implementations using non-shaded protobuf classes (#2026 )	2020-06-12 23:16:33 +05:30
Ayush Saxena	b735a77717	HDFS-15398. EC: hdfs client hangs due to exception during addBlock. Contributed by Hongbing Wang.	2020-06-10 12:06:16 +05:30
Ayush Saxena	8525874561	HDFS-15211. EC: File write hangs during close in case of Exception during updatePipeline. Contributed by Ayush Saxena. *Added missed test file.	2020-06-09 18:53:33 +05:30
Uma Maheswara Rao G	3ca15292c5	HDFS-15394. Add all available fs.viewfs.overload.scheme.target.<scheme>.impl classes in core-default.xml bydefault. Contributed by Uma Maheswara Rao G.	2020-06-06 08:11:57 -07:00
Ayush Saxena	cc671b16f7	HDFS-15389. DFSAdmin should close filesystem and dfsadmin -setBalancerBandwidth should work with ViewFSOverloadScheme. Contributed by Ayush Saxena	2020-06-05 22:19:38 -07:00
Uma Maheswara Rao G	76fa0222f0	HDFS-15330. Document the ViewFSOverloadScheme details in ViewFS guide. Contributed by Uma Maheswara Rao G.	2020-06-05 10:58:21 -07:00
Ayush Saxena	2326123705	HDFS-15359. EC: Allow closing a file with committed blocks. Contributed by Ayush Saxena.	2020-06-05 19:15:05 +05:30
Toshihiro Suzuki	545a0a147c	HDFS-15386 ReplicaNotFoundException keeps happening in DN after removing multiple DN's data directories (#2052 ) Contributed by Toshihiro Suzuki.	2020-06-05 11:11:49 +01:00
Ayush Saxena	e8cb2ae409	HDFS-11041. Unable to unregister FsDatasetState MBean if DataNode is shutdown twice. Contributed by Wei-Chiu Chuang.	2020-06-03 12:47:15 +05:30
Ayush Saxena	f6453244ab	HDFS-14960. TestBalancerWithNodeGroup should not succeed with DFSNetworkTopology. Contributed by Jim Brennan.	2020-06-03 12:16:36 +05:30
Uma Maheswara Rao G	ed83c865dd	HDFS-15321. Make DFSAdmin tool to work with ViewFileSystemOverloadScheme. Contributed by Uma Maheswara Rao G.	2020-06-02 11:09:26 -07:00
Ayush Saxena	ae13a5ccbe	HDFS-10792. RedundantEditLogInputStream should log caught exceptions. Contributed by Wei-Chiu Chuang.	2020-05-31 17:10:59 +05:30
Wanqiang Ji	d9838f2d42	HADOOP-17055. Remove residual code of Ozone (#2039 )	2020-05-29 16:49:18 +09:00
Ayush Saxena	a838d871a7	HDFS-15368. TestBalancerWithHANameNodes#testBalancerWithObserver failed occasionally. Contributed by Xiaoqiao He.	2020-05-28 20:52:58 +05:30
Ayush Saxena	9b38be43c6	HDFS-13183. Addendum: Standby NameNode process getBlocks request to reduce Active load. Contributed by Xiaoqiao He.	2020-05-28 13:00:04 +05:30
Inigo Goiri	2148a8fe64	HDFS-15362. FileWithSnapshotFeature#updateQuotaAndCollectBlocks should collect all distinct blocks. Contributed by hemanthboyina.	2020-05-27 11:06:13 -07:00
Ayush Saxena	6c9f75cf16	HDFS-15373. Fix number of threads in IPCLoggerChannel#createParallelExecutor. Contributed by Ayush Saxena.	2020-05-26 16:26:57 +05:30
Ayush Saxena	f43a152b97	HDFS-15369. Refactor method VolumeScanner#runLoop(). Contributed by Yang Yun.	2020-05-24 18:03:41 +05:30
Ayush Saxena	f4901d0778	HDFS-15355. Make the default block storage policy ID configurable. Contributed by Yang Yun.	2020-05-24 17:59:16 +05:30
Inigo Goiri	6e04b00df1	HDFS-12288. Fix DataNode's xceiver count calculation. Contributed by Lisheng Sun.	2020-05-23 09:58:19 -07:00
Ayush Saxena	e0ae232f66	HDFS-15093. RENAME.TO_TRASH is ignored When RENAME.OVERWRITE is specified. Contributed by Ayush Saxena.	2020-05-23 18:38:32 +05:30
Ayush Saxena	f99fcb26ab	HDFS-15288. Add Available Space Rack Fault Tolerant BPP. Contributed by Ayush Saxena.	2020-05-23 18:29:31 +05:30
Takanobu Asanuma	4d22d1c58f	HDFS-15363. BlockPlacementPolicyWithNodeGroup should validate if it is initialized by NetworkTopologyWithNodeGroup. Contributed by hemanthboyina.	2020-05-23 17:28:38 +09:00
Uma Maheswara Rao G	4734c77b4b	HDFS-15322. Make NflyFS to work when ViewFsOverloadScheme's scheme and target uris schemes are same. Contributed by Uma Maheswara Rao G.	2020-05-21 21:34:58 -07:00
leosunli	be374faf42	HDFS-13639. SlotReleaser is not fast enough (#1885 )	2020-05-21 13:21:17 -07:00
Abhishek Das	ce4ec74453	HADOOP-17024. ListStatus on ViewFS root (ls "/") should list the linkFallBack root (configured target root). Contributed by Abhishek Das.	2020-05-18 22:27:12 -07:00
Chen Liang	7bb902bc0d	HDFS-15293. Relax the condition for accepting a fsimage when receiving a checkpoint. Contributed by Chen Liang	2020-05-18 10:58:52 -07:00
Ayush Saxena	c84e6beada	HDFS-14999. Avoid Potential Infinite Loop in DFSNetworkTopology. Contributed by Ayush Saxena.	2020-05-18 22:24:34 +05:30
Wei-Chiu Chuang	2abcf7762a	HDFS-15202 Boost short circuit cache (rebase PR-1884) (#2016 )	2020-05-18 09:23:09 -07:00
Wei-Chiu Chuang	4525292d41	Revert "HDFS-15202 Boost short circuit cache (rebase PR-1884) (#2016 )" This reverts commit `86e6aa8eec`.	2020-05-18 09:22:05 -07:00
Wei-Chiu Chuang	50caba1a92	HDFS-15207. VolumeScanner skip to scan blocks accessed during recent scan peroid. Contributed by Yang Yun.	2020-05-18 08:40:38 -07:00
He Xiaoqiao	a3f44dacc1	HDFS-13183. Standby NameNode process getBlocks request to reduce Active load. Contributed by Xiaoqiao He. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2020-05-18 07:09:43 -07:00
pustota2009	86e6aa8eec	HDFS-15202 Boost short circuit cache (rebase PR-1884) (#2016 ) Added parameter dfs.client.short.circuit.num improving HDFS-client's massive reading performance by create few instances ShortCircuit caches instead of one. It helps avoid locks and lets CPU do job.	2020-05-18 07:04:04 -07:00
Ayush Saxena	6e416a83d1	HDFS-15358. RBF: Unify router datanode UI with namenode datanode UI. Contributed by Ayush Saxena.	2020-05-17 03:06:27 +05:30
Ayush Saxena	178336f8a8	HDFS-15356. Unify configuration `dfs.ha.allow.stale.reads` to DFSConfigKeys. Contributed by Xiaoqiao He.	2020-05-16 16:35:06 +05:30
Uma Maheswara Rao G	ac4a2e11d9	HDFS-15306. Make mount-table to read from central place ( Let's say from HDFS). Contributed by Uma Maheswara Rao G.	2020-05-14 17:29:35 -07:00
Xiaoyu Yao	3cacf1ce56	HDFS-15344. DataNode#checkSuperuserPrivilege should use UGI#getGroups after HADOOP-13442. (#2004 )	2020-05-13 11:47:19 -07:00
Surendra Singh Lilhore	743c2e9071	HDFS-15316. Deletion failure should not remove directory from snapshottables. Contributed by hemanthboyina	2020-05-13 15:01:07 +05:30
Elixir Kook	a3f945fb84	HADOOP-17035. fixed typos (timeout, interruped) (#2007 ) Co-authored-by: Sungpeo Kook <elixir.kook@kakaocorp.com>	2020-05-12 10:50:04 -05:00
S O'Donnell	29dddb8a14	HDFS-15255. Consider StorageType when DatanodeManager#sortLocatedBlock(). Contributed by Lisheng Sun.	2020-05-12 15:07:51 +01:00
Takanobu Asanuma	928b81a533	HDFS-15350. Set dfs.client.failover.random.order to true as default. (#2008 )	2020-05-12 09:04:03 -05:00
Ayush Saxena	8dad38c0be	HDFS-14367. EC: Parameter maxPoolSize in striped reconstruct thread pool isn't affecting number of threads. Contributed by Guo Lei.	2020-05-12 18:34:26 +05:30
Ayush Saxena	0fe49036e5	HDFS-15243. Add an option to prevent sub-directories of protected directories from deletion. Contributed by liuyanyu.	2020-05-12 13:11:31 +05:30
Ayush Saxena	4c53fb9ce1	HDFS-15338. listOpenFiles() should throw InvalidPathException in case of invalid paths. Contributed by Jinglun.	2020-05-11 16:48:34 +05:30
Ayush Saxena	cb64e993c2	HDFS-15347. Replace the deprecated method shaHex. Contributed by bianqi.	2020-05-10 00:11:46 +05:30
Ayush Saxena	d59de27c01	HDFS-15339. TestHDFSCLI fails for user names with the dot/dash character. Contributed by Yan Xiaole.	2020-05-08 00:53:49 +05:30
Inigo Goiri	130f89e068	HDFS-15332. Quota Space consumed was wrong in truncate with Snapshots. Contributed by hemanthboyina.	2020-05-06 13:18:32 -07:00
Ayush Saxena	7fddf4855e	HDFS-14283. DFSInputStream to prefer cached replica. Contributed by Lisheng Sun.	2020-05-06 16:55:04 +05:30
Wei-Chiu Chuang	adecdb8b53	HDFS-15334. INodeAttributeProvider's new API checkPermissionWithContext not getting called in for authorization. (#1998 ) Reviewed-by: Arpit Agarwal <arp@apache.org>	2020-05-05 16:04:54 -07:00
Uma Maheswara Rao G	9c8236d04d	HDFS-15305. Extend ViewFS and provide ViewFileSystemOverloadScheme implementation with scheme configurable. Contributed by Uma Maheswara Rao G.	2020-05-04 17:55:40 -07:00
Konstantin V Shvachko	ebb878bab9	HDFS-15323. StandbyNode fails transition to active due to insufficient transaction tailing. Contributed by Konstantin V Shvachko.	2020-05-04 10:29:50 -07:00
Ayush Saxena	8dace8ff3a	HDFS-15328. Use DFSConfigKeys MONITOR_CLASS_DEFAULT constant. Contributed by bianqi.	2020-05-03 23:59:17 +05:30
Ayush Saxena	d6fc482a54	Erasure Coding: metrics xmitsInProgress become to negative. Contributed by maobaolong and Toshihiko Uchida.	2020-05-03 19:09:14 +05:30
Mingliang Liu	44de193bec	HDFS-15325. TestRefreshCallQueue is failing due to changed CallQueue constructor. Contributed by Fengnan Li	2020-05-02 20:56:29 -07:00
Akira Ajisaka	f40dacdc2e	HDFS-15320. StringIndexOutOfBoundsException in HostRestrictingAuthorizationFilter (#1992 ) Signed-off-by: Mingliang Liu <liuml07@apache.org>	2020-05-02 13:02:27 -07:00
Shashikant Banerjee	82343790ee	HDFS-15313. Ensure inodes in active filesytem are not deleted during snapshot delete. Contributed by Shashikant Banerjee.	2020-05-01 12:16:02 +05:30
Mingliang Liu	263c76b678	HADOOP-17011. Tolerate leading and trailing spaces in fs.defaultFS. Contributed by Ctest Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>	2020-04-30 14:15:28 -07:00
S O'Donnell	9ca6298a9a	HDFS-15285. The same distance and load nodes don't shuffle when consider DataNode load. Contributed by Lisheng Sun.	2020-04-29 15:56:06 +01:00
Akira Ajisaka	5e0eda5d5f	HDFS-15286. Concat on a same file deleting the file. Contributed by hemanthboyina.	2020-04-28 06:17:14 +09:00
Toshihiro Suzuki	62c26b91fd	HDFS-15298 Fix the findbugs warnings introduced in HDFS-15217 (#1979 )	2020-04-27 17:20:47 +09:00
Mingliang Liu	74d4a18735	HDFS-15297. TestNNHandlesBlockReportPerStorage::blockReport_02 fails intermittently. Contributed by Ayush Saxena	2020-04-25 15:43:02 -07:00
Dhiraj	453c9c9a67	HDFS-15281. Make sure ZKFC uses dfs.namenode.rpc-address to bind to host address (#1964 ) Contributed by Dhiraj Hegde. Signed-off-by: Mingliang Liu <liuml07@apache.org> Signed-off-by: Inigo Goiri <inigoiri@apache.org>	2020-04-25 13:04:32 -07:00
Wei-Chiu Chuang	453771f170	HDFS-15295. AvailableSpaceBlockPlacementPolicy should use chooseRandomWithStorageTypeTwoTrial() for better performance. Contributed by Jinglun.	2020-04-24 11:53:24 -07:00
Mingliang Liu	5958af4dbc	HDFS-15276. Concat on INodeRefernce fails with illegal state exception. Contributed by hemanthboyina	2020-04-22 13:31:02 -07:00
Toshihiro Suzuki	1824aee9da	HDFS-15217 Add more information to longest write/read lock held log	2020-04-18 13:52:07 -07:00
Ayush Saxena	37d6582223	HDFS-15266. Add missing DFSOps Statistics in WebHDFS. Contributed by Ayush Saxena.	2020-04-17 13:32:02 +05:30
Stephen O'Donnell	3481895f8a	HDFS-15283. Cache pool MAXTTL is not persisted and restored on cluster restart. Contributed by Stephen O'Donnell. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2020-04-16 20:18:25 -07:00
Ayush Saxena	cc5c1da7c1	HDFS-15277. Parent directory in the explorer does not support all path formats. Contributed by Jianfei Jiang.	2020-04-15 23:01:58 +05:30
Wei-Chiu Chuang	061afcdf30	HDFS-15269. NameNode should check the authorization API version only … (#1945 ) Reviewed-by: Takanobu Asanuma <tasanuma@apache.org> Reviewed-by: Akira Ajisaka <aajisaka@apache.org>	2020-04-09 09:19:35 -07:00
Ayush Saxena	1189af4746	HDFS-15263. Fix the logic of scope and excluded scope in Network Topology. Contributed by Ayush Saxena.	2020-04-08 15:55:14 +05:30
Toshihiro Suzuki	c12ddbd1de	HDFS-15249 ThrottledAsyncChecker is not thread-safe. (#1922 )	2020-04-07 14:38:09 +09:00
Inigo Goiri	0b855b9f35	HDFS-15256. Fix typo in DataXceiverServer#run(). Contributed by Lisheng Sun.	2020-04-06 13:05:33 -07:00
He Xiaoqiao	d3b5951572	HDFS-15242. Add metrics for operations hold lock times of FsDatasetImpl. Contributed by Xiaoqiao He. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org> Reviewed-by: Inigo Goiri <inigoiri@apache.org>	2020-04-01 16:36:46 -07:00
Ayush Saxena	960c9ebaea	HDFS-15245. Improve JournalNode web UI. Contributed by Jianfei Jiang.	2020-03-30 02:40:25 +05:30
Inigo Goiri	f7a94ec0a4	HDFS-15239. Add button to go to the parent directory in the explorer. Contributed by hemanthboyina.	2020-03-29 08:17:02 -07:00
Ayush Saxena	696a663cd0	HDFS-15193. Improving the error message for missing `dfs.namenode.rpc-address.$NAMESERVICE`. Contributed by Ctest.	2020-03-29 00:01:22 +05:30
Chen Liang	f531a4a487	HDFS-15191. EOF when reading legacy buffer in BlockTokenIdentifier. Contributed by Steven Rand.	2020-03-27 12:55:38 -07:00
Wei-Chiu Chuang	0fa7bf47df	HDFS-15234. Add a default method body for the INodeAttributeProvider#checkPermissionWithContext API. (#1909 )	2020-03-25 16:03:26 -07:00
Inigo Goiri	cdcb77a2c5	HDFS-15075. Remove process command timing from BPServiceActor. Contributed by Xiaoqiao He.	2020-03-25 11:30:54 -07:00
Arpit Agarwal	a700803a18	HDFS-15154. Allow only hdfs superusers the ability to assign HDFS storage policies. Contributed by Siddharth Wagle. Change-Id: I32d6dd2837945b8fc026a759aa367c55daefe348	2020-03-25 10:28:30 -07:00
Toshihiro Suzuki	d353b30baf	HDFS-15215. The Timestamp for longest write/read lock held log is wrong	2020-03-24 14:50:15 -07:00
Ayush Saxena	ea87d60493	HDFS-13377. The owner of folder can set quota for his sub folder. Contributed by Yang Yun.	2020-03-24 22:56:09 +05:30
Karthik Palanisamy	5250cd6db3	HDFS-15201 SnapshotCounter hits MaxSnapshotID limit (#1870 )	2020-03-24 15:15:46 +05:30
He Xiaoqiao	af64ce2f4a	HDFS-15113. Addendum: Missing IBR when NameNode restart if open processCommand async feature. Contributed by Xiaoqiao He. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2020-03-23 12:43:51 -07:00
Ayush Saxena	62c8015683	HDFS-15227. NPE if the last block changes from COMMITTED to COMPLETE during FSCK. Contributed by Ayush Saxena.	2020-03-22 11:44:18 +05:30
Takanobu Asanuma	b74e47e31a	HDFS-15214. WebHDFS: Add snapshot counts to Content Summary. Contributed by hemanthboyina.	2020-03-22 01:44:55 +09:00
Ayush Saxena	f9bb2a8cc5	HDFS-15200. Delete Corrupt Replica Immediately Irrespective of Replicas On Stale Storage. Contributed by Ayush Saxena.	2020-03-19 22:26:34 +05:30
Ayush Saxena	bb41ddaf1e	HDFS-15223. FSCK fails if one namenode is not available. Contributed by Ayush Saxena.	2020-03-19 21:23:13 +05:30
Ayush Saxena	654db35fa2	HDFS-14919. Provide Non DFS Used per DataNode in DataNode UI. Contributed by Lisheng Sun.	2020-03-18 21:00:45 +05:30
Wei-Chiu Chuang	096533c2dc	HDFS-15208. Suppress bogus AbstractWadlGeneratorGrammarGenerator in KMS stderr in hdfs. Contributed by Wei-Chiu Chuang. Change-Id: I7abeaf4b65e2a4758356aa69ab8930f7e993077d	2020-03-18 13:27:13 +00:00
Ayush Saxena	8a7c54995a	HDFS-15159. Prevent adding same DN multiple times in PendingReconstructionBlocks. Contributed by hemanthboyina.	2020-03-15 16:16:27 +05:30
Wei-Chiu Chuang	7b637a3341	HDFS-14820. The default 8KB buffer of BlockReaderRemote#newBlockReader#BufferedOutputStream is too big. Contributed by Lisheng Sun.	2020-03-13 19:01:23 -07:00
He Xiaoqiao	e9955bb8ff	HDFS-15113. Missing IBR when NameNode restart if open processCommand async feature. Contributed by Xiaoqiao He. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org> Reviewed-by: Brahma Reddy Battula <brahma@apache.org> Reviewed-by: Inigo Goiri <inigoiri@apache.org>	2020-03-13 18:55:06 -07:00
Wei-Chiu Chuang	4b95c242ec	HDFS-14743. Enhance INodeAttributeProvider/ AccessControlEnforcer Interface in HDFS to support Authorization of mkdir, rm, rmdir, copy, move etc... (#1829 ) Reviewed-by: Xiaoyu Yao <xyao@apache.org>	2020-03-13 11:29:44 -07:00
Ayush Saxena	fb1d7635ae	HDFS-15155. writeIoRate of DataNodeVolumeMetrics is never used. Contributed by Haibin Huang.	2020-03-13 21:10:03 +05:30
S O'Donnell	25497799cb	HDFS-15216. Wrong Use Case of -showprogress in fsck. Contributed by Ravuri Sushma sree.	2020-03-13 15:24:19 +00:00
Wei-Chiu Chuang	20903f72b4	HDFS-15039. Cache meta file length of FinalizedReplica to reduce call File.length(). Contributed by Yang Yun.	2020-03-12 19:23:12 -07:00
Inigo Goiri	0695f7a538	HDFS-14612. SlowDiskReport won't update when SlowDisks is always empty in heartbeat. Contributed by Haibin Huang.	2020-03-12 17:22:11 -07:00
Surendra Singh Lilhore	f736408a83	HDFS-14442. Disagreement between HAUtil.getAddressOfActive and RpcInvocationHandler.getConnectionId. Contributed by Ravuri Sushma sree.	2020-03-12 18:59:17 +05:30
Steve Loughran	c734d69a55	HADOOP-16898. Batch listing of multiple directories via an (unstable) interface Contributed by Steve Loughran. This moves the new API of HDFS-13616 into a interface which is implemented by HDFS RPC filesystem client (not WebHDFS or any other connector) This new interface, BatchListingOperations, is in hadoop-common, so applications do not need to be compiled with HDFS on the classpath. They must cast the FS into the interface. instanceof can probe the client for having the new interface -the patch also adds a new path capability to probe for this. The FileSystem implementation is cut; tests updated as appropriate. All new interfaces/classes/constants are marked as @unstable. Change-Id: I5623c51f2c75804f58f915dd7e60cb2cffdac681	2020-03-09 14:51:16 +00:00
Inigo Goiri	4062217189	HDFS-15212. TestEncryptionZones.testVersionAndSuiteNegotiation fails in trunk. Contributed by Ayush Saxena.	2020-03-06 10:14:53 -08:00
Inigo Goiri	3afd4cbe89	HDFS-14977. Quota Usage and Content summary are not same in Truncate with Snapshot. Contributed by hemanthboyina.	2020-03-04 10:13:23 -08:00
Inigo Goiri	1d4d0fcbe1	HDFS-15204. TestRetryCacheWithHA testRemoveCacheDescriptor fails intermittently. Contributed by Ahmed Hussein.	2020-03-04 10:02:54 -08:00
Akshay Nehe	bbd704bb82	HADOOP-16899. Update HdfsDesign.md to reduce ambiguity. (#1871 )	2020-03-04 11:07:31 +09:00
Ayush Saxena	1a636da041	HDFS-15033. Support to save replica cached files to other place and make expired time configurable. Contributed by Yang Yun.	2020-02-29 09:55:29 +05:30
Inigo Goiri	97b797c314	HDFS-15149. TestDeadNodeDetection test cases time-out. Contributed by Lisheng Sun.	2020-02-28 18:47:22 -08:00
Surendra Singh Lilhore	fccfb02672	HDFS-15199. NPE in BlockSender. Contributed by Ayush Saxena.	2020-02-28 18:33:06 +05:30
Ayush Saxena	429da635ec	HDFS-15186. Erasure Coding: Decommission may generate the parity block's content with all 0 in some case. Contributed by Yao Guangdong.	2020-02-28 00:31:55 +05:30
Ayush Saxena	cd2c6b1aac	HDFS-15124. Crashing bugs in NameNode when using a valid configuration for . Contributed by Ctest.	2020-02-27 22:19:35 +05:30
Surendra Singh Lilhore	2059f255d3	HDFS-15167. Block Report Interval shouldn't be reset apart from first Block Report. Contributed by Ayush Saxena.	2020-02-27 14:08:42 +05:30
Ayush Saxena	033a3d7ff9	HDFS-15111. stopStandbyServices() should log which service state it is transitioning from. Contributed by Xieming Li.	2020-02-27 04:02:26 +05:30
Ayush Saxena	209630472a	HDFS-15120. Refresh BlockPlacementPolicy at runtime. Contributed by Jinglun.	2020-02-27 02:22:24 +05:30
Kihwal Lee	352a4ec16d	HDFS-15147. LazyPersistTestCase wait logic is flawed. Contributed by Ahmed Hussein.	2020-02-26 09:33:29 -06:00
Stephen O'Donnell	900430b990	HDFS-14861. Reset LowRedundancyBlocks Iterator periodically. Contributed by Stephen O'Donnell. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2020-02-25 13:28:50 -08:00
Wei-Chiu Chuang	1c5d2f1fdc	HDFS-15174. Optimize ReplicaCachingGetSpaceUsed by reducing unnecessary io operations. Contributed by Lisheng Sun.	2020-02-24 13:39:16 -08:00
Ayush Saxena	7f8685f476	HDFS-15187. CORRUPT replica mismatch between namenodes after failover. Contributed by Ayush Saxena.	2020-02-24 20:38:04 +05:30
Ayush Saxena	93b8f453b9	HDFS-15166. Remove redundant field fStream in ByteStringLog. Contributed by Xieming Li.	2020-02-24 20:17:01 +05:30
Ayush Saxena	b5698e0c33	HDFS-15176. Enable GcTimePercentage Metric in NameNode's JvmMetrics. Contributed by Jinglun.	2020-02-24 00:07:18 +05:30
Ayush Saxena	9eb7a8bdf8	HDFS-15041. Make MAX_LOCK_HOLD_MS and full queue size configurable. Contributed by zhuqi.	2020-02-23 14:25:39 +05:30
Ayush Saxena	ba9025c7cd	HDFS-15182. TestBlockManager#testOneOfTwoRacksDecommissioned() fail in trunk. Contributed by Lisheng Sun.	2020-02-22 14:27:26 +05:30
Konstantin V Shvachko	ecbcb058b8	HDFS-14731. [FGL] Remove redundant locking on NameNode. Contributed by Konstantin V Shvachko.	2020-02-21 17:53:37 -08:00
Konstantin V Shvachko	6f84269bcd	HDFS-15185. StartupProgress reports edits segments until the entire startup completes. Contributed by Konstantin V Shvachko.	2020-02-21 10:51:14 -08:00
Masatake Iwasaki	2338d25dc7	HDFS-15052. WebHDFS getTrashRoot leads to OOM due to FileSystem object creation. (#1758 )	2020-02-21 11:56:07 +09:00
Inigo Goiri	ec7507162c	HDFS-15165. In Du missed calling getAttributesProvider. Contributed by Bharat Viswanadham.	2020-02-19 11:33:58 -08:00
Ayush Saxena	ac4b556e2d	HDFS-13739. Add option to disable rack local write preference. Contributed by Ayush Saxena.	2020-02-19 08:20:59 +05:30
Ayush Saxena	84f7638840	HADOOP-13666. Supporting rack exclusion in countNumOfAvailableNodes in NetworkTopology. Contributed by Inigo Goiri.	2020-02-18 00:43:33 +05:30
Surendra Singh Lilhore	810783d443	HDFS-15135. EC : ArrayIndexOutOfBoundsException in BlockRecoveryWorker#RecoveryTaskStriped. Contributed by Ravuri Sushma sree.	2020-02-16 12:15:03 +05:30
Ayush Saxena	c75756fe13	HDFS-15164. Fix TestDelegationTokensWithHA. Contributed by Ayush Saxena.	2020-02-14 22:07:24 +05:30
Surendra Singh Lilhore	a98352ced1	HDFS-15086. Block scheduled counter never get decremet if the block got deleted before replication. Contributed by hemanthboyina.	2020-02-13 16:57:41 +05:30
Akira Ajisaka	0ddb5f0881	HDFS-13989. RBF: Add FSCK to the Router (#1832 ) Co-authored-by: Inigo Goiri <inigoiri@apache.org>	2020-02-13 10:06:07 +09:00
Kihwal Lee	9b8a78d97b	HDFS-14758. Make lease hard limit configurable and reduce the default. Contributed by hemanthboyina.	2020-02-11 12:40:00 -06:00
Stephen O'Donnell	d7c136b9ed	HDFS-15150. Introduce read write lock to Datanode. Contributed Stephen O'Donnell. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2020-02-11 08:00:15 -08:00
Ayush Saxena	6191d4b4a0	HDFS-15158. The number of failed volumes mismatch with volumeFailures of Datanode metrics. Contributed by Yang Yun.	2020-02-09 23:32:22 +05:30
Ayush Saxena	d23317b102	HDFS-15115. Namenode crash caused by NPE in BlockPlacementPolicyDefault when dynamically change logger to debug. Contributed by wangzhixiang	2020-02-08 10:33:57 +05:30
Vinayakumar B	7dac7e1d13	HADOOP-16596. [pb-upgrade] Use shaded protobuf classes from hadoop-thirdparty dependency (#1635 ). Contributed by Vinayakumar B.	2020-02-07 14:51:24 +05:30
Chen Liang	ce7b8b5634	HDFS-15148. dfs.namenode.send.qop.enabled should not apply to primary NN port. Contributed by Chen Liang.	2020-02-04 12:12:35 -08:00
Kihwal Lee	10a60fbe20	HDFS-12491. Support wildcard in CLASSPATH for libhdfs. Contributed by Muhammad Samir Khan.	2020-02-04 12:22:35 -06:00
Stephen O'Donnell	1e3a0b0d93	HDFS-7175. Client-side SocketTimeoutException during Fsck. Contributed by Stephen O'Donnell, Akira Ajisaka. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org> Co-authored-by: Akira Ajisaka <aajisaka@apache.org>	2020-01-31 16:13:02 -08:00
Kihwal Lee	799d4c1cf4	HDFS-15146. TestBalancerRPCDelay.testBalancerRPCDelay fails intermittently. Contributed by Ahmed Hussein.	2020-01-29 11:00:27 -06:00
Chen Liang	483397c7f7	[SBN Read] Slow clients when Observer reads are enabled but there are no Observers on the cluster. Conntributed by Chen Liang	2020-01-28 15:20:36 -08:00
Chen Liang	3e86807802	Revert "[SBN Read] Slow clients when Observer reads are enabled but there are no Observers on the cluster. Contributed by Chen Liang." This reverts commit `ff8ff0f7e5`.	2020-01-28 15:19:47 -08:00
Chen Liang	ff8ff0f7e5	[SBN Read] Slow clients when Observer reads are enabled but there are no Observers on the cluster. Contributed by Chen Liang.	2020-01-28 15:14:58 -08:00
Inigo Goiri	1839c467f6	HDFS-13179. TestLazyPersistReplicaRecovery#testDnRestartWithSavedReplicas fails intermittently. Contributed by Ahmed Hussein.	2020-01-28 10:10:35 -08:00
Ayush Saxena	87c198468b	HDFS-14993. checkDiskError doesn't work during datanode startup. Contributed by Yang Yun.	2020-01-28 23:09:38 +05:30
Ayush Saxena	f876dc228b	HDFS-15143. LocatedStripedBlock returns wrong block type. Contributed by Ayush Saxena.	2020-01-28 19:34:21 +05:30
Ayush Saxena	6d008c0d39	HDFS-15128. Unit test failing to clean testing data and crashed future Maven test run due to failure in TestDataNodeVolumeFailureToleration. Contributed by Ctest	2020-01-25 11:47:29 +05:30
Kihwal Lee	d10f77e3c9	HDFS-15119. Allow expiration of cached locations in DFSInputStream. Contributed by Ahmed Hussein.	2020-01-24 09:15:27 -06:00
Ayush Saxena	92c58901d7	HDFS-15117. EC: Add getECTopologyResultForPolicies to DistributedFileSystem. Contributed by Ayush Saxena	2020-01-23 18:18:34 +05:30
Kihwal Lee	bd03053ea2	HDFS-14968. Add ability to log stale datanodes. Contributed by Ahmed Hussein.	2020-01-22 09:14:38 -06:00
Inigo Goiri	8cfc3673dc	HDFS-15092. TestRedudantBlocks#testProcessOverReplicatedAndRedudantBlock sometimes fails. Contributed by Fei Hui.	2020-01-21 13:29:20 -08:00
Inigo Goiri	b657822b98	HDFS-15126. TestDatanodeRegistration#testForcedRegistration fails intermittently. Contributed by Ahmed Hussein.	2020-01-21 13:22:53 -08:00
Akira Ajisaka	1defe3a65a	HADOOP-16753. Refactor HAAdmin. Contributed by Xieming Li.	2020-01-21 10:58:32 +09:00
Chao Sun	d7c4f8ab21	HDFS-13616. Batch listing of multiple directories (#1725 )	2020-01-15 17:22:47 -08:00
Ayush Saxena	cebce0a348	HDFS-14578. AvailableSpaceBlockPlacementPolicy always prefers local node. Contributed by Ayush Saxena.	2020-01-11 10:29:11 +05:30
Chen Liang	26a969ec73	HDFS-15099. [SBN Read] checkOperation(WRITE) should throw ObserverRetryOnActiveException for ObserverNode. Contributed by Chen Liang.	2020-01-10 17:06:59 -08:00
Kihwal Lee	5fb901ac40	HDFS-15095. Fix TestDecommissioningStatus. Contributed by Ahmed Hussein.	2020-01-10 12:43:21 -06:00
Ayush Saxena	b32757c616	HDFS-15107. dfs.client.server-defaults.validity.period.ms to support time units. Contributed by Ayush Saxena.	2020-01-10 08:14:56 +05:30
Surendra Singh Lilhore	bf45f3b80a	HDFS-14957. INodeReference Space Consumed was not same in QuotaUsage and ContentSummary. Contributed by hemanthboyina.	2020-01-09 12:04:05 +05:30
Rakesh Radhakrishnan	7030722e5d	HDFS-15080. Fix the issue in reading persistent memory cached data with an offset. Contributed by Feilong He.	2020-01-08 14:25:17 +05:30
Masatake Iwasaki	aba3f6c3e1	HDFS-15077. Fix intermittent failure of TestDFSClientRetries#testLeaseRenewSocketTimeout. (#1797 )	2020-01-08 16:45:39 +09:00
Masatake Iwasaki	a43c177f1d	HDFS-15072. HDFS MiniCluster fails to start when run in directory path with a %. (#1775 )	2020-01-08 11:28:34 +09:00
Ayush Saxena	f8644fbe9f	HDFS-15091. Cache Admin and Quota Commands Should Check SuperUser Before Taking Lock. Contributed by Ayush Saxena.	2020-01-04 19:02:59 +05:30
Masatake Iwasaki	037ec8cfb1	HDFS-15068. DataNode could meet deadlock if invoke refreshVolumes when register. Contributed by Aiphago. Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>	2020-01-04 01:55:36 +09:00
Rakesh Radhakrishnan	d79cce20ab	HDFS-14740. Recover data blocks from persistent memory read cache during datanode restarts. Contributed by Feilong He.	2020-01-02 11:44:00 +05:30
Takanobu Asanuma	074050ca59	HDFS-15063. HttpFS: getFileStatus doesn't return ecPolicy. Contributed by hemanthboyina.	2020-01-01 11:26:38 +09:00
Surendra Singh Lilhore	ee51eadda0	HDFS-15074. DataNode.DataTransfer thread should catch all the expception and log it. Contributed by hemanthboyina.	2019-12-29 11:15:54 +05:30
Takanobu Asanuma	dc32f583af	HDFS-14934. [SBN Read] Standby NN throws many InterruptedExceptions when dfs.ha.tail-edits.period is 0. Contributed by Ayush Saxena.	2019-12-28 21:32:15 +09:00
Ayush Saxena	8730a7bf60	HDFS-15003. RBF: Make Router support storage type quota. Contributed by Jinglun.	2019-12-27 09:22:58 +05:30
Ayush Saxena	80f91d14ab	HDFS-14997. Addendum: BPServiceActor processes commands from NameNode asynchronously. Contributed by Xiaoqiao He.	2019-12-27 09:12:44 +05:30
Ayush Saxena	300505c562	HDFS-15054. Delete Snapshot not updating new modification time. Contributed by hemanthboyina.	2019-12-25 12:42:50 +05:30
Konstantin V Shvachko	b98ac2a3af	HDFS-15076. Fix tests that hold FSDirectory lock, without holding FSNamesystem lock. Contributed by Konstantin V Shvachko.	2019-12-24 12:08:34 -08:00
Inigo Goiri	52d7b745c6	HDFS-15062. Add LOG when sendIBRs failed. Contributed by Fei Hui.	2019-12-19 09:42:17 -08:00
Inigo Goiri	b86895485d	HDFS-14997. BPServiceActor processes commands from NameNode asynchronously. Contributed by Xiaoqiao He.	2019-12-19 09:34:43 -08:00
Shashikant Banerjee	fdd96e46d1	HDFS-15012. NN fails to parse Edit logs after applying HDFS-13101. Contributed by Shashikant Banerjee.	2019-12-18 22:50:46 +05:30
Inigo Goiri	24080666e5	HDFS-14908. LeaseManager should check parent-child relationship when filter open files. Contributed by Jinglun.	2019-12-16 18:41:45 -08:00
Ayush Saxena	7a87007545	HDFS-15038. TestFsck testFsckListCorruptSnapshotFiles is failing in trunk. Contributed by hemanthboyina.	2019-12-15 00:33:32 +05:30
Chen Liang	65c4660bcd	HDFS-15036. Active NameNode should not silently fail the image transfer. Contributed by Chen Liang.	2019-12-12 10:22:05 -08:00
Masatake Iwasaki	bdd00f10b4	HDFS-15047. Document the new decommission monitor (HDFS-14854). (#1755 )	2019-12-13 00:35:41 +09:00
Takanobu Asanuma	93bb368094	HDFS-14983. RBF: Add dfsrouteradmin -refreshSuperUserGroupsConfiguration command option. Contributed by Xieming Li	2019-12-12 10:56:19 +09:00
Erik Krogen	c174d50b30	HDFS-15032. Properly handle InvocationTargetExceptions in the proxy created by ProxyCombiner. This fixes a bug encountered by the HDFS balancer when used with Observer Nodes. Contributed by Erik Krogen.	2019-12-11 09:39:37 -08:00
Stephen O'Donnell	c93cb6790e	HDFS-14854. Create improved decommission monitor implementation. Contributed by Stephen O'Donnell. Reviewed-by: Inigo Goiri <inigoiri@apache.org> Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2019-12-10 17:16:43 -08:00
Masatake Iwasaki	11cd5b6e39	HDFS-15028. Keep the capacity of volume and reduce a system call. Contributed by Yang Yun. Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>	2019-12-07 08:46:45 +09:00
Wei-Chiu Chuang	ecd461f940	HDFS-14751. Synchronize on diffs in DirectoryScanner. Contributed by Lisheng Sun.	2019-12-06 15:10:40 -08:00
Wei-Chiu Chuang	313b76f8e9	HDFS-14476. lock too long when fix inconsistent blocks between disk and in-memory. Contributed by Sean Chow.	2019-12-06 15:10:40 -08:00
Ayush Saxena	705b172b95	HDFS-14998. [SBN read] Update Observer Namenode doc for ZKFC after HDFS-14130. Contributed by Fei Hui.	2019-12-06 23:04:09 +05:30
Ayush Saxena	83a14559e5	HDFS-15023. [SBN read] ZKFC should check the state before joining the election. Contributed by Fei Hui.	2019-12-05 18:52:01 +05:30
Ayush Saxena	0c217feed8	HDFS-15026. TestPendingReconstruction#testPendingReconstruction() fail in trunk. Contributed by Lisheng Sun.	2019-12-03 09:18:09 +05:30
Ayush Saxena	6b2d6d4aaf	HDFS-15009. FSCK -list-corruptfileblocks return Invalid Entries. Contributed by hemanthboyina.	2019-11-29 23:55:37 +05:30
Surendra Singh Lilhore	44f7b9159d	HDFS-15013. Reduce NameNode overview tab response time. Contributed by HuangTao.	2019-11-29 00:42:31 +05:30
Surendra Singh Lilhore	0384687811	HDFS-15010. BlockPoolSlice#addReplicaThreadPool static pool should be initialized by static method. Contributed by Surendra Singh Lilhore.	2019-11-28 23:49:35 +05:30
Ayush Saxena	46166bd8d1	HDFS-14961. [SBN read] Prevent ZKFC changing Observer Namenode state. Contributed by Ayush Saxena.	2019-11-28 18:28:20 +05:30
Yiqun Lin	c3659f8f94	HDFS-15019. Refactor the unit test of TestDeadNodeDetection. Contributed by Lisheng Sun.	2019-11-28 14:41:49 +08:00
Yiqun Lin	2b452b4e60	HDFS-14986. ReplicaCachingGetSpaceUsed throws ConcurrentModificationException. Contributed by Aiphago.	2019-11-28 10:43:35 +08:00
Yiqun Lin	c8bef4d6a6	HDFS-14649. Add suspect probe for DeadNodeDetector. Contributed by Lisheng Sun.	2019-11-27 10:57:20 +08:00
Ayush Saxena	049940e77b	HDFS-14519. NameQuota is not update after concat operation, so namequota is wrong. Contributed by Ranith Sardar.	2019-11-23 02:30:02 +05:30
Takanobu Asanuma	b25e94ce29	HDFS-14924. RenameSnapshot not updating new modification time. Contributed by hemanthboyina	2019-11-22 13:48:02 +09:00
Yiqun Lin	9b6906fe91	HDFS-14651. DeadNodeDetector checks dead node periodically. Contributed by Lisheng Sun.	2019-11-22 10:53:55 +08:00
Surendra Singh Lilhore	26270196a2	HDFS-14940. HDFS Balancer : Do not allow to set balancer maximum network bandwidth more than 1TB. Contributed by hemanthboyina.	2019-11-22 00:31:25 +05:30
Ayush Saxena	98d249dcda	HDFS-14996. RBF: GetFileStatus fails for directory with EC policy set in case of multiple destinations. Contributed by Ayush Saxena.	2019-11-21 12:08:42 +05:30
Surendra Singh Lilhore	fd264b8265	HDFS-14995. Use log variable directly instead of passing as argument in InvalidateBlocks#printBlockDeletionTime(). Contributed by Lisheng Sun.	2019-11-21 00:00:42 +05:30
He Xiaoqiao	0b50aa29fd	HDFS-14952. Skip safemode if blockTotal is 0 in new NN. Contributed by Xiaoqiao He. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org> Reviewed-by: Mukul Kumar Singh <msingh@apache.org>	2019-11-19 12:16:57 -08:00
Surendra Singh Lilhore	c870514740	HDFS-14992. TestOfflineEditsViewer is failing in Trunk. Contributed by hemanthboyina.	2019-11-20 00:07:43 +05:30
Ayush Saxena	34cb595d2e	HDFS-14967. TestWebHDFS fails in Windows. Contributed by Renukaprasad C.	2019-11-18 18:49:58 +05:30
Yiqun Lin	b3119b9ab6	HDFS-14648. Implement DeadNodeDetector basic model. Contributed by Lisheng Sun.	2019-11-16 11:32:41 +08:00
Hui Fei	67f2c491fe	HDFS-14802. The feature of protect directories should be used in RenameOp (#1669 )	2019-11-15 12:35:37 -08:00
He Xiaoqiao	c892a879dd	HDFS-14882. Consider DataNode load when #getBlockLocation. Contributed by Xiaoqiao He. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org> Reviewed-by: Inigo Goiri <inigoiri@apache.org> Reviewed-by: Istvan Fajth <pifta@cloudera.com>	2019-11-15 12:16:31 -08:00
Erik Krogen	b2cc8b6b4a	HDFS-14973. More strictly enforce Balancer/Mover/SPS throttling of getBlocks RPCs to NameNodes. Contributed by Erik Krogen.	2019-11-15 10:10:37 -08:00
Erik Krogen	586defe711	HDFS-14979 Allow Balancer to submit getBlocks calls to Observer Nodes when possible. Contributed by Erik Krogen.	2019-11-13 11:46:35 -08:00
Chao Sun	97ec34e117	HDFS-14959: [SBNN read] access time should be turned off (#1706 ) Contributed by Chao Sun.	2019-11-12 11:35:50 -08:00
Inigo Goiri	40150da1e1	HDFS-14922. Prevent snapshot modification time got change on startup. Contributed by hemanthboyina.	2019-11-12 10:53:54 -08:00
Ayush Saxena	77934bc07b	HDFS-14975. Add CR for SetECPolicyCommand usage. Contributed by Fei Hui.	2019-11-11 11:05:34 +05:30
Takanobu Asanuma	6663d6a5c2	HDFS-14928. UI: unifying the WebUI across different components. Contributed by Xieming Li	2019-11-11 14:04:57 +09:00
Surendra Singh Lilhore	320008bb7c	HDFS-14720. DataNode shouldn't report block as bad block if the block length is Long.MAX_VALUE. Contributed by hemanthboyina.	2019-11-09 22:14:35 +05:30
Ayush Saxena	247584eb63	HDFS-14958. TestBalancerWithNodeGroup is not using NetworkTopologyWithNodeGroup. Contributed by Jim Brennan.	2019-11-07 22:49:01 +05:30
Chen Liang	dd900259c4	HDFS-14941. Potential editlog race condition can cause corrupted file. Contributed by Chen Liang and Konstantin Shvachko.	2019-11-06 09:56:19 -08:00
Chen Liang	9d0d580031	HDFS-14806. Bootstrap standby may fail if with in-progress tailing. Contributed by Chen Liang and Erik Krogen.	2019-11-06 09:30:04 -08:00
Ayush Saxena	9e287054a8	HDFS-14942. Change Log Level to debug in JournalNodeSyncer#syncWithJournalAtIndex. Contributed by Lisheng Sun.	2019-11-06 22:02:03 +05:30
Ayush Saxena	dcf55838ae	HDFS-14880. Correct the sequence of statistics & exit message in balencer. Contributed by Renukaprasad C.	2019-11-06 21:45:03 +05:30
Surendra Singh Lilhore	c36014165c	HDFS-14384. When lastLocatedBlock token expire, it will take 1~3s second to refetch it. Contributed by Surendra Singh Lilhore.	2019-11-06 19:28:55 +05:30
Inigo Goiri	bfb8f28cc9	HDFS-14775. Add Timestamp for longest FSN write/read lock held log. Contributed by Chen Zhang.	2019-11-05 09:19:52 -08:00
Ayush Saxena	b643a1cbe8	HDFS-14938. Add check if excludedNodes contain scope in DFSNetworkTopology#chooseRandomWithStorageType(). Contributed by Lisheng Sun.	2019-11-05 09:23:20 +05:30
Ayush Saxena	2ffec347eb	HDFS-14946. Erasure Coding: Block recovery failed during decommissioning. Contributed by Fei Hui.	2019-11-05 01:37:15 +05:30
Wei-Chiu Chuang	eb73ba6ed5	HDFS-14945. Revise PacketResponder's log. Contributed by Xudong Cao.	2019-11-04 09:40:42 -08:00
Zhao Junwang	de6b8b0c0b	HDFS-14925. Rename operation should check nest snapshot (#1670 ) If the src directory or any of its descendant is snapshottable and the dst directory or any of its ancestors is snapshottable, we consider this as nested snapshot, which should be denied. Reviewed-by: Shashikant Banerjee <shashikant@apache.org>	2019-11-01 16:37:53 -07:00
Ayush Saxena	7d7acb004a	HDFS-13736. BlockPlacementPolicyDefault can not choose favored nodes when 'dfs.namenode.block-placement-policy.default.prefer-local-node' set to false. Contributed by hu xiaodong.	2019-11-01 22:50:10 +05:30
Surendra Singh Lilhore	02009c3bb7	HDFS-14768. EC : Busy DN replica should be consider in live replica check. Contributed by guojh.	2019-11-01 22:15:05 +05:30
Ayush Saxena	9d25ae7669	HDFS-14920. Erasure Coding: Decommission may hang If one or more datanodes are out of service during decommission. Contributed by Fei Hui.	2019-10-31 23:49:37 +05:30
Ayush Saxena	d9fbedc4ae	HDFS-14936. Add getNumOfChildren() for interface InnerNode. Contributed by Lisheng Sun.	2019-10-31 23:39:27 +05:30
Ayush Saxena	fa4904cdca	HDFS-14935. Refactor DFSNetworkTopology#isNodeInScope. Contributed by Lisheng Sun.	2019-10-29 12:30:24 +05:30
Eric Yang	30ed24a421	HDFS-14730. Removed unused configuration dfs.web.authentication.filter. Contributed by Chen Zhang	2019-10-28 19:27:32 -04:00
Eric Badger	9ef6ed9c1c	HDFS-14931. hdfs crypto commands limit column width. Contributed by Eric Badger	2019-10-28 16:37:04 +00:00
zhaoyim	30db895b59	HDFS-14308. DFSStripedInputStream curStripeBuf is not freed by unbuffer() (#1667 ) Reviewed-by: Aravindan Vijayan <avijayan@cloudera.com> Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>	2019-10-25 13:09:14 -07:00
Takanobu Asanuma	862526530a	HDFS-14933. Fixing a typo in documentation of Observer NameNode. Contributed by Xieming Li	2019-10-25 15:23:05 +09:00
Takanobu Asanuma	0db0f1e399	HDFS-14917. Change the ICON of "Decommissioned & dead" datanode on "dfshealth.html". Contributed by Xieming Li	2019-10-25 10:27:38 +09:00
Wei-Chiu Chuang	a1b4eebcc9	HDFS-14910. Rename Snapshot with Pre Descendants Fail With IllegalArgumentException. Contributed by Wei-Chiu Chuang. (#1672 ) Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>	2019-10-24 13:09:52 -07:00
Vinayakumar B	ee699dc26c	HDFS-14921. Remove SuperUser Check in Setting Storage Policy in FileStatus During Listing. Contributed by Ayush Saxena.	2019-10-24 12:14:09 +05:30
Mukul Kumar Singh	a901405ad8	HDFS-14884. Add sanity check that zone key equals feinfo key while setting Xattrs. Contributed by Mukul Kumar Singh. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2019-10-22 13:31:15 -07:00
Ayush Saxena	6020505943	HDFS-14915. Move Superuser Check Before Taking Lock For Encryption API. Contributed by Ayush Saxena.	2019-10-22 18:44:22 +05:30
Ayush Saxena	19f35cfd57	HDFS-14918. Remove useless getRedundancyThread from BlockManagerTestUtil. Contributed by Fei Hui.	2019-10-22 18:34:02 +05:30
Wei-Chiu Chuang	72003b19bf	HDFS-13901. INode access time is ignored because of race between open and rename. Contributed by Jinglun.	2019-10-21 17:31:43 -07:00
Kihwal Lee	c4e27ef773	HDFS-12749. DN may not send block report to NN after NN restart. Contributed by Xiaoqiao He.	2019-10-21 16:33:15 -05:00
Ayush Saxena	74c2329fc3	HDFS-14913. Correct the value of available count in DFSNetworkTopology#chooseRandomWithStorageType(). Contributed by Ayush Saxena.	2019-10-21 18:35:12 +05:30
Ayush Saxena	447f46d962	HDFS-14847. Erasure Coding: Blocks are over-replicated while EC decommissioning. Contributed by Fei Hui.	2019-10-20 06:10:59 +05:30
Yuming Wang	3d41f33018	HADOOP-16152. Upgrade Eclipse Jetty version to 9.4.x. Contributed by Yuming Wang, Siyao Meng. Co-authored-By: Siyao Meng <smeng@cloudera.com> Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2019-10-18 08:25:02 -07:00
Surendra Singh Lilhore	54dc6b7d72	HDFS-14909. DFSNetworkTopology#chooseRandomWithStorageType() should not decrease storage count for excluded node which is already part of excluded scope. Contributed by Surendra Singh Lilhore.	2019-10-17 22:28:47 +05:30
Ayush Saxena	5527d79adb	HDFS-14810. Review FSNameSystem editlog sync. Contributed by Xiaoqiao He.	2019-10-17 21:56:30 +05:30
Surendra Singh Lilhore	336abbd873	HDFS-14886. In NameNode Web UI's Startup Progress page, Loading edits always shows 0 sec. Contributed by hemanthboyina.	2019-10-14 23:31:43 +05:30
Ashvin	fabd41fa48	HDFS-14856. Fetch file ACLs while mounting external store. (#1478 )	2019-10-14 09:44:56 -07:00
Ayush Saxena	5f4641a120	HDFS-14238. A log in NNThroughputBenchmark should change log level to INFO instead of ERROR. Contributed by Shen Yinjie.	2019-10-12 21:06:23 +05:30
Surendra Singh Lilhore	d76e2655ac	HDFS-14754. Erasure Coding : The number of Under-Replicated Blocks never reduced(addendum). Contributed by Surendra Singh Lilhore.	2019-10-09 23:20:28 +05:30
Chen Liang	72ae371e7a	HDFS-14509. DN throws InvalidToken due to inequality of password when upgrade NN 2.x to 3.x. Contributed by Yuxuan Wang and Konstantin Shvachko.	2019-10-08 11:56:52 -07:00
Ayush Saxena	91320b4461	HDFS-14859. Prevent unnecessary evaluation of costly operation getNumLiveDataNodes when dfs.namenode.safemode.min.datanodes is not zero. Contributed by Srinivasu Majeti.	2019-10-08 14:14:14 +05:30
Surendra Singh Lilhore	382967be51	HDFS-14373. EC : Decoding is failing when block group last incomplete cell fall in to AlignedStripe. Contributed by Surendra Singh Lilhore.	2019-10-08 00:14:30 +05:30
Eric Yang	aa24add8f0	HDFS-14890. Fixed namenode and journalnode startup on Windows. Contributed by Siddharth Wagle	2019-10-04 13:13:10 -04:00
Takanobu Asanuma	b23bdaf085	HDFS-14879. Header was wrong in Snapshot web UI. Contributed by hemanthboyina	2019-10-04 16:47:06 +09:00
Stephen O'Donnell	c99a12167f	HDFS-14637. Namenode may not replicate blocks to meet the policy after enabling upgradeDomain. Contributed by Stephen O'Donnell. Reviewed-by: Ayush Saxena <ayushsaxena@apache.org> Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2019-10-03 22:13:50 -07:00
Virajith Jalaparti	844b766da5	HDFS-14889. Ability to check if a block has a replica on provided storage. Contributed by Ashvin Agrawal. (#1573 )"	2019-10-03 21:32:48 -07:00
Ayush Saxena	a3fe404244	HDFS-14881. Safemode 'forceExit' option, doesn’t shown in help message. Contributed by Renukaprasad C.	2019-10-03 22:34:55 +05:30
Chen Liang	1303255aee	HDFS-14858. [SBN read] Allow configurably enable/disable AlignmentContext on NameNode. Contributed by Chen Liang.	2019-10-02 16:38:20 -07:00
Akira Ajisaka	3df733c250	HDFS-14885. UI: Fix a typo on WebUI of DataNode. Contributed by Xieming Li.	2019-10-02 11:03:16 +09:00
Wei-Chiu Chuang	6ef6594c7e	HDFS-14492. Snapshot memory leak. Contributed by Wei-Chiu Chuang. (#1370 ) * HDFS-14492. Snapshot memory leak. Contributed by Wei-Chiu Chuang. Change-Id: I9e5e450c07ad70aa1905973896c4f627042dbd37 * Fix checkstyle Change-Id: I16d4bd4f03a971e1ed36cf57d89dc42357ef8fbf	2019-10-01 21:16:46 +05:30
Konstantin V Shvachko	b3275ab1f2	HDFS-14305. Fix serial number calculation in BlockTokenSecretManager to avoid token key ID overlap between NameNodes. Contributed by Konstantin V Shvachko.	2019-09-30 16:48:10 -07:00
Konstantin V Shvachko	760b523e58	Revert "HDFS-14305. Fix serial number calculation in BlockTokenSecretManager to avoid token key ID overlap between NameNodes. Contributed by He Xiaoqiao." This reverts commit `0feba4396f`.	2019-09-29 13:11:31 -07:00
Ayush Saxena	2200871607	HDFS-14876. Remove unused imports from TestBlockMissingException.java and TestClose.java. Contributed by Lisheng Sun.	2019-09-28 22:03:53 +05:30
Ayush Saxena	c4c8d5fd0e	HDFS-14849. Erasure Coding: the internal block is replicated many times when datanode is decommissioning. Contributed by HuangTao.	2019-09-28 21:44:44 +05:30
Ayush Saxena	0d5d0b914a	Revert "HDFS-14849. Erasure Coding: the internal block is replicated many times when datanode is decommissioning. Contributed by HuangTao." This reverts commit `ce58c05f1d`.	2019-09-28 21:32:15 +05:30
Sahil Takiar	13b427fc05	HDFS-14564: Add libhdfs APIs for readFully; add readFully to ByteBufferPositionedReadable (#963 ) Contributed by Sahil Takiar. Reviewed-by: Siyao Meng <smeng@cloudera.com>	2019-09-27 12:25:03 -07:00
Ayush Saxena	ce58c05f1d	HDFS-14849. Erasure Coding: the internal block is replicated many times when datanode is decommissioning. Contributed by HuangTao.	2019-09-27 18:58:19 +05:30
Ayush Saxena	a3f68933d7	HDFS-14873. Fix dfsadmin doc for triggerBlockReport. Contributed by Fei Hui.	2019-09-26 19:54:07 +05:30
Ayush Saxena	66400c1cbb	HDFS-14808. EC: Improper size values for corrupt ec block in LOG. Contributed by Ayush Saxena.	2019-09-25 01:31:15 +05:30
Ayush Saxena	eb96a3093e	HDFS-14655. [SBN Read] Namenode crashes if one of The JN is down. Contributed by Ayush Saxena.	2019-09-25 01:16:30 +05:30
Inigo Goiri	afa1006a53	HDFS-14843. Double Synchronization in BlockReportLeaseManager. Contributed by David Mollitor.	2019-09-24 09:58:42 -07:00
Ayush Saxena	c30e495557	HDFS-14853. NPE in DFSNetworkTopology#chooseRandomWithStorageType() when the excludedNode is not present. Contributed by Ranith Sardar.	2019-09-23 21:22:50 +05:30
Vinayakumar B	07c81e9bfc	HADOOP-16558. [COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes (#1494 ). Contributed by Vinayakumar B.	2019-09-23 12:37:47 +05:30
Rakesh Radhakrishnan	659c88801d	HDFS-14818. Check native pmdk lib by 'hadoop checknative' command. Contributed by Feilong He.	2019-09-22 22:02:54 +05:30
Inigo Goiri	3f223bebfa	HDFS-14844. Make buffer of BlockReaderRemote#newBlockReader#BufferedOutputStream configurable. Contributed by Lisheng Sun.	2019-09-20 11:45:01 -07:00
Vinayakumar B	1654497f98	HADOOP-16557. [pb-upgrade] Upgrade protobuf.version to 3.7.1 (#1432 ) HADOOP-16557. [pb-upgrade] Upgrade protobuf.version to 3.7.1. Contributed by Vinayakumar B.	2019-09-20 16:08:30 +05:30
Chen Liang	4ed0aefe9f	HDFS-14822. [SBN read] Revisit GlobalStateIdContext locking when getting server state id. Contributed by Chen Liang.	2019-09-18 16:47:46 -07:00
Wei-Chiu Chuang	5dd859a8a0	HDFS-14836. FileIoProvider should not increase FileIoErrors metric in datanode volume metric. Contributed by Aiphago. Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>	2019-09-18 09:34:01 -07:00
Inigo Goiri	f580a87079	HDFS-14795. Add Throttler for writing block. Contributed by Lisheng Sun.	2019-09-17 14:55:34 -07:00
Ayush Saxena	e04b8a46c3	HDFS-14799. Do Not Call Map containsKey In Conjunction with get. Contributed by hemanthboyina.	2019-09-14 10:39:20 +05:30
Stephen O'Donnell	06ad6540b3	HDFS-14821. Make HDFS-14617 (fsimage sub-sections) off by default. Contributed by Stephen O'Donnell. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2019-09-13 10:36:45 -07:00
Surendra Singh Lilhore	4852a90e4b	HDFS-14754. Erasure Coding : The number of Under-Replicated Blocks never reduced. Contributed by hemanthboyina.	2019-09-13 00:17:13 +05:30
Surendra Singh Lilhore	2ff2a7f612	HDFS-14798. Synchronize invalidateBlocks in DatanodeDescriptor. Contributed by hemanthboyina.	2019-09-12 19:43:18 +05:30
Surendra Singh Lilhore	d1c303a497	HDFS-14699. Erasure Coding: Storage not considered in live replica when replication streams hard limit reached to threshold. Contributed by Zhao Yi Ming.	2019-09-12 19:11:50 +05:30
Akira Ajisaka	68612a0410	HDFS-14840. Use Java Conccurent Instead of Synchronization in BlockPoolTokenSecretManager. Contributed by David Mollitor.	2019-09-12 12:41:57 +09:00
Erik Krogen	c92a3e94d8	HADOOP-15565. Add an inner FS cache to ViewFileSystem, separate from the global cache, to avoid file system leaks. Contributed by Jinglun.	2019-09-06 10:22:28 -07:00
Surendra Singh Lilhore	d98c54816d	HDFS-13913. LazyPersistFileScrubber.run() should log meaningful warn message. Contributed by Daniel Green.	2019-09-06 18:12:42 +05:30
Surendra Singh Lilhore	fa7f03fc56	HDFS-14826. dfs.ha.zkfc.port property duplicated in hdfs-default.xml. Contributed by Renukaprasad C.	2019-09-06 17:55:30 +05:30
Inigo Goiri	494d75eb2b	HDFS-14784. Add more methods to WebHdfsTestUtil to support tests outside of package. Contributed by Chen Zhang.	2019-09-05 21:15:17 -07:00
Inigo Goiri	72d8b92ba5	HDFS-12904. Add DataTransferThrottler to the Datanode transfers. Contributed by Lisheng Sun.	2019-09-05 11:44:02 -07:00
Xiaoyu Yao	3c117163a3	HDFS-14633. The StorageType quota and consume in QuotaFeature is not handled for rename. Contributed by Jinglun.	2019-09-03 09:29:58 -07:00
Ayush Saxena	91b01a1db7	HDFS-14807. SetTimes updates all negative values apart from -1. Contributed by Ayush Saxena.	2019-09-03 18:08:42 +05:30
Surendra Singh Lilhore	5ff76cb8bc	HDFS-14630. Configuration.getTimeDurationHelper() should not log time unit warning in info log. Contributed by hemanthboyina.	2019-09-03 12:37:09 +05:30
Stephen O'Donnell	915cbc91c0	HDFS-14706. Checksums are not checked if block meta file is less than 7 bytes. Contributed by Stephen O'Donnell. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2019-09-02 09:47:04 -07:00
Wei-Chiu Chuang	d207aba026	Revert "HDFS-14706. Checksums are not checked if block meta file is less than 7 bytes. Contributed by Stephen O'Donnell." This reverts commit `7bebad61d9`.	2019-09-02 09:43:44 -07:00
Ayush Saxena	c3abfcefdd	HDFS-13843. RBF: Add optional parameter -d for detailed listing of mount points. Contributed by Ayush Saxena.	2019-09-02 07:24:04 +05:30
Surendra Singh Lilhore	96f7dc1992	HDFS-14796. Define LOG instead of BlockManager.LOG in ErasureCodingWork/ReplicationWork. Contributed by Fei Hui.	2019-08-30 14:31:05 +05:30
Ayush Saxena	e220dac15c	HDFS-12212. Options.Rename.To_TRASH is considered even when Options.Rename.NONE is specified. Contributed by Vinayakumar B.	2019-08-30 08:40:13 +05:30
leosunli	915271245b	Make DataNodePeerMetrics#minOutlierDetectionSamples configurable (#1314 ). Contributed by Lisheng Sun. Signed-off-by: sunlisheng <sunlisheng@xiaomi.com>	2019-08-29 18:35:26 -07:00
Stephen O'Donnell	7bebad61d9	HDFS-14706. Checksums are not checked if block meta file is less than 7 bytes. Contributed by Stephen O'Donnell. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2019-08-29 17:38:07 -07:00
Istvan Fajth	fcb7884bfc	HDFS-8178. QJM doesn't move aside stale inprogress edits files. Contributed by Istvan Fajth. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2019-08-29 16:56:14 -07:00
Inigo Goiri	3b22fcd377	HDFS-14256. Review Logging of NameNode Class. Contributed by David Mollitor.	2019-08-29 14:21:09 -07:00
Inigo Goiri	ffca734c62	HDFS-14104. Review getImageTxIdToRetain. Contributed by David Mollitor.	2019-08-29 14:18:35 -07:00
He Xiaoqiao	f600fbb6c4	HDFS-11246. FSNameSystem#logAuditEvent should be called outside the read or write locks. Contributed by He Xiaoqiao, Kuhu Shukla. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org> Co-authored-by: Kuhu Shukla <kshukla@apache.org>	2019-08-29 10:10:52 -07:00
Surendra Singh Lilhore	29bd6f3fc3	HDFS-8631. WebHDFS : Support setQuota. Contributed by Chao Sun.	2019-08-28 23:58:23 +05:30
CR Hota	6e37d65b03	HDFS-14760. Log INFO mode if snapshot usage and actual usage differ. Contributed by CR Hota. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2019-08-27 16:20:25 -07:00
He Xiaoqiao	dde9399b37	HDFS-14497. Addendum: Write lock held by metasave impact following RPC processing. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2019-08-27 15:26:59 -07:00
Shashikant Banerjee	ce8eb1283a	Revert "HDDS-1610. applyTransaction failure should not be lost on restart. Contributed by Shashikant Banerjee." This reverts commit `62445021d5` as it has unintended changes in DirectoryWithSnapshotFeature class..	2019-08-27 23:23:44 +05:30
Jonathan Hung	8ab7020e64	HDFS-14779. Fix logging error in TestEditLog#testMultiStreamsLoadEditWithConfMaxTxns	2019-08-27 10:30:31 -07:00
Arpit Agarwal	07e3cf952e	HDFS-2470. NN should automatically set permissions on dfs.namenode.*.dir. Contributed by Siddharth Wagle.	2019-08-26 15:57:55 -07:00
Chen Liang	ebef99dcf4	HDFS-14674. [SBN read] Got an unexpected txid when tail editlog. Contributed by wangzhaohui.	2019-08-23 12:18:43 -07:00
Erik Krogen	d699022fce	HDFS-13977. Override shouldForceSync in QuorumOutputStream to allow for proper auto-sync behavior. Contributed by Erik Krogen.	2019-08-23 11:33:21 -07:00
Akira Ajisaka	bd7baea5a5	HDFS-14396. Failed to load image from FSImageFile when downgrade from 3.x to 2.x. Contributed by Fei Hui.	2019-08-23 15:28:27 +09:00
Akira Ajisaka	abc8fde4ca	HDFS-13596. NN restart fails after RollingUpgrade from 2.x to 3.x. Contributed by Fei Hui.	2019-08-23 14:44:34 +09:00
Stephen O'Donnell	b67812ea21	HDFS-14617. Improve fsimage load time by writing sub-sections to the fsimage index (#1028 ). Contributed by Stephen O'Donnell. Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>	2019-08-22 17:09:57 -07:00
Stephen O'Donnell	93daf69f90	HDFS-14675. Increase Balancer Defaults Further. Contributed by Stephen O'Donnell. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org> Reviewed-by: Gabor Bota <gabota@apache.org>	2019-08-22 15:21:33 -07:00
Inigo Goiri	e04dcfdc57	HDFS-14583. FileStatus#toString() will throw IllegalArgumentException. Contributed by xuzq.	2019-08-22 10:22:38 -07:00
Surendra Singh Lilhore	76790a1e67	HDFS-14358. Provide LiveNode and DeadNode filter in DataNode UI. Contributed by hemanthboyina.	2019-08-22 12:27:45 +05:30
Ayush Saxena	0f598aed13	HDFS-14276. [SBN read] Reduce tailing overhead. Contributed by Wei-Chiu Chuang.	2019-08-22 02:25:06 +05:30
Wei-Chiu Chuang	57f7370174	Revert "HDFS-14476. lock too long when fix inconsistent blocks between disk and in-memory. Contributed by Sean Chow." This reverts commit `b58a35f374`.	2019-08-21 08:59:49 -07:00
Surendra Singh Lilhore	3a145e2918	HDFS-14582. Failed to start DN with ArithmeticException when NULL checksum used. Contributed by Surendra Singh Lilhore. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2019-08-20 15:55:26 -07:00
Sandeep Kumar	269b543367	HADOOP-16523. Minor spell mistake in comment : Line number 101 (#388 )	2019-08-20 13:36:23 -07:00
Wei-Chiu Chuang	4cb22cd867	HDFS-14311. Multi-threading conflict at layoutVersion when loading block pool storage. Contributed by Yicong Cai.	2019-08-20 10:37:11 -07:00
Sunil G	bd92462321	HDFS-14729. Upgrade Bootstrap and jQuery versions used in HDFS UIs. Contributed by Vivek Ratnavel Subramanian. This closes #1297	2019-08-20 18:42:19 +05:30
Shashikant Banerjee	62445021d5	HDDS-1610. applyTransaction failure should not be lost on restart. Contributed by Shashikant Banerjee.	2019-08-20 14:37:01 +05:30
Wei-Chiu Chuang	360a96f342	HDFS-13709. Report bad block to NN when transfer block encounter EIO exception. Contributed by Chen Zhang.	2019-08-19 13:08:55 -07:00
HUAN-PING SU	abae6ff2a2	HDFS-14746. Trivial test code update after HDFS-14687. Contributed by kevin su. Reviewed-by: Surendra Singh Lilhore <surendralilhore@apache.org> Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2019-08-19 11:14:51 -07:00
Surendra Singh Lilhore	b8db5b9a98	HDFS-14687. Standby Namenode never come out of safemode when EC files are being written. Contributed by Surendra Singh Lilhore. Reviewed-by: Siyao Meng <smeng@cloudera.com> Reviewed-by: Wei-Chiu CHuang <weichiu@apache.org> Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2019-08-18 18:13:35 -07:00
Wei-Chiu Chuang	b58a35f374	HDFS-14476. lock too long when fix inconsistent blocks between disk and in-memory. Contributed by Sean Chow.	2019-08-18 17:46:00 -07:00
Wei-Chiu Chuang	d873ddd656	HDFS-14127. Add a description about the observer read configuration. Contributed by xiangheng.	2019-08-17 04:39:22 -07:00
Wei-Chiu Chuang	528378784f	HDFS-12012. Fix spelling mistakes in BPServiceActor.java. Contributed by chencan.	2019-08-17 04:37:05 -07:00
hunshenshi	a38b9e137e	HDFS-14456:HAState#prepareToEnterState neednt a lock (#770 ) Contributed by hunshenshi.	2019-08-16 14:53:06 -07:00
LeonGao91	9a1d8cfaf5	HDFS-14678. Allow triggerBlockReport to a specific namenode. (#1252 ). Contributed by Leon Gao.	2019-08-16 08:00:51 -07:00
Surendra Singh Lilhore	34681643e9	HDFS-14719. Correct the safemode threshold value in BlockManagerSafeMode. Contributed by hemanthboyina.	2019-08-15 12:12:18 +05:30
Shashikant Banerjee	0a85af959c	HDFS-13101. Yet another fsimage corruption related to snapshot. Contributed by Shashikant Banerjee.	2019-08-15 10:16:25 +05:30
Siyao Meng	3c0382f1b9	HDFS-14595. HDFS-11848 breaks API compatibility. Contributed by Siyao Meng. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org> Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>	2019-08-14 07:30:35 -07:00
Wei-Chiu Chuang	c13ec7ab66	HDFS-14491. More Clarity on Namenode UI Around Blocks and Replicas. Contributed by Siyao Meng.	2019-08-13 17:15:26 -07:00
Wei-Chiu Chuang	633b7c1cfe	HDFS-14625. Make DefaultAuditLogger class in FSnamesystem to Abstract. Contributed by hemanthboyina.	2019-08-13 16:50:49 -07:00
Masatake Iwasaki	da0006fe04	HDFS-14423. Percent (%) and plus (+) characters no longer work in WebHDFS. Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>	2019-08-14 08:39:40 +09:00
Ayush Saxena	e9b6b81de4	HDFS-13505. Turn on HDFS ACLs by default. Contributed by Siyao Meng.	2019-08-13 19:17:10 +05:30
Ayush Saxena	454420e4f2	HDFS-14708. TestLargeBlockReport#testBlockReportSucceedsWithLargerLengthLimit fails in trunk. Contributed by Lisheng Sun.	2019-08-13 08:27:20 +05:30
Siyao Meng	c92b49876a	HDFS-14148. HDFS OIV ReverseXML SnapshotSection parser throws exception when there are more than one snapshottable directory (#1274 ) Contributed by Siyao Meng.	2019-08-12 17:26:20 -07:00
Yiqun Lin	8a77a224c7	HDFS-13359. DataXceiver hung due to the lock in FsDatasetImpl#getBlockInputStream. Contributed by Yiqun Lin. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2019-08-09 18:40:28 -07:00
Siyao Meng	e02ffed1b1	HDFS-12125. Document the missing EC removePolicy command (#1258 ) Contributed by Wenxin He, Siyao Meng. Co-authored-by: Wenxin He <wenxin.he@gmail.com>	2019-08-09 18:00:22 -07:00
Wei-Chiu Chuang	ce3c5a3e3b	HDFS-14623. In NameNode Web UI, for Head the file (first 32K) old data is showing. Contributed by hemanthboyina.	2019-08-09 15:41:37 -07:00
Wei-Chiu Chuang	865021b8c9	HDFS-14195. OIV: print out storage policy id in oiv Delimited output. Contributed by Wang, Xinglong.	2019-08-09 15:37:29 -07:00
Rakesh Radhakrishnan	f6fa865d6f	HDFS-14700. Clean up pmem cache before setting pmem cache capacity. Contributed by Feilong He.	2019-08-09 14:07:54 +05:30
Wei-Chiu Chuang	6ad9a11494	HDFS-14693. NameNode should log a warning when EditLog IPC logger's pending size exceeds limit. Contributed by Xudong Cao.	2019-08-08 13:50:30 -07:00
Wei-Chiu Chuang	2265872c2d	HDFS-14705. Remove unused configuration dfs.min.replication. Contributed by CR Hota.	2019-08-08 13:48:29 -07:00
Wei-Chiu Chuang	23f91f68b8	HDFS-14662. Document the usage of the new Balancer "asService" parameter. Contributed by Chen Zhang.	2019-08-08 13:45:29 -07:00
Stephen O'Donnell	b0799148cf	HDFS-14459. ClosedChannelException silently ignored in FsVolumeList.addBlockPool(). Contributed by Stephen O'Donnell. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2019-08-08 13:36:39 -07:00
Ayush Saxena	11f750e6a7	HDFS-14616. Add the warn log when the volume available space isn't enough. Contributed by liying.	2019-08-08 00:49:58 +05:30
Ayush Saxena	de64253892	HDFS-14608. DataNode#DataTransfer should be named. Contributed by Inigo Goiri.	2019-08-08 00:37:56 +05:30
Erik Krogen	827dbb11e2	HDFS-14370. Add exponential backoff to the edit log tailer to avoid spinning on empty edit tail requests. Contributed by Erik Krogen.	2019-08-07 09:25:58 -07:00
Yiqun Lin	a5bb1e8ee8	HDFS-14313. Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du. Contributed by Lisheng Sun.	2019-08-07 10:18:11 +08:00
Lokesh Jain	954ff36360	HDFS-14692. Upload button should not encode complete url. Contributed by Lokesh Jain. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2019-08-06 15:30:50 -07:00
Stephen O'Donnell	35e0a01d7b	HDFS-14557. JournalNode error: Can't scan a pre-transactional edit log. Contributed by Stephen O'Donnell. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2019-08-06 07:57:46 -07:00
Ayush Saxena	11272159bb	HDFS-14679. Failed to add erasure code policies with example template. Contributed by Yuan Zhou.	2019-08-06 08:42:15 +05:30
Ayush Saxena	8f40856f76	HDFS-14669. TestDirectoryScanner#testDirectoryScannerInFederatedCluster fails intermittently in trunk. Contributed by qiang Liu.	2019-08-03 12:15:41 +05:30
Ayush Saxena	e503db5f44	HDFS-12826. Document Saying the RPC port, But it's required IPC port in HDFS Federation Document. Contributed by usharani.	2019-08-03 12:05:46 +05:30
Erik Krogen	e7a0b8aa83	HDFS-14462 Ensure WebHDFS client throws the correct exception during writes. Contributed by Simbarashe Dzinamarira.	2019-08-02 11:48:31 -07:00
Wei-Chiu Chuang	b964b81f85	HDFS-14685. DefaultAuditLogger doesn't print CallerContext. Contributed by xuzq.	2019-08-02 10:54:32 -07:00
Wei-Chiu Chuang	c2d00c8450	HDFS-13131. Modifying testcase testEnableAndDisableErasureCodingPolicy. Contributed by chencan.	2019-08-01 19:11:43 -07:00
Siyao Meng	99bf1dc9eb	HDFS-14683. WebHDFS: Add erasureCodingPolicy field to GETCONTENTSUMMARY response (#1189 ) Contributed by Siyao Meng.	2019-08-01 17:14:07 -07:00
Wei-Chiu Chuang	32607dbd98	HDFS-14631.The DirectoryScanner doesn't fix the wrongly placed replica. Contributed by Jinglun.	2019-08-01 15:48:36 -07:00
Wei-Chiu Chuang	99f88c30cb	HDFS-14419. Avoid repeated calls to the listOpenFiles function. Contributed by HuangTao.	2019-07-30 16:50:06 -07:00
Chao Sun	3ae775d740	HDFS-14034. Support getQuotaUsage API in WebHDFS. Contributed by Chao Sun. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2019-07-30 16:01:17 -07:00
Erik Krogen	1f26cc8705	HDFS-13783. Add an option to the Balancer to make it run as a long-running service. Contributed by Chen Zhang.	2019-07-30 15:42:55 -07:00
Inigo Goiri	7849bdcf70	HDFS-14449. Expose total number of DT in JMX for Namenode. Contributed by Fengnan Li.	2019-07-30 13:45:27 -07:00
Ayush Saxena	8deced0073	HDFS-14677. TestDataNodeHotSwapVolumes#testAddVolumesConcurrently fails intermittently in trunk. Contributed by Chen Zhang.	2019-07-30 22:17:39 +05:30
Wei-Chiu Chuang	8053085388	HDFS-14429. Block remain in COMMITTED but not COMPLETE caused by Decommission. Contributed by Yicong Cai.	2019-07-29 14:31:34 -07:00
Ayush Saxena	02bd02b5af	HDFS-14660. [SBN Read] ObserverNameNode should throw StandbyException for requests not from ObserverProxyProvider. Contributed by Chao Sun.	2019-07-28 08:41:42 +05:30
Ayush Saxena	ce99cc31e9	HDFS-14303. Addendum: check block directory logic not correct when there is only meta file, print no meaning warn log. Contributed by qiang Liu.	2019-07-26 08:31:00 +05:30
Masatake Iwasaki	6b8107ad97	HDFS-14135. TestWebHdfsTimeouts Fails intermittently in trunk. Contributed by Ayush Saxena. Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>	2019-07-26 09:35:23 +09:00
Ayush Saxena	62deab17a3	HDFS-14647. NPE during secure namenode startup. Contributed by Fengnan Li.	2019-07-25 06:51:07 +05:30
Ayush Saxena	377f95bbe8	HDFS-13693. Remove unnecessary search in INodeDirectory.addChild during image loading. Contributed by Lisheng Sun.	2019-07-23 08:37:55 +05:30
Chen Liang	5e6cc6fe8a	HDFS-12979. [SBN read] StandbyNode should upload FsImage to ObserverNode after checkpointing. Contributed by Chen Liang.	2019-07-17 14:37:16 -07:00
Ayush Saxena	af0665c7f5	HDFS-13647. Fix the description of storageType option for space quota. Contributed by Takanobu Asanuma.	2019-07-17 23:22:32 +05:30
Ayush Saxena	f77d54c243	HDFS-14642. processMisReplicatedBlocks does not return correct processed count. Contributed by Stephen O'Donnell.	2019-07-16 08:14:27 +05:30
Rakesh Radhakrishnan	30a8f840f1	HDFS-14357. Update documentation for HDFS cache on SCM support. Contributed by Feilong He.	2019-07-15 13:18:23 +05:30
Rakesh Radhakrishnan	e98adb00b7	HDFS-14458. Report pmem stats to namenode. Contributed by Feilong He.	2019-07-15 13:02:37 +05:30
Shashikant Banerjee	f9fab9f22a	HDFS-14499. Misleading REM_QUOTA value with snapshot and trash feature enabled for a directory. Contributed by Shashikant Banerjee.	2019-07-12 15:42:06 +05:30
Masatake Iwasaki	00dd843a1a	HDFS-14466. Add a regression test for HDFS-14323. Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2019-07-12 10:04:55 +09:00
Chen Liang	8fb5ca3f40	HDFS-14611. Move handshake secret field from Token to BlockAccessToken. Contributed by Chen Liang.	2019-07-11 13:23:25 -07:00
Inigo Goiri	5747f6cff5	HDFS-14624. When decommissioning a node, log remaining blocks to replicate periodically. Contributed by Stephen O'Donnell.	2019-07-11 08:55:44 -07:00
Inigo Goiri	eccc9a40de	HDFS-12703. Exceptions are fatal to decommissioning monitor. Contributed by He Xiaoqiao.	2019-07-10 11:11:52 -07:00
Ajay Yadav	cdb20adfcc	HDDS-1611. Evaluate ACL on volume bucket key and prefix to authorize access. Contributed by Ajay Kumar. (#973 )	2019-07-10 11:03:58 -07:00
Yiqun Lin	993dc8726b	HDFS-14632. Reduce useless #getNumLiveDataNodes call in SafeModeMonitor. Contributed by He Xiaoqiao.	2019-07-10 10:53:34 +08:00
Erik Krogen	4632708148	HDFS-14547. Improve memory efficiency of quotas when storage type quotas are not set. Contributed by Jinglun.	2019-07-08 14:46:25 -07:00
Wei-Chiu Chuang	ec851e4db2	HDFS-12862. CacheDirective becomes invalid when NN restart or failover. Contributed by Wang XL.	2019-07-08 15:23:46 +08:00
leosunli	9c90729486	HDFS-13694. Making md5 computing being in parallel with image loading.	2019-07-05 10:17:37 -07:00
Masatake Iwasaki	1c254a8ec7	HDFS-14629. Property value Hard Coded in DNConf.java. Contributed by hemanthboyina.	2019-07-04 23:20:42 +09:00
Weiwei Yang	729cb3aefe	HDFS-12748. NameNode memory leak when accessing webhdfs GETHOMEDIRECTORY. Contributed by Weiwei Yang.	2019-07-04 09:37:24 +08:00
paulward24	d8bac50e12	HDFS-14610. HashMap is not thread safe. Field storageMap is typically synchronized by storageMap. However, in one place, field storageMap is not protected with synchronized. (#1015 )	2019-07-01 13:54:49 -07:00
paulward24	d203045c30	HDFS-14618. Incorrect synchronization of ArrayList field (ArrayList is thread-unsafe). Contributed by Paul Ward.	2019-06-28 16:06:43 -07:00
Inigo Goiri	4a212242d9	HDFS-14599. HDFS-12487 breaks test TestDiskBalancer.testDiskBalancerWithFedClusterWithOneNameServiceEmpty. Contributed by He Xiaoqiao.	2019-06-27 10:00:05 -07:00
Anu Engineer	041e7a7dee	HDFS-14598. Findbugs warning caused by HDFS-12487. Contributed by He Xiaoqiao.	2019-06-24 19:07:22 -07:00
Ajay Kumar	95c94dcca7	HADOOP-16350. Ability to tell HDFS client not to request KMS Information from NameNode. Ccontributed by Greg Senia, Ajay Kumar.	2019-06-24 11:59:07 -07:00
Ayush Saxena	dd8c2b92df	HDFS-13853. RBF: RouterAdmin update cmd is overwriting the entry not updating the existing. Contributed by Ayush Saxena.	2019-06-24 22:03:04 +05:30
Ayush Saxena	6c42d40504	HDFS-14316. RBF: Support unavailable subclusters for mount points with multiple destinations. Contributed by Inigo Goiri.	2019-06-24 22:03:04 +05:30
Giovanni Matteo Fumarola	2a2d5eb441	HDFS-14334. RBF: Use human readable format for long numbers in the Router UI. Contributed by Inigo Goiri.	2019-06-24 22:03:03 +05:30
Giovanni Matteo Fumarola	8b8ff5ccbc	HDFS-14249. RBF: Tooling to identify the subcluster location of a file. Contributed by Inigo Goiri.	2019-06-24 22:03:03 +05:30
Inigo Goiri	f4e2bfce58	HDFS-13856. RBF: RouterAdmin should support dfsrouteradmin -refreshRouterArgs command. Contributed by yanghuafeng.	2019-06-24 22:03:02 +05:30
Yiqun Lin	8f6f9d9c83	HDFS-13443. RBF: Update mount table cache immediately after changing (add/update/remove) mount table entries. Contributed by Mohammad Arshad.	2019-06-24 22:03:02 +05:30
Wei-Chiu Chuang	645d67bc4f	HDFS-14074. DataNode runs async disk checks maybe throws NullPointerException, and DataNode failed to register to NameSpace. Contributed by guangyi lu.	2019-06-21 18:27:03 -07:00
Wei-Chiu Chuang	1524e2e6c5	HDFS-12487. FsDatasetSpi.isValidBlock() lacks null pointer check inside and neither do the callers. Contributed by liumi.	2019-06-21 18:17:36 -07:00
Lokesh Jain	272b96d243	HDFS-13893. DiskBalancer: no validations for Disk balancer commands. Contributed by Lokesh Jain. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2019-06-20 20:23:05 -07:00
Takanobu Asanuma	98d2065643	HDFS-12564. Add the documents of swebhdfs configurations on the client side. Contributed by Takanobu Asanuma. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2019-06-20 20:17:24 -07:00
Surendra Singh Lilhore	5962a518bd	HDFS-14581. Appending to EC files crashes NameNode. Contributed by Surendra Singh Lilhore. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2019-06-20 09:40:29 -07:00
Wei-Chiu Chuang	71ecd2e411	HDFS-14303. check block directory logic not correct when there is only meta file, print no meaning warn log. Contributed by qiang Liu.	2019-06-19 10:27:53 -07:00
David Mollitor	f5ecc0bc08	HDFS-14103. Review Logging of BlockPlacementPolicyDefault. Contributed by David Mollitor. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2019-06-19 10:06:40 -07:00
Wei-Chiu Chuang	d3ac516665	Revert "HDFS-13287. TestINodeFile#testGetBlockType results in NPE when run alone. Contributed by Virajith Jalaparti." This reverts commit `a1c3868c4f`.	2019-06-19 06:27:02 -07:00
Wei-Chiu Chuang	48e564f7e2	HDFS-14537. Journaled Edits Cache is not cleared when formatting the JN. Contributed by Ranith Sardar.	2019-06-18 22:28:21 -07:00
Virajith Jalaparti	a1c3868c4f	HDFS-13287. TestINodeFile#testGetBlockType results in NPE when run alone. Contributed by Virajith Jalaparti. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2019-06-18 22:17:49 -07:00

... 9 10 11 12 13 ...

8069 Commits