big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
23,780 Commits 2 Branches 2 Tags 264 MiB
9ca6298a9a
Commit Graph

8138 Commits

Author SHA1 Message Date
S O'Donnell
9ca6298a9a HDFS-15285. The same distance and load nodes don't shuffle when consider DataNode load. Contributed by Lisheng Sun. 2020-04-29 15:56:06 +01:00
Akira Ajisaka
5e0eda5d5f
HDFS-15286. Concat on a same file deleting the file. Contributed by hemanthboyina. 2020-04-28 06:17:14 +09:00
Toshihiro Suzuki
62c26b91fd
HDFS-15298 Fix the findbugs warnings introduced in HDFS-15217 (#1979) 2020-04-27 17:20:47 +09:00
Mingliang Liu
74d4a18735
HDFS-15297. TestNNHandlesBlockReportPerStorage::blockReport_02 fails intermittently. Contributed by Ayush Saxena 2020-04-25 15:43:02 -07:00
Dhiraj
453c9c9a67
HDFS-15281. Make sure ZKFC uses dfs.namenode.rpc-address to bind to host address (#1964) 
Contributed by Dhiraj Hegde.

Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Inigo Goiri <inigoiri@apache.org>
 2020-04-25 13:04:32 -07:00
Wei-Chiu Chuang
453771f170 HDFS-15295. AvailableSpaceBlockPlacementPolicy should use chooseRandomWithStorageTypeTwoTrial() for better performance. Contributed by Jinglun. 2020-04-24 11:53:24 -07:00
Mingliang Liu
5958af4dbc
HDFS-15276. Concat on INodeRefernce fails with illegal state exception. Contributed by hemanthboyina 2020-04-22 13:31:02 -07:00
Toshihiro Suzuki
1824aee9da
HDFS-15217 Add more information to longest write/read lock held log 2020-04-18 13:52:07 -07:00
Ayush Saxena
37d6582223 HDFS-15266. Add missing DFSOps Statistics in WebHDFS. Contributed by Ayush Saxena. 2020-04-17 13:32:02 +05:30
Stephen O'Donnell
3481895f8a HDFS-15283. Cache pool MAXTTL is not persisted and restored on cluster restart. Contributed by Stephen O'Donnell. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2020-04-16 20:18:25 -07:00
Ayush Saxena
cc5c1da7c1 HDFS-15277. Parent directory in the explorer does not support all path formats. Contributed by Jianfei Jiang. 2020-04-15 23:01:58 +05:30
Wei-Chiu Chuang
061afcdf30
HDFS-15269. NameNode should check the authorization API version only … (#1945) 
Reviewed-by: Takanobu Asanuma <tasanuma@apache.org>
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
 2020-04-09 09:19:35 -07:00
Ayush Saxena
1189af4746 HDFS-15263. Fix the logic of scope and excluded scope in Network Topology. Contributed by Ayush Saxena. 2020-04-08 15:55:14 +05:30
Toshihiro Suzuki
c12ddbd1de
HDFS-15249 ThrottledAsyncChecker is not thread-safe. (#1922) 2020-04-07 14:38:09 +09:00
Inigo Goiri
0b855b9f35 HDFS-15256. Fix typo in DataXceiverServer#run(). Contributed by Lisheng Sun. 2020-04-06 13:05:33 -07:00
He Xiaoqiao
d3b5951572 HDFS-15242. Add metrics for operations hold lock times of FsDatasetImpl. Contributed by Xiaoqiao He. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
 2020-04-01 16:36:46 -07:00
Ayush Saxena
960c9ebaea HDFS-15245. Improve JournalNode web UI. Contributed by Jianfei Jiang. 2020-03-30 02:40:25 +05:30
Brahma Reddy Battula
8914cf9167 Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30
Inigo Goiri
f7a94ec0a4 HDFS-15239. Add button to go to the parent directory in the explorer. Contributed by hemanthboyina. 2020-03-29 08:17:02 -07:00
Ayush Saxena
696a663cd0 HDFS-15193. Improving the error message for missing dfs.namenode.rpc-address.$NAMESERVICE. Contributed by Ctest. 2020-03-29 00:01:22 +05:30
Chen Liang
f531a4a487 HDFS-15191. EOF when reading legacy buffer in BlockTokenIdentifier. Contributed by Steven Rand. 2020-03-27 12:55:38 -07:00
Wei-Chiu Chuang
0fa7bf47df
HDFS-15234. Add a default method body for the INodeAttributeProvider#checkPermissionWithContext API. (#1909) 2020-03-25 16:03:26 -07:00
Inigo Goiri
cdcb77a2c5 HDFS-15075. Remove process command timing from BPServiceActor. Contributed by Xiaoqiao He. 2020-03-25 11:30:54 -07:00
Arpit Agarwal
a700803a18
HDFS-15154. Allow only hdfs superusers the ability to assign HDFS storage policies. Contributed by Siddharth Wagle. 
Change-Id: I32d6dd2837945b8fc026a759aa367c55daefe348
 2020-03-25 10:28:30 -07:00
Toshihiro Suzuki
d353b30baf
HDFS-15215. The Timestamp for longest write/read lock held log is wrong 2020-03-24 14:50:15 -07:00
Ayush Saxena
ea87d60493 HDFS-13377. The owner of folder can set quota for his sub folder. Contributed by Yang Yun. 2020-03-24 22:56:09 +05:30
Karthik Palanisamy
5250cd6db3
HDFS-15201 SnapshotCounter hits MaxSnapshotID limit (#1870) 2020-03-24 15:15:46 +05:30
He Xiaoqiao
af64ce2f4a HDFS-15113. Addendum: Missing IBR when NameNode restart if open processCommand async feature. Contributed by Xiaoqiao He. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2020-03-23 12:43:51 -07:00
Ayush Saxena
62c8015683 HDFS-15227. NPE if the last block changes from COMMITTED to COMPLETE during FSCK. Contributed by Ayush Saxena. 2020-03-22 11:44:18 +05:30
Takanobu Asanuma
b74e47e31a HDFS-15214. WebHDFS: Add snapshot counts to Content Summary. Contributed by hemanthboyina. 2020-03-22 01:44:55 +09:00
Ayush Saxena
f9bb2a8cc5 HDFS-15200. Delete Corrupt Replica Immediately Irrespective of Replicas On Stale Storage. Contributed by Ayush Saxena. 2020-03-19 22:26:34 +05:30
Ayush Saxena
bb41ddaf1e HDFS-15223. FSCK fails if one namenode is not available. Contributed by Ayush Saxena. 2020-03-19 21:23:13 +05:30
Ayush Saxena
654db35fa2 HDFS-14919. Provide Non DFS Used per DataNode in DataNode UI. Contributed by Lisheng Sun. 2020-03-18 21:00:45 +05:30
Wei-Chiu Chuang
096533c2dc
HDFS-15208. Suppress bogus AbstractWadlGeneratorGrammarGenerator in KMS stderr in hdfs. 
Contributed by Wei-Chiu Chuang.

Change-Id: I7abeaf4b65e2a4758356aa69ab8930f7e993077d
 2020-03-18 13:27:13 +00:00
Ayush Saxena
8a7c54995a HDFS-15159. Prevent adding same DN multiple times in PendingReconstructionBlocks. Contributed by hemanthboyina. 2020-03-15 16:16:27 +05:30
Wei-Chiu Chuang
7b637a3341 HDFS-14820. The default 8KB buffer of BlockReaderRemote#newBlockReader#BufferedOutputStream is too big. Contributed by Lisheng Sun. 2020-03-13 19:01:23 -07:00
He Xiaoqiao
e9955bb8ff HDFS-15113. Missing IBR when NameNode restart if open processCommand async feature. Contributed by Xiaoqiao He. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Brahma Reddy Battula <brahma@apache.org>
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
 2020-03-13 18:55:06 -07:00
Wei-Chiu Chuang
4b95c242ec
HDFS-14743. Enhance INodeAttributeProvider/ AccessControlEnforcer Interface in HDFS to support Authorization of mkdir, rm, rmdir, copy, move etc... (#1829) 
Reviewed-by: Xiaoyu Yao <xyao@apache.org>
 2020-03-13 11:29:44 -07:00
Ayush Saxena
fb1d7635ae HDFS-15155. writeIoRate of DataNodeVolumeMetrics is never used. Contributed by Haibin Huang. 2020-03-13 21:10:03 +05:30
S O'Donnell
25497799cb HDFS-15216. Wrong Use Case of -showprogress in fsck. Contributed by Ravuri Sushma sree. 2020-03-13 15:24:19 +00:00
Wei-Chiu Chuang
20903f72b4 HDFS-15039. Cache meta file length of FinalizedReplica to reduce call File.length(). Contributed by Yang Yun. 2020-03-12 19:23:12 -07:00
Inigo Goiri
0695f7a538 HDFS-14612. SlowDiskReport won't update when SlowDisks is always empty in heartbeat. Contributed by Haibin Huang. 2020-03-12 17:22:11 -07:00
Surendra Singh Lilhore
f736408a83 HDFS-14442. Disagreement between HAUtil.getAddressOfActive and RpcInvocationHandler.getConnectionId. Contributed by Ravuri Sushma sree. 2020-03-12 18:59:17 +05:30
Steve Loughran
c734d69a55
HADOOP-16898. Batch listing of multiple directories via an (unstable) interface 
Contributed by Steve Loughran.

This moves the new API of HDFS-13616 into a interface which is implemented by
HDFS RPC filesystem client (not WebHDFS or any other connector)

This new interface, BatchListingOperations, is in hadoop-common,
so applications do not need to be compiled with HDFS on the classpath.
They must cast the FS into the interface.

instanceof can probe the client for having the new interface -the patch
also adds a new path capability to probe for this.

The FileSystem implementation is cut; tests updated as appropriate.

All new interfaces/classes/constants are marked as @unstable.

Change-Id: I5623c51f2c75804f58f915dd7e60cb2cffdac681
 2020-03-09 14:51:16 +00:00
Inigo Goiri
4062217189 HDFS-15212. TestEncryptionZones.testVersionAndSuiteNegotiation fails in trunk. Contributed by Ayush Saxena. 2020-03-06 10:14:53 -08:00
Inigo Goiri
3afd4cbe89 HDFS-14977. Quota Usage and Content summary are not same in Truncate with Snapshot. Contributed by hemanthboyina. 2020-03-04 10:13:23 -08:00
Inigo Goiri
1d4d0fcbe1 HDFS-15204. TestRetryCacheWithHA testRemoveCacheDescriptor fails intermittently. Contributed by Ahmed Hussein. 2020-03-04 10:02:54 -08:00
Akshay Nehe
bbd704bb82
HADOOP-16899. Update HdfsDesign.md to reduce ambiguity. (#1871) 2020-03-04 11:07:31 +09:00
Ayush Saxena
1a636da041 HDFS-15033. Support to save replica cached files to other place and make expired time configurable. Contributed by Yang Yun. 2020-02-29 09:55:29 +05:30
Inigo Goiri
97b797c314 HDFS-15149. TestDeadNodeDetection test cases time-out. Contributed by Lisheng Sun. 2020-02-28 18:47:22 -08:00