Virajith Jalaparti
ea97fe250c
HDFS-15436. Default mount table name used by ViewFileSystem should be configurable ( #2100 )
...
* HDFS-15436. Default mount table name used by ViewFileSystem should be configurable
* Replace Constants.CONFIG_VIEWFS_DEFAULT_MOUNT_TABLE use in tests
* Address Uma's comments on PR#2100
* Sort lists in test to match without concern to order
* Address comments, fix checkstyle and fix failing tests
* Fix checkstyle
(cherry picked from commit bed0a3a374
)
2020-06-27 16:22:50 -07:00
Uma Maheswara Rao G
81e33d22a0
HDFS-15429. mkdirs should work when parent dir is an internalDir and fallback configured. Contributed by Uma Maheswara Rao G.
...
(cherry picked from commit d5e1bb6155
)
2020-06-27 15:42:36 -07:00
Uma Maheswara Rao G
29a8ee4be6
HDFS-15427. Merged ListStatus with Fallback target filesystem and InternalDirViewFS. Contributed by Uma Maheswara Rao G.
...
(cherry picked from commit 7c02d1889b
)
2020-06-27 15:42:14 -07:00
Uma Maheswara Rao G
5f67c3f3ca
HDFS-15418. ViewFileSystemOverloadScheme should represent mount links as non symlinks. Contributed by Uma Maheswara Rao G.
...
(cherry picked from commit b27810aa60
)
2020-06-27 15:41:48 -07:00
Stephen O'Donnell
7613191fcd
HDFS-15372. Files in snapshots no longer see attribute provider permissions. Contributed by Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 630ac9e7e5fdbc3ce86476cf0167255ab9b0470a)
2020-06-18 06:45:28 -07:00
Wei-Chiu Chuang
10880dc396
Revert "HDFS-15372. Files in snapshots no longer see attribute provider permissions. Contributed by Stephen O'Donnell."
...
This reverts commit 0b9e5ea592
.
2020-06-18 06:45:28 -07:00
S O'Donnell
3dc9db3aed
HDFS-15406. Improve the speed of Datanode Block Scan. Contributed by hemanthboyina
...
(cherry picked from commit 123777823e
)
2020-06-18 12:29:12 +01:00
Uma Maheswara Rao G
120ee793fc
HDFS-15387. FSUsage#DF should consider ViewFSOverloadScheme in processPath. Contributed by Uma Maheswara Rao G.
...
(cherry picked from commit 785b1def95
)
2020-06-16 20:02:44 -07:00
Ayush Saxena
bee2846bee
HDFS-15389. DFSAdmin should close filesystem and dfsadmin -setBalancerBandwidth should work with ViewFSOverloadScheme. Contributed by Ayush Saxena
...
(cherry picked from commit cc671b16f7
)
2020-06-16 16:54:40 -07:00
Uma Maheswara Rao G
418580446b
HDFS-15330. Document the ViewFSOverloadScheme details in ViewFS guide. Contributed by Uma Maheswara Rao G.
...
(cherry picked from commit 76fa0222f0
)
2020-06-16 16:54:01 -07:00
Uma Maheswara Rao G
0b5e202614
HDFS-15321. Make DFSAdmin tool to work with ViewFileSystemOverloadScheme. Contributed by Uma Maheswara Rao G.
...
(cherry picked from commit ed83c865dd
)
2020-06-16 16:53:38 -07:00
Uma Maheswara Rao G
8e71e85af7
HDFS-15322. Make NflyFS to work when ViewFsOverloadScheme's scheme and target uris schemes are same. Contributed by Uma Maheswara Rao G.
...
(cherry picked from commit 4734c77b4b
)
2020-06-16 16:53:10 -07:00
Abhishek Das
5b248de42d
HADOOP-17024. ListStatus on ViewFS root (ls "/") should list the linkFallBack root (configured target root). Contributed by Abhishek Das.
...
(cherry picked from commit ce4ec74453
)
2020-06-16 16:52:29 -07:00
Uma Maheswara Rao G
544996c857
HDFS-15306. Make mount-table to read from central place ( Let's say from HDFS). Contributed by Uma Maheswara Rao G.
...
(cherry picked from commit ac4a2e11d9
)
2020-06-16 16:50:57 -07:00
Stephen O'Donnell
0b9e5ea592
HDFS-15372. Files in snapshots no longer see attribute provider permissions. Contributed by Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 730a39d138
)
2020-06-16 16:02:07 -07:00
Takanobu Asanuma
3b8418250f
HDFS-15403. NPE in FileIoProvider#transferToSocketFully. Contributed by hemanthboyina.
...
(cherry picked from commit f41a144077
)
2020-06-15 09:17:40 +09:00
Vinayakumar B
534b15caf9
HADOOP-17046. Support downstreams' existing Hadoop-rpc implementations using non-shaded protobuf classes ( #2026 )
2020-06-12 23:20:10 +05:30
Ayush Saxena
043628dcf1
HDFS-15398. EC: hdfs client hangs due to exception during addBlock. Contributed by Hongbing Wang.
2020-06-10 12:09:34 +05:30
Ayush Saxena
2d4faa39e8
HDFS-15211. EC: File write hangs during close in case of Exception during updatePipeline. Contributed by Ayush Saxena.
...
*Added missed test file.
2020-06-09 18:47:24 +05:30
Toshihiro Suzuki
ec8f3714e0
HDFS-15386. ReplicaNotFoundException keeps happening in DN after removing multiple DN's data directories ( #2052 )
...
Contributed by Toshihiro Suzuki.
(cherry picked from commit 545a0a147c
)
2020-06-05 11:17:13 +01:00
Wanqiang Ji
9b84a637b7
HADOOP-17055. Remove residual code of Ozone ( #2039 )
...
(cherry picked from commit d9838f2d42
)
2020-05-29 16:50:10 +09:00
Ayush Saxena
6b7040a1cb
HDFS-13183. Addendum: Standby NameNode process getBlocks request to reduce Active load. Contributed by Xiaoqiao He.
2020-05-28 13:04:22 +05:30
Ayush Saxena
718c8a5868
HDFS-15369. Refactor method VolumeScanner#runLoop(). Contributed by Yang Yun.
2020-05-24 18:06:19 +05:30
Takanobu Asanuma
c44d8b8bdf
HDFS-15363. BlockPlacementPolicyWithNodeGroup should validate if it is initialized by NetworkTopologyWithNodeGroup. Contributed by hemanthboyina.
...
(cherry picked from commit 4d22d1c58f
)
2020-05-23 17:33:39 +09:00
Chen Liang
e452163a06
HDFS-15293. Relax the condition for accepting a fsimage when receiving a checkpoint. Contributed by Chen Liang
...
(cherry picked from commit 7bb902bc0d
)
2020-05-19 09:52:25 +09:00
He Xiaoqiao
eb045ea056
HDFS-13183. Standby NameNode process getBlocks request to reduce Active load. Contributed by Xiaoqiao He.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit a3f44dacc1
)
2020-05-18 09:59:12 -07:00
Ayush Saxena
3915d1afc7
HDFS-15356. Unify configuration dfs.ha.allow.stale.reads
to DFSConfigKeys. Contributed by Xiaoqiao He.
...
(cherry picked from commit 178336f8a8
)
2020-05-18 09:58:59 -07:00
Wei-Chiu Chuang
240cba7e6d
HDFS-15202 Boost short circuit cache (rebase PR-1884) ( #2016 )
...
(cherry picked from commit 2abcf7762a
)
2020-05-18 09:41:11 -07:00
Wei-Chiu Chuang
289f8acc64
Revert "HDFS-15202 Boost short circuit cache (rebase PR-1884) ( #2016 )"
...
This reverts commit ad9a6a0ee3
.
2020-05-18 09:41:03 -07:00
Wei-Chiu Chuang
53d22fdb88
Revert "HDFS-13183. Standby NameNode process getBlocks request to reduce Active load. Contributed by Xiaoqiao He."
...
This reverts commit acae31aa28
.
2020-05-18 09:39:57 -07:00
Wei-Chiu Chuang
032ccba67c
HDFS-15207. VolumeScanner skip to scan blocks accessed during recent scan peroid. Contributed by Yang Yun.
...
(cherry picked from commit 50caba1a92
)
2020-05-18 08:42:20 -07:00
He Xiaoqiao
acae31aa28
HDFS-13183. Standby NameNode process getBlocks request to reduce Active load. Contributed by Xiaoqiao He.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit a3f44dacc1
)
2020-05-18 07:10:27 -07:00
pustota2009
ad9a6a0ee3
HDFS-15202 Boost short circuit cache (rebase PR-1884) ( #2016 )
...
Added parameter dfs.client.short.circuit.num improving HDFS-client's massive reading performance by create few instances ShortCircuit caches instead of one. It helps avoid locks and lets CPU do job.
(cherry picked from commit 86e6aa8eec
)
2020-05-18 07:05:03 -07:00
Surendra Singh Lilhore
7cef7109de
YARN-10265. Upgrade Netty-all dependency to latest version 4.1.50 to fix ARM support issue. Contributed by liusheng
2020-05-14 15:17:13 +05:30
Ayush Saxena
74887ab2ad
YARN-9898. Dependency netty-all-4.1.27.Final doesn't support ARM platform. Contributed by liusheng.
2020-05-14 00:37:09 +05:30
S O'Donnell
433aaeefa4
HDFS-15255. Consider StorageType when DatanodeManager#sortLocatedBlock(). Contributed by Lisheng Sun.
2020-05-12 15:25:05 +01:00
Ayush Saxena
ef41229f70
HDFS-14367. EC: Parameter maxPoolSize in striped reconstruct thread pool isn't affecting number of threads. Contributed by Guo Lei.
2020-05-12 18:43:37 +05:30
Ayush Saxena
61a4cd5539
HDFS-15338. listOpenFiles() should throw InvalidPathException in case of invalid paths. Contributed by Jinglun.
2020-05-11 16:52:23 +05:30
Ayush Saxena
817814105a
HDFS-15339. TestHDFSCLI fails for user names with the dot/dash character. Contributed by Yan Xiaole.
2020-05-08 00:57:06 +05:30
Uma Maheswara Rao G
edf52d29f1
HDFS-15305. Extend ViewFS and provide ViewFileSystemOverloadScheme implementation with scheme configurable. Contributed by Uma Maheswara Rao G.
...
(cherry picked from commit 9c8236d04d
)
2020-05-06 15:13:33 -07:00
Ayush Saxena
4e0d99c4d6
HDFS-14283. DFSInputStream to prefer cached replica. Contributed by Lisheng Sun.
2020-05-06 17:02:40 +05:30
Wei-Chiu Chuang
78c97907cb
HDFS-15334. INodeAttributeProvider's new API checkPermissionWithContext not getting called in for authorization. ( #1998 )
...
Reviewed-by: Arpit Agarwal <arp@apache.org>
(cherry picked from commit adecdb8b53
)
2020-05-05 21:25:56 -07:00
Konstantin V Shvachko
8809b43fb9
HDFS-15323. StandbyNode fails transition to active due to insufficient transaction tailing. Contributed by Konstantin V Shvachko.
...
(cherry picked from commit ebb878bab9
)
2020-05-04 10:45:18 -07:00
Ayush Saxena
df16146e7f
Erasure Coding: metrics xmitsInProgress become to negative. Contributed by maobaolong and Toshihiko Uchida.
2020-05-03 19:17:14 +05:30
Akira Ajisaka
e32e1384d9
HDFS-15320. StringIndexOutOfBoundsException in HostRestrictingAuthorizationFilter ( #1992 )
...
Signed-off-by: Mingliang Liu <liuml07@apache.org>
2020-05-02 13:59:58 -07:00
Akira Ajisaka
dfa7f160a5
Preparing for 3.3.1 development
2020-04-30 13:33:42 +09:00
S O'Donnell
a34174fea3
HDFS-15285. The same distance and load nodes don't shuffle when consider DataNode load. Contributed by Lisheng Sun.
...
(cherry picked from commit 9ca6298a9a
)
2020-04-29 16:01:48 +01:00
Akira Ajisaka
d504574d82
HDFS-15286. Concat on a same file deleting the file. Contributed by hemanthboyina.
...
(cherry picked from commit 5e0eda5d5f
)
2020-04-28 06:18:25 +09:00
Mingliang Liu
1db1992b31
HDFS-15297. TestNNHandlesBlockReportPerStorage::blockReport_02 fails intermittently. Contributed by Ayush Saxena
2020-04-25 15:44:01 -07:00
Dhiraj
1c19107ce8
HDFS-15281. Make sure ZKFC uses dfs.namenode.rpc-address to bind to host address ( #1964 )
...
Contributed by Dhiraj Hegde.
Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Inigo Goiri <inigoiri@apache.org>
2020-04-25 13:06:08 -07:00