big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
24,356 Commits 2 Branches 2 Tags 264 MiB
4331c88352
Commit Graph

969 Commits

Author SHA1 Message Date
LeonGao
9a9ab5b48e
HDFS-15548. Allow configuring DISK/ARCHIVE storage types on same device mount (#2288). Contributed by Leon Gao. 2020-11-09 15:06:16 -08:00
Takanobu Asanuma
30f06e0c74
HDFS-15639. [JDK 11] Fix Javadoc errors in hadoop-hdfs-client. (#2394) 2020-10-20 19:08:59 +09:00
Ayush Saxena
1e3a6efcef
HADOOP-17288. Use shaded guava from thirdparty. (#2342). Contributed by Ayush Saxena. 2020-10-17 12:01:18 +05:30
Ayush Saxena
cc57eebe45
HDFS-15633. Avoid redundant RPC calls for getDiskStatus. (#2386). Contributed by Ayush Saxena. 2020-10-16 10:36:24 +05:30
Konstantin V Shvachko
b3786d6c3c HDFS-15567. [SBN Read] HDFS should expose msync() API to allow downstream applications call it explicitly. Contributed by Konstantin V Shvachko. 2020-10-12 17:26:49 -07:00
bshashikant
16aea11c94
HDFS-15611. Add list Snapshot command in WebHDFS. (#2355) 2020-10-07 10:34:32 +05:30
Doroszlai, Attila
1cfe5916e2 HADOOP-16990. Update Mockserver. Contributed by Attila Doroszlai. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2020-10-06 14:58:42 -07:00
Siyao Meng
43b0c0b054
HDFS-15607. Create trash dir when allowing snapshottable dir (#2352) 2020-10-05 05:02:00 -07:00
Vinayakumar B
82b86e3754
HDFS-15098. Add SM4 encryption method for HDFS. Contributed by liusheng 2020-09-27 19:27:13 +05:30
Uma Maheswara Rao G
899dea2a21
HDFS-15598: ViewHDFS#canonicalizeUri should not be restricted to DFS only API. (#2339). Contributed by Uma Maheswara Rao G. 
Co-authored-by: Uma Maheswara Rao G <umagangumalla@cloudera.com>
 2020-09-25 21:21:01 -07:00
Uma Maheswara Rao G
3ccc962b99
HDFS-15596: ViewHDFS#create(f, permission, cflags, bufferSize, replication, blockSize, progress, checksumOpt) should not be restricted to DFS only. (#2333). Contributed by Uma Maheswara Rao G. 
Co-authored-by: Uma Maheswara Rao G <umagangumalla@cloudera.com>
 2020-09-24 07:07:48 -07:00
huangtianhua
ff59fbb8b0
HDFS-15025. Applying NVDIMM storage media to HDFS (#2189) 
* HDFS-15025. Applying NVDIMM storage media to HDFS
Co-authored-by: YaYun-Wang <yywangyayun@163.com>
Co-authored-by: YaYun-Wang <34060507+YaYun-Wang@users.noreply.github.com>
 2020-09-24 14:27:04 +05:30
Uma Maheswara Rao G
7bba4c609c
HDFS-15585: ViewDFS#getDelegationToken should not throw UnsupportedOperationException. (#2312). Contributed by Uma Maheswara Rao G. 2020-09-18 15:18:10 +05:30
Takanobu Asanuma
5c5b2ed7c7 HDFS-15576. Erasure Coding: Add rs and rs-legacy codec test for addPolicies. Contributed by Fei Hui. 2020-09-16 01:36:46 +09:00
Siyao Meng
6d3312de47
HDFS-15539. When disallowing snapshot on a dir, throw exception if its trash root is not empty (#2258) 2020-09-14 13:31:34 -07:00
imbajin
89428f142f
HDFS-15551. Tiny Improve for DeadNode detector (#2265) 
Contributed by imbajin.

Reviewed-by: leosunli <lisheng.sun08@gmail.com>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2020-09-11 15:44:03 +08:00
1996fanrui
56ebabd426
HADOOP-17222. Create socket address leveraging URI cache (#2241) 
Contributed by fanrui.

Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2020-09-10 22:30:52 -07:00
Siyao Meng
43c52d6495
HDFS-15563. Incorrect getTrashRoot return value when a non-snapshottable dir prefix matches the path of a snapshottable dir (#2295) 
Change-Id: Iefcbc14ca939b61f0c1befc9a94d686be1684ddc
 2020-09-10 15:55:23 +05:30
He Xiaoqiao
1d6d0d8207 HDFS-14694. Call recoverLease on DFSOutputStream close exception. Contributed by Lisheng Sun. 
Co-authored-by: Chen Zhang <chzhang1987@gmail.com>

Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
 2020-09-09 20:09:13 +08:00
Uma Maheswara Rao G
ac7d4623ae
HDFS-15558: ViewDistributedFileSystem#recoverLease should call super.recoverLease when there are no mounts configured (#2275) Contributed by Uma Maheswara Rao G. 2020-09-07 11:36:13 -07:00
Uma Maheswara Rao G
dd013f2fdf
HDFS-15533: Provide DFS API compatible class, but use ViewFileSystemOverloadScheme inside. (#2229). Contributed by Uma Maheswara Rao G. 2020-08-19 09:30:41 -07:00
Vivek Ratnavel Subramanian
cb50e3fcf7
HDFS-15496. Add UI for deleted snapshots (#2212) 2020-08-13 22:36:15 +05:30
Siyao Meng
3fd3aeb621
HDFS-15492. Make trash root inside each snapshottable directory (#2176) 2020-08-11 08:52:16 -07:00
Akira Ajisaka
32895f4f7e
HDFS-15507. [JDK 11] Fix javadoc errors in hadoop-hdfs-client module. Contributed by Xieming Li. 2020-08-11 13:49:56 +09:00
Takanobu Asanuma
1d5ccc790b
HDFS-15512. Remove smallBufferSize in DFSClient. (#2191) 2020-08-06 22:47:12 +05:30
Akira Ajisaka
c40cbc57fa
HADOOP-17091. [JDK11] Fix Javadoc errors (#2098) 2020-08-03 10:46:51 +09:00
bshashikant
d8a2df25ad
HDFS-15498. Show snapshots deletion status in snapList cmd. (#2181) 2020-08-02 21:59:56 +05:30
bshashikant
68287371cc
HDFS-15488. Add a command to list all snapshots for a snaphottable root with snapshot Ids. (#2166) 2020-07-29 21:33:25 +05:30
Ayush Saxena
6bcb24d269 HADOOP-17100. Replace Guava Supplier with Java8+ Supplier in Hadoop. Contributed by Ahmed Hussein. 2020-07-18 14:33:43 +05:30
Steve Loughran
b9fa5e0182
HDFS-13934. Multipart uploaders to be created through FileSystem/FileContext. 
Contributed by Steve Loughran.

Change-Id: Iebd34140c1a0aa71f44a3f4d0fee85f6bdf123a3
 2020-07-13 13:30:02 +01:00
He Xiaoqiao
4f26454a7d HDFS-15425. Review Logging of DFSClient. Contributed by Hongbing Wang. 2020-07-08 00:02:39 +08:00
Yiqun Lin
9cbd76cc77 HDFS-15346. FedBalance tool implementation. Contributed by Jinglun. 2020-06-18 13:33:25 +08:00
Eric Yang
81d8a887b0
SPNEGO TLS verification 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2020-06-15 10:56:16 +09:00
Vinayakumar B
e154084770
HADOOP-17046. Support downstreams' existing Hadoop-rpc implementations using non-shaded protobuf classes (#2026) 2020-06-12 23:16:33 +05:30
Uma Maheswara Rao G
93b121a971
HADOOP-17060. Clarify listStatus and getFileStatus behaviors inconsistent in the case of ViewFs implementation for isDirectory. Contributed by Uma Maheswara Rao G. 2020-06-10 15:00:02 -07:00
Ayush Saxena
b735a77717 HDFS-15398. EC: hdfs client hangs due to exception during addBlock. Contributed by Hongbing Wang. 2020-06-10 12:06:16 +05:30
Ayush Saxena
f4901d0778 HDFS-15355. Make the default block storage policy ID configurable. Contributed by Yang Yun. 2020-05-24 17:59:16 +05:30
Ayush Saxena
e0ae232f66 HDFS-15093. RENAME.TO_TRASH is ignored When RENAME.OVERWRITE is specified. Contributed by Ayush Saxena. 2020-05-23 18:38:32 +05:30
leosunli
be374faf42
HDFS-13639. SlotReleaser is not fast enough (#1885) 2020-05-21 13:21:17 -07:00
Wei-Chiu Chuang
2abcf7762a HDFS-15202 Boost short circuit cache (rebase PR-1884) (#2016) 2020-05-18 09:23:09 -07:00
Wei-Chiu Chuang
4525292d41 Revert "HDFS-15202 Boost short circuit cache (rebase PR-1884) (#2016)" 
This reverts commit 86e6aa8eec.
 2020-05-18 09:22:05 -07:00
pustota2009
86e6aa8eec
HDFS-15202 Boost short circuit cache (rebase PR-1884) (#2016) 
Added parameter dfs.client.short.circuit.num improving HDFS-client's massive reading performance by create few instances ShortCircuit caches instead of one. It helps avoid locks and lets CPU do job.
 2020-05-18 07:04:04 -07:00
Surendra Singh Lilhore
1958cb7c2b YARN-10265. Upgrade Netty-all dependency to latest version 4.1.50 to fix ARM support issue. Contributed by liusheng 2020-05-14 15:09:34 +05:30
Ayush Saxena
0918433b4d YARN-9898. Dependency netty-all-4.1.27.Final doesn't support ARM platform. Contributed by liusheng. 2020-05-14 00:36:20 +05:30
Akira Ajisaka
8ffc356b1e
Revert "SPNEGO TLS verification" 
This reverts commit ba66f3b454.
 2020-05-13 17:14:14 +09:00
S O'Donnell
29dddb8a14 HDFS-15255. Consider StorageType when DatanodeManager#sortLocatedBlock(). Contributed by Lisheng Sun. 2020-05-12 15:07:51 +01:00
Takanobu Asanuma
928b81a533
HDFS-15350. Set dfs.client.failover.random.order to true as default. (#2008) 2020-05-12 09:04:03 -05:00
Ayush Saxena
4c53fb9ce1 HDFS-15338. listOpenFiles() should throw InvalidPathException in case of invalid paths. Contributed by Jinglun. 2020-05-11 16:48:34 +05:30
Ayush Saxena
aab9e0b16e HDFS-15250. Setting dfs.client.use.datanode.hostname to true can crash the system because of unhandled UnresolvedAddressException. Contributed by Ctest. 2020-05-10 11:43:30 +05:30
Ayush Saxena
7fddf4855e HDFS-14283. DFSInputStream to prefer cached replica. Contributed by Lisheng Sun. 2020-05-06 16:55:04 +05:30