big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
22,918 Commits 2 Branches 2 Tags 264 MiB
3f223bebfa
Commit Graph

496 Commits

Author SHA1 Message Date
Inigo Goiri
3f223bebfa HDFS-14844. Make buffer of BlockReaderRemote#newBlockReader#BufferedOutputStream configurable. Contributed by Lisheng Sun. 2019-09-20 11:45:01 -07:00
Inigo Goiri
f580a87079 HDFS-14795. Add Throttler for writing block. Contributed by Lisheng Sun. 2019-09-17 14:55:34 -07:00
Stephen O'Donnell
06ad6540b3 HDFS-14821. Make HDFS-14617 (fsimage sub-sections) off by default. Contributed by Stephen O'Donnell. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-09-13 10:36:45 -07:00
Surendra Singh Lilhore
fa7f03fc56 HDFS-14826. dfs.ha.zkfc.port property duplicated in hdfs-default.xml. Contributed by Renukaprasad C. 2019-09-06 17:55:30 +05:30
Inigo Goiri
72d8b92ba5 HDFS-12904. Add DataTransferThrottler to the Datanode transfers. Contributed by Lisheng Sun. 2019-09-05 11:44:02 -07:00
Surendra Singh Lilhore
5ff76cb8bc HDFS-14630. Configuration.getTimeDurationHelper() should not log time unit warning in info log. Contributed by hemanthboyina. 2019-09-03 12:37:09 +05:30
leosunli
915271245b Make DataNodePeerMetrics#minOutlierDetectionSamples configurable (#1314). Contributed by Lisheng Sun. 
Signed-off-by: sunlisheng <sunlisheng@xiaomi.com>
 2019-08-29 18:35:26 -07:00
Arpit Agarwal
07e3cf952e
HDFS-2470. NN should automatically set permissions on dfs.namenode.*.dir. Contributed by Siddharth Wagle. 2019-08-26 15:57:55 -07:00
Stephen O'Donnell
b67812ea21 HDFS-14617. Improve fsimage load time by writing sub-sections to the fsimage index (#1028). Contributed by Stephen O'Donnell. 
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2019-08-22 17:09:57 -07:00
Stephen O'Donnell
93daf69f90 HDFS-14675. Increase Balancer Defaults Further. Contributed by Stephen O'Donnell. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Gabor Bota <gabota@apache.org>
 2019-08-22 15:21:33 -07:00
Wei-Chiu Chuang
d873ddd656 HDFS-14127. Add a description about the observer read configuration. Contributed by xiangheng. 2019-08-17 04:39:22 -07:00
Ayush Saxena
e9b6b81de4 HDFS-13505. Turn on HDFS ACLs by default. Contributed by Siyao Meng. 2019-08-13 19:17:10 +05:30
Erik Krogen
827dbb11e2 HDFS-14370. Add exponential backoff to the edit log tailer to avoid spinning on empty edit tail requests. Contributed by Erik Krogen. 2019-08-07 09:25:58 -07:00
Erik Krogen
1f26cc8705 HDFS-13783. Add an option to the Balancer to make it run as a long-running service. Contributed by Chen Zhang. 2019-07-30 15:42:55 -07:00
Inigo Goiri
3ab77d9bc9 HDFS-14201. Ability to disallow safemode NN to become active. Contributed by Xiao Liang and He Xiaoqiao. 2019-06-18 09:58:29 -07:00
Weiwei Yang
e3172221ea HDFS-12770. Add doc about how to disable client socket cache. Contributed by Weiwei Yang. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-06-17 21:48:16 -07:00
Anu Engineer
101d5b5f86 HDFS-14234. Limit WebHDFS to specifc user, host, directory triples. 
Contributed by Clay B.
 2019-06-10 17:55:16 -07:00
He Xiaoqiao
bd46bdf9f9 HDFS-14553. Make queue size of BlockReportProcessingThread configurable. Contributed by He Xiaoqiao. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-06-10 17:21:52 -07:00
Rakesh Radhakrishnan
9b0aace1e6 HDFS-14401. Refine the implementation for HDFS cache on SCM. Contributed by Feilong He. 2019-05-08 17:20:21 +05:30
Puleya7
13907d8479 HDFS-14432. dfs.datanode.shared.file.descriptor.paths duplicated in hdfs-default.xml 
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
 2019-04-17 21:39:06 +09:00
Chen Liang
626fec652b HDFS-13699. Add DFSClient sending handshake token to DataNode, and allow DataNode overwrite downstream QOP. Contributed by Chen Liang. 2019-04-12 17:37:51 -07:00
Inigo Goiri
7b5b783f66 HDFS-14327. Using FQDN instead of IP to access servers with DNS resolving. Contributed by Fengnan Li. 2019-04-03 16:11:13 -07:00
Uma Maheswara Rao G
35ff31dd94 HDFS-14355 : Implement HDFS cache on SCM by using pure java mapped byte buffer. Contributed by Feilong He. 2019-03-30 23:33:25 -07:00
Erik Krogen
1bc282e0b3 HDFS-14317. Ensure checkpoints are created when in-progress edit log tailing is enabled with a period shorter than the log roll period. Contributed by Ekanth Sethuramalingam. 2019-03-07 08:41:23 -08:00
Yongjun Zhang
f7a27cdee4 HDFS-14118. Support using DNS to resolve nameservices to IP addresses. Contributed by Fengnan Li. 2019-02-23 09:35:36 -08:00
Chen Liang
024c87291c HDFS-13617. Allow wrapping NN QOP into token in encrypted message. Contributed by Chen Liang 2019-02-13 12:40:31 -08:00
Erik Krogen
00e99c6594 HDFS-13609. [SBN read] Edit Tail Fast Path Part 3: NameNode-side changes to support tailing edits via RPC. Contributed by Erik Krogen. 2018-12-24 09:33:59 -08:00
Erik Krogen
c81ac2ff02 HDFS-13607. [SBN read] Edit Tail Fast Path Part 1: Enhance JournalNode with an in-memory cache of recent edit transactions. Contributed by Erik Krogen. 2018-12-24 09:33:59 -08:00
Vinayakumar B
c03024a530 HDFS-14113. EC : Add Configuration to restrict UserDefined Policies. Contributed by Ayush Saxena. 2018-12-06 18:20:28 +05:30
Tsz Wo Nicholas Sze
0081b02e35 HDFS-14112. Avoid recursive call to external authorizer for getContentSummary. 2018-11-29 13:55:21 -08:00
Vinayakumar B
68d5dfdc78 HDFS-14056. Fix error messages in HDFS-12716. Contributed by Ayush Saxena. 2018-11-16 18:05:10 +05:30
Chen Liang
635786a511 HDFS-13566. Add configurable additional RPC listener to NameNode. Contributed by Chen Liang. 2018-10-23 14:53:45 -07:00
Inigo Goiri
88cce32551 HDFS-9872. HDFS bytes-default configurations should accept multiple size units. Contributed by Yiqun Lin. 2018-10-19 10:38:04 -07:00
Kitti Nanasi
10185d9a77 HDFS-13882. Set a maximum delay for retrying locateFollowingBlock. Contributed by Kitti Nanasi. 
Signed-off-by: Xiao Chen <xiao@apache.org>
 2018-10-10 08:55:32 -07:00
Nanda kumar
b39b802422 HDFS-13949. Correct the description of dfs.datanode.disk.check.timeout in hdfs-default.xml. Contributed by Toshihiro Suzuki. 2018-10-10 12:29:07 +05:30
Yiqun Lin
5689355783 HDFS-13768. Adding replicas to volume map makes DataNode start slowly. Contributed by Surendra Singh Lilhore. 2018-10-02 09:43:14 +08:00
Yiqun Lin
0da49642fc HDFS-13884. Improve the description of the setting dfs.image.compress. Contributed by Ryan Wu. 2018-09-10 13:57:36 +08:00
Hrishikesh Gadre
335a8139f5 HDFS-13820. Add an ability to disable CacheReplicationMonitor. Contributed by Hrishikesh Gadre. 
Signed-off-by: Xiao Chen <xiao@apache.org>
 2018-09-07 14:59:06 -07:00
Yiqun Lin
b9b964d253 HDFS-13831. Make block increment deletion number configurable. Contributed by Ryan Wu. 2018-08-27 14:55:46 +08:00
Uma Maheswara Rao G
39ed3a66db HDFS-13076: [SPS]: Cleanup work for HDFS-10285 merge. Contributed by Rakesh R. 2018-08-12 03:06:06 -07:00
Rakesh Radhakrishnan
2acc50b826 HDFS-13165: [SPS]: Collects successfully moved block details via IBR. Contributed by Rakesh R. 2018-08-12 03:06:05 -07:00
Surendra Singh Lilhore
75ccc1396b HDFS-13166: [SPS]: Implement caching mechanism to keep LIVE datanodes to minimize costly getLiveDatanodeStorageReport() calls. Contributed by Rakesh R. 2018-08-12 03:06:05 -07:00
Uma Maheswara Rao G
3b83110d5e HDFS-13057: [SPS]: Revisit configurations to make SPS service modes internal/external/none. Contributed by Rakesh R. 2018-08-12 03:06:04 -07:00
Surendra Singh Lilhore
c561cb316e HDFS-12106: [SPS]: Improve storage policy satisfier configurations. Contributed by Surendra Singh Lilhore. 2018-08-12 03:06:02 -07:00
Uma Maheswara Rao G
00eceed233 HDFS-12570: [SPS]: Refactor Co-ordinator datanode logic to track the block storage movements. Contributed by Rakesh R. 2018-08-12 03:06:01 -07:00
Uma Maheswara Rao G
bfd3f8bd8a HDFS-12291: [SPS]: Provide a mechanism to recursively iterate and satisfy storage policy of all the files under the given dir. Contributed by Surendra Singh Lilhore. 2018-08-12 03:06:01 -07:00
Uma Maheswara Rao G
0e820f16af HDFS-12214: [SPS]: Fix review comments of StoragePolicySatisfier feature. Contributed by Rakesh R. 2018-08-12 03:06:01 -07:00
Rakesh Radhakrishnan
422f870607 HDFS-11289. [SPS]: Make SPS movement monitor timeouts configurable. Contributed by Uma Maheswara Rao G 2018-08-12 03:05:56 -07:00
Rakesh Radhakrishnan
5179d99b7e HDFS-11123. [SPS] Make storage policy satisfier daemon work on/off dynamically. Contributed by Uma Maheswara Rao G 2018-08-12 03:05:55 -07:00
Rakesh Radhakrishnan
cd5262aba0 HDFS-10885. [SPS]: Mover tool should not be allowed to run when Storage Policy Satisfier is on. Contributed by Wei Zhou 2018-08-12 03:05:55 -07:00