big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
22,019 Commits 2 Branches 2 Tags 264 MiB
5740eea081
Commit Graph

8197 Commits

Author SHA1 Message Date
Inigo Goiri
5740eea081 HDFS-14513. FSImage which is saving should be clean while NameNode shutdown. Contributed by He Xiaoqiao. 2019-06-11 11:48:35 -07:00
Anu Engineer
101d5b5f86 HDFS-14234. Limit WebHDFS to specifc user, host, directory triples. 
Contributed by Clay B.
 2019-06-10 17:55:16 -07:00
He Xiaoqiao
bd46bdf9f9 HDFS-14553. Make queue size of BlockReportProcessingThread configurable. Contributed by He Xiaoqiao. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-06-10 17:21:52 -07:00
Wei-Chiu Chuang
0d160a0ba8 HDFS-10210. Remove the defunct startKdc profile from hdfs. Contributed by Wei-Chiu Chuang, Akira Ajisaka. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Co-authored-by: Akira Ajisaka <aajisaka@apache.org>
 2019-06-10 14:13:53 -07:00
Hanisha Koneru
7217494f40 HDFS-10659. Namenode crashes after Journalnode re-installation in an HA cluster due to missing paxos directory. Contributed by star, Hanisha Koneru. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-06-10 13:45:37 -07:00
Shweta Yakkali
d6d95d2686 HDFS-14494. Move Server logging of StatedId inside receiveRequestState(). Contributed by Shweta Yakkali. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-06-10 10:23:18 -07:00
Inigo Goiri
e1dfc060f8 HDFS-14486. The exception classes in some throw statements do not accurately describe why they are thrown. Contributed by Ayush Saxena. 2019-06-06 11:59:53 -07:00
Inigo Goiri
944adc61b1 HDFS-14527. Stop all DataNodes may result in NN terminate. Contributed by He Xiaoqiao. 2019-06-06 10:20:28 -07:00
Eric Yang
294695dd57 HADOOP-16314. Make sure all web end points are covered by the same authentication filter. 
Contributed by Prabhu Joseph
 2019-06-05 18:55:13 -04:00
Sammi Chen
d1aad44490 HDFS-14356. Implement HDFS cache on SCM with native PMDK libs. Contributed by Feilong He. 2019-06-05 21:33:00 +08:00
huzheng
ea3b0a1844 HDFS-14535. The default 8KB buffer in requestFileDescriptors#BufferedOutputStream is causing lots of heap allocation in HBase when using short-circut read 2019-06-04 08:48:05 -07:00
Takanobu Asanuma
35f1014b3e HDFS-13654. Use a random secret when a secret file doesn't exist in HttpFS. This should be default. 2019-05-31 10:29:24 +09:00
Kihwal Lee
9122b9b649 HDFS-14521. Suppress setReplication logging. Contributed by Kihwal Lee 2019-05-30 17:28:45 -05:00
He Xiaoqiao
33c62f8f4e HDFS-14497. Write lock held by metasave impact following RPC processing. Contributed by He Xiaoqiao. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-05-30 13:30:35 -07:00
Shweta Yakkali
6f5a36c13c HADOOP-13656. fs -expunge to take a filesystem. Contributed by Shweta. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-05-30 13:21:58 -07:00
Ayush Saxena
c1caab40f2 HDFS-14512. ONE_SSD policy will be violated while write data with DistributedFileSystem.create(....favoredNodes). Contributed by Ayush Saxena. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-05-29 20:56:38 -07:00
Akira Ajisaka
afd844059c HADOOP-16331. Fix ASF License check in pom.xml 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2019-05-29 17:25:13 +09:00
Eric Yang
d78854b928 HDFS-14434. Ignore user.name query parameter in secure WebHDFS. 
Contributed by KWON BYUNGCHANG
 2019-05-28 17:31:35 -04:00
Akira Ajisaka
9f933e6446
HADOOP-16323. https everywhere in Maven settings. 2019-05-27 15:24:59 +09:00
Rakesh Radhakrishnan
37900c5639 HDFS-14402. Use FileChannel.transferTo() method for transferring block to SCM cache. Contributed by Feilong He. 2019-05-26 14:30:11 +05:30
Erik Krogen
55e0c134f0 HDFS-14500. NameNode StartupProgress should not allow new steps in an already-completed phase. Contributed by Erik Krogen. 2019-05-24 13:12:59 -07:00
Konstantin V Shvachko
c31b7b8d52 HDFS-14502. keepResults option in NNThroughputBenchmark should call saveNamespace(). Contributed by Konstantin V Shvachko. 2019-05-24 12:34:29 -07:00
Inigo Goiri
c9393dd17f Revert "HDFS-14353. Erasure Coding: metrics xmitsInProgress become to negative. Contributed by maobaolong." 
This reverts commit 1cb2eb0df3.
 2019-05-24 10:23:47 -07:00
Christopher Gregorian
f96a2df38d HADOOP-16266. Add more fine-grained processing time metrics to the RPC layer. Contributed by Christopher Gregorian. 2019-05-23 10:28:37 -07:00
Siyao Meng
9dff6eff81 HDFS-14507. Document -blockingDecommission option for hdfs dfsadmin -listOpenFiles 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2019-05-22 13:22:26 +09:00
Inigo Goiri
1cb2eb0df3 HDFS-14353. Erasure Coding: metrics xmitsInProgress become to negative. Contributed by maobaolong. 2019-05-20 17:22:58 -07:00
Srinivasu Majeti
3e5e5b028a HDFS-14323. Distcp fails in Hadoop 3.x when 2.x source webhdfs url has special characters in hdfs file path. Contributed by Srinivasu Majeti. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-05-17 19:20:28 +02:00
Virajith Jalaparti
77170e70d1 HDFS-14390. Provide kerberos support for AliasMap service used by Provided storage. Contributed by Ashvin Agrawal 2019-05-15 12:15:47 -07:00
Sahil Takiar
e2dfdaee7c HDFS-14482: Crash when using libhdfs with bad classpath 2019-05-14 11:04:06 -07:00
Inigo Goiri
389e640f0c HADOOP-16161. NetworkTopology#getWeightUsingNetworkLocation return unexpected result. Contributed by He Xiaoqiao. 2019-05-13 11:46:16 -07:00
Erik Krogen
5847e00143 HDFS-14245. [SBN read] Enable ObserverReadProxyProvider to work with non-ClientProtocol proxy types. Contributed by Erik Krogen. 2019-05-13 08:39:59 -07:00
Rakesh Radhakrishnan
9b0aace1e6 HDFS-14401. Refine the implementation for HDFS cache on SCM. Contributed by Feilong He. 2019-05-08 17:20:21 +05:30
Surendra Singh Lilhore
69b903bbd8 HDFS-14372. NPE while DN is shutting down. Contributed by lujie. 2019-05-05 16:33:52 +05:30
Surendra Singh Lilhore
e424392a62 HDFS-14438. Fix typo in OfflineEditsVisitorFactory. Contributed by bianqi. 2019-05-05 13:38:09 +05:30
Chen Liang
1d59cc490c HADOOP-16292. Refactor checkTrustAndSend in SaslDataTransferClient to make it cleaner. Contributed by Sherwood Zheng. 2019-05-03 10:14:17 -07:00
Vinayakumar B
f1875b205e HADOOP-16059. Use SASL Factories Cache to Improve Performance. Contributed by Ayush Saxena. 2019-05-03 11:22:14 +05:30
Shweta
d6b7609c96 HDFS-14453. Improve Bad Sequence Number Error Message. Contributed by Shweta. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-05-02 14:25:02 -07:00
Giovanni Matteo Fumarola
7a3188d054 HADOOP-16282. Avoid FileStream to improve performance. Contributed by Ayush Saxena. 2019-05-02 12:58:42 -07:00
Inigo Goiri
865c328930 HDFS-14460. DFSUtil#getNamenodeWebAddr should return HTTPS address based on policy configured. Contributed by CR Hota. 2019-05-02 10:09:21 -07:00
Siyao Meng
7cb46f035a HDFS-14463. Add Log Level link under NameNode and DataNode Web UI Utilities dropdown. Contributed by Siyao Meng. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-05-01 16:48:44 -07:00
Sahil Takiar
4877f0aa51 HDFS-3246: pRead equivalent for direct read path (#597) 
HDFS-3246: pRead equivalent for direct read path

Contributed by Sahil Takiar
 2019-04-30 14:52:16 -07:00
Arpit Agarwal
4b4200f1f8
HDFS-13677. Dynamic refresh Disk configuration results in overwriting VolumeMap. Contributed by xuzq. 2019-04-29 14:49:35 -07:00
Sean Mackrory
a703dae25e HADOOP-16222. Fix new deprecations after guava 27.0 update in trunk. Contributed by Gabor Bota. 2019-04-24 10:39:00 -06:00
Erik Krogen
174b7d3126 HDFS-14435. [SBN Read] Enable ObserverReadProxyProvider to gracefully handle StandbyException when fetching HAServiceState. Contributed by Erik Krogen. 2019-04-22 13:29:56 -07:00
Inigo Goiri
5321235fe8 HDFS-14445. TestTrySendErrorReportWhenNNThrowsIOException fails in trunk. Contributed by Ayush Saxena. 2019-04-22 13:23:42 -07:00
lys0716
685cb83e4c HDFS-14433. Remove the extra empty space in the DataStreamer logging. Contributed by Yishuang Lu. (#747) 2019-04-17 10:38:48 -07:00
Puleya7
13907d8479 HDFS-14432. dfs.datanode.shared.file.descriptor.paths duplicated in hdfs-default.xml 
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
 2019-04-17 21:39:06 +09:00
Inigo Goiri
be6c8014e6 HDFS-14418. Remove redundant super user priveledge checks from namenode. Contributed by Ayush Saxena. 2019-04-16 10:34:31 -07:00
Sahil Takiar
2382f63fc0
HADOOP-14747. S3AInputStream to implement CanUnbuffer. 
Author:    Sahil Takiar <stakiar@cloudera.com>
 2019-04-12 18:12:02 -07:00
Chen Liang
626fec652b HDFS-13699. Add DFSClient sending handshake token to DataNode, and allow DataNode overwrite downstream QOP. Contributed by Chen Liang. 2019-04-12 17:37:51 -07:00