Inigo Goiri
d8313b2274
HDFS-14850. Optimize FileSystemAccessService#getFileSystemConfiguration. Contributed by Lisheng Sun.
2019-09-28 17:20:44 -07:00
Ayush Saxena
2200871607
HDFS-14876. Remove unused imports from TestBlockMissingException.java and TestClose.java. Contributed by Lisheng Sun.
2019-09-28 22:03:53 +05:30
Ayush Saxena
c4c8d5fd0e
HDFS-14849. Erasure Coding: the internal block is replicated many times when datanode is decommissioning. Contributed by HuangTao.
2019-09-28 21:44:44 +05:30
Ayush Saxena
0d5d0b914a
Revert "HDFS-14849. Erasure Coding: the internal block is replicated many times when datanode is decommissioning. Contributed by HuangTao."
...
This reverts commit ce58c05f1d
.
2019-09-28 21:32:15 +05:30
Sahil Takiar
13b427fc05
HDFS-14564: Add libhdfs APIs for readFully; add readFully to ByteBufferPositionedReadable ( #963 ) Contributed by Sahil Takiar.
...
Reviewed-by: Siyao Meng <smeng@cloudera.com>
2019-09-27 12:25:03 -07:00
Ayush Saxena
ce58c05f1d
HDFS-14849. Erasure Coding: the internal block is replicated many times when datanode is decommissioning. Contributed by HuangTao.
2019-09-27 18:58:19 +05:30
Zhankun Tang
2870668cfa
Make upstream aware of 3.1.3 release.
2019-09-27 16:44:07 +08:00
Chen Liang
2adcc3c932
HDFS-14785. [SBN read] Change client logging to be less aggressive. Contributed by Chen Liang.
2019-09-26 15:48:19 -07:00
Inigo Goiri
b1e55cfb55
HDFS-14461. RBF: Fix intermittently failing kerberos related unit test. Contributed by Xiaoqiao He.
2019-09-26 15:26:23 -07:00
Ayush Saxena
1a2a352ecd
HDFS-11934. Add assertion to TestDefaultNameNodePort#testGetAddressFromConf. Contributed by Nikhil Navadiya.
2019-09-26 20:45:37 +05:30
Ayush Saxena
a3f68933d7
HDFS-14873. Fix dfsadmin doc for triggerBlockReport. Contributed by Fei Hui.
2019-09-26 19:54:07 +05:30
Takanobu Asanuma
bdaaa3bbf2
HDFS-14832. RBF: Add Icon for ReadOnly False. Contributed by hemanthboyina
2019-09-25 23:48:49 +09:00
Steve Loughran
e346e3638c
HADOOP-15691 Add PathCapabilities to FileSystem and FileContext.
...
Contributed by Steve Loughran.
This complements the StreamCapabilities Interface by allowing applications to probe for a specific path on a specific instance of a FileSystem client
to offer a specific capability.
This is intended to allow applications to determine
* Whether a method is implemented before calling it and dealing with UnsupportedOperationException.
* Whether a specific feature is believed to be available in the remote store.
As well as a common set of capabilities defined in CommonPathCapabilities,
file systems are free to add their own capabilities, prefixed with
fs. + schema + .
The plan is to identify and document more capabilities -and for file systems which add new features, for a declaration of the availability of the feature to always be available.
Note
* The remote store is not expected to be checked for the feature;
It is more a check of client API and the client's configuration/knowledge
of the state of the remote system.
* Permissions are not checked.
Change-Id: I80bfebe94f4a8bdad8f3ac055495735b824968f5
2019-09-25 12:16:41 +01:00
Prabhu Joseph
3f89084ac7
HDFS-14845. Ignore AuthenticationFilterInitializer for HttpFSServerWebServer and honor hadoop.http.authentication configs.
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-09-25 10:19:30 +09:00
Ayush Saxena
66400c1cbb
HDFS-14808. EC: Improper size values for corrupt ec block in LOG. Contributed by Ayush Saxena.
2019-09-25 01:31:15 +05:30
Ayush Saxena
eb96a3093e
HDFS-14655. [SBN Read] Namenode crashes if one of The JN is down. Contributed by Ayush Saxena.
2019-09-25 01:16:30 +05:30
Inigo Goiri
afa1006a53
HDFS-14843. Double Synchronization in BlockReportLeaseManager. Contributed by David Mollitor.
2019-09-24 09:58:42 -07:00
Inigo Goiri
816d3cb908
HDFS-14837. Review of Block.java. Contributed by David Mollitor.
2019-09-24 09:54:09 -07:00
Wei-Chiu Chuang
43203b466d
HDFS-14868. RBF: Fix typo in TestRouterQuota. Contributed by Jinglun.
...
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-09-24 09:38:58 -07:00
Ayush Saxena
c30e495557
HDFS-14853. NPE in DFSNetworkTopology#chooseRandomWithStorageType() when the excludedNode is not present. Contributed by Ranith Sardar.
2019-09-23 21:22:50 +05:30
Vinayakumar B
07c81e9bfc
HADOOP-16558. [COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes ( #1494 ). Contributed by Vinayakumar B.
2019-09-23 12:37:47 +05:30
Rohith Sharma K S
4c0a7a9e13
Make upstream aware of 3.2.1 release.
2019-09-23 06:20:54 +00:00
Rakesh Radhakrishnan
659c88801d
HDFS-14818. Check native pmdk lib by 'hadoop checknative' command. Contributed by Feilong He.
2019-09-22 22:02:54 +05:30
Ayush Saxena
aa938662f9
HDFS-14833. RBF: Router Update Doesn't Sync Quota. Contributed by Ayush Saxena.
2019-09-21 09:53:13 +05:30
Inigo Goiri
3f223bebfa
HDFS-14844. Make buffer of BlockReaderRemote#newBlockReader#BufferedOutputStream configurable. Contributed by Lisheng Sun.
2019-09-20 11:45:01 -07:00
Vinayakumar B
1654497f98
HADOOP-16557. [pb-upgrade] Upgrade protobuf.version to 3.7.1 ( #1432 )
...
HADOOP-16557. [pb-upgrade] Upgrade protobuf.version to 3.7.1. Contributed by Vinayakumar B.
2019-09-20 16:08:30 +05:30
Inigo Goiri
a79f286c6f
HDFS-14609. RBF: Security should use common AuthenticationFilter. Contributed by Chen Zhang.
2019-09-19 11:00:10 -07:00
Malcolm Taylor
56248f9d87
HADOOP-16556. Fix some alerts raised by LGTM.
...
Contributed by Malcolm Taylor.
Change-Id: Ic60c3f4681dd9d48b3afcba7520bd1e4d3cc4231
2019-09-19 16:00:05 +01:00
Chen Liang
4ed0aefe9f
HDFS-14822. [SBN read] Revisit GlobalStateIdContext locking when getting server state id. Contributed by Chen Liang.
2019-09-18 16:47:46 -07:00
Wei-Chiu Chuang
5dd859a8a0
HDFS-14836. FileIoProvider should not increase FileIoErrors metric in datanode volume metric. Contributed by Aiphago.
...
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
2019-09-18 09:34:01 -07:00
Sahil Takiar
3cf6e4272f
HDFS-14846: libhdfs tests are failing on trunk due to jni usage bugs
...
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-09-17 15:51:39 -07:00
Inigo Goiri
f580a87079
HDFS-14795. Add Throttler for writing block. Contributed by Lisheng Sun.
2019-09-17 14:55:34 -07:00
Ayush Saxena
e04b8a46c3
HDFS-14799. Do Not Call Map containsKey In Conjunction with get. Contributed by hemanthboyina.
2019-09-14 10:39:20 +05:30
Stephen O'Donnell
06ad6540b3
HDFS-14821. Make HDFS-14617 (fsimage sub-sections) off by default. Contributed by Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-09-13 10:36:45 -07:00
Surendra Singh Lilhore
4852a90e4b
HDFS-14754. Erasure Coding : The number of Under-Replicated Blocks never reduced. Contributed by hemanthboyina.
2019-09-13 00:17:13 +05:30
Surendra Singh Lilhore
2ff2a7f612
HDFS-14798. Synchronize invalidateBlocks in DatanodeDescriptor. Contributed by hemanthboyina.
2019-09-12 19:43:18 +05:30
Surendra Singh Lilhore
d1c303a497
HDFS-14699. Erasure Coding: Storage not considered in live replica when replication streams hard limit reached to threshold. Contributed by Zhao Yi Ming.
2019-09-12 19:11:50 +05:30
Akira Ajisaka
68612a0410
HDFS-14840. Use Java Conccurent Instead of Synchronization in BlockPoolTokenSecretManager. Contributed by David Mollitor.
2019-09-12 12:41:57 +09:00
Takanobu Asanuma
c255333e20
HDFS-14838. RBF: Display RPC (instead of HTTP) Port Number in RBF web UI. Contributed by Xieming Li
2019-09-11 16:54:08 +09:00
Takanobu Asanuma
524b553a5f
HDFS-14835. RBF: Secured Router should not run when it can't initialize DelegationTokenSecretManager. ( #1414 )
2019-09-11 10:32:07 +09:00
Erik Krogen
c92a3e94d8
HADOOP-15565. Add an inner FS cache to ViewFileSystem, separate from the global cache, to avoid file system leaks. Contributed by Jinglun.
2019-09-06 10:22:28 -07:00
Surendra Singh Lilhore
d98c54816d
HDFS-13913. LazyPersistFileScrubber.run() should log meaningful warn message. Contributed by Daniel Green.
2019-09-06 18:12:42 +05:30
Surendra Singh Lilhore
fa7f03fc56
HDFS-14826. dfs.ha.zkfc.port property duplicated in hdfs-default.xml. Contributed by Renukaprasad C.
2019-09-06 17:55:30 +05:30
Inigo Goiri
494d75eb2b
HDFS-14784. Add more methods to WebHdfsTestUtil to support tests outside of package. Contributed by Chen Zhang.
2019-09-05 21:15:17 -07:00
Inigo Goiri
72d8b92ba5
HDFS-12904. Add DataTransferThrottler to the Datanode transfers. Contributed by Lisheng Sun.
2019-09-05 11:44:02 -07:00
Ayush Saxena
2f70b52a5b
HDFS-14812. RBF: MountTableRefresherService should load cache when refresh. Contributed by xuzq.
2019-09-05 08:29:56 +05:30
Surendra Singh Lilhore
05704754a0
HDFS-14777. RBF: Set ReadOnly is failing for mount Table but actually readonly succeed to set. Contributed by Ranith Sardar.
2019-09-04 15:09:09 +05:30
Xiaoyu Yao
3c117163a3
HDFS-14633. The StorageType quota and consume in QuotaFeature is not handled for rename. Contributed by Jinglun.
2019-09-03 09:29:58 -07:00
Ayush Saxena
91b01a1db7
HDFS-14807. SetTimes updates all negative values apart from -1. Contributed by Ayush Saxena.
2019-09-03 18:08:42 +05:30
Surendra Singh Lilhore
5ff76cb8bc
HDFS-14630. Configuration.getTimeDurationHelper() should not log time unit warning in info log. Contributed by hemanthboyina.
2019-09-03 12:37:09 +05:30