big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
23,189 Commits 2 Branches 2 Tags 264 MiB
ceb9c6175e
Commit Graph

23189 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Abhishek Modi
4d3c580b03 YARN-9859. Refactoring of OpportunisticContainerAllocator. Contributed by Abhishek Modi. 2019-09-30 23:40:15 +05:30
bshashikant
98ca07ebed HDDS-2207. Update Ratis to latest snapshot. Contributed by Shashikant Bannerjee. (#1550) 2019-09-30 20:36:27 +05:30
dchitlangia
b46d82339f
HDDS-2202. Remove unused import in OmUtils 
Closes #1543
 2019-09-30 16:02:30 +02:00
Shashikant Banerjee
a530ac3f50
HDDS-2153. Add a config to tune max pending requests in Ratis leader 
Closes #1474
 2019-09-30 15:42:04 +02:00
cxorm
d6b0a8da77
HDDS-2183. Container and pipline subcommands of scmcli should be grouped 
Closes #1532
 2019-09-30 14:38:01 +02:00
Konstantin V Shvachko
760b523e58 Revert "HDFS-14305. Fix serial number calculation in BlockTokenSecretManager to avoid token key ID overlap between NameNodes. Contributed by He Xiaoqiao." 
This reverts commit 0feba4396f.
 2019-09-29 13:11:31 -07:00
Sneha Vijayarajan
c0edc848a8
HADOOP-16548 : Disable Flush() over config 2019-09-28 20:39:42 -07:00
Inigo Goiri
d8313b2274 HDFS-14850. Optimize FileSystemAccessService#getFileSystemConfiguration. Contributed by Lisheng Sun. 2019-09-28 17:20:44 -07:00
Ayush Saxena
2200871607 HDFS-14876. Remove unused imports from TestBlockMissingException.java and TestClose.java. Contributed by Lisheng Sun. 2019-09-28 22:03:53 +05:30
Ayush Saxena
c4c8d5fd0e HDFS-14849. Erasure Coding: the internal block is replicated many times when datanode is decommissioning. Contributed by HuangTao. 2019-09-28 21:44:44 +05:30
Ayush Saxena
0d5d0b914a Revert "HDFS-14849. Erasure Coding: the internal block is replicated many times when datanode is decommissioning. Contributed by HuangTao." 
This reverts commit ce58c05f1d.
 2019-09-28 21:32:15 +05:30
Bharat Viswanadham
14b4fbc019
HDDS-1146. Adding container related metrics in SCM. (#1541) 2019-09-27 15:24:30 -07:00
Sahil Takiar
13b427fc05 HDFS-14564: Add libhdfs APIs for readFully; add readFully to ByteBufferPositionedReadable (#963) Contributed by Sahil Takiar. 
Reviewed-by: Siyao Meng <smeng@cloudera.com>
 2019-09-27 12:25:03 -07:00
Ayush Saxena
ce58c05f1d HDFS-14849. Erasure Coding: the internal block is replicated many times when datanode is decommissioning. Contributed by HuangTao. 2019-09-27 18:58:19 +05:30
Doroszlai, Attila
a93a139b5d
HDDS-2185. createmrenv failure not reflected in acceptance test result 
Closes #1533
 2019-09-27 12:47:25 +02:00
Sammi Chen
8a9ede5cff HADOOP-15616. Incorporate Tencent Cloud COS File System Implementation. Contributed by Yang Yu. 2019-09-27 16:53:13 +08:00
Zhankun Tang
2870668cfa Make upstream aware of 3.1.3 release. 2019-09-27 16:44:07 +08:00
Doroszlai, Attila
9bf7a6e5b2 HDDS-2149. Replace findbugs with spotbugs 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-09-26 22:56:05 -07:00
Doroszlai, Attila
0371e953ac HDDS-2179. ConfigFileGenerator fails with Java 10 or newer 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-09-26 21:59:36 -07:00
Bharat Viswanadham
b6ef8cc120 HDD-2193. Adding container related metrics in SCM. 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-09-26 21:42:20 -07:00
dchitlangia
c55ac6a1c7 HDDS-2174. Delete GDPR Encryption Key from metadata when a Key is deleted 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-09-26 21:10:34 -07:00
Chen Liang
2adcc3c932 HDFS-14785. [SBN read] Change client logging to be less aggressive. Contributed by Chen Liang. 2019-09-26 15:48:19 -07:00
Inigo Goiri
b1e55cfb55 HDFS-14461. RBF: Fix intermittently failing kerberos related unit test. Contributed by Xiaoqiao He. 2019-09-26 15:26:23 -07:00
Anu Engineer
06998a1126
HDDS-2180. Add Object ID and update ID on VolumeList Object. (#1526) 2019-09-26 09:41:03 -07:00
Eric Badger
18a8c2404e YARN-9857. TestDelegationTokenRenewer throws NPE but tests pass. Contributed by Ahmed Hussein 2019-09-26 16:15:08 +00:00
Ayush Saxena
1a2a352ecd HDFS-11934. Add assertion to TestDefaultNameNodePort#testGetAddressFromConf. Contributed by Nikhil Navadiya. 2019-09-26 20:45:37 +05:30
Ayush Saxena
a3f68933d7 HDFS-14873. Fix dfsadmin doc for triggerBlockReport. Contributed by Fei Hui. 2019-09-26 19:54:07 +05:30
Vivek Ratnavel Subramanian
7b6219a8d1
HDDS-2182. Fix checkstyle violations introduced by HDDS-1738 
Closes #1529
 2019-09-26 12:09:28 +02:00
Gabor Bota
587a8eeec8 HDFS-14874. Fix TestHDFSCLI and TestDFSShell test break because of logging change in mkdir (#1522). Contributed by Gabor Bota. 2019-09-26 10:15:13 +05:30
Jonathan Hung
606e341c1a Addendum to YARN-9730. Support forcing configured partitions to be exclusive based on app node label 2019-09-25 17:54:53 -07:00
Márton Elek
f647185905 HDDS-2067. Create generic service facade with tracing/metrics/logging support 
Signed-off-by: Anu Engineer <aengineer@apache.org>
Co-Authored-By: Doroszlai, Attila <6454655+adoroszlai@users.noreply.github.com>
 2019-09-25 17:19:22 -07:00
Takanobu Asanuma
bdaaa3bbf2 HDFS-14832. RBF: Add Icon for ReadOnly False. Contributed by hemanthboyina 2019-09-25 23:48:49 +09:00
Xieming Li
c89d22d13a
HADOOP-16602. mvn package fails in hadoop-aws. 
Contributed by Xieming Li.

Follow-up to HADOOP-16445

Change-Id: I72c62d55b734a0f67556844f398ef4a50d9ea585
 2019-09-25 14:15:35 +01:00
Doroszlai, Attila
16f626f7f0
HDDS-2165. Freon fails if bucket does not exists 
Closes #1503
 2019-09-25 13:31:35 +02:00
Steve Loughran
e346e3638c HADOOP-15691 Add PathCapabilities to FileSystem and FileContext. 
Contributed by Steve Loughran.

This complements the StreamCapabilities Interface by allowing applications to probe for a specific path on a specific instance of a FileSystem client
to offer a specific capability.

This is intended to allow applications to determine

* Whether a method is implemented before calling it and dealing with UnsupportedOperationException.
* Whether a specific feature is believed to be available in the remote store.

As well as a common set of capabilities defined in CommonPathCapabilities,
file systems are free to add their own capabilities, prefixed with
 fs. + schema + .

The plan is to identify and document more capabilities -and for file systems which add new features, for a declaration of the availability of the feature to always be available.

Note

* The remote store is not expected to be checked for the feature;
  It is more a check of client API and the client's configuration/knowledge
  of the state of the remote system.
* Permissions are not checked.

Change-Id: I80bfebe94f4a8bdad8f3ac055495735b824968f5
 2019-09-25 12:16:41 +01:00
Yi Sheng
e6fb6ee94d
HDDS-1738. Add nullable annotation for OMResponse classes 
Closes #1499
 2019-09-25 12:14:07 +02:00
Doroszlai, Attila
8baebb54e1
HDDS-2171. Dangling links in test report due to incompatible realpath 
Closes #1515
 2019-09-25 12:02:42 +02:00
Szilard Nemeth
c72457787d YARN-6715. Fix documentation about NodeHealthScriptRunner. Contributed by Peter Bacsko 2019-09-25 10:36:22 +02:00
Szilard Nemeth
bec0864394 YARN-9808. Zero length files in container log output haven't got a header. Contributed by Adam Antal 2019-09-25 10:28:34 +02:00
Prabhu Joseph
3f89084ac7
HDFS-14845. Ignore AuthenticationFilterInitializer for HttpFSServerWebServer and honor hadoop.http.authentication configs. 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2019-09-25 10:19:30 +09:00
Vivek Ratnavel Subramanian
a3463812ef HDDS-2168. TestOzoneManagerDoubleBufferWithOMResponse sometimes fails with out of memory error (#1509) 2019-09-24 16:28:25 -07:00
Bharat Viswanadham
6917754ba7
HDDS-2172.Ozone shell should remove description about REST protocol support. Contributed by Siddharth Wagle. 2019-09-24 16:06:10 -07:00
Jonathan Hung
c2731d4b63 YARN-9730. Support forcing configured partitions to be exclusive based on app node label 2019-09-24 13:51:32 -07:00
Ayush Saxena
66400c1cbb HDFS-14808. EC: Improper size values for corrupt ec block in LOG. Contributed by Ayush Saxena. 2019-09-25 01:31:15 +05:30
Ayush Saxena
eb96a3093e HDFS-14655. [SBN Read] Namenode crashes if one of The JN is down. Contributed by Ayush Saxena. 2019-09-25 01:16:30 +05:30
Anu Engineer
f16cf877e5
HDDS-2170. Add Object IDs and Update ID to Volume Object (#1510) 2019-09-24 11:01:05 -07:00
Inigo Goiri
afa1006a53 HDFS-14843. Double Synchronization in BlockReportLeaseManager. Contributed by David Mollitor. 2019-09-24 09:58:42 -07:00
Inigo Goiri
816d3cb908 HDFS-14837. Review of Block.java. Contributed by David Mollitor. 2019-09-24 09:54:09 -07:00
Wei-Chiu Chuang
43203b466d HDFS-14868. RBF: Fix typo in TestRouterQuota. Contributed by Jinglun. 
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-09-24 09:38:58 -07:00
Márton Elek
91f50b98ca
HDDS-2167. Hadoop31-mr acceptance test is failing due to the shading 
Closes #1507
 2019-09-24 17:52:36 +02:00