big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
21,164 Commits 2 Branches 2 Tags 264 MiB
a536eb5c41
Commit Graph

21164 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Nanda kumar
a536eb5c41 HDDS-360. Use RocksDBStore and TableStore for SCM Metadata. 
Contributed by Anu Engineer.
 2019-02-12 14:26:19 +05:30
Yiqun Lin
26e60135f5 HDDS-1074. Remove dead variable from KeyOutputStream#addKeyLocationInfo. Contributed by Siddharth Wagle. 2019-02-12 16:28:50 +08:00
Ajay Kumar
d48e61dd36 HDDS-1012. Add Default CertificateClient implementation. Contributed by Ajay Kumar 2019-02-11 18:59:47 -08:00
Giovanni Matteo Fumarola
7536488bbd YARN-996. REST API support for node resource configuration. Contributed by Inigo Goiri. 2019-02-11 15:27:43 -08:00
Suma Shivaprasad
1ce2e91c4b YARN-9229. Document docker registry deployment with NFS Gateway. Contributed by Eric Yang. 2019-02-11 14:54:30 -08:00
Masatake Iwasaki
6c999fe4b0 HADOOP-16098. Fix javadoc warnings in hadoop-aws. Contributed by Masatake Iwasaki. 2019-02-12 06:07:47 +09:00
Xiaoyu Yao
ca4e46a05e HDDS-1075. Fix CertificateUtil#parseRSAPublicKey charsetName. Contributed by Siddharth Wagle. 2019-02-11 12:09:14 -08:00
Siyao Meng
5c10630ad8 HDFS-14261. Kerberize JournalNodeSyncer unit test. Contributed by Siyao Meng. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-02-11 11:02:32 -08:00
Inigo Goiri
0ceb1b70f3 HDFS-14260. Replace synchronized method in BlockReceiver with atomic value. Contributed by BELUGA BEHR. 2019-02-11 10:09:44 -08:00
Mukul Kumar Singh
73b67b2df5 HDDS-1040. Add blockade Tests for client failures. Contributed by Nilotpal Nandi. 2019-02-11 20:08:25 +05:30
Márton Elek
e7d1ae52d2 HDDS-1017. Use distributed tracing to indentify performance problems in Ozone. Contributed by Elek, Marton. 2019-02-11 12:18:26 +01:00
Weiwei Yang
0a1637c750 YARN-8555. Parameterize TestSchedulingRequestContainerAllocation(Async) to cover both PC handler options. Contributed by Prabhu Joseph. 2019-02-11 15:53:50 +08:00
Weiwei Yang
2b7f828d46 YARN-9252. Allocation Tag Namespace support in Distributed Shell. Contributed by Prabhu Joseph. 2019-02-11 14:42:21 +08:00
Bharat Viswanadham
a1414582a8 HDDS-1077. TestSecureOzoneCluster does not config OM HTTP keytab. Contributed by Xiaoyu Yao. 2019-02-10 19:34:53 -08:00
Bharat Viswanadham
ed99da850e HDDS-1078. TestRatisPipelineProvider failing because of node count mismatch. Contributed by Mukul Kumar Singh. 2019-02-10 19:17:08 -08:00
Shashikant Banerjee
965d26c9c7 HDDS-1026. Reads should fail over to alternate replica. Contributed by Shashikant Banerjee. 2019-02-10 10:53:16 +05:30
Xiaoyu Yao
e50dc7ee59 HDDS-1018. Update the base image of krb5 container for the secure ozone cluster. Contributed by Xiaoyu Yao. 2019-02-08 12:45:49 -08:00
Yiqun Lin
fb8c997a68 HDDS-1048. Remove SCMNodeStat from SCMNodeManager and use storage information from DatanodeInfo#StorageReportProto. Contributed by Nanda kumar. 2019-02-08 23:49:37 +08:00
Masatake Iwasaki
e0ab1bdece YARN-9282. Typo in javadoc of class LinuxContainerExecutor: hadoop.security.authetication should be 'authentication'. Contributed by Charan Hebri. 2019-02-09 00:26:50 +09:00
Yiqun Lin
177131793a HDFS-14172. Avoid NPE when SectionName#fromString returns null. Contributed by Xiang Li. 2019-02-08 20:51:30 +08:00
Nanda kumar
394a9f7d5c HDDS-1033. Add FSStatistics for OzoneFileSystem. 
Contributed by Mukul Kumar Singh.
 2019-02-08 17:33:37 +05:30
Márton Elek
df7b7dadf9 HDDS-1073. Fix FindBugs issues on OzoneBucketStub#createMultipartKey. Contributed by Aravindan Vijayan. 2019-02-08 11:35:55 +01:00
Nanda kumar
0c1bc4dcee HDDS-981. Block allocation should involve pipeline selection and then container selection. 
Contributed by Lokesh Jain.
 2019-02-08 15:44:41 +05:30
Bharat Viswanadham
a140a890c6 HDDS-1069. Temporarily disable the security acceptance tests by default in Ozone. Contributed by Marton Elek. 2019-02-07 18:05:05 -08:00
Surendra Singh Lilhore
4be87353e3 HDFS-14140. JournalNodeSyncer authentication is failing in secure cluster. Contributed by Surendra Singh Lilhore. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-02-07 16:51:59 -08:00
Da Zhou
1f1655028e
HADOOP-15954. ABFS: Enable owner and group conversion for MSI and login user using OAuth. 
Contributed by Da Zhou and Junhua Gu.
 2019-02-07 21:58:21 +00:00
Steve Loughran
668817a6ce
Revert "HADOOP-15954. ABFS: Enable owner and group conversion for MSI and login user using OAuth." 
(accidentally mixed in two patches)

This reverts commit fa8cd1bf28.
 2019-02-07 21:57:22 +00:00
Ranith Sardar
546c5d70ef
HADOOP-16032. Distcp It should clear sub directory ACL before applying new ACL on. 2019-02-07 21:48:07 +00:00
Bharat Viswanadham
75e8441c61 HDDS-1071. Make Ozone s3 acceptance test suite centos compatible. Contributed by Elek Marton. 2019-02-07 08:53:11 -08:00
Eric E Payne
d1ca9432dd YARN-7171: RM UI should sort memory / cores numerically. Contributed by Ahmed Hussein 2019-02-07 16:38:11 +00:00
Márton Elek
a65aca2fef HDDS-922. Create isolated classloder to use ozonefs with any older hadoop versions. Contributed by Elek, Marton. 2019-02-07 17:02:03 +01:00
Mukul Kumar Singh
214112b2d7 HDDS-1010. ContainerSet#getContainerMap should be renamed. Contributed by Supratim Deka. 2019-02-07 18:06:23 +05:30
Andrew Olson
de804e53b9
HADOOP-15281. Distcp to add no-rename copy option. 
Contributed by Andrew Olson.
 2019-02-07 10:07:22 +00:00
Vinayakumar B
3c96a03dea YARN-8498. Yarn NodeManager OOM Listener Fails Compilation on Ubuntu 18.04. Contributed by Ayush Saxena. 2019-02-07 13:02:35 +05:30
Rohith Sharma K S
7fa62e150c YARN-8219. Add application launch time to ATSV2. Contributed by Abhishek Modi. 2019-02-07 09:58:53 +05:30
Rohith Sharma K S
1e5e08d83b YARN-7627. [ATSv2] When passing a non-number as metricslimit, the error message is wrong. Contributed by Charan Hebri. 2019-02-07 09:58:53 +05:30
Bharat Viswanadham
912d9f790d HDDS-1044. Client doesn't propogate correct error code to client on out of disk space. Contributed by Yiqun Lin. 2019-02-06 20:02:17 -08:00
Siyao Meng
6aa63452b3 HDFS-14242. OIV WebImageViewer: NPE when param op is not specified. Contributed by Siyao Meng. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-02-06 12:19:21 -08:00
Erik Krogen
d3de8e162b HDFS-14250. [SBN read]. msync should always direct to active NameNode to get latest stateID. Contributed by Chao Sun. 2019-02-06 09:54:47 -08:00
Weiwei Yang
711d22f166 YARN-9253. Add UT to verify Placement Constraint in Distributed Shell. Contributed by Prabhu Joseph. 2019-02-06 22:36:51 +08:00
Mukul Kumar Singh
911790cc26 HDDS-1027. Add blockade Tests for datanode isolation and scm failures. Contributed by Nilotpal Nandi. 2019-02-06 11:32:38 +05:30
Kitti Nanasi
49ddd8a6ed HDFS-14231. DataXceiver#run() should not log exceptions caused by InvalidToken exception as an error. Contributed by Kitti Nanasi. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-02-05 15:14:19 -08:00
Wangda Tan
308f3168fa Make upstream aware of 3.1.2 release 
Change-Id: I397bc6ef75498726df4763bd07a8bf8fe1c38365
 2019-02-05 14:03:18 -08:00
Da Zhou
fa8cd1bf28
HADOOP-15954. ABFS: Enable owner and group conversion for MSI and login user using OAuth. 
Contributed by Da Zhou and Junhua Gu.
 2019-02-05 19:23:15 +00:00
Suma Shivaprasad
2044967e75 YARN-9246 NPE when executing a command yarn node -status or -states without additional arguments. Contributed by Masahiro Tanaka 2019-02-05 09:39:38 -08:00
Márton Elek
194f0b49fb HDDS-631. Ozone classpath shell command is not working. Contributed by Elek, Marton. 2019-02-05 16:20:30 +01:00
Márton Elek
9f2da01591 HDDS-776. Make OM initialization resilient to dns failures. Contributed by Doroszlai, Attila. 2019-02-05 16:11:19 +01:00
Da Zhou
ba9efe06fa
HADOOP-16074. WASB: Update container not found error code. 
Contributed by Da Zhou.
 2019-02-05 14:39:45 +00:00
Márton Elek
9ace37b119 HDDS-987. MultipartUpload: S3API for list parts of a object. Contributed by Bharat Viswanadham. 2019-02-05 13:08:36 +01:00
Steve Loughran
f365957c63
HADOOP-15229. Add FileSystem builder-based openFile() API to match createFile(); 
S3A to implement S3 Select through this API.

The new openFile() API is asynchronous, and implemented across FileSystem and FileContext.

The MapReduce V2 inputs are moved to this API, and you can actually set must/may
options to pass in.

This is more useful for setting things like s3a seek policy than for S3 select,
as the existing input format/record readers can't handle S3 select output where
the stream is shorter than the file length, and splitting plain text is suboptimal.
Future work is needed there.

In the meantime, any/all filesystem connectors are now free to add their own filesystem-specific
configuration parameters which can be set in jobs and used to set filesystem input stream
options (seek policy, retry, encryption secrets, etc).

Contributed by Steve Loughran
 2019-02-05 11:51:02 +00:00