big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
24,510 Commits 2 Branches 2 Tags 264 MiB
5fc61d2e56
Commit Graph

24510 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Szilard Nemeth
c79a5f2d99 HADOOP-16580. Disable retry of FailoverOnNetworkExceptionRetry in case of AccessControlException. Contributed by Adam Antal 2019-10-16 13:29:06 +02:00
Steve Loughran
bbcf0b91d6 HADOOP-16478. S3Guard bucket-info fails if the caller lacks s3:GetBucketLocation. 
Contributed by Steve Loughran.

Includes HADOOP-16651. S3 getBucketLocation() can return "US" for us-east.

Change-Id: Ifc0dca76e51495ed1a8fc0f077b86bf125deff40
 2019-10-16 09:41:33 +01:00
Wei-Chiu Chuang
c39e9fc9a3 HADOOP-15169. "hadoop.ssl.enabled.protocols" should be considered in httpserver2. Contributed by Brahma Reddy Battula, Wei-Chiu Chuang. 
Reviewed-by: Xiaoyu Yao <xyao@apache.org>
Co-authored-By: Brahma Reddy Battula <brahma@apache.org>
 2019-10-15 13:55:23 -07:00
Wei-Chiu Chuang
85af77c757 HADOOP-16643. Update netty4 to the latest 4.1.42. Contributed by Lisheng Sun. 2019-10-15 08:55:14 -07:00
Szilard Nemeth
a8ef03e961 YARN-9840. Capacity scheduler: add support for Secondary Group rule mapping. Contributed by Manikandan R 2019-10-15 17:22:52 +02:00
Surendra Singh Lilhore
336abbd873 HDFS-14886. In NameNode Web UI's Startup Progress page, Loading edits always shows 0 sec. Contributed by hemanthboyina. 2019-10-14 23:31:43 +05:30
Ashvin
fabd41fa48 HDFS-14856. Fetch file ACLs while mounting external store. (#1478) 2019-10-14 09:44:56 -07:00
Steve Loughran
74e5018d87 HADOOP-16635. S3A "directories only" scan still does a HEAD. 
Contributed by Steve Loughran.

Change-Id: I5e41d7f721364c392e1f4344db83dfa8c5aa06ce
 2019-10-14 17:05:52 +01:00
Steve Loughran
dee9e97075 Revert "HADOOP-15870. S3AInputStream.remainingInFile should use nextReadPos." 
This reverts commit 7a4b3d42c4.

The patch broke TestRouterWebHDFSContractSeek as it turns out that
WebHDFSInputStream.available() is always 0.
 2019-10-14 16:56:50 +01:00
Szilard Nemeth
5cc7873a47 YARN-9699. Migration tool that help to generate CS config based on FS config [Phase 1]. Contributed by Peter Bacsko 2019-10-14 17:19:39 +02:00
Ayush Saxena
5f4641a120 HDFS-14238. A log in NNThroughputBenchmark should change log level to INFO instead of ERROR. Contributed by Shen Yinjie. 2019-10-12 21:06:23 +05:30
Ayush Saxena
6e5cd5273f HDFS-14899. Use Relative URLS in Hadoop HDFS RBF. Contributed by David Mollitor. 2019-10-12 13:26:47 +05:30
Ayush Saxena
44bc095998 HADOOP-13907. Fix TestWebDelegationToken#testKerberosDelegationTokenAuthenticator on Windows. Contributed by Kitti Nanasi. 2019-10-12 13:16:51 +05:30
Shweta Yakkali
c561a70c49 HDDS-2213.Reduce key provider loading log level in OzoneFileSystem#getAdditionalTokenIssuers (#1556) 2019-10-11 10:23:19 -07:00
Szilard Nemeth
ec86f42e40 YARN-8453. Additional Unit tests to verify queue limit and max-limit with multiple resource types. Contributed by Adam Antal 2019-10-11 14:01:19 +02:00
Szilard Nemeth
62b5cefaea YARN-9836. General usability improvements in showSimulationTrace.html. Contributed by Adam Antal 2019-10-11 13:39:27 +02:00
Gabor Bota
4a700c20d5
HADOOP-16520. Race condition in DDB table init and waiting threads. (#1576). Contributed by Gabor Bota. 
Fixes HADOOP-16349. DynamoDBMetadataStore.getVersionMarkerItem() to log at info/warn on retry

Change-Id: Ia83e92b9039ccb780090c99c41b4f71ef7539d35
 2019-10-11 12:08:47 +02:00
Xiaoyu Yao
f267917ce3 HDDS-2282. scmcli pipeline list command throws NullPointerException. Contributed by Xiaoyu Yao. (#1642) 2019-10-10 22:33:59 -07:00
Bharat Viswanadham
9c72bf4621
HDDS-1986. Fix listkeys API. (#1588) 2019-10-10 16:49:06 -07:00
Eric Yang
31e0122f4d YARN-9860. Enable service mode for Docker containers on YARN 
Contributed by Prabhu Joseph and Shane Kumpf
 2019-10-10 19:02:02 -04:00
lqjacklee
7a4b3d42c4
HADOOP-15870. S3AInputStream.remainingInFile should use nextReadPos. 
Contributed by lqjacklee.

Change-Id: I32bb00a683102e7ff8ff8ce0b8d9c3195ca7381c
 2019-10-10 21:58:42 +01:00
Bharat Viswanadham
957253fea6
HDDS-1984. Fix listBucket API. (#1555) 2019-10-10 12:33:55 -07:00
Bharat Viswanadham
4850b3aa86 HDDS-2269. Provide config for fair/non-fair for OM RW Lock. (#1623) 2019-10-10 23:28:19 +05:30
Steve Loughran
effe6087a5
HADOOP-16650. ITestS3AClosedFS failing. 
Contributed by Steve Loughran.

Change-Id: Ia9bb84bd6455e210a54cfe9eb944feeda8b58da9
 2019-10-10 17:32:25 +01:00
Ayush Saxena
104ccca916 HDFS-14900. Fix build failure of hadoop-hdfs-native-client. Contributed by Masatake Iwasaki. 2019-10-10 21:36:02 +05:30
Siddharth
a031388a2e HDDS-2266. Avoid evaluation of LOG.trace and LOG.debug statement in the read/write path. (#1633) 2019-10-10 15:30:11 +05:30
Ayush Saxena
eeb58a07e2 HDFS-14898. Use Relative URLS in Hadoop HDFS HTTP FS. Contributed by David Mollitor. 2019-10-10 01:03:33 +05:30
Surendra Singh Lilhore
d76e2655ac HDFS-14754. Erasure Coding : The number of Under-Replicated Blocks never reduced(addendum). Contributed by Surendra Singh Lilhore. 2019-10-09 23:20:28 +05:30
Doroszlai, Attila
2d81abce5e
HDDS-2265. integration.sh may report false negative 
Closes #1608
 2019-10-09 16:17:40 +02:00
Márton Elek
b034350f3d
Squashed commit of the following: 
commit 9d6a1b97f365b22392ceac8d958cf041c3515bd6
Author: Doroszlai, Attila <adoroszlai@apache.org>
Date:   Mon Oct 7 13:42:35 2019 +0200

    HDDS-2265. integration.sh may report false negative
Revert "HDDS-2217. Remove log4j and audit configuration from the docker-config files"

This reverts commit 4b0a5bca46.
 2019-10-09 16:16:44 +02:00
Doroszlai, Attila
4b0a5bca46
HDDS-2217. Remove log4j and audit configuration from the docker-config files 
Closes #1582
 2019-10-09 15:51:00 +02:00
Chris Teoh
1f954e6798
HDDS-2217. Remove log4j and audit configuration from the docker-config files 
Closes #1582
 2019-10-09 15:46:16 +02:00
Szilard Nemeth
6f1ab95168 YARN-9128. Use SerializationUtils from apache commons to serialize / deserialize ResourceMappings. Contributed by Zoltan Siegl 2019-10-09 13:26:26 +02:00
Szilard Nemeth
35f093f5b3 YARN-9356. Add more tests to ratio method in TestResourceCalculator. Contributed by Zoltan Siegl 2019-10-09 13:09:09 +02:00
Sunil G
be901f4962 YARN-9873. Mutation API Config Change need to update Version Number. Contributed by Prabhu Joseph 2019-10-09 15:53:14 +05:30
Istvan Fajth
1d27930407 HDDS-2233 - Remove ByteStringHelper and refactor the code to the place where it used (#1596) 2019-10-09 11:28:47 +05:30
Bharat Viswanadham
87d9f3668c
HDDS-2244. Use new ReadWrite lock in OzoneManager. (#1589) 2019-10-08 13:03:14 -07:00
Chen Liang
72ae371e7a HDFS-14509. DN throws InvalidToken due to inequality of password when upgrade NN 2.x to 3.x. Contributed by Yuxuan Wang and Konstantin Shvachko. 2019-10-08 11:56:52 -07:00
Jonathan Hung
5462d2176f YARN-9760. Support configuring application priorities on a workflow level. Contributed by Varun Saxena 2019-10-08 11:20:13 -07:00
Siddharth
15a9beed1b HDDS-2260. Avoid evaluation of LOG.trace and LOG.debug statement in the read/write path (HDDS). (#1612) 2019-10-08 11:13:53 -07:00
Ayush Saxena
91320b4461 HDFS-14859. Prevent unnecessary evaluation of costly operation getNumLiveDataNodes when dfs.namenode.safemode.min.datanodes is not zero. Contributed by Srinivasu Majeti. 2019-10-08 14:14:14 +05:30
Ayush Saxena
761594549e HDFS-14814. RBF: RouterQuotaUpdateService supports inherited rule. Contributed by Jinglun. 2019-10-08 14:01:44 +05:30
Anu Engineer
4fdf016358 HDDS-2245. Use dynamic ports for SCM in TestSecureOzoneCluster 
Contributed by kevin su.
 2019-10-07 15:41:42 -07:00
Doroszlai, Attila
012d897e5b HDDS-2262. SLEEP_SECONDS: command not found 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-10-07 14:38:54 -07:00
Doroszlai, Attila
aaa94c3da6 HDDS-2259. Container Data Scrubber computes wrong checksum 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-10-07 14:30:23 -07:00
Doroszlai, Attila
cfba6ac951 HDDS-2264. Improve output of TestOzoneContainer 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-10-07 14:10:57 -07:00
Doroszlai, Attila
1877312440 HDDS-2238. Container Data Scrubber spams log in empty cluster 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-10-07 13:59:49 -07:00
Surendra Singh Lilhore
382967be51 HDFS-14373. EC : Decoding is failing when block group last incomplete cell fall in to AlignedStripe. Contributed by Surendra Singh Lilhore. 2019-10-08 00:14:30 +05:30
Doroszlai, Attila
9685a6c0e5 HDDS-2239. Fix TestOzoneFsHAUrls (#1600) 2019-10-07 10:17:25 -07:00
bilaharith
1a77a15fe4
HADOOP-16587. Make ABFS AAD endpoints configurable. 
Contributed by Bilahari T H.

This also addresses HADOOP-16498: AzureADAuthenticator cannot authenticate
in China.

Change-Id: I2441dd48b50b59b912b0242f7f5a4418cf94a87c
 2019-10-07 13:07:46 +01:00