big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
23,719 Commits 2 Branches 2 Tags 264 MiB
5b92d73a74
Commit Graph

5159 Commits

Author SHA1 Message Date
Steve Loughran
958764479d
HADOOP-16777. Add Tez to LimitedPrivate of ClusterStorageCapacityExceededException 
Contributed by Wang Yan.

Change-Id: I92dfe7079ba8ebe89d70255bb845309be0603a8e
 2020-01-02 15:49:42 +00:00
Rakesh Radhakrishnan
d79cce20ab HDFS-14740. Recover data blocks from persistent memory read cache during datanode restarts. Contributed by Feilong He. 2020-01-02 11:44:00 +05:30
Ayush Saxena
926d0b48f0 HDFS-15081. Typo in RetryCache#waitForCompletion annotation. Contributed by Fei Hui. 2019-12-27 18:32:15 +05:30
Chandra Sanivarapu
d80d86264d HADOOP-16042. Update the link to HadoopJavaVersion (#1784) 2019-12-27 17:18:10 +09:00
Vinayakumar B
e2a5448d2b
HADOOP-16774. TestDiskChecker and TestReadWriteDiskValidator fails when run with -Pparallel-tests (#1776). Contributed by Vinayakumar B. 2019-12-21 00:08:15 +05:30
Ayush Saxena
ab32762f43 HADOOP-16755. Fix broken link in single node cluster setup documentation. Contributed by Denes Gerencser. 2019-12-16 08:44:28 +05:30
Kengo Seki
fd7de2b82a HADOOP-16764. Rewrite Python example codes using Python3 (#1762) 2019-12-16 11:04:20 +09:00
Szilard Nemeth
631dbbc6f2 YARN-9923. Introduce HealthReporter interface to support multiple health checker files. Contributed by Adam Antal 2019-12-15 17:28:04 +01:00
Mingliang Liu
d12ad9e8ad
HADOOP-16757. Increase timeout unit test rule for MetadataStoreTestBase (#1757) 
Contributed by Mingliang Liu.

Signed-off-by: Steve Loughran <stevel@apache.org>
 2019-12-13 08:19:27 -08:00
Erik Krogen
c174d50b30 HDFS-15032. Properly handle InvocationTargetExceptions in the proxy created by ProxyCombiner. This fixes a bug encountered by the HDFS balancer when used with Observer Nodes. Contributed by Erik Krogen. 2019-12-11 09:39:37 -08:00
Mingliang Liu
19512b21e3
HADOOP-16735. Make it clearer in config default that EnvironmentVariableCredentialsProvider supports AWS_SESSION_TOKEN. Contributed by Mingliang Liu 
This closes #1733
 2019-12-05 17:37:17 -08:00
Ayush Saxena
83a14559e5 HDFS-15023. [SBN read] ZKFC should check the state before joining the election. Contributed by Fei Hui. 2019-12-05 18:52:01 +05:30
Aravindan Vijayan
f1ab7f18c4 HADOOP-16718. Allow disabling Server Name Indication (SNI) for Jetty. Contributed by Aravindan Vijayan. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Siyao Meng <smeng@cloudera.com>
 2019-12-03 15:47:16 -08:00
Yiqun Lin
2b452b4e60 HDFS-14986. ReplicaCachingGetSpaceUsed throws ConcurrentModificationException. Contributed by Aiphago. 2019-11-28 10:43:35 +08:00
Sahil Takiar
3161813482 HADOOP-16685: FileSystem#listStatusIterator does not check if given path exists (#1695) 2019-11-26 11:11:26 -08:00
Akira Ajisaka
da9435182a
HADOOP-16717. Remove GenericsUtil isLog4jLogger dependency on Log4jLoggerAdapter. Contributed by Xieming Li. 2019-11-25 14:36:27 +09:00
Siyao Meng
1a0c0e42e7 HADOOP-16276. Fix jsvc startup command in hadoop-functions.sh due to jsvc >= 1.0.11 changed default current working directory (#1272) 2019-11-20 16:44:44 -08:00
David Mollitor
6f899e9030 HADOOP-15852. Refactor QuotaUsage. Contributed by David Mollitor. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-11-20 14:40:10 -08:00
Erik Krogen
be77231452 HADOOP-16700. Track response time separately in Server.Call to avoid negative RpcQueueTime values. Contributed by xuzq. 2019-11-20 07:56:33 -08:00
Steve Loughran
1415ad3800
HADOOP-16707. NPE in UGI.getCurrentUser in ITestAbfsIdentityTransformer setup. 
Contributed by Steve Loughran.

Change-Id: I38fdba2fa70e534d78b15e61de19368912588b0c
 2019-11-20 14:32:12 +00:00
Abhishek Modi
4050471b45 HADOOP-16657. Move remaining log4j APIs over to slf4j in hadoop-common. Contributed by Minni Mittal. 2019-11-19 08:01:17 +05:30
Szilard Nemeth
27642367ef YARN-9011. Race condition during decommissioning. Contributed by Peter Bacsko 2019-11-18 16:29:39 +01:00
Ayush Saxena
7f811722b7 HADOOP-16712. Config ha.failover-controller.active-standby-elector.zk.op.retries is not in core-default.xml. Contributed by Xieming Li. 2019-11-18 19:00:25 +05:30
Hui Fei
67f2c491fe HDFS-14802. The feature of protect directories should be used in RenameOp (#1669) 2019-11-15 12:35:37 -08:00
He Xiaoqiao
c892a879dd HDFS-14882. Consider DataNode load when #getBlockLocation. Contributed by Xiaoqiao He. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>

Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: Istvan Fajth <pifta@cloudera.com>
 2019-11-15 12:16:31 -08:00
Akira Ajisaka
92c28c100e
HADOOP-15097. AbstractContractDeleteTest::testDeleteNonEmptyDirRecursive with misleading path. Contributed by Xieming Li. 2019-11-15 13:49:57 +09:00
belugabehr
d0302d3849 HADOOP-16691. Unify Logging in UserGroupInformation. 
Unify Logging in UserGroupInformation.
 2019-11-14 10:28:26 -08:00
Lukas Majercak
c73334a924 HADOOP-16705. MBeanInfoBuilder puts unnecessary memory pressure on the system with a debug log. 
MBeanInfoBuilder's get() method DEBUG logs all the MBeanAttributeInfo attributes that it gathered. This can have a high memory churn that can be easily avoided.
 2019-11-14 09:54:02 -08:00
Steve Loughran
a139a87610
HADOOP-16708. HadoopExecutors cleanup to only log at debug. 
Contributed by David Mollitor.

Change-Id: Ib501a3ed6ed299ccd3c773c4dc086441a5728e73
 2019-11-14 17:51:39 +00:00
Surendra Singh Lilhore
73a386ab76 HADOOP-16585. [Tool:NNloadGeneratorMR] Multiple threads are using same id for creating file LoadGenerator#write. Contributed by Ranith Sardar. 2019-11-14 01:45:46 +05:30
Xudong Cao
df6b3162c1 HADOOP-16677. Recalculate the remaining timeout millis correctly while throwing an InterupptedException in SocketIOWithTimeout. (#1687) 2019-11-12 14:19:39 -08:00
Siyao Meng
3ab98eefa5 HADOOP-16656. Document FairCallQueue configs in core-default.xml. Contributed by Siyao Meng. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-11-12 11:34:24 -08:00
Steve Loughran
990063d2af
HADOOP-16665. Filesystems to be closed if they failed during initialize(). 
Contributed by Steve Loughran.

This FileSystem instantiation so if an IOException or RuntimeException is
raised in the invocation of FileSystem.initialize() then a best-effort
attempt is made to close the FS instance; exceptions raised that there
are swallowed.

The S3AFileSystem is also modified to do its own cleanup if an
IOException is raised during its initialize() process, it being the
FS we know has the "potential" to leak threads, especially in
extension points (e.g AWS Authenticators) which spawn threads.

Change-Id: Ib84073a606c9d53bf53cbfca4629876a03894f04
 2019-11-12 18:17:21 +00:00
Szilard Nemeth
3d249301f4 HADOOP-16683. Disable retry of FailoverOnNetworkExceptionRetry in case of wrapped AccessControlException. Contributed by Adam Antal 2019-11-09 15:43:53 +01:00
Chen Liang
dd900259c4 HDFS-14941. Potential editlog race condition can cause corrupted file. Contributed by Chen Liang and Konstantin Shvachko. 2019-11-06 09:56:19 -08:00
Wei-Chiu Chuang
ee8addbec4 Revert "HADOOP-16656. Document FairCallQueue configs in core-default.xml. Contributed by Siyao Meng." 
This reverts commit f9b99d2f24.
 2019-11-05 15:41:52 -08:00
belugabehr
6f0190d8e4 HADOOP-16678: Review of ArrayWritable (#1692) 2019-11-04 14:24:44 -08:00
Gabor Bota
dca19fc3aa
HADOOP-16484. S3A to warn or fail if S3Guard is disabled (#1661). Contributed by Gabor Bota. 2019-11-04 12:55:36 +01:00
Jonathan Hung
ef9d12df24 Add 2.10.0 release notes for HDFS-12943 2019-10-31 20:33:59 -07:00
Siyao Meng
f9b99d2f24 HADOOP-16656. Document FairCallQueue configs in core-default.xml. Contributed by Siyao Meng. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-10-31 12:59:46 -07:00
Ayush Saxena
d9fbedc4ae HDFS-14936. Add getNumOfChildren() for interface InnerNode. Contributed by Lisheng Sun. 2019-10-31 23:39:27 +05:30
Szilard Nemeth
8c9173c87c HADOOP-16510. [hadoop-common] Fix order of actual and expected expression in assert statements. Contributed by Adam Antal 2019-10-31 14:35:20 +01:00
Jonathan Hung
e6137d0e50 Make upstream aware of 2.10.0 release 2019-10-30 20:58:32 -07:00
Steve Loughran
bd498bac86
HADOOP-16669. TestRawLocalFileSystemContract.testPermission fails if no native library. 
Contributed by Steve Loughran.

Change-Id: I622e36f17a63c775c5e33e93b8b26e461d0bc451
 2019-10-29 17:09:26 +00:00
Fei Hui
7be5508d9b HDFS-14923. Remove dead code from HealthMonitor. Contributed by Fei Hui. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-10-25 15:06:16 -07:00
hongdongdong
2eba262472
HADOOP-16615. Add password check for credential provider, 
Contributed by hongdongdong.

Change-Id: Iaac01bc8594860064a80c822a0e47981243ab7e1
 2019-10-24 18:48:27 +01:00
Ayush Saxena
2ae4b33d48 HADOOP-16662. Remove unnecessary InnerNode check in NetworkTopology#add(). Contributed by Lisheng Sun. 2019-10-19 11:05:54 +05:30
Mate Szalay-Beko
6d92aa7c30 HADOOP-16579. Upgrade to Curator 4.2.0 and ZooKeeper 3.5.5 (#1656). Contributed by Norbert Kalmár, Mate Szalay-Beko 
* HADOOP-16579 - Upgrade to Apache Curator 4.2.0 and ZooKeeper 3.5.5

- Add a static initializer for the unit tests using ZooKeeper to enable
the four-letter-words diagnostic telnet commands. (this is an interface
that become disabled by default, so to keep the ZooKeeper 3.4.x behavior
we enabled it for the tests)
- Also fix ZKFailoverController to look for relevant fail-over ActiveAttempt
records. The new ZooKeeper seems to respond quicker during the fail-over
tests than the ZooKeeper, so we made sure to catch all the relevant records
by adding a new parameter to ZKFailoverontroller.waitForActiveAttempt().

Co-authored-by: Norbert Kalmár <nkalmar@cloudera.com>
 2019-10-18 13:26:20 -07:00
Yuming Wang
3d41f33018 HADOOP-16152. Upgrade Eclipse Jetty version to 9.4.x. Contributed by Yuming Wang, Siyao Meng. 
Co-authored-By: Siyao Meng <smeng@cloudera.com>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-10-18 08:25:02 -07:00
Szilard Nemeth
c79a5f2d99 HADOOP-16580. Disable retry of FailoverOnNetworkExceptionRetry in case of AccessControlException. Contributed by Adam Antal 2019-10-16 13:29:06 +02:00
Wei-Chiu Chuang
c39e9fc9a3 HADOOP-15169. "hadoop.ssl.enabled.protocols" should be considered in httpserver2. Contributed by Brahma Reddy Battula, Wei-Chiu Chuang. 
Reviewed-by: Xiaoyu Yao <xyao@apache.org>
Co-authored-By: Brahma Reddy Battula <brahma@apache.org>
 2019-10-15 13:55:23 -07:00
Steve Loughran
dee9e97075 Revert "HADOOP-15870. S3AInputStream.remainingInFile should use nextReadPos." 
This reverts commit 7a4b3d42c4.

The patch broke TestRouterWebHDFSContractSeek as it turns out that
WebHDFSInputStream.available() is always 0.
 2019-10-14 16:56:50 +01:00
Ayush Saxena
44bc095998 HADOOP-13907. Fix TestWebDelegationToken#testKerberosDelegationTokenAuthenticator on Windows. Contributed by Kitti Nanasi. 2019-10-12 13:16:51 +05:30
lqjacklee
7a4b3d42c4
HADOOP-15870. S3AInputStream.remainingInFile should use nextReadPos. 
Contributed by lqjacklee.

Change-Id: I32bb00a683102e7ff8ff8ce0b8d9c3195ca7381c
 2019-10-10 21:58:42 +01:00
Jonathan Hung
5462d2176f YARN-9760. Support configuring application priorities on a workflow level. Contributed by Varun Saxena 2019-10-08 11:20:13 -07:00
Steve Loughran
1921e94292
HADOOP-16458. LocatedFileStatusFetcher.getFileStatuses failing intermittently with S3 
Contributed by Steve Loughran.

Includes
-S3A glob scans don't bother trying to resolve symlinks
-stack traces don't get lost in getFileStatuses() when exceptions are wrapped
-debug level logging of what is up in Globber
-Contains HADOOP-13373. Add S3A implementation of FSMainOperationsBaseTest.
-ITestRestrictedReadAccess tests incomplete read access to files.

This adds a builder API for constructing globbers which other stores can use
so that they too can skip symlink resolution when not needed.

Change-Id: I23bcdb2783d6bd77cf168fdc165b1b4b334d91c7
 2019-10-01 18:11:05 +01:00
Sahil Takiar
13b427fc05 HDFS-14564: Add libhdfs APIs for readFully; add readFully to ByteBufferPositionedReadable (#963) Contributed by Sahil Takiar. 
Reviewed-by: Siyao Meng <smeng@cloudera.com>
 2019-09-27 12:25:03 -07:00
Zhankun Tang
2870668cfa Make upstream aware of 3.1.3 release. 2019-09-27 16:44:07 +08:00
Gabor Bota
587a8eeec8 HDFS-14874. Fix TestHDFSCLI and TestDFSShell test break because of logging change in mkdir (#1522). Contributed by Gabor Bota. 2019-09-26 10:15:13 +05:30
Steve Loughran
e346e3638c HADOOP-15691 Add PathCapabilities to FileSystem and FileContext. 
Contributed by Steve Loughran.

This complements the StreamCapabilities Interface by allowing applications to probe for a specific path on a specific instance of a FileSystem client
to offer a specific capability.

This is intended to allow applications to determine

* Whether a method is implemented before calling it and dealing with UnsupportedOperationException.
* Whether a specific feature is believed to be available in the remote store.

As well as a common set of capabilities defined in CommonPathCapabilities,
file systems are free to add their own capabilities, prefixed with
 fs. + schema + .

The plan is to identify and document more capabilities -and for file systems which add new features, for a declaration of the availability of the feature to always be available.

Note

* The remote store is not expected to be checked for the feature;
  It is more a check of client API and the client's configuration/knowledge
  of the state of the remote system.
* Permissions are not checked.

Change-Id: I80bfebe94f4a8bdad8f3ac055495735b824968f5
 2019-09-25 12:16:41 +01:00
Szilard Nemeth
c72457787d YARN-6715. Fix documentation about NodeHealthScriptRunner. Contributed by Peter Bacsko 2019-09-25 10:36:22 +02:00
Prabhu Joseph
3f89084ac7
HDFS-14845. Ignore AuthenticationFilterInitializer for HttpFSServerWebServer and honor hadoop.http.authentication configs. 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2019-09-25 10:19:30 +09:00
Gabor Bota
aa664d7259
HADOOP-16138. hadoop fs mkdir / of nonexistent abfs container raises NPE (#1302). Contributed by Gabor Bota. 
Change-Id: I2f637865c871e400b95fe7ddaa24bf99fa192023
 2019-09-23 13:29:01 +02:00
Vinayakumar B
07c81e9bfc
HADOOP-16558. [COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes (#1494). Contributed by Vinayakumar B. 2019-09-23 12:37:47 +05:30
Rohith Sharma K S
4c0a7a9e13 Make upstream aware of 3.2.1 release. 2019-09-23 06:20:54 +00:00
Rakesh Radhakrishnan
659c88801d HDFS-14818. Check native pmdk lib by 'hadoop checknative' command. Contributed by Feilong He. 2019-09-22 22:02:54 +05:30
Yuval Degani
dd0834696a HADOOP-16581. Revise ValueQueue to correctly replenish queues that go below the watermark (#1463) 
In the existing implementation, the ValueQueue::getAtMost() method will only trigger a refill on a key queue if it has gone empty, instead of triggering a refill when it has gone below the watermark. Revise the test suite to correctly verify this behavior.
 2019-09-20 09:55:48 -07:00
Vinayakumar B
1654497f98
HADOOP-16557. [pb-upgrade] Upgrade protobuf.version to 3.7.1 (#1432) 
HADOOP-16557. [pb-upgrade] Upgrade protobuf.version to 3.7.1. Contributed by Vinayakumar B.
 2019-09-20 16:08:30 +05:30
Takanobu Asanuma
298f07abce HADOOP-16069. Support configure ZK_DTSM_ZK_KERBEROS_PRINCIPAL in ZKDelegationTokenSecretManager using principal with Schema /_HOST. Contributed by luhuachao. 2019-09-20 13:55:34 +09:00
Kihwal Lee
d4205dce17 HADOOP-16582. LocalFileSystem's mkdirs() does not work as expected under viewfs. Contributed by Kihwal Lee 2019-09-19 08:23:35 -05:00
Weiwei Yang
f8c14326ee YARN-2255. YARN Audit logging not added to log4j.properties. Contributed by Aihua Xu. 2019-09-18 09:12:28 +08:00
Sahil Takiar
3cf6e4272f HDFS-14846: libhdfs tests are failing on trunk due to jni usage bugs 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-09-17 15:51:39 -07:00
Sahil Takiar
55ce454ce4
HADOOP-16371: Option to disable GCM for SSL connections when running on Java 8. 
Contributed by Sahil Takiar.

This moves the SSLSocketFactoryEx class from hadoop-azure into hadoop-common
as the DelegatingSSLSocketFactory and binds the S3A connector to it so that
it can avoid using those HTTPS algorithms which are underperformant on Java 8.

Change-Id: Ie9e6ac24deac1aa05e136e08899620efa7d22abd
 2019-09-17 11:32:03 +01:00
Steve Loughran
9221704f85
HADOOP-16490. Avoid/handle cached 404s during S3A file creation. 
Contributed by Steve Loughran.

This patch avoids issuing any HEAD path request when creating a file with overwrite=true,
so 404s will not end up in the S3 load balancers unless someone calls getFileStatus/exists/isFile
in their own code.

The Hadoop FsShell CommandWithDestination class is modified to not register uncreated files
for deleteOnExit(), because that calls exists() and so can place the 404 in the cache, even
after S3A is patched to not do it itself.

Because S3Guard knows when a file should be present, it adds a special FileNotFound retry policy
independently configurable from other retry policies; it is also exponential, but with
different parameters. This is because every HEAD request will refresh any 404 cached in
the S3 Load Balancers. It's not enough to retry: we have to have a suitable gap between
attempts to (hopefully) ensure any cached entry wil be gone.

The options and values are:

fs.s3a.s3guard.consistency.retry.interval: 2s
fs.s3a.s3guard.consistency.retry.limit: 7

The S3A copy() method used during rename() raises a RemoteFileChangedException which is not caught
so not downgraded to false. Thus: when a rename is unrecoverable, this fact is propagated.

Copy operations without S3Guard lack the confidence that the file exists, so don't retry the same way:
it will fail fast with a different error message. However, because create(path, overwrite=false) no
longer does HEAD path, we can at least be confident that S3A itself is not creating those cached
404 markers.

Change-Id: Ia7807faad8b9a8546836cb19f816cccf17cca26d
 2019-09-11 16:46:25 +01:00
Daisuke Kobayashi
bc2d3a71d6 HADOOP-16549. Remove Unsupported SSL/TLS Versions from Docs/Properties. Contributed by Daisuke Kobayashi. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
 2019-09-10 10:51:47 +08:00
Sneha Vijayarajan
147f98629c
HADOOP-16438. ADLS Gen1 OpenSSL config control. 
Contributed by Sneha Vijayarajan.

Change-Id: Ib79ea6b4a90ad068033e175f3f59c5185868872d
 2019-09-09 17:09:32 +01:00
Jungtaek Lim (HeartSaVioR)
bb0b922a71
HADOOP-16255. Add ChecksumFs.rename(path, path, boolean) 
Contributed by Jungtaek Lim

Change-Id: If00a4d7d30456c08eb2b0f7e2b242197bc4ee05d
 2019-09-06 21:53:00 +01:00
Erik Krogen
a23417533e HADOOP-16531. Log more timing information for slow RPCs. Contributed by Chen Zhang. 2019-09-06 10:28:21 -07:00
Erik Krogen
c92a3e94d8 HADOOP-15565. Add an inner FS cache to ViewFileSystem, separate from the global cache, to avoid file system leaks. Contributed by Jinglun. 2019-09-06 10:22:28 -07:00
Steve Loughran
511df1e837 HADOOP-16430. S3AFilesystem.delete to incrementally update s3guard with deletions 
Contributed by Steve Loughran.

This overlaps the scanning for directory entries with batched calls to S3 DELETE and updates of the S3Guard tables.
It also uses S3Guard to list the files to delete, so find newly created files even when S3 listings are not use consistent.

For path which the client considers S3Guard to be authoritative, we also do a recursive LIST of the store and delete files; this is to find unindexed files and do guarantee that the delete(path, true) call really does delete everything underneath.

Change-Id: Ice2f6e940c506e0b3a78fa534a99721b1698708e
 2019-09-05 14:25:15 +01:00
Erik Krogen
337e9b794d HADOOP-16268. Allow StandbyException to be thrown as CallQueueOverflowException when RPC call queue is filled. Contributed by CR Hota. 2019-09-04 08:22:02 -07:00
Surendra Singh Lilhore
5ff76cb8bc HDFS-14630. Configuration.getTimeDurationHelper() should not log time unit warning in info log. Contributed by hemanthboyina. 2019-09-03 12:37:09 +05:30
Stephen O'Donnell
915cbc91c0 HDFS-14706. Checksums are not checked if block meta file is less than 7 bytes. Contributed by Stephen O'Donnell. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-09-02 09:47:04 -07:00
Surendra Singh Lilhore
29bd6f3fc3 HDFS-8631. WebHDFS : Support setQuota. Contributed by Chao Sun. 2019-08-28 23:58:23 +05:30
Akira Ajisaka
55cc115878 HADOOP-16527. Add a whitelist of endpoints to skip Kerberos authentication (#1336) Contributed by Akira Ajisaka. 2019-08-28 14:28:41 +09:00
Steve Loughran
61b2df2331
HADOOP-16470. Make last AWS credential provider in default auth chain EC2ContainerCredentialsProviderWrapper. 
Contributed by Steve Loughran.

Contains HADOOP-16471. Restore (documented) fs.s3a.SharedInstanceProfileCredentialsProvider.

Change-Id: I06b99b57459cac80bf743c5c54f04e59bb54c2f8
 2019-08-22 17:27:56 +01:00
Akira Ajisaka
30ce8546f1
HADOOP-16496. Apply HDDS-1870 (ConcurrentModification at PrometheusMetricsSink) to Hadoop common. 
This closes #1317

Reviewed-by: Bharat Viswanadham <bharat@apache.org>
 2019-08-21 10:10:11 +09:00
David Mollitor
a707bb7c1b HADOOP-15246. SpanReceiverInfo - Prefer ArrayList over LinkedList. Contributed by David Mollitor. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-08-18 18:44:35 -07:00
Wei-Chiu Chuang
971a4c8e83 HDFS-14523. Remove excess read lock for NetworkToplogy. Contributed by Wu Weiwei. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Chen Liang <cliang@apache.org>
 2019-08-16 17:19:58 -07:00
Erik Krogen
e356e4f4b7 HADOOP-16391 Add a prefix to the metric names for MutableRatesWithAggregation used for deferred RPC metrics to avoid collision with non-deferred metrics. Contributed by Bilwa S T. 2019-08-16 09:01:44 -07:00
Wei-Chiu Chuang
5882cf94ea HADOOP-16504. Increase ipc.server.listen.queue.size default from 128 to 256. Contributed by Lisheng Sun. 2019-08-15 15:20:54 -07:00
Akira Ajisaka
0f8add8a60
HADOOP-16495. Fix invalid metric types in PrometheusMetricsSink (#1244) 2019-08-14 12:24:03 +09:00
Masatake Iwasaki
da0006fe04 HDFS-14423. Percent (%) and plus (+) characters no longer work in WebHDFS. 
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
 2019-08-14 08:39:40 +09:00
Inigo Goiri
6b4564f1d5 HADOOP-16453. Update how exceptions are handled in NetUtils. Contributed by Lisheng Sun. 2019-08-11 20:34:36 -07:00
Steve Loughran
e25a5c2eab HADOOP-16499. S3A retry policy to be exponential (#1246). Contributed by Steve Loughran. 2019-08-09 15:52:37 +02:00
Szilard Nemeth
df30d8ea09 YARN-9727: Allowed Origin pattern is discouraged if regex contains *. Contributed by Zoltan Siegl 2019-08-09 09:34:23 +02:00
Yiqun Lin
a5bb1e8ee8 HDFS-14313. Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du. Contributed by Lisheng Sun. 2019-08-07 10:18:11 +08:00
Wei-Chiu Chuang
8cef9f89f4 HDFS-14652. Addendum: HealthMonitor connection retry times should be configurable. Contributed by Chen Zhang. 2019-08-06 15:24:11 -07:00
Eric Yang
22430c10e2 HADOOP-16457. Fixed Kerberos activation in ServiceAuthorizationManager. 
Contributed by Prabhu Joseph
 2019-08-06 17:04:17 -04:00
Jianfei Jiang
71aad60e51
HDFS-14691. Wrong usage hint for hadoop fs command "test". 
Contributed by Jianfei Jiang.

Change-Id: I9f5e89721ff210641375fbf42a70043f0d74458e
 2019-08-05 13:08:47 +01:00