hadoop

Author	SHA1	Message	Date
Szilard Nemeth	8c9173c87c	HADOOP-16510. [hadoop-common] Fix order of actual and expected expression in assert statements. Contributed by Adam Antal	2019-10-31 14:35:20 +01:00
Jonathan Hung	e6137d0e50	Make upstream aware of 2.10.0 release	2019-10-30 20:58:32 -07:00
Steve Loughran	bd498bac86	HADOOP-16669. TestRawLocalFileSystemContract.testPermission fails if no native library. Contributed by Steve Loughran. Change-Id: I622e36f17a63c775c5e33e93b8b26e461d0bc451	2019-10-29 17:09:26 +00:00
Fei Hui	7be5508d9b	HDFS-14923. Remove dead code from HealthMonitor. Contributed by Fei Hui. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2019-10-25 15:06:16 -07:00
hongdongdong	2eba262472	HADOOP-16615. Add password check for credential provider, Contributed by hongdongdong. Change-Id: Iaac01bc8594860064a80c822a0e47981243ab7e1	2019-10-24 18:48:27 +01:00
Ayush Saxena	2ae4b33d48	HADOOP-16662. Remove unnecessary InnerNode check in NetworkTopology#add(). Contributed by Lisheng Sun.	2019-10-19 11:05:54 +05:30
Mate Szalay-Beko	6d92aa7c30	HADOOP-16579. Upgrade to Curator 4.2.0 and ZooKeeper 3.5.5 (#1656 ). Contributed by Norbert Kalmár, Mate Szalay-Beko * HADOOP-16579 - Upgrade to Apache Curator 4.2.0 and ZooKeeper 3.5.5 - Add a static initializer for the unit tests using ZooKeeper to enable the four-letter-words diagnostic telnet commands. (this is an interface that become disabled by default, so to keep the ZooKeeper 3.4.x behavior we enabled it for the tests) - Also fix ZKFailoverController to look for relevant fail-over ActiveAttempt records. The new ZooKeeper seems to respond quicker during the fail-over tests than the ZooKeeper, so we made sure to catch all the relevant records by adding a new parameter to ZKFailoverontroller.waitForActiveAttempt(). Co-authored-by: Norbert Kalmár <nkalmar@cloudera.com>	2019-10-18 13:26:20 -07:00
Yuming Wang	3d41f33018	HADOOP-16152. Upgrade Eclipse Jetty version to 9.4.x. Contributed by Yuming Wang, Siyao Meng. Co-authored-By: Siyao Meng <smeng@cloudera.com> Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2019-10-18 08:25:02 -07:00
Szilard Nemeth	c79a5f2d99	HADOOP-16580. Disable retry of FailoverOnNetworkExceptionRetry in case of AccessControlException. Contributed by Adam Antal	2019-10-16 13:29:06 +02:00
Wei-Chiu Chuang	c39e9fc9a3	HADOOP-15169. "hadoop.ssl.enabled.protocols" should be considered in httpserver2. Contributed by Brahma Reddy Battula, Wei-Chiu Chuang. Reviewed-by: Xiaoyu Yao <xyao@apache.org> Co-authored-By: Brahma Reddy Battula <brahma@apache.org>	2019-10-15 13:55:23 -07:00
Steve Loughran	dee9e97075	Revert "HADOOP-15870. S3AInputStream.remainingInFile should use nextReadPos." This reverts commit `7a4b3d42c4`. The patch broke TestRouterWebHDFSContractSeek as it turns out that WebHDFSInputStream.available() is always 0.	2019-10-14 16:56:50 +01:00
Ayush Saxena	44bc095998	HADOOP-13907. Fix TestWebDelegationToken#testKerberosDelegationTokenAuthenticator on Windows. Contributed by Kitti Nanasi.	2019-10-12 13:16:51 +05:30
lqjacklee	7a4b3d42c4	HADOOP-15870. S3AInputStream.remainingInFile should use nextReadPos. Contributed by lqjacklee. Change-Id: I32bb00a683102e7ff8ff8ce0b8d9c3195ca7381c	2019-10-10 21:58:42 +01:00
Jonathan Hung	5462d2176f	YARN-9760. Support configuring application priorities on a workflow level. Contributed by Varun Saxena	2019-10-08 11:20:13 -07:00
Steve Loughran	1921e94292	HADOOP-16458. LocatedFileStatusFetcher.getFileStatuses failing intermittently with S3 Contributed by Steve Loughran. Includes -S3A glob scans don't bother trying to resolve symlinks -stack traces don't get lost in getFileStatuses() when exceptions are wrapped -debug level logging of what is up in Globber -Contains HADOOP-13373. Add S3A implementation of FSMainOperationsBaseTest. -ITestRestrictedReadAccess tests incomplete read access to files. This adds a builder API for constructing globbers which other stores can use so that they too can skip symlink resolution when not needed. Change-Id: I23bcdb2783d6bd77cf168fdc165b1b4b334d91c7	2019-10-01 18:11:05 +01:00
Sahil Takiar	13b427fc05	HDFS-14564: Add libhdfs APIs for readFully; add readFully to ByteBufferPositionedReadable (#963 ) Contributed by Sahil Takiar. Reviewed-by: Siyao Meng <smeng@cloudera.com>	2019-09-27 12:25:03 -07:00
Zhankun Tang	2870668cfa	Make upstream aware of 3.1.3 release.	2019-09-27 16:44:07 +08:00
Gabor Bota	587a8eeec8	HDFS-14874. Fix TestHDFSCLI and TestDFSShell test break because of logging change in mkdir (#1522 ). Contributed by Gabor Bota.	2019-09-26 10:15:13 +05:30
Steve Loughran	e346e3638c	HADOOP-15691 Add PathCapabilities to FileSystem and FileContext. Contributed by Steve Loughran. This complements the StreamCapabilities Interface by allowing applications to probe for a specific path on a specific instance of a FileSystem client to offer a specific capability. This is intended to allow applications to determine * Whether a method is implemented before calling it and dealing with UnsupportedOperationException. * Whether a specific feature is believed to be available in the remote store. As well as a common set of capabilities defined in CommonPathCapabilities, file systems are free to add their own capabilities, prefixed with fs. + schema + . The plan is to identify and document more capabilities -and for file systems which add new features, for a declaration of the availability of the feature to always be available. Note * The remote store is not expected to be checked for the feature; It is more a check of client API and the client's configuration/knowledge of the state of the remote system. * Permissions are not checked. Change-Id: I80bfebe94f4a8bdad8f3ac055495735b824968f5	2019-09-25 12:16:41 +01:00
Szilard Nemeth	c72457787d	YARN-6715. Fix documentation about NodeHealthScriptRunner. Contributed by Peter Bacsko	2019-09-25 10:36:22 +02:00
Prabhu Joseph	3f89084ac7	HDFS-14845. Ignore AuthenticationFilterInitializer for HttpFSServerWebServer and honor hadoop.http.authentication configs. Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2019-09-25 10:19:30 +09:00
Gabor Bota	aa664d7259	HADOOP-16138. hadoop fs mkdir / of nonexistent abfs container raises NPE (#1302 ). Contributed by Gabor Bota. Change-Id: I2f637865c871e400b95fe7ddaa24bf99fa192023	2019-09-23 13:29:01 +02:00
Vinayakumar B	07c81e9bfc	HADOOP-16558. [COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes (#1494 ). Contributed by Vinayakumar B.	2019-09-23 12:37:47 +05:30
Rohith Sharma K S	4c0a7a9e13	Make upstream aware of 3.2.1 release.	2019-09-23 06:20:54 +00:00
Rakesh Radhakrishnan	659c88801d	HDFS-14818. Check native pmdk lib by 'hadoop checknative' command. Contributed by Feilong He.	2019-09-22 22:02:54 +05:30
Yuval Degani	dd0834696a	HADOOP-16581. Revise ValueQueue to correctly replenish queues that go below the watermark (#1463 ) In the existing implementation, the ValueQueue::getAtMost() method will only trigger a refill on a key queue if it has gone empty, instead of triggering a refill when it has gone below the watermark. Revise the test suite to correctly verify this behavior.	2019-09-20 09:55:48 -07:00
Vinayakumar B	1654497f98	HADOOP-16557. [pb-upgrade] Upgrade protobuf.version to 3.7.1 (#1432 ) HADOOP-16557. [pb-upgrade] Upgrade protobuf.version to 3.7.1. Contributed by Vinayakumar B.	2019-09-20 16:08:30 +05:30
Takanobu Asanuma	298f07abce	HADOOP-16069. Support configure ZK_DTSM_ZK_KERBEROS_PRINCIPAL in ZKDelegationTokenSecretManager using principal with Schema /_HOST. Contributed by luhuachao.	2019-09-20 13:55:34 +09:00
Kihwal Lee	d4205dce17	HADOOP-16582. LocalFileSystem's mkdirs() does not work as expected under viewfs. Contributed by Kihwal Lee	2019-09-19 08:23:35 -05:00
Weiwei Yang	f8c14326ee	YARN-2255. YARN Audit logging not added to log4j.properties. Contributed by Aihua Xu.	2019-09-18 09:12:28 +08:00
Sahil Takiar	3cf6e4272f	HDFS-14846: libhdfs tests are failing on trunk due to jni usage bugs Signed-off-by: Anu Engineer <aengineer@apache.org>	2019-09-17 15:51:39 -07:00
Sahil Takiar	55ce454ce4	HADOOP-16371: Option to disable GCM for SSL connections when running on Java 8. Contributed by Sahil Takiar. This moves the SSLSocketFactoryEx class from hadoop-azure into hadoop-common as the DelegatingSSLSocketFactory and binds the S3A connector to it so that it can avoid using those HTTPS algorithms which are underperformant on Java 8. Change-Id: Ie9e6ac24deac1aa05e136e08899620efa7d22abd	2019-09-17 11:32:03 +01:00
Steve Loughran	9221704f85	HADOOP-16490. Avoid/handle cached 404s during S3A file creation. Contributed by Steve Loughran. This patch avoids issuing any HEAD path request when creating a file with overwrite=true, so 404s will not end up in the S3 load balancers unless someone calls getFileStatus/exists/isFile in their own code. The Hadoop FsShell CommandWithDestination class is modified to not register uncreated files for deleteOnExit(), because that calls exists() and so can place the 404 in the cache, even after S3A is patched to not do it itself. Because S3Guard knows when a file should be present, it adds a special FileNotFound retry policy independently configurable from other retry policies; it is also exponential, but with different parameters. This is because every HEAD request will refresh any 404 cached in the S3 Load Balancers. It's not enough to retry: we have to have a suitable gap between attempts to (hopefully) ensure any cached entry wil be gone. The options and values are: fs.s3a.s3guard.consistency.retry.interval: 2s fs.s3a.s3guard.consistency.retry.limit: 7 The S3A copy() method used during rename() raises a RemoteFileChangedException which is not caught so not downgraded to false. Thus: when a rename is unrecoverable, this fact is propagated. Copy operations without S3Guard lack the confidence that the file exists, so don't retry the same way: it will fail fast with a different error message. However, because create(path, overwrite=false) no longer does HEAD path, we can at least be confident that S3A itself is not creating those cached 404 markers. Change-Id: Ia7807faad8b9a8546836cb19f816cccf17cca26d	2019-09-11 16:46:25 +01:00
Daisuke Kobayashi	bc2d3a71d6	HADOOP-16549. Remove Unsupported SSL/TLS Versions from Docs/Properties. Contributed by Daisuke Kobayashi. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org> Reviewed-by: Akira Ajisaka <aajisaka@apache.org>	2019-09-10 10:51:47 +08:00
Sneha Vijayarajan	147f98629c	HADOOP-16438. ADLS Gen1 OpenSSL config control. Contributed by Sneha Vijayarajan. Change-Id: Ib79ea6b4a90ad068033e175f3f59c5185868872d	2019-09-09 17:09:32 +01:00
Jungtaek Lim (HeartSaVioR)	bb0b922a71	HADOOP-16255. Add ChecksumFs.rename(path, path, boolean) Contributed by Jungtaek Lim Change-Id: If00a4d7d30456c08eb2b0f7e2b242197bc4ee05d	2019-09-06 21:53:00 +01:00
Erik Krogen	a23417533e	HADOOP-16531. Log more timing information for slow RPCs. Contributed by Chen Zhang.	2019-09-06 10:28:21 -07:00
Erik Krogen	c92a3e94d8	HADOOP-15565. Add an inner FS cache to ViewFileSystem, separate from the global cache, to avoid file system leaks. Contributed by Jinglun.	2019-09-06 10:22:28 -07:00
Steve Loughran	511df1e837	HADOOP-16430. S3AFilesystem.delete to incrementally update s3guard with deletions Contributed by Steve Loughran. This overlaps the scanning for directory entries with batched calls to S3 DELETE and updates of the S3Guard tables. It also uses S3Guard to list the files to delete, so find newly created files even when S3 listings are not use consistent. For path which the client considers S3Guard to be authoritative, we also do a recursive LIST of the store and delete files; this is to find unindexed files and do guarantee that the delete(path, true) call really does delete everything underneath. Change-Id: Ice2f6e940c506e0b3a78fa534a99721b1698708e	2019-09-05 14:25:15 +01:00
Erik Krogen	337e9b794d	HADOOP-16268. Allow StandbyException to be thrown as CallQueueOverflowException when RPC call queue is filled. Contributed by CR Hota.	2019-09-04 08:22:02 -07:00
Surendra Singh Lilhore	5ff76cb8bc	HDFS-14630. Configuration.getTimeDurationHelper() should not log time unit warning in info log. Contributed by hemanthboyina.	2019-09-03 12:37:09 +05:30
Stephen O'Donnell	915cbc91c0	HDFS-14706. Checksums are not checked if block meta file is less than 7 bytes. Contributed by Stephen O'Donnell. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2019-09-02 09:47:04 -07:00
Surendra Singh Lilhore	29bd6f3fc3	HDFS-8631. WebHDFS : Support setQuota. Contributed by Chao Sun.	2019-08-28 23:58:23 +05:30
Akira Ajisaka	55cc115878	HADOOP-16527. Add a whitelist of endpoints to skip Kerberos authentication (#1336 ) Contributed by Akira Ajisaka.	2019-08-28 14:28:41 +09:00
Steve Loughran	61b2df2331	HADOOP-16470. Make last AWS credential provider in default auth chain EC2ContainerCredentialsProviderWrapper. Contributed by Steve Loughran. Contains HADOOP-16471. Restore (documented) fs.s3a.SharedInstanceProfileCredentialsProvider. Change-Id: I06b99b57459cac80bf743c5c54f04e59bb54c2f8	2019-08-22 17:27:56 +01:00
Akira Ajisaka	30ce8546f1	HADOOP-16496. Apply HDDS-1870 (ConcurrentModification at PrometheusMetricsSink) to Hadoop common. This closes #1317 Reviewed-by: Bharat Viswanadham <bharat@apache.org>	2019-08-21 10:10:11 +09:00
David Mollitor	a707bb7c1b	HADOOP-15246. SpanReceiverInfo - Prefer ArrayList over LinkedList. Contributed by David Mollitor. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2019-08-18 18:44:35 -07:00
Wei-Chiu Chuang	971a4c8e83	HDFS-14523. Remove excess read lock for NetworkToplogy. Contributed by Wu Weiwei. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org> Reviewed-by: Chen Liang <cliang@apache.org>	2019-08-16 17:19:58 -07:00
Erik Krogen	e356e4f4b7	HADOOP-16391 Add a prefix to the metric names for MutableRatesWithAggregation used for deferred RPC metrics to avoid collision with non-deferred metrics. Contributed by Bilwa S T.	2019-08-16 09:01:44 -07:00
Wei-Chiu Chuang	5882cf94ea	HADOOP-16504. Increase ipc.server.listen.queue.size default from 128 to 256. Contributed by Lisheng Sun.	2019-08-15 15:20:54 -07:00
Akira Ajisaka	0f8add8a60	HADOOP-16495. Fix invalid metric types in PrometheusMetricsSink (#1244 )	2019-08-14 12:24:03 +09:00
Masatake Iwasaki	da0006fe04	HDFS-14423. Percent (%) and plus (+) characters no longer work in WebHDFS. Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>	2019-08-14 08:39:40 +09:00
Inigo Goiri	6b4564f1d5	HADOOP-16453. Update how exceptions are handled in NetUtils. Contributed by Lisheng Sun.	2019-08-11 20:34:36 -07:00
Steve Loughran	e25a5c2eab	HADOOP-16499. S3A retry policy to be exponential (#1246 ). Contributed by Steve Loughran.	2019-08-09 15:52:37 +02:00
Szilard Nemeth	df30d8ea09	YARN-9727: Allowed Origin pattern is discouraged if regex contains *. Contributed by Zoltan Siegl	2019-08-09 09:34:23 +02:00
Yiqun Lin	a5bb1e8ee8	HDFS-14313. Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du. Contributed by Lisheng Sun.	2019-08-07 10:18:11 +08:00
Wei-Chiu Chuang	8cef9f89f4	HDFS-14652. Addendum: HealthMonitor connection retry times should be configurable. Contributed by Chen Zhang.	2019-08-06 15:24:11 -07:00
Eric Yang	22430c10e2	HADOOP-16457. Fixed Kerberos activation in ServiceAuthorizationManager. Contributed by Prabhu Joseph	2019-08-06 17:04:17 -04:00
Jianfei Jiang	71aad60e51	HDFS-14691. Wrong usage hint for hadoop fs command "test". Contributed by Jianfei Jiang. Change-Id: I9f5e89721ff210641375fbf42a70043f0d74458e	2019-08-05 13:08:47 +01:00
Wei-Chiu Chuang	61180f4656	HADOOP-15942. Change the logging level form DEBUG to ERROR for RuntimeErrorException in JMXJsonServlet. Contributed by Anuhan Torgonshar.	2019-08-02 14:58:24 -07:00
Wei-Chiu Chuang	797d14e816	Revert "HADOOP-16336. finish variable is unused in ZStandardCompressor. Contributed by cxorm." This reverts commit `076618677d`.	2019-08-02 08:25:41 -07:00
Wei-Chiu Chuang	e872ceb810	HADOOP-15865. ConcurrentModificationException in Configuration.overlay() method. Contributed by Oleksandr Shevchenko.	2019-08-01 19:56:51 -07:00
Wei-Chiu Chuang	d086d058d8	HDFS-14652. HealthMonitor connection retry times should be configurable. Contributed by Chen Zhang.	2019-08-01 16:13:10 -07:00
Wei-Chiu Chuang	b94eba9f11	HADOOP-12282. Connection thread's name should be updated after address changing is detected. Contributed by Lisheng Sun.	2019-08-01 15:50:43 -07:00
He Xiaoqiao	f86de6f76a	HDFS-13529. Fix default trash policy emptier trigger time correctly. Contributed by He Xiaoqiao. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2019-08-01 14:54:39 -07:00
Akira Ajisaka	8bda91d20a	HADOOP-16398. Exports Hadoop metrics to Prometheus (#1170 )	2019-07-31 10:11:36 -07:00
Wei-Chiu Chuang	0f2dad6679	HDFS-14569. Result of crypto -listZones is not formatted properly. Contributed by hemanthboyina.	2019-07-30 16:52:42 -07:00
Siyao Meng	c75f16db79	HADOOP-16452. Increase ipc.maximum.data.length default from 64MB to 128MB. Contributed by Siyao Meng. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2019-07-30 11:22:45 -07:00
Akira Ajisaka	cbfa3f3e98	HADOOP-16435. RpcMetrics should not retained forever. Contributed by Zoltan Haindrich.	2019-07-29 17:37:26 -07:00
Erik Krogen	62efb63006	HADOOP-16245. Restrict the effect of LdapGroupsMapping SSL configurations to avoid interfering with other SSL connections. Contributed by Erik Krogen.	2019-07-26 11:16:58 -07:00
Gopal V	aebac6d2d2	HADOOP-16461. Regression: FileSystem cache lock parses XML within the lock. Contributed by Gopal V. Change-Id: If6654f850e9c24ee0d9519a46fd6269b18e1a7a4	2019-07-26 11:32:13 +01:00
Steve Loughran	07530314c2	HADOOP-9844. NPE when trying to create an error message response of SASL RPC This closes #55 Change-Id: I10a20380565fa89762f4aa564b2f1c83b9aeecdc Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2019-07-26 17:53:18 +09:00
sunlisheng	a1251addff	HADOOP-16431. Remove useless log in IOUtils.java and ExceptionDiags.java. This closes #1091 Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2019-07-24 10:04:39 +09:00
Steve Loughran	4317d33232	HADOOP-16380. S3Guard to determine empty directory status for all non-root directories. Contributed by Steve Loughran and Gabor Bota. This * Asks S3Guard to determine the empty directory status. * Has S3A's root directory rm("/") command to always return false (as abfs does) * Documents that object stores MAY do this * Overloads ContractTestUtils.assertDeleted to let assertions declare that the source directory does not need to exist. This stops inconsistencies in directory listings failing a root test. It avoids a recent regression (HADOOP-16279) where if there was a tombstone above the first element found in a directory listing, the directory would be considered empty, when in fact there were child entries. That could downgrade an rm(path, recursive) to a no-op, while also confusing rename(src, dest), as dest could be mistaken for an empty directory and so permit the copy above it, rather than reject it "destination path exists and is not empty". Change-Id: I136a3d1a5a48a67e6155d790a40ff558d0d2c108	2019-07-23 14:52:03 +01:00
S O'Donnell	eb36b09cb7	HADOOP-16443. Improve help text for setfacl --set option. Contributed by S O'Donnell. Change-Id: I1da46c4c414a5d2b07ee15867508f0799440a413	2019-07-23 10:24:07 +01:00
Sean Mackrory	7f1b76ca35	HADOOP-13868. [s3a] New default for S3A multi-part configuration (#1125 )	2019-07-19 09:49:59 -06:00
Josh Rosen	d545f9c290	HADOOP-16437 documentation typo fix: fs.s3a.experimental.input.fadvise Fix fs.s3a.experimental.fadvise to fs.s3a.experimental.input.fadvise Contributed by: Josh Rosen	2019-07-18 23:19:38 +01:00
Gopal V	b4466a3b0a	HADOOP-16341. ShutDownHookManager: Regressed performance on Hook removals after HADOOP-15679 Contributed by Gopal V and Atilla Magyar. Change-Id: I066d5eece332a1673594de0f9b484443f95530ec	2019-07-17 13:50:02 +01:00
Steve Loughran	b15ef7dc3d	HADOOP-16384: S3A: Avoid inconsistencies between DDB and S3. Contributed by Steve Loughran Contains - HADOOP-16397. Hadoop S3Guard Prune command to support a -tombstone option. - HADOOP-16406. ITestDynamoDBMetadataStore.testProvisionTable times out intermittently This patch doesn't fix the underlying problem but it * changes some tests to clean up better * does a lot more in logging operations in against DDB, if enabled * adds an entry point to dump the state of the metastore and s3 tables (precursor to fsck) * adds a purge entry point to help clean up after a test run has got a store into a mess * s3guard prune command adds -tombstone option to only clear tombstones The outcome is that tests should pass consistently and if problems occur we have better diagnostics. Change-Id: I3eca3f5529d7f6fec398c0ff0472919f08f054eb	2019-07-12 13:02:25 +01:00
Chen Liang	8fb5ca3f40	HDFS-14611. Move handshake secret field from Token to BlockAccessToken. Contributed by Chen Liang.	2019-07-11 13:23:25 -07:00
Steve Loughran	6a3433bffd	HADOOP-16357. TeraSort Job failing on S3 DirectoryStagingCommitter: destination path exists. Contributed by Steve Loughran. This patch * changes the default for the staging committer to append, as we get for the classic FileOutputFormat committer * adds a check for the dest path being a file not a dir * adds tests for this * Changes AbstractCommitTerasortIT. to not use the simple parser, so fails if the file is present. Change-Id: Id53742958ed1cf321ff96c9063505d64f3254f53	2019-07-11 18:15:34 +01:00
Inigo Goiri	aa9f0850e8	HADOOP-16385. Namenode crashes with 'RedundancyMonitor thread received Runtime exception'. Contributed by Ayush Saxena.	2019-07-02 13:22:30 -07:00
Christopher Gregorian	129576f628	HDFS-14403. Cost-based extension to the RPC Fair Call Queue. Contributed by Christopher Gregorian.	2019-06-24 12:09:17 -07:00
Ajay Kumar	95c94dcca7	HADOOP-16350. Ability to tell HDFS client not to request KMS Information from NameNode. Ccontributed by Greg Senia, Ajay Kumar.	2019-06-24 11:59:07 -07:00
Ayush Saxena	b52fd05d42	HDFS-13404. Addendum: RBF: TestRouterWebHDFSContractAppend.testRenameFileBeingAppended fail. Contributed by Takanobu Asanuma.	2019-06-24 22:03:04 +05:30
Ayush Saxena	ba221dc50e	HDFS-14526. RBF: Update the document of RBF related metrics. Contributed by Takanobu Asanuma.	2019-06-24 22:03:04 +05:30
Takanobu Asanuma	559cb11551	HDFS-13404. RBF: TestRouterWebHDFSContractAppend.testRenameFileBeingAppended fails.	2019-06-24 22:03:03 +05:30
Surendra Singh Lilhore	f40e10b349	HDFS-14129. RBF: Create new policy provider for router. Contributed by Ranith Sardar.	2019-06-24 22:03:03 +05:30
Prabhu Joseph	371452e260	HADOOP-15989. Synchronized at CompositeService#removeService is not required. Contributed by Prabhu Joseph. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2019-06-21 17:05:13 -07:00
Wei-Chiu Chuang	840d02ca5b	HADOOP-14385. HttpExceptionUtils#validateResponse swallows exceptions. Contributed by Wei-Chiu Chuang. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2019-06-20 19:57:48 -07:00
Steve Loughran	e02eb24e0a	HADOOP-15183. S3Guard store becomes inconsistent after partial failure of rename. Contributed by Steve Loughran. Change-Id: I825b0bc36be960475d2d259b1cdab45ae1bb78eb	2019-06-20 09:56:40 +01:00
Inigo Goiri	3ab77d9bc9	HDFS-14201. Ability to disallow safemode NN to become active. Contributed by Xiao Liang and He Xiaoqiao.	2019-06-18 09:58:29 -07:00
Shweta Yakkali	dd4a7633ec	HADOOP-16156. [Clean-up] Remove NULL check before instanceof and fix checkstyle in InnerNodeImpl. Contributed by Shweta Yakkali. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2019-06-17 23:47:57 -07:00
Adam Antal	56c621c68f	HADOOP-15914. hadoop jar command has no help argument. Contributed by Adam Antal. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2019-06-17 22:56:00 -07:00
Wei-Chiu Chuang	10311c30b0	HADOOP-14807. should prevent the possibility of NPE about ReconfigurableBase.java. Contributed by hu xiaodong.	2019-06-17 19:55:56 -07:00
Wei-Chiu Chuang	1e92db5a1e	HDFS-11949. Add testcase for ensuring that FsShell cann't move file to the target directory that file exists. Contributed by legend.	2019-06-17 18:29:34 -07:00
Gabor Bota	f9cc9e1621	HADOOP-16279. S3Guard: Implement time-based (TTL) expiry for entries (and tombstones). Contributed by Gabor Bota. Change-Id: I73a2d2861901dedfe7a0e783b310fbb95e7c1af9	2019-06-16 17:05:01 +01:00
cxorm	076618677d	HADOOP-16336. finish variable is unused in ZStandardCompressor. Contributed by cxorm.	2019-06-14 20:05:20 -07:00
dineshchitlangia	c7554ffd5c	HADOOP-16373. Fix typo in FileSystemShell#test documentation (#968 )	2019-06-14 14:36:04 -07:00
Takanobu Asanuma	54f9f75a44	HADOOP-16369. Fix zstandard shortname misspelled as zts. Contributed by Jonathan Eagles.	2019-06-14 10:17:25 +09:00
Eric Yang	4ea6c2f457	HADOOP-16354. Enable AuthFilter as default for WebHDFS. Contributed by Prabhu Joseph	2019-06-11 18:41:08 -04:00
Wanqiang Ji	b417a4c854	MAPREDUCE-7214. Remove unused pieces related to `mapreduce.job.userlog.retain.hours` Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2019-06-11 18:40:35 +09:00
Shweta Yakkali	d6d95d2686	HDFS-14494. Move Server logging of StatedId inside receiveRequestState(). Contributed by Shweta Yakkali. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2019-06-10 10:23:18 -07:00
Erik Krogen	76b94c274f	HADOOP-16345. Fix a potential NPE when instantiating FairCallQueue metrics. Contributed by Erik Krogen.	2019-06-07 14:20:44 -07:00
Steve Loughran	4e38dafde4	HADOOP-15563. S3Guard to support creating on-demand DDB tables. Contributed by Steve Loughran Change-Id: I2262b5b9f52e42ded8ed6f50fd39756f96e77087	2019-06-07 18:26:10 +01:00
Huan-Ping Su	14552d19e3	HADOOP-16344. Make DurationInfo public unstable. Second attempt: imports fixed up. Contributed by Huan-Ping Su and Kai Xie. Change-Id: Ib2a85dd4b2d12b840692cc854cc53ddb3bdde7d5	2019-06-07 12:55:17 +01:00
Akira Ajisaka	3ea4f41d9f	MAPREDUCE-6794. Remove unused properties from TTConfig.java	2019-06-07 10:27:41 +09:00
Steve Loughran	1f244b4227	Revert "HADOOP-16344. Make DurationInfo public unstable." This reverts commit `829848ba2e`. Change-Id: Ied91250e191b2ba701a8fc697c78b3756ce76be8	2019-06-06 17:49:31 +01:00
Huan-Ping Su	829848ba2e	HADOOP-16344. Make DurationInfo public unstable. Contributed by Huan-Ping Su Change-Id: I64a94cf382f9db78b4ef49a3912f25f0d906af7c	2019-06-06 12:52:49 +01:00
Eric Yang	294695dd57	HADOOP-16314. Make sure all web end points are covered by the same authentication filter. Contributed by Prabhu Joseph	2019-06-05 18:55:13 -04:00
Sammi Chen	d1aad44490	HDFS-14356. Implement HDFS cache on SCM with native PMDK libs. Contributed by Feilong He.	2019-06-05 21:33:00 +08:00
Steve Loughran	309501c6fa	Revert "HADOOP-16050: s3a SSL connections should use OpenSSL" This reverts commit `b067f8acaa`. Change-Id: I584b050a56c0e6f70b11fa3f7db00d5ac46e7dd8	2019-06-05 13:54:55 +01:00
Steve Loughran	7724d8031b	Revert "HADOOP-16321: ITestS3ASSL+TestOpenSSLSocketFactory failing with java.lang.UnsatisfiedLinkErrors" This reverts commit `5906268f0d`.	2019-06-05 13:54:42 +01:00
Steve Loughran	827a84778a	HADOOP-16266. Add more fine-grained processing time metrics to the RPC layer -follow-on patch. This follow-on patch to HADOOP-16266 fixes up the problem where logs were being full of stack traces because the timeout passed down to select was in nanos, whereas the API expected millis. Contributed by Erik Krogen. Change-Id: I5c6e9ddf68127b1d7e0ca0e179d036eb9941e445	2019-06-04 15:56:47 +01:00
Shweta Yakkali	6f5a36c13c	HADOOP-13656. fs -expunge to take a filesystem. Contributed by Shweta. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2019-05-30 13:21:58 -07:00
Akira Ajisaka	afd844059c	HADOOP-16331. Fix ASF License check in pom.xml Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>	2019-05-29 17:25:13 +09:00
Akira Ajisaka	9f933e6446	HADOOP-16323. https everywhere in Maven settings.	2019-05-27 15:24:59 +09:00
Shweta	93d7630062	HDFS-14451. Incorrect header or version mismatch log message. Contributed by Shweta. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2019-05-24 18:42:46 +02:00
Christopher Gregorian	f96a2df38d	HADOOP-16266. Add more fine-grained processing time metrics to the RPC layer. Contributed by Christopher Gregorian.	2019-05-23 10:28:37 -07:00
Eric Yang	ea0b1d8fba	HADOOP-16287. Implement ProxyUserAuthenticationFilter for web protocol impersonation. Contributed by Prabhu Joseph	2019-05-23 11:36:32 -04:00
Akira Ajisaka	a771e2a638	HADOOP-12948. Remove the defunct startKdc profile from hadoop-common. Contributed by Wei-Chiu Chuang.	2019-05-23 13:59:42 +09:00
Sahil Takiar	5906268f0d	HADOOP-16321: ITestS3ASSL+TestOpenSSLSocketFactory failing with java.lang.UnsatisfiedLinkErrors	2019-05-21 11:30:45 -06:00
Ben Roling	a36274d699	HADOOP-16085. S3Guard: use object version or etags to protect against inconsistent read after replace/overwrite. Contributed by Ben Roling. S3Guard will now track the etag of uploaded files and, if an S3 bucket is versioned, the object version. You can then control how to react to a mismatch between the data in the DynamoDB table and that in the store: warn, fail, or, when using versions, return the original value. This adds two new columns to the table: etag and version. This is transparent to older S3A clients -but when such clients add/update data to the S3Guard table, they will not add these values. As a result, the etag/version checks will not work with files uploaded by older clients. For a consistent experience, upgrade all clients to use the latest hadoop version.	2019-05-19 22:29:54 +01:00
Alexis Daboville	4cb3da6ac7	HADOOP-16248. MutableQuantiles leak memory under heavy load. Contributed by Alexis Daboville,	2019-05-17 15:15:22 +01:00
Sahil Takiar	b067f8acaa	HADOOP-16050: s3a SSL connections should use OpenSSL (cherry picked from commit aebf229c175dfa19fff3b31e9e67596f6c6124fa)	2019-05-16 08:57:54 -06:00
David Mollitor	2713dcf6e9	HADOOP-16307. Intern User Name and Group Name in FileStatus. Author: David Mollitor	2019-05-16 16:02:07 +02:00
Bharat Viswanadham	d4c8858586	HADOOP-16247. NPE in FsUrlConnection. Contributed by Karthik Palanisamy.	2019-05-15 17:41:36 -07:00
Inigo Goiri	389e640f0c	HADOOP-16161. NetworkTopology#getWeightUsingNetworkLocation return unexpected result. Contributed by He Xiaoqiao.	2019-05-13 11:46:16 -07:00
Akira Ajisaka	f257497b0f	HADOOP-16299. [JDK 11] Build fails without specifying -Djavac.version=11 Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>	2019-05-09 14:49:46 +09:00
Prabhu Joseph	96dc5cedfe	HADOOP-16293. AuthenticationFilterInitializer doc has speudo instead of pseudo. Author: Prabhu Joseph	2019-05-08 10:18:20 +01:00
Peter Bacsko	713e8a27ae	HADOOP-16238. Add the possbility to set SO_REUSEADDR in IPC Server Listener. Contributed by Peter Bacsko. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2019-05-07 17:48:27 -07:00
Siyao Meng	93f2283a69	HADOOP-16289. Allow extra jsvc startup option in hadoop_start_secure_daemon in hadoop-functions.sh. Contributed by Siyao Meng. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2019-05-06 15:47:33 -07:00
Vinayakumar B	f1875b205e	HADOOP-16059. Use SASL Factories Cache to Improve Performance. Contributed by Ayush Saxena.	2019-05-03 11:22:14 +05:30
Giovanni Matteo Fumarola	7a3188d054	HADOOP-16282. Avoid FileStream to improve performance. Contributed by Ayush Saxena.	2019-05-02 12:58:42 -07:00
Sahil Takiar	4877f0aa51	HDFS-3246: pRead equivalent for direct read path (#597 ) HDFS-3246: pRead equivalent for direct read path Contributed by Sahil Takiar	2019-04-30 14:52:16 -07:00
Ben Roling	0af4011580	HADOOP-16221. S3Guard: add option to fail operation on metadata write failure.	2019-04-30 11:53:26 +01:00
Sean Mackrory	a703dae25e	HADOOP-16222. Fix new deprecations after guava 27.0 update in trunk. Contributed by Gabor Bota.	2019-04-24 10:39:00 -06:00
Anu Engineer	f4ab9370f5	HADOOP-16026:Replace incorrect use of system property user.name. Contributed by Dinesh Chitlangia.	2019-04-22 14:02:13 -07:00
Inigo Goiri	fb1c549139	HDFS-14374. Expose total number of delegation tokens in AbstractDelegationTokenSecretManager. Contributed by CR Hota.	2019-04-22 13:32:08 -07:00
Erik Krogen	1ddb48872f	HADOOP-16265. Fix bug causing Configuration#getTimeDuration to use incorrect units when the default value is used. Contributed by starphin.	2019-04-22 08:16:57 -07:00
Kenneth Yang	b1120d27ab	HADOOP-16249. Make CallerContext LimitedPrivate scope to Public. Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2019-04-16 11:18:21 +09:00
Sahil Takiar	2382f63fc0	HADOOP-14747. S3AInputStream to implement CanUnbuffer. Author: Sahil Takiar <stakiar@cloudera.com>	2019-04-12 18:12:02 -07:00
Chen Liang	626fec652b	HDFS-13699. Add DFSClient sending handshake token to DataNode, and allow DataNode overwrite downstream QOP. Contributed by Chen Liang.	2019-04-12 17:37:51 -07:00
Arpit Agarwal	87407553ef	HADOOP-16243. Change Log Level to trace in NetUtils.java. Contributed by chencan.	2019-04-10 13:21:04 -07:00
Giovanni Matteo Fumarola	813cee1a18	HDFS-14420. Fix typo in KeyShell console. Contributed by Hu Xiaodong.	2019-04-10 11:18:40 -07:00
Todd Lipcon	65deb1ac42	HADOOP-16179. hadoop-common pom should not depend on kerb-simplekdc The hadoop-common pom currently has a dependency on kerb-simplekdc. In fact, the only classes used from Kerby are in kerb-core and kerb-util (which is a transitive dependency frmo kerb-core). Depending on kerb-simplekdc pulls a bunch of other unnecessary classes into the hadoop-common classpath. This changes the hadoop-common pom to depend only on kerb-core. hadoop-minikdc already had the appropriate dependency on kerb-simplekdc so it continues to pull in what it needs. Signed-off-by: Todd Lipcon <todd@apache.org>	2019-04-10 08:49:46 -07:00
Akira Ajisaka	bb8dda2bf9	HADOOP-12890. Fix typo in AbstractService. Contributed by Gabor Liptak.	2019-04-08 15:26:12 +09:00
Akira Ajisaka	ab645b3caa	HADOOP-14635. Javadoc correction for AccessControlList#buildACL. Contributed by Yeliang Cang.	2019-04-08 15:18:45 +09:00
Akira Ajisaka	72f4b9cd68	HADOOP-15242. Fix typos in hadoop-functions.sh. Contributed by Ray Chiang.	2019-04-08 13:20:21 +09:00
David Mollitor	c90736350b	HADOOP-16208. Do Not Log InterruptedException in Client. Contributed by David Mollitor.	2019-04-04 16:15:57 +01:00
Inigo Goiri	7b5b783f66	HDFS-14327. Using FQDN instead of IP to access servers with DNS resolving. Contributed by Fengnan Li.	2019-04-03 16:11:13 -07:00
Siyao Meng	e62cbcbc83	HADOOP-16011. OsSecureRandom very slow compared to other SecureRandom implementations. Contributed by Siyao Meng. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2019-04-03 14:29:52 -07:00
Steve Loughran	366186d999	HADOOP-16233. S3AFileStatus to declare that isEncrypted() is always true (#685 ) This is needed to fix up some confusion about caching of job.addCache() handling of S3A paths; all parent dirs -the files are downloaded by the NM without using the DTs of the user submitting the job. This means that when you submit jobs to an EC2 cluster with lower IAM permissions than the user, cached resources don't get downloaded and the job doesn't start. Production code changes: * S3AFileStatus Adds "true" to the superclass's encrypted flag during construction. Tests * Base AbstractContractOpenTest can control whether zero byte files created in tests are encrypted. Not done via an XML attribute, just a subclass point. Thoughts? * Verify that the filecache considers paths to not have the permissions which trigger reduce-privilege downloads * And extend ITestDelegatedMRJob to test a completely different bucket (open street map), to verify that cached resources do get their tokens picked up Docs: * Advise FS developers to say all files are encrypted. It's otherwise harmless and it'll stop other people seeing impossible to debug error messages on app launch. Contributed by Steve Loughran. Change-Id: Ifaae4c9d735ccc5eafeebd2584b65daf2d4e5da3	2019-04-03 21:23:40 +01:00
Gabor Bota	d7979079ea	HADOOP-16210. Update guava to 27.0-jre in hadoop-project trunk. Contributed by Gabor Bota.	2019-04-03 12:59:39 -06:00
Sahil Takiar	3b0c5016b2	HDFS-14394: Add -std=c99 / -std=gnu99 to libhdfs compile flags Signed-off-by: Todd Lipcon <todd@apache.org>	2019-04-03 10:56:33 -07:00
Akira Ajisaka	aaaf856f4b	HADOOP-16226. new Path(String str) does not remove all the trailing slashes of str	2019-04-03 13:16:59 +09:00
Lokesh Jain	cf268114c9	HDFS-13960. hdfs dfs -checksum command should optionally show block size in output. Contributed by Lokesh Jain. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2019-04-02 12:24:55 -07:00
Steve Loughran	61d19110d4	HADOOP-16218. Findbugs warning of null param to non-nullable method in Configuration with Guava update. (#655 ) Change-Id: I461e518ce9a4730b91a8138ad55b39e9a4b0a4b8	2019-04-02 09:15:11 +01:00
Akira Ajisaka	ebd0d21538	HADOOP-16225. Fix links to the developer mailing lists in DownstreamDev.md. Contributed by Wanqiang Ji.	2019-04-02 10:51:47 +09:00
Xiaoyu Yao	f41f938b2e	HADOOP-16199. KMSLoadBlanceClientProvider does not select token correctly. Contributed by Xiaoyu Yao. This closes #642.	2019-03-28 21:55:31 -07:00
Gabor Bota	b5db238383	HADOOP-15999. S3Guard: Better support for out-of-band operations. Author: Gabor Bota	2019-03-28 15:59:25 +00:00
David Mollitor	d18d0859eb	HADOOP-16181. HadoopExecutors shutdown Cleanup. Author: David Mollitor <david.mollitor@cloudera.com>	2019-03-22 10:29:27 +00:00
David Mollitor	246ab77f28	HADOOP-16196. Path Parameterize Comparable. Author: David Mollitor <david.mollitor@cloudera.com>	2019-03-22 10:26:24 +00:00
Eric Yang	5f6e225166	YARN-9363. Replaced debug logging with SLF4J parameterized log message. Contributed by Prabhu Joseph	2019-03-18 13:57:18 -04:00
Eric Yang	5446e3cb8a	HADOOP-16167. Fixed Hadoop shell script for Ubuntu 18. Contributed by Daniel Templeton	2019-03-18 13:04:49 -04:00
Erik Krogen	8c95cb9d6b	HADOOP-16192. Fix CallQueue backoff bugs: perform backoff when add() is used and update backoff when refreshed.	2019-03-18 08:13:43 -07:00
Shweta Yakkali	2db38abffc	HDFS-14328. [Clean-up] Remove NULL check before instanceof in TestGSet (Contributed by Shweta Yakkali via Daniel Templeton) Change-Id: I5b9f0e66664714d7c5bbfa30492a09f770626711	2019-03-18 15:10:26 +01:00
Eric Yang	2064ca015d	YARN-9349. Changed logging to use slf4j api. Contributed by Prabhu Joseph	2019-03-15 19:20:59 -04:00
Ben Roling	6fa229891e	HADOOP-15625. S3A input stream to use etags/version number to detect changed source files. Author: Ben Roling <ben.roling@gmail.com> Initial patch from Brahma Reddy Battula.	2019-03-13 20:37:11 +00:00
Erik Krogen	66357574ae	HDFS-14346. Add better time precision to Configuration#getTimeDuration, allowing return unit and default unit to be specified independently. Contributed by Chao Sun.	2019-03-13 13:15:56 -07:00
Matt Foley	f74159c8fc	HADOOP-16166. TestRawLocalFileSystemContract fails with build Docker container running on Mac. Also provided similar fix for Windows.	2019-03-13 09:33:24 -07:00
Konstantin V Shvachko	2a54feabb2	HDFS-14347. [SBN Read] Restore a comment line mistakenly removed in ProtobufRpcEngine. Contributed by Fengnan Li.	2019-03-11 18:59:15 -07:00
Konstantin V Shvachko	4ad295a4f1	HDFS-14270.[SBN Read] Add trace level logging for stateId in RPC Server. Contributed by Shweta Yakkali.	2019-03-11 13:48:06 -07:00
Steve Loughran	0cbe9ad8c2	HADOOP-16109. Parquet reading S3AFileSystem causes EOF Nobody gets seek right. No matter how many times they think they have. Reproducible test from: Dave Christianson Fixed seek() logic: Steve Loughran	2019-03-09 16:00:34 +00:00
Praveen Krishna	2b94e51a8f	HADOOP-16114. NetUtils#canonicalizeHost gives different value for same host. Author: Praveen Krishna <praveenkrishna@tutanota.com>	2019-03-07 11:06:34 +00:00
Vivek Ratnavel Subramanian	a55fc36299	HDDS-1093. Configuration tab in OM/SCM ui is not displaying the correct values.	2019-03-06 17:43:57 -08:00
Sahil Takiar	618e009ac0	HDFS-14111. hdfsOpenFile on HDFS causes unnecessary IO from file offset 0. Contributed by Sahil Takiar. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2019-03-06 15:04:06 -08:00
Wei-Chiu Chuang	6192c1fe3b	Revert "HDFS-14111. hdfsOpenFile on HDFS causes unnecessary IO from file offset 0. Contributed by Sahil Takiar." This reverts commit `f5a4b43a49`.	2019-03-06 15:02:18 -08:00
Sahil Takiar	f5a4b43a49	HDFS-14111. hdfsOpenFile on HDFS causes unnecessary IO from file offset 0. Contributed by Sahil Takiar. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2019-03-06 14:58:45 -08:00
Eric Yang	3c5b7136e2	HADOOP-16150. Added concat method to ChecksumFS as unsupported operation. Contributed by Steve Loughran (cherry picked from commit 8b517e7ad670aa8ee0b73ce5a572f36ce63eabee)	2019-03-05 13:32:00 -05:00
Stephen O'Donnell	686c0141ef	HADOOP-16140. hadoop fs expunge to add -immediate option to purge trash immediately. Contributed by Stephen O'Donnell. Signed-off-by: Steve Loughran <stevel@apache.org>	2019-03-05 14:09:00 +00:00
Prabhu Joseph	e40e2d6ad5	YARN-7243. Moving logging APIs over to slf4j in hadoop-yarn-server-resourcemanager. Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2019-03-05 14:10:08 +09:00
coder-chenzhi	fe7551f21b	HADOOP-16162. Remove unused Job Summary Appender configurations from log4j.properties This closes #551 Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2019-03-05 13:41:33 +09:00
David Mollitor	9fcd89ab93	HADOOP-16148. Cleanup LineReader Unit Test. Contributed by David Mollitor. Signed-off-by: Steve Loughran <stevel@apache.org>	2019-03-04 23:08:12 +00:00
Ajay Kumar	0d61facd37	HADOOP-15889. Add hadoop.token configuration parameter to load tokens. Contributed by Íñigo Goiri	2019-02-28 10:34:28 -08:00
Eric Yang	feccd282fe	HADOOP-16107. Update ChecksumFileSystem createFile/openFile API to generate checksum. Contributed by Steve Loughran	2019-02-27 15:53:41 -05:00
Tsz Wo Nicholas Sze	9192f71e21	HADOOP-16127. In ipc.Client, put a new connection could happen after stop.	2019-02-26 15:14:21 -08:00
Abhishek Modi	52b2eab575	HADOOP-16093. Move DurationInfo from hadoop-aws to hadoop-common org.apache.hadoop.util. Contributed by Abhishek Modi	2019-02-26 17:10:41 +00:00
Surendra Singh Lilhore	59ba3552d3	HDFS-14299. ViewFs: Correct error message for read only operations. Contributed by hu xiaodong.	2019-02-26 12:40:32 +05:30
Konstantin V Shvachko	a6ab37192a	HDFS-14130. [SBN read] Make ZKFC ObserverNode aware. Contributed by xiangheng and Konstantin Shvachko.	2019-02-25 14:35:02 -08:00
Inigo Goiri	ba4e7bd192	HADOOP-16125. Support multiple bind users in LdapGroupsMapping. Contributed by Lukas Majercak.	2019-02-25 13:39:13 -08:00
Tsz Wo Nicholas Sze	0edb0c51dc	HADOOP-16126. ipc.Client.stop() may sleep too long to wait for all connections.	2019-02-25 13:15:28 -08:00
Yongjun Zhang	f7a27cdee4	HDFS-14118. Support using DNS to resolve nameservices to IP addresses. Contributed by Fengnan Li.	2019-02-23 09:35:36 -08:00
Daryn Sharp	a87e458432	HADOOP-15813. Enable more reliable SSL connection reuse. Contributed by Daryn Sharp. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2019-02-20 18:17:04 -08:00
Shweta Yakkali	371a6db59a	HDFS-14273. Fix checkstyle issues in BlockLocation's method javadoc (Contributed by Shweta Yakkali via Daniel Templeton) Change-Id: I546aa4a0fe7f83b53735acd9925f366b2f1a00e2	2019-02-20 15:36:53 -08:00
Akira Ajisaka	dabfeab785	YARN-9308. fairscheduler-statedump.log gets generated regardless of service again after the merge of HDFS-7240. Contributed by Wilfred Spiegelenburg.	2019-02-15 14:49:49 +09:00
Erik Krogen	64f28f9efa	HDFS-14162. [SBN read] Allow Balancer to work with Observer node. Add a new ProxyCombiner allowing for multiple related protocols to be combined. Allow AlignmentContext to be passed in NameNodeProxyFactory. Contributed by Erik Krogen.	2019-02-14 11:22:04 -08:00
Siyao Meng	fd026863b1	HDFS-14241. Provide feedback on successful renameSnapshot and deleteSnapshot. Contributed by Siyao Meng. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2019-02-13 19:48:02 -08:00
Chen Liang	024c87291c	HDFS-13617. Allow wrapping NN QOP into token in encrypted message. Contributed by Chen Liang	2019-02-13 12:40:31 -08:00
Vinayakumar B	00c5ffaee2	HADOOP-16108. Tail Follow Interval Should Allow To Specify The Sleep Interval To Save Unnecessary RPC's. Contributed by Ayush Saxena.	2019-02-13 16:44:32 +05:30
Yiqun Lin	7b11b404a3	HADOOP-16097. Provide proper documentation for FairCallQueue. Contributed by Erik Krogen.	2019-02-13 11:16:04 +08:00
Steve Loughran	668817a6ce	Revert "HADOOP-15954. ABFS: Enable owner and group conversion for MSI and login user using OAuth." (accidentally mixed in two patches) This reverts commit `fa8cd1bf28`.	2019-02-07 21:57:22 +00:00
Wangda Tan	308f3168fa	Make upstream aware of 3.1.2 release Change-Id: I397bc6ef75498726df4763bd07a8bf8fe1c38365	2019-02-05 14:03:18 -08:00
Da Zhou	fa8cd1bf28	HADOOP-15954. ABFS: Enable owner and group conversion for MSI and login user using OAuth. Contributed by Da Zhou and Junhua Gu.	2019-02-05 19:23:15 +00:00
Steve Loughran	f365957c63	HADOOP-15229. Add FileSystem builder-based openFile() API to match createFile(); S3A to implement S3 Select through this API. The new openFile() API is asynchronous, and implemented across FileSystem and FileContext. The MapReduce V2 inputs are moved to this API, and you can actually set must/may options to pass in. This is more useful for setting things like s3a seek policy than for S3 select, as the existing input format/record readers can't handle S3 select output where the stream is shorter than the file length, and splitting plain text is suboptimal. Future work is needed there. In the meantime, any/all filesystem connectors are now free to add their own filesystem-specific configuration parameters which can be set in jobs and used to set filesystem input stream options (seek policy, retry, encryption secrets, etc). Contributed by Steve Loughran	2019-02-05 11:51:02 +00:00
Steve Loughran	7f46d13dac	HADOOP-16079. Token.toString faulting if any token listed can't load. Contributed by Steve Loughran.	2019-02-01 14:31:47 +00:00
Inigo Goiri	bcc3a79f58	HADOOP-16084. Fix the comment for getClass in Configuration. Contributed by Fengnan Li.	2019-01-31 10:06:05 -08:00
Akira Ajisaka	1129288cf5	HADOOP-14178. Move Mockito up to version 2.23.4. Contributed by Akira Ajisaka and Masatake Iwasaki.	2019-01-29 18:29:56 -08:00
Hanisha Koneru	b3bc94ebfd	HDFS-14236. Lazy persist copy/ put fails with ViewFs.	2019-01-29 16:45:44 -08:00
Wei-Chiu Chuang	d1714c20e9	Revert "HDFS-14084. Need for more stats in DFSClient. Contributed by Pranay Singh." This reverts commit `1d523279da`.	2019-01-29 15:43:09 -08:00
Pranay Singh	1d523279da	HDFS-14084. Need for more stats in DFSClient. Contributed by Pranay Singh. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2019-01-25 09:02:57 -08:00
Eric Yang	0dd35e218f	HADOOP-15922. Fixed DelegationTokenAuthenticator URL decoding for doAs user. Contributed by He Xiaoqiao	2019-01-22 18:59:36 -05:00
Akira Ajisaka	a463cf75a0	HADOOP-15787. [JDK11] TestIPC.testRTEDuringConnectionSetup fails. Contributed by Zsolt Venczel.	2019-01-22 10:19:05 +09:00
Sunil G	2e2508b8e3	Make 3.2.0 aware to other branches	2019-01-21 21:24:51 +05:30
Surendra Singh Lilhore	dacc1a759e	HDFS-14175. EC: Native XOR decoder should reset the output buffer before using it. Contributed by Ayush Saxena.	2019-01-18 13:55:12 +05:30
Akira Ajisaka	01405df87f	HADOOP-15941. Addendum patch. Contributed by Takanobu Asanuma.	2019-01-16 10:28:08 +09:00
luhuachao	ccc43621ab	HADOOP-16019. ZKDelegationTokenSecretManager won't log exception message occured in function setJaasConfiguration. Contributed by luhuachao.	2019-01-15 13:15:18 +00:00
Steve Loughran	6d0bffe17e	HADOOP-14556. S3A to support Delegation Tokens. Contributed by Steve Loughran and Daryn Sharp.	2019-01-14 17:59:27 +00:00
Inigo Goiri	35fa3bd685	HADOOP-16045. Don't run TestDU on Windows. Contributed by Lukas Majercak.	2019-01-11 18:07:19 -08:00
Chen Liang	bf08f4abae	HADOOP-15481. Emit FairCallQueue stats as metrics. Contributed by Christopher Gregorian.	2019-01-11 14:01:23 -08:00
Chen Liang	01cb958af4	HADOOP-16013. DecayRpcScheduler decay thread should run as a daemon. Contributed by Erik Krogen.	2019-01-11 12:51:07 -08:00
Giovanni Matteo Fumarola	fb8932a727	HADOOP-16029. Consecutive StringBuilder.append can be reused. Contributed by Ayush Saxena.	2019-01-11 10:54:49 -08:00
Akira Ajisaka	d4ca907da6	HADOOP-16016. TestSSLFactory#testServerWeakCiphers fails on Java 1.8.0_191 or upper	2019-01-11 11:24:05 +09:00
Kihwal Lee	33c009a45d	HADOOP-15909. KeyProvider class should implement Closeable. Contributed by Kuhu Shukla.	2019-01-10 13:21:37 -06:00
Jason Lowe	c634589ab2	Revert "HDFS-14084. Need for more stats in DFSClient. Contributed by Pranay Singh." This reverts commit `ecdeaa7e6a`.	2019-01-09 17:24:58 -06:00
Takanobu Asanuma	709ddb10e5	HADOOP-15941. [JDK 11] Compilation failure: package com.sun.jndi.ldap is not visible.	2019-01-09 13:55:25 +09:00
shwetayakkali@cloudera.com	4ab5260b7e	HDFS-14132. Add BlockLocation.isStriped() to determine if block is replicated or Striped (Contributed by Shweta Yakkali via Daniel Templeton) Change-Id: I0ed8996a0bae2ad2c7d3513143195533f7191af8	2019-01-08 17:04:57 -08:00
Akira Ajisaka	7f78397036	Revert "HADOOP-14556. S3A to support Delegation Tokens." This reverts commit `d7152332b3`.	2019-01-08 14:51:30 +09:00
Steve Loughran	d7152332b3	HADOOP-14556. S3A to support Delegation Tokens. Contributed by Steve Loughran.	2019-01-07 13:18:03 +00:00
Eric Yang	d43af8b3db	HADOOP-15996. Improved Kerberos username mapping strategy in Hadoop. Contributed by Bolke de Bruin	2019-01-04 17:54:15 -05:00
Inigo Goiri	f4e18242bd	HADOOP-16028. Fix NetworkTopology chooseRandom function to support excluded nodes. Contributed by Sihai Ke.	2019-01-04 09:56:58 -08:00
Sunil G	51427cbdfb	HADOOP-15997. KMS client uses wrong UGI after HADOOP-14445. Contributed by Wei-Chiu Chuang.	2019-01-04 21:49:06 +05:30
Pranay Singh	ecdeaa7e6a	HDFS-14084. Need for more stats in DFSClient. Contributed by Pranay Singh. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2019-01-03 09:56:07 -08:00
Akira Ajisaka	57198bc511	HADOOP-15962. The buffer size is small when unpacking tar archives. Contributed by BELUGA BEHR.	2018-12-28 15:38:47 +09:00
Konstantin V Shvachko	bd2a59e508	HDFS-14170. [SBN read] Fix checkstyle warnings related to SBN reads. Contributed by Konstantin V Shvachko.	2018-12-24 09:39:20 -08:00
Chao Sun	fa8550337d	HDFS-14149. [SBN read] Fix annotations on new interfaces/classes for SBN reads. Contributed by Chao Sun.	2018-12-24 09:34:01 -08:00
Erik Krogen	ef3e1929dd	HDFS-14146. [SBN read] Handle exceptions from and prevent handler threads from blocking within internalQueueCall. Contributed by Chao Sun.	2018-12-24 09:34:01 -08:00
Chen Liang	cae279610b	HDFS-14138. [SBN read] Description errors in the comparison logic of transaction ID. Contributed by xiangheng.	2018-12-24 09:34:01 -08:00
Erik Krogen	b73fb70f97	HDFS-13873. [SBN read] ObserverNode should reject read requests when it is too far behind. Contributed by Konstantin Shvachko.	2018-12-24 09:34:01 -08:00
Chao Sun	cbc2f932eb	HDFS-14067. [SBN read] Allow manual failover between standby and observer. Contributed by Chao Sun.	2018-12-24 09:34:01 -08:00
Chao Sun	8c49135078	HDFS-13924. [SBN read] Handle BlockMissingException when reading from observer. Contributed by Chao Sun.	2018-12-24 09:34:01 -08:00
Chen Liang	a65bb97f5d	HDFS-13791. Limit logging frequency of edit tail related statements. Contributed by Erik Krogen.	2018-12-24 09:34:00 -08:00
Chen Liang	d2e85b0b6d	HDFS-13880. Add mechanism to allow certain RPC calls to bypass sync. Contributed by Chen Liang.	2018-12-24 09:34:00 -08:00
Chen Liang	66ec3a67e8	HDFS-13767. Add msync server implementation. Contributed by Chen Liang.	2018-12-24 09:34:00 -08:00
Plamen Jeliazkov	e27708c2da	HDFS-13665. [SBN read] Move RPC response serialization into Server.doResponse(). Contributed by Plamen Jeliazkov.	2018-12-24 09:34:00 -08:00
Chao Sun	64b7cf59bd	HDFS-12976. [SBN read] Introduce ObserverReadProxyProvider. Contributed by Chao Sun.	2018-12-24 09:33:59 -08:00
Erik Krogen	151c8ddbe4	HDFS-13608. [SBN read] Edit Tail Fast Path Part 2: Add ability for JournalNode to serve edits via RPC. Contributed by Erik Krogen.	2018-12-24 09:33:59 -08:00
Plamen Jeliazkov	e880660a20	HDFS-13399. [SBN read] Make Client field AlignmentContext non-static. Contributed by Plamen Jeliazkov.	2018-12-24 09:33:59 -08:00
Erik Krogen	091ad974cd	HDFS-13286. [SBN read] Add haadmin commands to transition between standby and observer. Contributed by Chao Sun.	2018-12-24 09:33:59 -08:00
Erik Krogen	9bf0696c73	HDFS-13331. [SBN read] Add lastSeenStateId to RpcRequestHeader. Contributed by Plamen Jeliazkov.	2018-12-24 09:33:59 -08:00
Plamen Jeliazkov	c9d73437e8	HDFS-12977. [SBN read] Add stateId to RPC headers. Contributed by Plamen Jeliazkov.	2018-12-24 09:33:59 -08:00
Yiqun Lin	feb2664ac4	HDFS-13946. Log longest FSN write/read lock held stack trace.	2018-12-22 23:09:59 +08:00
Surendra Singh Lilhore	763e96ea2b	HDFS-14166. Ls with -e option not giving the result in proper format. Contributed by Shubham Dewan.	2018-12-22 12:22:24 +05:30
Jason Lowe	3961690037	HADOOP-15973. Configuration: Included properties are not cached if resource is a stream. Contributed by Eric Payne	2018-12-20 09:29:48 -06:00
Takanobu Asanuma	d963575918	HDFS-13661. Ls command with e option fails when the filesystem is not HDFS.	2018-12-20 10:03:33 +09:00
Surendra Singh Lilhore	788e7473a4	HDFS-14096. [SPS] : Add Support for Storage Policy Satisfier in ViewFs. Contributed by Ayush Saxena.	2018-12-17 11:24:57 +05:30
Gabor Bota	1ea29b7385	HADOOP-16000. Remove TLSv1 and SSLv2Hello from the default value of hadoop.ssl.enabled.protocols Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2018-12-15 10:53:52 +09:00
Giovanni Matteo Fumarola	76efeacd5f	HADOOP-15995. Add ldap.bind.password.alias in LdapGroupsMapping to distinguish aliases when using multiple providers through CompositeGroupsMapping. Contributed by Lukas Majercak.	2018-12-12 11:10:24 -08:00
Steve Loughran	202926ac33	HADOOP-15808. Harden Token service loader use. Contributed by Steve Loughran.	2018-12-11 17:33:49 +00:00
Giovanni Matteo Fumarola	aa89492f29	Revert "HADOOP-15852. Refactor QuotaUsage. Contributed by Beluga Behr." This reverts commit `fb9deed41d`.	2018-12-04 12:57:28 -08:00
Steve Loughran	db2d8b01c6	HADOOP-15966. Hadoop Kerberos broken on macos as java.security.krb5.realm is reset. Contributed by Steve Loughran	2018-12-04 15:35:43 +00:00
Giovanni Matteo Fumarola	fb9deed41d	HADOOP-15852. Refactor QuotaUsage. Contributed by Beluga Behr.	2018-12-03 13:25:02 -08:00
Giovanni Matteo Fumarola	c9a3aa64dc	HADOOP-15950. Failover for LdapGroupsMapping. Contributed by Lukas Majercak.	2018-12-03 12:10:05 -08:00
Eric Yang	38ea3814bd	HADOOP-15922. Revert patch 004.	2018-11-30 12:49:46 -05:00
Akira Ajisaka	c9bfca217f	MAPREDUCE-7165. mapred-site.xml is misformatted in single node setup document. Contributed by Zhaohui Xin.	2018-11-30 13:13:23 +09:00
Steve Loughran	d0edd37269	HADOOP-15959. Revert "HADOOP-12751. While using kerberos Hadoop incorrectly assumes names with '@' to be non-simple" This reverts commit `829a2e4d27`.	2018-11-29 17:52:11 +00:00
Ewan Higgs	c1d24f8483	HDFS-13713. Add specification of Multipart Upload API to FS specification, with contract tests. Contributed by Ewan Higgs and Steve Loughran.	2018-11-29 15:12:17 +00:00
Eric Yang	a9d96948ed	HADOOP-15922. Fixed doAsUser decoding for DelegationTokenAuthenticationFilter. Contributed by He Xiaoqiao	2018-11-26 13:49:19 -05:00
Vinayakumar B	23b441c225	HADOOP-15948. Inconsistency in get and put syntax if filename/dirname contains space. Contributed by Ayush Saxena.	2018-11-26 18:11:01 +05:30
Márton Elek	6934a65402	HADOOP-15358. SFTPConnectionPool connections leakage. Contributed by Mikhail Pryakhin.	2018-11-23 09:48:26 +01:00
Akira Ajisaka	1a00b4e325	Add 2.9.2 release notes and changes documents.	2018-11-20 14:25:18 +09:00
Inigo Goiri	460a94a10f	HDFS-14045. Use different metrics in DataNode to better measure latency of heartbeat/blockReports/incrementalBlockReports of Active/Standby NN. Contributed by Jiandan Yang.	2018-11-15 10:58:57 -08:00
Surendra Singh Lilhore	75291e6d53	HADOOP-15869. BlockDecompressorStream#decompress should not return -1 in case of IOException. Contributed by Surendra Singh Lilhore	2018-11-13 20:22:58 +05:30
Steve Loughran	3c9d97b8f7	HADOOP-15110. Gauges are getting logged in exceptions from AutoRenewalThreadForUserCreds. Contributed by LiXin Ge.	2018-11-12 12:58:05 +00:00
Giovanni Matteo Fumarola	89b49167a5	HADOOP-15903. Allow HttpServer2 to discover resources in /static when symlinks are used. Contributed by Inigo Goiri.	2018-11-08 14:52:24 -08:00
Jason Lowe	0b6625a973	MAPREDUCE-7148. Fast fail jobs when exceeds dfs quota limitation. Contributed by Wang Yan	2018-11-07 08:20:49 -06:00
Jason Lowe	ab6aa4c726	YARN-8865. RMStateStore contains large number of expired RMDelegationToken. Contributed by Wilfred Spiegelenburg	2018-11-06 08:40:59 -06:00
Lars Francke	cb8d679c95	HADOOP-15687. Credentials class should allow access to aliases. Author: Lars Francke <lars.francke@gmail.com>	2018-11-03 16:21:29 +00:00
Giovanni Matteo Fumarola	44e37b4fd9	HADOOP-15885. Add base64 (urlString) support to DTUtil. Contributed by Inigo Goiri.	2018-11-02 10:54:12 -07:00
Steve Loughran	62d98ca92a	HADOOP-15855. Review hadoop credential doc, including object store details. Contributed by Steve Loughran.	2018-10-30 15:58:04 +00:00
Wei-Chiu Chuang	63e7134df6	Revert "HADOOP-15864. Job submitter / executor fail when SBN domain name can not resolved. Contributed by He Xiaoqiao." This reverts commit `fb2b72e6fc`.	2018-10-28 17:04:59 -07:00
Hrishikesh Gadre	bfb9adc2b9	HADOOP-9567. Provide auto-renewal for keytab based logins. Contributed by Hrishikesh Gadre, Gary Helmling and Harsh J. Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>	2018-10-27 08:59:47 -07:00
Wei-Chiu Chuang	fb2b72e6fc	HADOOP-15864. Job submitter / executor fail when SBN domain name can not resolved. Contributed by He Xiaoqiao.	2018-10-25 09:33:31 -07:00
Akira Ajisaka	8f97d6f2cd	HADOOP-15822. zstd compressor can fail with a small output buffer. Contributed by Jason Lowe.	2018-10-24 13:49:28 +09:00
Haibo Chen	823bb5dda8	MAPREDUCE-4669. MRAM web UI does not work with HTTPS. (Contributed by Robert Kanter)	2018-10-23 15:28:13 -07:00
Chen Liang	635786a511	HDFS-13566. Add configurable additional RPC listener to NameNode. Contributed by Chen Liang.	2018-10-23 14:53:45 -07:00
Inigo Goiri	dd268a64d3	Revert "HADOOP-15836. Review of AccessControlList. Contributed by BELUGA BEHR." This reverts commit `00254d7b8c`.	2018-10-23 09:23:03 -07:00
Steve Loughran	52cb766ad0	HADOOP-15866. Renamed HADOOP_SECURITY_GROUP_SHELL_COMMAND_TIMEOUT keys break compatibility. Contributed by Wei-Chiu Chuang.	2018-10-22 11:21:12 +01:00
Eric Yang	e2a9fa8448	HADOOP-15821. Move YARN Registry to Hadoop Registry. Contributed by Íñigo Goiri	2018-10-19 19:46:48 -04:00
Inigo Goiri	00254d7b8c	HADOOP-15836. Review of AccessControlList. Contributed by BELUGA BEHR.	2018-10-19 13:57:43 -07:00
Jitendra Pandey	d93d515af5	HDDS-651. Rename o3 to o3fs for Filesystem.	2018-10-17 14:19:17 -07:00
Jason Lowe	9abda83947	HADOOP-15859. ZStandardDecompressor.c mistakes a class for an instance. Contributed by Jason Lowe	2018-10-17 14:38:42 -05:00
Adam Antal	24dc068a36	HADOOP-11100. Support to configure ftpClient.setControlKeepAliveTimeout. Contributed by Adam Antal. Signed-off-by: Xiao Chen <xiao@apache.org>	2018-10-17 11:34:50 -07:00
Steve Loughran	41b3603b5b	HADOOP-15861. Move DelegationTokenIssuer to the right path. Contributed by Wei-Chiu Chuang.	2018-10-17 11:01:53 +01:00
Haibo Chen	c2288ac45b	YARN-8448. AM HTTPS Support for AM communication with RMWeb proxy. (Contributed by Robert Kanter)	2018-10-16 13:36:26 -07:00
Robert Kanter	7fe1a40a6b	HADOOP-15853. TestConfigurationDeprecation leaves behind a temp file, resulting in a license issue (ayushtkn via rkanter)	2018-10-15 15:02:37 -07:00
Xiao Chen	b6fc72a025	HADOOP-14445. Addendum: Use DelegationTokenIssuer to create KMS delegation tokens that can authenticate to all KMS instances.	2018-10-15 10:51:55 -07:00
Anu Engineer	3c1fe073d2	HDDS-445. Create a logger to print out all of the incoming requests. Contributed by Bharat Viswanadham.	2018-10-12 16:27:54 -07:00
Xiao Chen	5ec86b445c	HADOOP-14445. Use DelegationTokenIssuer to create KMS delegation tokens that can authenticate to all KMS instances. Contributed by Daryn Sharp, Xiao Chen, Rushabh S Shah.	2018-10-12 09:35:52 -07:00
Robert Kanter	d91d47bc73	HADOOP-15717. TGT renewal thread does not log IOException (snemeth via rkanter)	2018-10-11 15:35:44 -07:00
Xiao Chen	64f2b32d57	HADOOP-15676. Cleanup TestSSLHttpServer. Contributed by Szilard Nemeth.	2018-10-11 15:12:36 -07:00
Takanobu Asanuma	7b57f2f71f	HADOOP-15785. [JDK10] Javadoc build fails on JDK 10 in hadoop-common. Contributed by Dinesh Chitlangia.	2018-10-11 13:51:51 +09:00
Robert Kanter	f261c31937	HADOOP-15708. Reading values from Configuration before adding deprecations make it impossible to read value with deprecated key (zsiegl via rkanter)	2018-10-10 18:51:37 -07:00
Steve Loughran	cdc4350718	HADOOP-15839. Review + update cloud store sensitive keys in hadoop.security.sensitive-config-keys. Contributed by Steve Loughran.	2018-10-10 17:00:37 +01:00
Akira Ajisaka	6fa3feb577	HADOOP-15832. Upgrade BouncyCastle to 1.60. Contributed by Robert Kanter.	2018-10-10 10:16:57 +09:00
Eric Yang	bf04f19456	YARN-8845. Removed unused hadoop.registry.rm reference. Contributed by Íñigo Goiri	2018-10-09 15:54:45 -04:00
Takanobu Asanuma	9bbeb52486	HADOOP-15775. [JDK9] Add missing javax.activation-api dependency. Contributed by Akira Ajisaka.	2018-10-09 13:06:14 +09:00
Jason Lowe	f13e231025	HADOOP-15820. ZStandardDecompressor native code sets an integer field as a long. Contributed by Jason Lowe	2018-10-05 09:06:02 -05:00
Aaron Fabbri	046b8768af	HADOOP-15621 S3Guard: Implement time-based (TTL) expiry for Authoritative Directory Listing. Contributed by Gabor Bota	2018-10-02 21:22:49 -07:00
Akira Ajisaka	31fa9bb799	HADOOP-15767. [JDK10] Building native package on JDK10 fails due to missing javah. Contributed by Takanobu Asanuma.	2018-10-03 11:29:18 +09:00
Sunil G	58fa96b697	Changed version in trunk to 3.3.0-SNAPSHOT.	2018-10-02 22:41:41 +05:30
Ewan Higgs	6fab6886f6	HDFS-13936. Multipart upload to HDFS to support 0 byte upload. Contributed by Ewan Higgs.	2018-10-02 14:05:35 +02:00
Akira Ajisaka	5c8d90763c	HADOOP-15764. Addendum patch: Fix NPE in SecurityUtil.	2018-09-28 10:03:22 +09:00
Akira Ajisaka	c62b440421	HADOOP-15693. Typo in Secure Mode documentation. Contributed by Thomas Levine.	2018-09-26 10:30:43 +09:00
Akira Ajisaka	93b0f540ed	HADOOP-15783. [JDK10] TestSFTPFileSystem.testGetModifyTime fails.	2018-09-25 16:03:28 +09:00
Sunil G	d060cbea48	HDFS-13937. Multipart Uploader APIs to be marked as private/unstable in 3.2.0. Contributed by Steve Loughran.	2018-09-24 21:19:47 +05:30
Sean Mackrory	0def61482b	Merge branch 'HADOOP-15407' into trunk	2018-09-22 21:19:12 -06:00
Ewan Higgs	429a07e08c	HADOOP-15764. [JDK10] Migrate from sun.net.dns.ResolverConfiguration to the replacement. Contributed by Akira Ajisaka.	2018-09-20 15:13:55 +02:00
Ewan Higgs	3da94a36e2	HADOOP-15756. [JDK10] Migrate from sun.net.util.IPAddressUtil to the replacement. Contributed by Akira Ajisaka.	2018-09-20 14:53:21 +02:00
Steve Loughran	646874c326	HADOOP-15748. S3 listing inconsistency can raise NPE in globber. Contributed by Steve Loughran.	2018-09-20 13:04:52 +01:00
Vinayakumar B	7ad27e97f0	HADOOP-15736. Trash : Negative Value For Deletion Interval Leads To Abnormal Behaviour. Contributed by Ayush Saxena.	2018-09-20 09:31:35 +05:30
Chen Liang	a30b4f9e71	HADOOP-15726. Create utility to limit frequency of log statements. Contributed by Erik Krogen.	2018-09-19 13:22:37 -07:00
Jason Lowe	e71f61ecb8	HADOOP-15755. StringUtils#createStartupShutdownMessage throws NPE when args is null. Contributed by Lokesh Jain and Dinesh Chitlangia	2018-09-18 15:57:33 -05:00
Steve Loughran	1cf38a38da	HADOOP-15744. AbstractContractAppendTest fails against HDFS on HADOOP-15407 branch. Contributed by Steve Loughran.	2018-09-18 10:56:56 +01:00
Yiqun Lin	ee051ef9fe	HADOOP-15742. Log if ipc backoff is enabled in CallQueueManager. Contributed by Ryan Wu.	2018-09-18 11:10:33 +08:00
Sean Mackrory	9475fd902a	HADOOP-15745. ABFS: Add ABFS configuration to ConfigRedactor.	2018-09-17 19:54:01 +00:00
Thomas Marquardt	81dc4a995c	HADOOP-15663. ABFS: Simplify configuration. Contributed by Da Zhou.	2018-09-17 19:54:01 +00:00
Thomas Marquardt	ce03a93f78	HADOOP-15446. ABFS: tune imports & javadocs; stabilise tests. Contributed by Steve Loughran and Da Zhou.	2018-09-17 19:54:01 +00:00
Steve Loughran	f044deedbb	HADOOP-15407. HADOOP-15540. Support Windows Azure Storage - Blob file system "ABFS" in Hadoop: Core Commit. Contributed by Shane Mainali, Thomas Marquardt, Zichen Sun, Georgi Chalakov, Esfandiar Manii, Amit Singh, Dana Kaban, Da Zhou, Junhua Gu, Saher Ahwal, Saurabh Pant, James Baker, Shaoyu Zhang, Lawrence Chen, Kevin Chen and Steve Loughran	2018-09-17 19:54:01 +00:00
Brahma Reddy Battula	ef5c776a42	HADOOP-15733. Correct the log when Invalid emptier Interval configured. Contributed by Ayush Saxena	2018-09-14 07:32:27 +05:30
Hanisha Koneru	291dcf2247	HDDS-423. Introduce an ozone specific log4j.properties. Contributed by Elek, Marton.	2018-09-13 16:37:44 -07:00
Takanobu Asanuma	5d084d7eca	HADOOP-12760. sun.misc.Cleaner has moved to a new location in OpenJDK 9. Contributed by Akira Ajisaka.	2018-09-13 20:42:09 +09:00
Steve Loughran	d7c0a08a1c	HADOOP-15426 Make S3guard client resilient to DDB throttle events and network failures (Contributed by Steve Loughran)	2018-09-12 21:04:49 -07:00
bibinchundatt	c44088ac19	YARN-8739. Fix jenkins issues for Node Attributes branch. Contributed by Sunil Govindan.	2018-09-12 16:01:01 +05:30
Naganarasimha	2475fb0a1e	YARN-6856. [YARN-3409] Support CLI for Node Attributes Mapping. Contributed by Naganarasimha G R.	2018-09-12 16:00:59 +05:30
Mingliang Liu	87f63b6479	HADOOP-14833. Remove s3a user:secret authentication. Contributed by Steve Loughran	2018-09-11 17:18:42 -07:00
Allen Wittenauer	4b8c2b1c1c	HADOOP-14671. Upgrade Apache Yetus to 0.8.0 Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2018-09-11 08:55:34 -07:00
Inigo Goiri	527288ef89	HDFS-13836. RBF: Handle mount table znode with null value. Contributed by yanghuafeng.	2018-09-06 16:47:54 -07:00
Vrushali C	962089ab59	HADOOP-15657 Registering MutableQuantiles via Metric annotation. Contributed by Sushil Ks	2018-09-05 20:53:21 -07:00
Giovanni Matteo Fumarola	9af96d4ed4	HADOOP-15707. Add IsActiveServlet to be used for Load Balancers. Contributed by Lukas Majercak.	2018-09-05 10:50:25 -07:00
Wei-Chiu Chuang	e780556ae9	HADOOP-15696. KMS performance regression due to too many open file descriptors after Jetty migration. Contributed by Wei-Chiu Chuang.	2018-09-05 09:52:35 -07:00
Steve Loughran	9e96ac666d	HADOOP-10219. ipc.Client.setupIOstreams() needs to check for ClientCache.stopClient requested shutdowns. Contributed by Kihwal Lee and Lukas Majercak.	2018-09-04 16:46:12 +01:00
Daniel Templeton	f2c2a68ec2	HADOOP-15706. Typo in compatibility doc: SHOUD -> SHOULD (Contributed by Laszlo Kollar via Daniel Templeton) Change-Id: I6e2459d0700df7f3bad4eac8297a11690191c3ba	2018-08-30 09:12:36 -07:00
Steve Loughran	2e6c1109dc	HADOOP-15667. FileSystemMultipartUploader should verify that UploadHandle has non-0 length. Contributed by Ewan Higgs	2018-08-30 14:33:16 +01:00
Daniel Templeton	d53a10b0a5	HADOOP-15705. Typo in the definition of "stable" in the interface classification Change-Id: I3eae2143400a534903db4f186400561fc8d2bd56	2018-08-29 13:59:32 -07:00
John Zhuge	f9c6fd9471	HADOOP-15633. fs.TrashPolicyDefault: Can't create trash directory. Contributed by Fei Hui.	2018-08-27 09:22:59 -07:00
Konstantin V Shvachko	a4121c71c2	HDFS-13848. Refactor NameNode failover proxy providers. Contributed by Konstantin Shvachko.	2018-08-24 18:27:30 -07:00
Takanobu Asanuma	bed8cb6979	HADOOP-14314. The OpenSolaris taxonomy link is dead in InterfaceClassification.md. Contributed by Rui Gao.	2018-08-23 21:54:38 +09:00
Steve Loughran	34577d2c21	HADOOP-15679. ShutdownHookManager shutdown time needs to be configurable & extended. Contributed by Steve Loughran.	2018-08-20 18:36:24 -07:00
Giovanni Matteo Fumarola	79c97f6a0b	HADOOP-14624. Add GenericTestUtils.DelayAnswer that accept slf4j logger API. Contributed by Ian Pickering and Wenxin He.	2018-08-17 14:40:00 -07:00
Xiao Chen	60ffec9f79	HADOOP-9214. Create a new touch command to allow modifying atime and mtime. Contributed by Hrishikesh Gadre.	2018-08-17 11:18:09 -07:00
Jason Lowe	a17eed1b87	HDFS-13822. speedup libhdfs++ build (enable parallel build). Contributed by Allen Wittenauer and Pradeep Ambati	2018-08-17 12:25:36 -05:00
Xiao Chen	8d7c93186e	HADOOP-15674. Test failure TestSSLHttpServer.testExcludedCiphers with TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 cipher suite. Contributed by Szilard Nemeth.	2018-08-17 10:09:23 -07:00
Márton Elek	77b015000a	HADOOP-8807. Update README and website to reflect HADOOP-8662. Contributed by Andras Bokor.	2018-08-17 11:10:29 +02:00
Xiao Chen	edeb2a356a	HADOOP-15655. Enhance KMS client retry behavior. Contributed by Kitti Nanasi.	2018-08-16 22:42:03 -07:00
Akira Ajisaka	3e3963b035	HADOOP-15552. Move logging APIs over to slf4j in hadoop-tools - Part2. Contributed by Ian Pickering.	2018-08-16 00:31:59 +09:00
Márton Elek	7e822ec246	HDDS-324. Addendum: remove the q letter which is accidentally added to the hadoop-functions.sh.	2018-08-14 20:54:33 +02:00
Xiaoyu Yao	b4031a8f1b	HDDS-324. Use pipeline name as Ratis groupID to allow datanode to report pipeline info. Contributed by Mukul Kumar Singh.	2018-08-13 12:47:49 -07:00
Steve Loughran	da9a39eed1	HADOOP-15583. Stabilize S3A Assumed Role support. Contributed by Steve Loughran.	2018-08-08 22:57:24 -07:00
Wangda Tan	49c687608b	Make 3.1.1 awared by other branches	2018-08-08 13:02:12 -07:00
Xiao Chen	9499df7b81	HDFS-13658. Expose HighestPriorityLowRedundancy blocks statistics. Contributed by Kitti Nanasi.	2018-08-08 10:40:20 -07:00
Ewan Higgs	2ec97abb2e	HADOOP-15576. S3A Multipart Uploader to work with S3Guard and encryption Originally contributed by Ewan Higgs with refinements by Steve Loughran.	2018-08-08 13:50:23 +02:00
Mukul Kumar Singh	6ed8593d18	HDDS-301. ozone command shell does not contain subcommand to run ozoneFS commands. Contributed by Nilotpal Nandi.	2018-08-07 16:09:53 +05:30
Yiqun Lin	2e4e02b4df	HDFS-13792. Fix FSN read/write lock metrics name. Contributed by Chao Sun.	2018-08-07 09:32:51 +08:00
Arpit Agarwal	f2e29acbfa	HADOOP-15476. fix logging for split-dns multihome . Contributed by Ajay Kumar.	2018-08-01 12:32:01 -07:00
Chen Liang	e8f952ef06	HADOOP-15637. LocalFs#listLocatedStatus does not filter out hidden .crc files. Contributed by Erik Krogen.	2018-07-30 10:25:07 -07:00
Eric Yang	77721f39e2	HADOOP-15593. Fixed NPE in UGI spawnAutoRenewalThreadForUserCreds. Contributed by Gabor Bota	2018-07-26 18:35:36 -04:00
Xiao Chen	be150a17b1	HDFS-13622. mkdir should print the parent directory in the error message when parent directories do not exist. Contributed by Shweta.	2018-07-26 10:24:32 -07:00
Yiqun Lin	9089790cab	HADOOP-15611. Log more details for FairCallQueue. Contributed by Ryan Wu.	2018-07-26 18:08:28 +08:00
Mukul Kumar Singh	5f0b924360	HADOOP-15395. DefaultImpersonationProvider fails to parse proxy user config if username has . in it. Contributed by Ajay Kumar.	2018-07-25 21:09:11 +05:30
Xiao Chen	81d59506e5	HADOOP-15609. Retry KMS calls when SSLHandshakeException occurs. Contributed by Kitti Nanasi.	2018-07-24 21:45:43 -07:00
Daniel Templeton	849c45db18	HDFS-13448. HDFS Block Placement - Ignore Locality for First Block Replica (Contributed by BELUGA BEHR via Daniel Templeton) Change-Id: I965d1cfa642ad24296038b83e3d5c9983545267d	2018-07-24 16:05:27 -07:00
Gera Shegalov	6bec03cfc8	HADOOP-15612. Improve exception when tfile fails to load LzoCodec. (gera)	2018-07-24 14:32:30 -07:00
bibinchundatt	8461278833	YARN-8544. [DS] AM registration fails when hadoop authorization is enabled. Contributed by Bibin A Chundatt.	2018-07-24 13:09:17 +05:30
Haibo Chen	17e26163ec	HADOOP-15586. Fix wrong log statement in AbstractService. (Szilard Nemeth via Haibo Chen)	2018-07-23 11:18:25 -07:00
Xiao Chen	993ec026d1	HADOOP-15596. Stack trace should not be printed out when running hadoop key commands. Contributed by Kitti Nanasi.	2018-07-20 19:46:46 -07:00
Haibo Chen	e6873dfde0	YARN-7300. DiskValidator is not used in LocalDirAllocator. (Szilard Nemeth via Haibo Chen)	2018-07-19 16:27:11 -07:00
Kihwal Lee	ccf2db7fc2	HADOOP-15614. TestGroupsCaching.testExceptionOnBackgroundRefreshHandled reliably fails. Contributed by Weiwei Yang.	2018-07-19 11:13:37 -05:00
Wei-Chiu Chuang	d215357718	HDFS-13485. DataNode WebHDFS endpoint throws NPE. Contributed by Siyao Meng.	2018-07-16 15:45:55 -07:00
Xiao Chen	d2874e0417	HDFS-13690. Improve error message when creating encryption zone while KMS is unreachable. Contributed by Kitti Nanasi.	2018-07-16 13:19:53 -07:00
Wei-Chiu Chuang	0c7a578927	HADOOP-15598. DataChecksum calculate checksum is contented on hashtable synchronization. Contributed by Prasanth Jayachandran.	2018-07-16 11:32:45 -07:00
Akira Ajisaka	88625f5cd9	HADOOP-15531. Use commons-text instead of commons-lang in some classes to fix deprecation warnings. Contributed by Takanobu Asanuma.	2018-07-13 11:42:12 -04:00
Sean Mackrory	4f3f9391b0	HADOOP-15316. GenericTestUtils can exceed maxSleepTime. Contributed by Adam Antal.	2018-07-12 17:24:01 +02:00
Wei-Chiu Chuang	162228e8db	HDFS-13723. Occasional "Should be different group" error in TestRefreshUserMappings#testGroupMappingRefresh. Contributed by Siyao Meng.	2018-07-11 10:02:08 -07:00
Akira Ajisaka	4a08ddfa68	HADOOP-15568. fix some typos in the .sh comments. Contributed by Steve Loughran.	2018-07-09 15:43:38 -04:00
Xiao Chen	def9d94a40	HADOOP-15591. KMSClientProvider should log KMS DT acquisition at INFO level. Contributed by Kitti Nanasi.	2018-07-09 12:01:52 -07:00
Yiqun Lin	7a68ac607c	HDFS-13674. Improve documentation on Metrics. Contributed by Chao Sun.	2018-07-09 14:27:34 +08:00
Arpit Agarwal	061b168529	HDDS-167. Rename KeySpaceManager to OzoneManager. Contributed by Arpit Agarwal.	2018-07-06 12:09:05 -07:00
Xiaoyu Yao	498e3bfb6b	HADOOP-15571. Multiple FileContexts created with the same configuration object should be allowed to have different umask. Contributed by Vinod Kumar Vavilapalli.	2018-07-06 11:56:09 -07:00
Andrew Wang	f51da9c4d1	HADOOP-15554. Improve JIT performance for Configuration parsing. Contributed by Todd Lipcon.	2018-07-02 18:31:21 +02:00
Eric E Payne	d36f6b9e93	HADOOP-15548: Randomize local dirs. Contributed by Jim Brennan.	2018-06-29 18:18:32 +00:00
Eric Yang	384764cdea	YARN-8409. Fixed NPE in ActiveStandbyElectorBasedElectorService. Contributed by Chandni Singh	2018-06-28 12:39:49 -04:00
Bharat Viswanadham	d3fa83a44b	HDFS-13705:The native ISA-L library loading failure should be made warning rather than an error message. Contributed by Shashikant Banerjee.	2018-06-28 09:21:56 -07:00
Akira Ajisaka	ddbff7c8d3	HADOOP-14313. Replace/improve Hadoop's byte[] comparator. Contributed by Vikas Vishwakarma.	2018-06-28 14:58:40 +09:00
Akira Ajisaka	2b2399d623	HADOOP-15495. Upgrade commons-lang version to 3.7 in hadoop-common-project and hadoop-tools. Contributed by Takanobu Asanuma.	2018-06-28 14:37:22 +09:00
Todd Lipcon	7a3c6e9c3c	HADOOP-15550. Avoid static initialization of ObjectMappers	2018-06-25 15:36:45 -07:00
Inigo Goiri	abc3e4bad9	HADOOP-15458. TestLocalFileSystem#testFSOutputStreamBuilder fails on Windows. Contributed by Xiao Liang.	2018-06-25 09:50:27 -07:00
Steve Loughran	1ba4e62304	HADOOP-14396. Add builder interface to FileContext. Contributed by Lei (Eddy) Xu.	2018-06-25 14:38:57 +01:00
Eric E Payne	6432128622	YARN-8444: NodeResourceMonitor crashes on bad swapFree value. Contributed by Jim Brennan.	2018-06-22 17:15:29 +00:00
Todd Lipcon	59de967954	HADOOP-15549. Upgrade to commons-configuration 2.1 regresses task CPU consumption	2018-06-21 10:32:52 -07:00
Todd Lipcon	43541a1890	HADOOP-15551. Avoid use of Arrays.stream in Configuration.addTags	2018-06-20 16:43:10 -07:00
Xiao Chen	32f867a6a9	HDFS-13682. Cannot create encryption zone after KMS auth token expires.	2018-06-20 15:58:26 -07:00
Eric Yang	2d87592fc6	HADOOP-15527. Improve delay check for stopping processes. Fixed script location. Contributed by Vinod Kumar Vavilapalli	2018-06-19 13:38:13 -04:00
Chris Douglas	980031bb04	HADOOP-13186. Multipart Uploader API. Contributed by Ewan Higgs	2018-06-17 11:54:26 -07:00
Surendra Singh Lilhore	3905fdb793	HADOOP-15523. Shell command timeout given is in seconds whereas it is taken as millisec while scheduling. Contributed by Bilwa S T.	2018-06-17 12:12:01 +05:30
Yiqun Lin	8e7548d33b	HDFS-13641. Add metrics for edit log tailing. Contributed by Chao Sun.	2018-06-13 20:05:55 +08:00
Eric Yang	108da85320	HADOOP-15527. Improve delay check for stopping processes. Contributed by Vinod Kumar Vavilapalli	2018-06-12 20:40:32 -04:00
Inigo Goiri	aeaf9fec62	HADOOP-15532. TestBasicDiskValidator fails with NoSuchFileException. Contributed by Giovanni Matteo Fumarola.	2018-06-12 14:16:14 -07:00
Rohith Sharma K S	2df73dace0	YARN-8405. RM zk-state-store.parent-path ACLs has been changed since HADOOP-14773. Contributed by Íñigo Goiri.	2018-06-12 17:13:09 +05:30
Yongjun Zhang	7c3dc39083	Update CHANGES, RELEASENOTES, and jdiff for 3.0.3 release. (cherry picked from commit 5a82f10e3254bc0745c2dda6bcef888d3ff0d1c3)	2018-06-11 15:13:18 -07:00
Sammi Chen	18201b882a	HADOOP-15499. Performance severe drops when running RawErasureCoderBenchmark with NativeRSRawErasureCoder. Contributed by Sammi Chen.	2018-06-11 13:53:37 +08:00
Steve Loughran	ef0118b91e	HADOOP-15520. Add tests for various org.apache.hadoop.util classes. Contributed by Arash Nabili	2018-06-09 15:33:38 +01:00
Inigo Goiri	12be8bad7d	HADOOP-15516. Add test cases to cover FileUtil#readLink. Contributed by Giovanni Matteo Fumarola.	2018-06-07 13:34:52 -07:00
Steve Loughran	f494f0b896	HADOOP-15512. Clean up Shell from JDK7 workarounds. Contributed by Zsolt Venczel.	2018-06-07 15:55:49 +01:00
Inigo Goiri	db81f85e84	HADOOP-15513. Add additional test cases to cover some corner cases for FileUtil#symlink. Contributed by Giovanni Matteo Fumarola.	2018-06-06 11:44:17 -07:00
Xiao Chen	ba4011d64f	HADOOP-15217. FsUrlConnection does not handle paths with spaces. Contributed by Joseph Fourny and Zsolt Venczel.	2018-06-05 21:18:31 -07:00
Chen Liang	1b0d4f4606	HDFS-13547. Add ingress port based sasl resolver. Contributed by Chen Liang.	2018-06-05 11:51:29 -07:00
Xiao Chen	6d5e87aec2	HADOOP-15507. Add MapReduce counters about EC bytes read.	2018-06-04 21:13:47 -07:00
Haibo Chen	04cf699dd5	YARN-8388. TestCGroupElasticMemoryController.testNormalExit() hangs on Linux. (Miklos Szegedi via Haibo Chen)	2018-06-04 14:23:08 -07:00
Haibo Chen	4880d890ee	YARN-8375. TestCGroupElasticMemoryController fails surefire build. (Miklos Szegedi via Haibo Chen)	2018-06-01 16:29:13 -07:00
Daniel Templeton	cba3194998	HDFS-13636. Cross-Site Scripting vulnerability in HttpServer2 (Contributed by Haibo Yan via Daniel Templeton) Change-Id: I28edde8125dd20d8d270f0e609d1c04d8173c8b7	2018-06-01 14:42:39 -07:00
Takanobu Asanuma	d1e2b80980	MAPREDUCE-7098. Upgrade common-langs version to 3.7 in hadoop-mapreduce-project Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2018-05-31 18:31:10 +09:00
Kihwal Lee	e3236a9680	Additional check when unpacking archives. Contributed by Wilfred Spiegelenburg.	2018-05-29 14:15:12 -05:00
Inigo Goiri	3c75f8e493	HADOOP-15497. TestTrash should use proper test path to avoid failing on Windows. Contributed by Anbang Hu.	2018-05-29 09:11:08 -07:00
Karthik Palanisamy	61df174e8b	HADOOP-15449. Increase default timeout of ZK session to avoid frequent NameNode failover Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2018-05-28 19:41:07 +09:00
Johan Gustavsson	d14e26b31f	HADOOP-15477. Make unjar in RunJar overrideable Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2018-05-28 17:29:59 +09:00
Xiao Chen	02322de3f9	HADOOP-15473. Configure serialFilter in KeyProvider to avoid UnrecoverableKeyException caused by JDK-8189997. Contributed by Gabor Bota.	2018-05-25 09:10:51 -07:00
Steve Loughran	bddfe796f2	HADOOP-15494. TestRawLocalFileSystemContract fails on Windows. Contributed by Anbang Hu.	2018-05-25 11:12:47 +01:00
Arpit Agarwal	51ce02bb54	HADOOP-15486. Make NetworkTopology#netLock fair. Contributed by Nanda kumar.	2018-05-23 10:30:12 -07:00
Robert Kanter	aa23d49fc8	HADOOP-15457. Add Security-Related HTTP Response Header in WEBUIs. (kanwaljeets via rkanter)	2018-05-23 10:24:09 -07:00
Sammi Chen	34e8b9f9a8	HDFS-13540. DFSStripedInputStream should only allocate new buffers when reading. Contributed by Xiao Chen.	2018-05-23 19:10:09 +08:00
Akira Ajisaka	745f203e57	Additional check when unpacking archives. Contributed by Jason Lowe and Akira Ajisaka.	2018-05-23 17:16:23 +09:00
Miklos Szegedi	3e5f7ea986	YARN-8310. Handle old NMTokenIdentifier, AMRMTokenIdentifier, and ContainerTokenIdentifier formats. Contributed by Robert Kanter.	2018-05-22 18:10:33 -07:00
Kihwal Lee	bcc8e76bad	HADOOP-15450. Avoid fsync storm triggered by DiskChecker and handle disk full situation. Contributed by Arpit Agarwal.	2018-05-22 11:20:51 -05:00
Mukul Kumar Singh	57c2feb0d3	HADOOP-15474. Rename properties introduced for <tags>. Contributed by Zsolt Venczel.	2018-05-22 13:33:31 +05:30
Rushabh Shah	73e9120ad7	Skip the proxy user check if the ugi has not been initialized. Contributed by Daryn Sharp	2018-05-21 12:33:00 -05:00
Xiao Chen	89f5911392	HADOOP-15154. Abstract new method assertCapability for StreamCapabilities testing. Contributed by Zsolt Venczel.	2018-05-18 11:29:20 -07:00
Inigo Goiri	8783613696	HDFS-13586. Fsync fails on directories on Windows. Contributed by Lukas Majercack.	2018-05-17 19:26:44 -07:00
Inigo Goiri	53b807a6a8	HDFS-13560. Insufficient system resources exist to complete the requested service for some tests on Windows. Contributed by Anbang Hu.	2018-05-17 17:03:23 -07:00
Steve Loughran	07d8505f75	HADOOP-15466. Correct units in adl.http.timeout. Contributed by Sean Mackrory,	2018-05-15 16:19:03 +01:00
Sammi Chen	f3f544b004	Add 2.9.1 release notes and changes documents	2018-05-14 15:14:02 +08:00
sammichen	66c9905b57	Revert "Add 2.9.1 release notes and changes documents" This reverts commit `e4dc346d65`.	2018-05-14 14:38:40 +08:00
littlezhou	e4dc346d65	Add 2.9.1 release notes and changes documents	2018-05-14 14:24:01 +08:00
Rushabh Shah	a5449d3631	HADOOP-15441. Log kms url and token service at debug level. Contributed by Gabor Bota	2018-05-12 12:19:13 -05:00
Haibo Chen	274eee3284	MAPREDUCE-7095. Race conditions in closing FadvisedChunkedFile. (Miklos Szegedi via Haibo Chen)	2018-05-10 10:46:55 -07:00
Inigo Goiri	1da8d4190d	HADOOP-15454. TestRollingFileSystemSinkWithLocal fails on Windows. Contributed by Xiao Liang.	2018-05-10 09:41:16 -07:00
Sean Mackrory	1cfe7506f7	HADOOP-15356. Make HTTP timeout configurable in ADLS connector. Contributed by Atul Sikaria and Sean Mackrory.	2018-05-09 15:15:51 -06:00
Anu Engineer	3a43ac2851	HDDS-1. Remove SCM Block DB. Contributed by Xiaoyu Yao.	2018-05-07 14:58:52 -07:00
Xiao Chen	a3a1552c33	Revert "HADOOP-14445. Delegation tokens are not shared between KMS instances. Contributed by Xiao Chen and Rushabh S Shah." This reverts commit `583fa6ed48`. Conflicts: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/kms/KMSTokenRenewer.java hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/TestKMS.java	2018-05-07 13:32:27 -07:00
Rushabh Shah	4cdbdce752	HADOOP-14841 Kms client should disconnect if unable to get output stream from connection. Contributed by Rushabh S Shah	2018-05-04 15:36:13 -05:00
Daniel Templeton	87c23ef643	HDFS-13481. TestRollingFileSystemSinkWithHdfs#testFlushThread: test failed intermittently (Contributed by Gabor Bota via Daniel Templeton) Change-Id: I9921981dfa69669fe7912dd2a31ae8b638283204	2018-05-02 17:13:40 -07:00
Steve Loughran	fe649bb305	HADOOP-12071. conftest is not documented. Contributed by Kengo Seki.	2018-05-02 13:33:56 +01:00
BELUGA BEHR	33768724ff	HADOOP-15377. Improve debug messages in MetricsConfig.java Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2018-05-02 17:09:22 +09:00
Steve Loughran	8f42dafcf8	HADOOP-15250. Split-DNS MultiHomed Server Network Cluster Network IPC Client Bind Addr Wrong Contributed by Ajay Kumar	2018-05-01 22:32:40 +01:00
Wei-Chiu Chuang	bff3d7b0cf	HADOOP-15382. Log kinit output in credential renewal thread. Contributed by Gabor Bota.	2018-04-27 10:05:55 -07:00
Ewan Higgs	84ecfe3ceb	HADOOP-14188. Remove the usage of org.mockito.internal.util.reflection.Whitebox Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2018-04-27 19:21:37 +09:00
Wangda Tan	13d389bf51	HADOOP-15411. AuthenticationFilter should use Configuration.getPropsWithPrefix instead of iterator. (Suma Shivaprasad via wangda) Change-Id: Ia19458a43aceb4ffdb1e8eccc519655235d92ba9	2018-04-26 13:54:40 -07:00
Daryn Sharp	bbc8dfcb8b	HADOOP-15402. Prevent double logout of UGI's LoginContext Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2018-04-26 13:54:40 -07:00
Owen O'Malley	2adda92de1	HDFS-7240 git commit fixup Signed-off-by: Owen O'Malley <omalley@apache.org>	2018-04-26 13:52:17 -07:00
Mukul Kumar Singh	8cc13ff962	HDFS-13425. Ozone: Clean-up of ozone related change from hadoop-common-project. Contributed by Lokesh Jain.	2018-04-26 05:36:04 -07:00
Mukul Kumar Singh	ec6c8742e5	HDFS-13415. Ozone: Remove cblock code from HDFS-7240. Contributed by Elek, Marton.	2018-04-26 05:36:04 -07:00
Anu Engineer	8b832f3c35	HDFS-13405. Ozone: Rename HDSL to HDDS. Contributed by Ajay Kumar, Elek Marton, Mukul Kumar Singh, Shashikant Banerjee and Anu Engineer.	2018-04-26 05:36:04 -07:00
Anu Engineer	4f37ff7616	HDFS-13341. Ozone: Move ozone specific ServiceRuntimeInfo utility to the framework. Contributed by Elek, Marton.	2018-04-26 05:36:04 -07:00
Anu Engineer	4e61bc431e	HDFS-13258. Ozone: restructure Hdsl/Ozone code to separated maven subprojects. Contributed by Elek Marton, Mukul Kumar Singh, Xiaoyu Yao, Ajay Kumar, Anu Engineer, Lokesh Jain, Nanda Kumar.	2018-04-26 05:36:04 -07:00
Anu Engineer	740a06cdd7	HDFS-12719. Ozone: Fix checkstyle, javac, whitespace issues in HDFS-7240 branch. Contributed by Mukul Kumar Singh.	2018-04-26 05:36:04 -07:00
Anu Engineer	06c6582694	HDFS-12585. Ozone: KSM UI: Add description for configs in UI. Contributed by Ajay Kumar.	2018-04-26 05:36:04 -07:00
Anu Engineer	0b3270a8de	HDFS-12513. Ozone: Create UI page to show Ozone configs by tag. Contributed by Ajay Kumar.	2018-04-26 05:36:04 -07:00
Chris Douglas	6d3d01d7ad	HADOOP-12077. Provide a multi-URI replication Inode for ViewFs. Contributed by Gera Shegalov	2018-04-26 05:36:04 -07:00
Anu Engineer	dcc21a4f93	HDFS-12005. Ozone: Web interface for SCM. Contributed by Elek, Marton.	2018-04-26 05:36:04 -07:00
Anu Engineer	acfae2716a	HDFS-12034. Ozone: Web interface for KSM. Contributed by Elek, Marton.	2018-04-26 05:36:04 -07:00
Yiqun Lin	8cf0c864c4	HDFS-12170. Ozone: OzoneFileSystem: KSM should maintain key creation time and modification time. Contributed by Mukul Kumar Singh.	2018-04-26 05:36:04 -07:00
Rakesh Radhakrishnan	665f7c8b69	HDFS-11794. Add ec sub command -listCodec to show currently supported ec codecs. Contributed by SammiChen.	2018-04-26 05:36:04 -07:00
Robert Kanter	7ab08a9c37	HADOOP-15390. Yarn RM logs flooded by DelegationTokenRenewer trying to renew KMS tokens (xiaochen via rkanter)	2018-04-23 15:44:15 -07:00
Lei Xu	42e82f0281	Update CHANGES, RELEASENOTES, and jdiff for 3.0.2 release. (cherry picked from commit f6ecb76d0b919b9836600fe28ec9e637b223cd54) Conflicts: hadoop-project-dist/pom.xml	2018-04-23 14:14:05 -07:00
Virajith Jalaparti	827d8593a5	HDFS-13311. RBF: TestRouterAdminCLI#testCreateInvalidEntry fails on Windows. Contributed by Inigo Goiri	2018-04-16 15:55:00 -07:00
Bharat Viswanadham	ec1e8c1a8c	HADOOP-15379. Make IrqHandler.bind() public. Contributed by Ajay Kumar	2018-04-12 21:51:20 -07:00
Aaron Fabbri	0d898b7bb8	HADOOP-12502 SetReplication OutOfMemoryError. Contributed by Vinayakumar B.	2018-04-11 17:19:56 -07:00
Xiao Chen	7c9cdad6d0	HDFS-13056. Expose file-level composite CRCs in HDFS which are comparable across different instances/layouts. Contributed by Dennis Huo.	2018-04-10 21:31:48 -07:00
Xiao Chen	583fa6ed48	HADOOP-14445. Delegation tokens are not shared between KMS instances. Contributed by Xiao Chen and Rushabh S Shah.	2018-04-10 15:38:25 -07:00
Jason Lowe	e81397545a	HADOOP-15357. Configuration.getPropsWithPrefix no longer does variable substitution. Contributed by Jim Brennan	2018-04-10 16:44:55 -05:00
Xiaoyu Yao	8ab776d61e	HADOOP-15340. Provide meaningful RPC server name for RpcMetrics. Contributed by Elek Marton.	2018-04-10 11:42:54 -07:00
Ewan Higgs	cef8eb7981	HADOOP-15376. Remove double semi colons on imports that make Clover fall over. Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2018-04-10 23:58:26 +09:00
Bharat	0006346abe	HADOOP-15328. Fix the typo in HttpAuthentication.md. Contributed by fang zhenyi	2018-04-09 16:37:49 -07:00
Mukul Kumar Singh	0b345b7653	HADOOP-15366. Add a helper shutdown routine in HadoopExecutor to ensure clean shutdown. Contributed by Shashikant Banerjee.	2018-04-07 16:29:01 +05:30
Wangda Tan	6cf023f9b7	Added CHANGES/RELEASES/Jdiff for 3.1.0 release Change-Id: Ied5067a996151c04d15cad46c46ac98b60c37b39 (cherry picked from commit 2d96570452a72569befdf9cfe9b90c9fa2e0e261)	2018-04-05 15:52:39 -07:00
Wei Yan	3121e8c293	HDFS-13353. RBF: TestRouterWebHDFSContractCreate failed. Contributed by Takanobu Asanuma.	2018-04-05 12:00:52 -07:00
Miklos Szegedi	e52539b46f	HADOOP-14855. Hadoop scripts may errantly believe a daemon is still running, preventing it from starting. Contributed by Robert Kanter.	2018-04-04 15:35:58 -07:00
Xiao Chen	1077392eaa	HADOOP-15355. TestCommonConfigurationFields is broken by HADOOP-15312. Contributed by LiXin Ge.	2018-04-02 23:10:08 -07:00
Xiao Chen	57374c4737	HADOOP-15317. Improve NetworkTopology chooseRandom's loop.	2018-04-02 22:48:26 -07:00
Steve Loughran	dde1579096	HADOOP-15146. Remove DataOutputByteBuffer. Contributed by BELUGA BEHR.	2018-04-02 15:52:40 +01:00
Konstantin V Shvachko	acfd764fcc	HADOOP-15253. Should update maxQueueSize when refresh call queue. Contributed by Tao Jie.	2018-03-30 18:55:35 -07:00
Konstantin V Shvachko	2216bde322	HADOOP-12862. LDAP Group Mapping over SSL can not specify trust store. Contributed by Wei-Chiu Chuang and Konstantin Shvachko.	2018-03-29 17:16:23 -07:00
Wei-Chiu Chuang	3fe41c65d8	HADOOP-15312. Undocumented KeyProvider configuration keys. Contributed by LiXin Ge.	2018-03-27 13:33:41 -07:00
Xiao Chen	c22d62b338	HADOOP-15313. TestKMS should close providers.	2018-03-26 15:59:32 -07:00
Xiaoyu Yao	22194f3d21	HADOOP-15339. Support additional key/value propereties in JMX bean registration. Contributed by Elek Marton.	2018-03-26 10:45:29 -07:00
Anu Engineer	28790b81ec	HADOOP-15295. Remove redundant logging related to tags from Configuration. Contributed by Ajay Kumar.	2018-03-23 21:17:18 -07:00
Lei Xu	f9427b73a2	Update releasenotes and changelogs for 3.0.1 release	2018-03-23 11:46:41 -07:00
Jitendra Pandey	4bea96f9a8	HADOOP-14067. VersionInfo should load version-info.properties from its own classloader. Contributed by Thejas M Nair.	2018-03-22 14:09:06 -07:00
Yufei Gu	268c29a5f5	HADOOP-15331. Fix a race condition causing parsing error of java.io.BufferedInputStream in class org.apache.hadoop.conf.Configuration. Contributed by Miklos Szegedi.	2018-03-22 11:04:37 -07:00
Yufei Gu	138c1ed566	HADOOP-15062. TestCryptoStreamsWithOpensslAesCtrCryptoCodec fails on Debian 9. Contributed by Miklos Szegedi.	2018-03-20 15:19:18 -07:00

... 8 9 10 11 12 ...

5518 Commits