big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
25,297 Commits 2 Branches 2 Tags 264 MiB
1e8c25d528
Commit Graph

5673 Commits

Author SHA1 Message Date
Petre Bogdan Stolojan
a218038960
HADOOP-17139 Re-enable optimized copyFromLocal implementation in S3AFileSystem (#3101) 
This work
* Defines the behavior of FileSystem.copyFromLocal in filesystem.md
* Implements a high performance implementation of copyFromLocalOperation
  for S3 
* Adds a contract test for the operation: AbstractContractCopyFromLocalTest
* Implements the contract tests for Local and S3A FileSystems

Contributed by: Bogdan Stolojan
 2021-07-30 19:42:08 +01:00
Tamas Domok
798a0837c1
YARN-10814. Fallback to RandomSecretProvider if the secret file is empty (#3206) 
The rest endpoint would be unusable with an empty secret file
(throwing IllegalArgumentExceptions).

Any IO error would have resulted in the same fallback path.

Co-authored-by: Tamas Domok <tdomok@cloudera.com>
 2021-07-30 12:16:46 +02:00
hchaverr
3c8a48e681 HADOOP-17819. Add extensions to ProtobufRpcEngine RequestHeaderProto. Contributed by Hector Sandoval Chaverri. (#3242) 2021-07-28 15:37:56 -07:00
Viraj Jasani
e001f8ee39
HADOOP-17814. Provide fallbacks for identity/cost providers and backoff enable (#3230) 
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2021-07-29 02:10:07 +09:00
jianghuazhu
fd13970d94
HDFS-16137.Improve the comments related to FairCallQueue#queues. (#3226) 
Co-authored-by: zhujianghua <zhujianghua@zhujianghuadeMacBook-Pro.local>
Reviewed-by: Viraj Jasani <vjasani@apache.org>
 2021-07-28 03:18:04 -07:00
Mehakmeet Singh
f813554769
HADOOP-13887. Support S3 client side encryption (S3-CSE) using AWS-SDK (#2706) 
This (big!) patch adds support for client side encryption in AWS S3,
with keys managed by AWS-KMS.

Read the documentation in encryption.md very, very carefully before
use and consider it unstable.

S3-CSE is enabled in the existing configuration option
"fs.s3a.server-side-encryption-algorithm":

fs.s3a.server-side-encryption-algorithm=CSE-KMS
fs.s3a.server-side-encryption.key=<KMS_KEY_ID>

You cannot enable CSE and SSE in the same client, although
you can still enable a default SSE option in the S3 console. 
  
* Filesystem list/get status operations subtract 16 bytes from the length
  of all files >= 16 bytes long to compensate for the padding which CSE
  adds.
* The SDK always warns about the specific algorithm chosen being
  deprecated. It is critical to use this algorithm for ranged
  GET requests to work (i.e. random IO). Ignore.
* Unencrypted files CANNOT BE READ.
  The entire bucket SHOULD be encrypted with S3-CSE.
* Uploading files may be a bit slower as blocks are now
  written sequentially.
* The Multipart Upload API is disabled when S3-CSE is active.

Contributed by Mehakmeet Singh
 2021-07-27 11:08:51 +01:00
Akira Ajisaka
4c35466359
HADOOP-17317. [JDK 11] Upgrade dnsjava to remove illegal access warnings (#2442) 
Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
 2021-07-24 14:48:25 +09:00
Viraj Jasani
3a52bfc5db
HADOOP-17808. ipc.Client to set interrupt flag after catching InterruptedException (#3219) 2021-07-23 12:38:55 +08:00
Viraj Jasani
e1d00addb5
HADOOP-16290. Enable RpcMetrics units to be configurable (#3198) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-07-19 23:55:49 -07:00
He Xiaoqiao
6ed7670a93
HDFS-16067. Support Append API in NNThroughputBenchmark. Contributed by Renukaprasad C. 2021-07-17 21:57:00 +08:00
Viraj Jasani
df44178eb6
HADOOP-17795. Provide fallbacks for callqueue.impl and scheduler.impl (#3192) 
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2021-07-14 20:58:32 +09:00
jianghuazhu
87e0000137
HADOOP-17672.Remove an invalid comment content in the FileContext class. (#2961) 
Co-authored-by: zhujianghua <zhujianghua@zhujianghuadeMacBook-Pro.local>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2021-07-14 00:15:02 -07:00
Abhishek Das
1dd03cc4b5 HADOOP-17028. ViewFS should initialize mounted target filesystems lazily. Contributed by Abhishek Das (#2260) 2021-07-13 18:11:50 -07:00
LeonGao
ea90c5117d
HDFS-15785. Datanode to support using DNS to resolve nameservices to IP addresses to get list of namenodes. (#2639) 
* Rebase trunk

* Fix to use FQDN and update config name

* Fix javac

* Style and trigger build

* Trigger Build after force push

* Trigger Build

* Fix config names
 2021-07-13 14:18:59 -07:00
Akira Ajisaka
c81f82e21d
HADOOP-12665. Document hadoop.security.token.service.use_ip. (#3187) 
Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
Reviewed-by: Chris Nauroth <cnauroth@apache.org>
 2021-07-12 10:16:13 +09:00
Artem Smotrakov
ba325a8ada
HADOOP-17793. Better token validation (#3189) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-07-10 13:42:31 +09:00
Akira Ajisaka
6ac5d8a1ed
HADOOP-17794. Add a sample configuration to use ZKDelegationTokenSecretManager in Hadoop KMS (#3190) 2021-07-09 22:48:47 +09:00
Viraj Jasani
618c9218ee
HADOOP-17788. Replace IOUtils#closeQuietly usages by Hadoop's own utility (#3171) 
Reviewed-by: Steve Loughran <stevel@apache.org>
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2021-07-08 16:03:40 +09:00
liangxs
a5db6831bc
HADOOP-17749. Remove lock contention in SelectorPool of SocketIOWithTimeout (#3080) 2021-07-06 09:11:03 +08:00
Rafal Wojdyla
f639fbc29f
HADOOP-17402. Add GCS config to the core-site (#2638) 
Contributed by Rafal Wojdyla
 2021-07-05 21:07:12 +01:00
Akira Ajisaka
20a4b1ae36
HADOOP-17331. [JDK 16] TestDNS fails (#2884) 2021-06-30 03:06:29 -07:00
Viraj Jasani
c488abbc79
HDFS-16075. Use empty array constants present in StorageType and DatanodeInfo to avoid creating redundant objects (#3115) 
Reviewed-by: Hui Fei <ferhui@apache.org>
 2021-06-21 10:25:12 +09:00
Takanobu Asanuma
9e7c7ad129
HADOOP-17760. Delete hadoop.ssl.enabled and dfs.https.enable from docs and core-default.xml (#3099) 
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
 2021-06-17 09:58:47 +09:00
Wei-Chiu Chuang
a77bf7cf07 Make upstream aware of 3.3.1 release 2021-06-15 15:43:36 +00:00
Mingliang Liu
6e5692e7e2
Revert "HDFS-16033 Fix issue of the StatisticsDataReferenceCleaner cleanUp (#3042)" 
This reverts commit 4a26a61ecd.
 2021-06-11 00:34:24 -07:00
Viraj Jasani
4ef27a596f
HADOOP-17753. Keep restrict-imports-enforcer-rule for Guava Lists in top level hadoop-main pom (#3087) 2021-06-11 12:15:52 +09:00
Steve Loughran
762a83e044
HADOOP-17631. Configuration ${env.VAR:-FALLBACK} to eval FALLBACK when restrictSystemProps=true (#2977) 
Contributed by Steve Loughran.
 2021-06-08 21:56:40 +01:00
Viraj Jasani
f4b24c68e7
HADOOP-17743. Replace Guava Lists usage by Hadoop's own Lists in hadoop-common, hadoop-tools and cloud-storage projects (#3072) 2021-06-07 13:24:09 +09:00
July
4a26a61ecd
HDFS-16033 Fix issue of the StatisticsDataReferenceCleaner cleanUp (#3042) 
Contributed by kaifeiYi (yikf).

Signed-off-by: Mingliang Liu <liuml07@apache.org>
Reviewed-by: Steve Loughran <stevel@apache.org>
 2021-06-04 13:36:09 -07:00
Viraj Jasani
59fc4061cb
HADOOP-17152. Provide Hadoop's own Lists utility to reduce dependency on Guava (#3061) 
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2021-06-03 18:56:00 +09:00
Konstantin V Shvachko
8ce30f51f9 HDFS-16040. RpcQueueTime metric counts requeued calls as unique events. Contributed by Simbarashe Dzinamarira. 2021-05-27 18:47:20 -07:00
Hui Fei
98a74e2351
HDFS-16041. TestErasureCodingCLI fails (#3052) 2021-05-26 20:12:17 +09:00
Viraj Jasani
986d0a4f1d
HADOOP-17732. Keep restrict-imports-enforcer-rule for Guava Sets in hadoop-main pom (#3049) 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2021-05-26 17:14:31 +09:00
Steve Loughran
832a3c6a89
HADOOP-17511. Add audit/telemetry logging to S3A connector (#2807) 
The S3A connector supports
"an auditor", a plugin which is invoked
at the start of every filesystem API call,
and whose issued "audit span" provides a context
for all REST operations against the S3 object store.

The standard auditor sets the HTTP Referrer header
on the requests with information about the API call,
such as process ID, operation name, path,
and even job ID.

If the S3 bucket is configured to log requests, this
information will be preserved there and so can be used
to analyze and troubleshoot storage IO.

Contributed by Steve Loughran.
 2021-05-25 10:25:41 +01:00
Mehakmeet Singh
c665ab02ed
HADOOP-17670. S3AFS and ABFS to log IOStats at DEBUG mode or optionally at INFO level in close() (#2963) 
When the S3A and ABFS filesystems are closed,
their IOStatistics are logged at debug in the log:

org.apache.hadoop.fs.statistics.IOStatisticsLogging

Set `fs.iostatistics.logging.level` to `info` for the statistics 
to be logged at info. (also: `warn` or `error` for even higher
log levels).


Contributed by: Mehakmeet Singh
 2021-05-24 13:02:11 +01:00
Vinayakumar B
2bbeae3240
HDFS-15790. Make ProtobufRpcEngineProtos and ProtobufRpcEngineProtos2 Co-Exist (#2767) 2021-05-24 02:45:39 -07:00
Viraj Jasani
c70ee2d548
HADOOP-17700. ExitUtil#halt info log should log HaltException (#3015) 
Reviewed-by: Steve Loughran <stevel@apache.org>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
 2021-05-22 19:06:58 +09:00
Viraj Jasani
e4062ad027
HADOOP-17115. Replace Guava Sets usage by Hadoop's own Sets in hadoop-common and hadoop-tools (#2985) 
Signed-off-by: Sean Busbey <busbey@apache.org>
 2021-05-20 10:47:04 -05:00
Hongbing Wang
f7247922b7
HDFS-16018. Optimize the display of hdfs "count -e" or "count -t" com… (#2994) 2021-05-20 11:23:54 +08:00
Xiaoyu Yao
86729e130f
HADOOP-17699. Remove hardcoded SunX509 usage from SSLFactory. (#3016) 2021-05-18 10:11:36 -07:00
Akira Ajisaka
35ca1dcb9d
HADOOP-17685. Fix junit deprecation warnings in hadoop-common module. (#2983) 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2021-05-13 14:22:25 +09:00
Viraj Jasani
fdd20a3cf4
HADOOP-17689. Avoid Potential NPE in org.apache.hadoop.fs (#3008) 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2021-05-13 00:05:58 +09:00
dependabot[bot]
29105ffb63
HADOOP-17683. Update commons-io to 2.8.0 (#2974) 
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Akira Ajisaka <aajisaka@apache.org>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2021-05-12 10:57:45 +09:00
Viraj Jasani
c80f07422f
HADOOP-17686. Avoid potential NPE by using Path#getParentPath API in hadoop-huaweicloud (#2990) 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2021-05-12 10:34:01 +09:00
Borislav Iordanov
91430889a5
HADOOP-17665 Ignore missing keystore configuration in reloading mechanism 2021-05-10 13:31:48 -07:00
hexiaoqiao
6d6766bc22
HADOOP-17690. Improve the log for The DecayRpcScheduler. Contributed by Bhavik Patel. 2021-05-10 14:11:36 +08:00
Ayush Saxena
7f93349ee7 HADOOP-17644. Add back the exceptions removed by HADOOP-17432 for compatibility. Contributed by Quan Li. 2021-05-09 03:40:08 +05:30
hchaverr
f40e3eb059 HADOOP-17680. Allow ProtobufRpcEngine to be extensible (#2905) Contributed by Hector Chaverri. 2021-05-06 16:42:27 -07:00
Viraj Jasani
b93e448f9a
HADOOP-11616. Remove workaround for Curator's ChildReaper requiring Guava 15+ (#2973) 
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-05-06 04:52:02 +09:00
Istvan Fajth
0d78d73973
HADOOP-17675. LdapGroupsMapping$LdapSslSocketFactory ClassNotFoundException (#2965) 2021-05-04 11:33:59 +01:00
kishendas
e571025f5b
HADOOP-17657: implement StreamCapabilities in SequenceFile.Writer and fall back to flush, if hflush is not supported (#2949) 
Co-authored-by: Kishen Das <kishen@cloudera.com>
Reviewed-by: Steve Loughran <stevel@apache.org>
 2021-05-04 01:20:56 -07:00
lfengnan
6e525ab81c
HDFS-15810. RBF: RBFMetrics's TotalCapacity out of bounds (#2910) 
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-05-02 19:18:47 +09:00
Wei-Chiu Chuang
b2e54762a4
HDFS-15624. fix the function of setting quota by storage type (#2377) (#2955) 
1. puts NVDIMM to the end of storage type enum to make sure compatibility.
2. adds check to make sure the software layout version is satisfied

Co-authored-by: su xu <kevinbrandon@163.com>
Co-authored-by: huangtianhua <huangtianhua223@gmail.com>
Co-authored-by: YaYun-Wang <34060507+YaYun-Wang@users.noreply.github.com>

Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Vinayakumar B <vinayakumarb@apache.org>

Change-Id: I3c58beef50730827a09b3c968e9ad637baa57d44
 2021-04-28 23:54:39 -07:00
Wei-Chiu Chuang
f41a368c14
HADOOP-11245. Update NFS gateway to use Netty4 (#2832) 
Reviewed-by: Tsz-Wo Nicholas Sze <szetszwo@apache.org>
 2021-04-28 23:43:11 -07:00
Wei-Chiu Chuang
90c6caf650 Revert "HDFS-15624. fix the function of setting quota by storage type (#2377)" 
This reverts commit 394b9f7a5c.

Ref: HDFS-15995.
Had to revert this commit, so we can commit HDFS-15566 (a critical bug preventing rolling upgrade to Hadoop 3.3)
Will re-work this fix again later.
 2021-04-26 11:27:15 +08:00
Steve Loughran
027c8fb257
HADOOP-17597. Optionally downgrade on S3A Syncable calls (#2801) 
Followup to HADOOP-13327, which changed S3A output stream hsync/hflush calls
to raise an exception.

Adds a new option fs.s3a.downgrade.syncable.exceptions

When true, calls to Syncable hsync/hflush on S3A output streams will
log once at warn (for entire process life, not just the stream), then
increment IOStats with the relevant operation counter

With the downgrade option false (default)
* IOStats are incremented
* The UnsupportedOperationException current raised includes a link to the
  JIRA.

Contributed by Steve Loughran.
 2021-04-23 18:44:41 +01:00
Mehakmeet Singh
6085f09db5
HADOOP-17471. ABFS to collect IOStatistics (#2731) 
The ABFS Filesystem and its input and output streams now implement
the IOStatisticSource interface and provide IOStatistics on
their interactions with Azure Storage.

This includes the min/max/mean durations of all REST API calls.

Contributed by Mehakmeet Singh <mehakmeet.singh@cloudera.com>
 2021-04-23 10:28:31 +01:00
Vinayakumar B
c4c0683dff
HADOOP-17505. public interface GroupMappingServiceProvider needs default impl for getGroupsSet() (#2661). Contributed by Vinayakumar B. 2021-04-22 01:02:03 +05:30
Viraj Jasani
9a261b0a16
HADOOP-17642. Remove appender EventCounter to avoid instantiation (#2922) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-04-17 20:47:44 +09:00
litao
af0448d37b
HDFS-15975. Use LongAdder instead of AtomicLong (#2907) 2021-04-17 16:49:09 +09:00
Masatake Iwasaki
2717203f85
HADOOP-17569. Building native code fails on Fedora 33. (#2886) 2021-04-15 21:33:18 +09:00
Viraj Jasani
9179638017
HADOOP-17524. Remove EventCounter and Log counters from JVM Metrics (#2909) 
Reviewed-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-04-15 18:04:46 +09:00
Akira Ajisaka
156ecc89be
HADOOP-17630. [JDK 15] TestPrintableString fails due to Unicode 13.0 support. (#2890) 
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
 2021-04-13 17:08:49 +09:00
Akira Ajisaka
2bd810a507
HADOOP-17608. Fix TestKMS failure (#2880) 
Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
 2021-04-12 12:53:58 +09:00
Brahma Reddy Battula
ae88174c29 HADOOP-17617. Incorrect representation of RESPONSE for Get Key Version in KMS index.md.vm file. Contributed by Ravuri Sushma sree 2021-04-07 23:49:17 +05:30
Wei-Chiu Chuang
dac60b8282
HADOOP-17621. hadoop-auth to remove jetty-server dependency. (#2865) 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
 2021-04-06 21:55:01 -07:00
He Xiaoqiao
3a89471c36
HADOOP-17613. Log not flushed fully when daemon shutdown. Contributed by Renukaprasad C. 2021-04-07 12:18:23 +08:00
Boyina, Hemanth Kumar
e86050fae5 HADOOP-17588. CryptoInputStream#close() should be syncronized. Contributed by RenukaPrasad C 2021-04-06 18:06:10 +05:30
Viraj Jasani
3f2682b92b
HADOOP-17622. Avoid usage of deprecated IOUtils#cleanup API. (#2862) 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2021-04-06 13:39:10 +09:00
zhuqi
5eaa1b7230
HADOOP-17619: Fix DelegationTokenRenewer#updateRenewalTime java doc error. (#2846). Contributed by Qi Zhu. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2021-04-04 14:52:03 +05:30
Brahma Reddy Battula
bc7689abf5 HADOOP-17587. Kinit with keytab should not display the keytab file's full path in any logs. Contributed by Ravuri Sushma sree. 2021-04-02 10:03:50 +05:30
Brahma Reddy Battula
478402cc74 HADOOP-17610. DelegationTokenAuthenticator prints token information. Contributed by Ravuri Sushma sree. 2021-04-02 09:56:00 +05:30
litao
4bd04126d6
HDFS-15892. Add metric for editPendingQ in FSEditLogAsync (#2770) 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2021-04-02 10:33:40 +09:00
Akira Ajisaka
a94a23ab9e Revert "HADOOP-17608. Fix NPE in TestKMS (#2828)" 
This reverts commit d69088a097.
 2021-04-01 04:24:26 +09:00
Borislav Iordanov
2c482fbacf HADOOP-16524. Automatic keystore reloading for HttpServer2 
Reapply of issue reverted first because it caused yarn failures and
then again because the commit message was incorrectly formatted
(and yet again because of commit message format).

Signed-off-by: stack <stack@apache.org>
 2021-03-31 10:46:35 -07:00
stack
22961a615d Revert "HADOOP-16524. Automatic keystore reloading for HttpServer2" 
This reverts commit a2975d2153.
 2021-03-31 10:43:09 -07:00
stack
a2975d2153 HADOOP-16524. Automatic keystore reloading for HttpServer2 
Reapply of issue reverted first because it caused yarn failures and
then again because the commit message was incorrectly formatted.
 2021-03-31 10:40:20 -07:00
stack
5183aaeda2 Revert "Hadoop 16524 - resubmission following some unit test fixes (#2693)" 
Revert to fix the summary message.

This reverts commit 9509bebf7f.
 2021-03-31 10:39:55 -07:00
Borislav Iordanov
9509bebf7f
Hadoop 16524 - resubmission following some unit test fixes (#2693) 
Signed-off-by: stack <stack@apache.org>
 2021-03-31 10:07:42 -07:00
Akira Ajisaka
d69088a097
HADOOP-17608. Fix NPE in TestKMS (#2828) 2021-03-31 09:57:31 -07:00
He Xiaoqiao
054e1c5e83
HDFS-15932. Improve the balancer error message when process exits abnormally. Contributed by Renukaprasad C. 2021-03-31 12:22:53 +08:00
Ayush Saxena
f5c1557288
HADOOP-17531.Addendum: DistCp: Reduce memory usage on copying huge directories. (#2820). Contributed by Ayush Saxena. 
Signed-off-by: Steve Loughran <stevel@apache.org>
 2021-03-27 03:01:41 +05:30
Akira Ajisaka
af1f9f43ea
HADOOP-17133. Implement HttpServer2 metrics (#2145) 2021-03-25 12:09:43 -07:00
touchida
95e6892675
HDFS-15759. EC: Verify EC reconstruction correctness on DataNode (#2585) 2021-03-24 16:56:09 +08:00
Takanobu Asanuma
b4d97a8dc7 HDFS-15902. Improve the log for HTTPFS server operation. Contributed by Bhavik Patel. 2021-03-24 17:51:35 +09:00
Ayush Saxena
03cfc85279
HADOOP-17531. DistCp: Reduce memory usage on copying huge directories. (#2732). Contributed by Ayush Saxena. 
Signed-off-by: Steve Loughran <stevel@apache.org>
 2021-03-24 02:36:26 +05:30
Jack Jiang
d8ec8ab965
HADOOP-17599. Remove NULL checks before instanceof (#2804) 2021-03-23 08:46:11 -07:00
Xiaoyu Yao
63eb289462
HADOOP-17598. Fix java doc issue introduced by HADOOP-17578. (#2802). Contributed by Xiaoyu Yao. 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2021-03-23 10:23:45 +05:30
kwangsun
c8d327a4f1
HADOOP-17952. Fix the wrong CIDR range example in Proxy User documentation. (#2780) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-03-22 11:43:32 +09:00
Xiaoyu Yao
b503de2328
HADOOP-17578. Improve UGI debug log to help troubleshooting TokenCach… (#2762) 2021-03-17 10:57:11 -07:00
Jim Brennan
299b8062f1 MAPREDUCE-7322. revisiting TestMRIntermediateDataEncryption. Contributed by Ahmed Hussein. 2021-03-15 20:13:17 +00:00
He Xiaoqiao
b1dc6c40a0
HADOOP-17585. Correct timestamp format in the docs for the touch command. Contributed by Stephen O'Donnell. 2021-03-14 18:09:50 +08:00
Masatake Iwasaki
0f6ba5e031
HADOOP-17514. Remove trace subcommand from hadoop CLI. (#2680) 2021-03-12 10:26:46 +09:00
Steve Loughran
bcd9c67082
HADOOP-16721. Improve S3A rename resilience (#2742) 
The S3A connector's rename() operation now raises FileNotFoundException if
the source doesn't exist; a FileAlreadyExistsException if the destination
exists and is unsuitable for the source file/directory.

When renaming to a path which does not exist, the connector no longer checks
for the destination parent directory existing -instead it simply verifies
that there is no file immediately above the destination path.
This is needed to avoid race conditions with delete() and rename()
calls working on adjacent subdirectories.

Contributed by Steve Loughran.
 2021-03-11 12:47:39 +00:00
Akira Ajisaka
23b343aed1
HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin (#2753) 
Removed findbugs from the hadoop build images and added spotbugs instead.
Upgraded SpotBugs to 4.2.2 and spotbugs-maven-plugin to 4.2.0.

Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
 2021-03-11 10:56:07 +09:00
Chao Sun
176bd88890
HADOOP-16080. hadoop-aws does not work with hadoop-client-api. (#2522) 
Contributed by Chao Sun.

(Cherry-picked via PR #2575)
 2021-03-09 20:01:29 +00:00
Haoze Wu
ef7ab535c5
HADOOP-17552. Change ipc.client.rpc-timeout.ms from 0 to 120000 by default to avoid potential hang. (#2727) 2021-03-06 22:26:16 +09:00
Akira Ajisaka
9fd2198daa
HADOOP-17546. Update Description of hadoop-http-auth-signature-secret in HttpAuthentication.md. Contributed by Ravuri Sushma sree. 2021-03-04 14:55:37 +09:00
S O'Donnell
c3b3b36dee HDFS-14013. Skip any credentials stored in HDFS when starting ZKFC. Contributed by Stephen O'Donnell 2021-03-01 11:36:41 +00:00
Ahmed Hussein
e04bcb3a06
MAPREDUCE-7320. organize test directories for ClusterMapReduceTestCase (#2722). Contributed by Ahmed Hussein 2021-02-26 13:42:33 -06:00