big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
27,146 Commits 2 Branches 2 Tags 264 MiB
3f98cb6741
Commit Graph

6041 Commits

Author SHA1 Message Date
PJ Fanning
56b928b86f
YARN-11498. Add exclusion for jettison everywhere jersey-json is loaded (#5786) 
All uses  of jersey-json in the yarn and other hadoop modules now
exclude the obsolete org.codehaus.jettison/jettison and so avoid
all security issues which can come from the library.

Contributed by PJ Fanning
 2023-09-13 18:10:24 +01:00
Steve Loughran
81d90fd65b
HADOOP-18073. S3A: Upgrade AWS SDK to V2 (#5995) 
This patch migrates the S3A connector to use the V2 AWS SDK.

This is a significant change at the source code level.
Any applications using the internal extension/override points in
the filesystem connector are likely to break.

This includes but is not limited to:
- Code invoking methods on the S3AFileSystem class
  which used classes from the V1 SDK.
- The ability to define the factory for the `AmazonS3` client, and
  to retrieve it from the S3AFileSystem. There is a new factory
  API and a special interface S3AInternals to access a limited
  set of internal classes and operations.
- Delegation token and auditing extensions.
- Classes trying to integrate with the AWS SDK.

All standard V1 credential providers listed in the option 
fs.s3a.aws.credentials.provider will be automatically remapped to their
V2 equivalent.

Other V1 Credential Providers are supported, but only if the V1 SDK is
added back to the classpath.  

The SDK Signing plugin has changed; all v1 signers are incompatible.
There is no support for the S3 "v2" signing algorithm.

Finally, the aws-sdk-bundle JAR has been replaced by the shaded V2
equivalent, "bundle.jar", which is now exported by the hadoop-aws module.

Consult the document aws_sdk_upgrade for the full details.

Contributed by Ahmar Suhail + some bits by Steve Loughran
 2023-09-11 14:30:25 +01:00
Szilard Nemeth
9342ecf6cc HADOOP-18870. CURATOR-599 change broke functionality introduced in HADOOP-18139 and HADOOP-18709. Contributed by Ferenc Erdelyi 2023-09-06 21:32:36 -04:00
huhaiyang
2831c7ce26
HADOOP-18880. Add some rpc related metrics to Metrics.md (#6015) Contributed by Yanghai Hu. 
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
 2023-09-05 17:34:05 +08:00
Steve Loughran
28c533a582 Revert "HADOOP-18860. Upgrade mockito version to 4.11.0 (#5977)" 
This reverts commit 1046f9cf98.
 2023-08-31 14:54:53 +01:00
Anmol Asrani
1046f9cf98
HADOOP-18860. Upgrade mockito version to 4.11.0 (#5977) 
As well as the POM update, this patch moves to the (renamed) verify methods. 
Backporting mockito test changes may now require cherrypicking this patch, otherwise
use the old method names.

Contributed by Anmol Asrani
 2023-08-29 12:12:27 +01:00
Chunyi Yang
42b4525f75
HDFS-17156. Client may receive old state ID which will lead to inconsistent reads. (#5951) 
Reviewed-by: Simbarashe Dzinamarira <sdzinamarira@linkedin.com>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2023-08-18 01:56:34 +09:00
hchaverri
ad2f45c64f
HDFS-17148. RBF: SQLDelegationTokenSecretManager must cleanup expired tokens in SQL (#5936) 2023-08-11 13:04:32 -07:00
Liangjun He
b6edcb9a84
HADOOP-18840. Add enQueue time to RpcMetrics (#5926). Contributed by Liangjun He. 
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Reviewed-by: Xing Lin <linxingnku@gmail.com>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2023-08-10 10:38:48 +08:00
hchaverri
bc48e5cbe8
HDFS-17128. Updating SQLDelegationTokenSecretManager to use LoadingCache so tokens are updated frequently. (#5897) Contributed by Hector Sandoval Chaverri. 
Reviewed-by: Simbarashe Dzinamarira <sdzinamarira@linkedin.com>
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
 2023-08-08 07:45:14 +08:00
WangYuanben
1e3e246934
HADOOP-18810. Document missing a lot of properties in core-default.xml. (#5912) Contributed by WangYuanben. 
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
 2023-08-08 07:37:26 +08:00
WangYuanben
440698eb07
HADOOP-18836. Some properties are missing from hadoop-policy.xml (#5922) 2023-08-07 20:03:23 +08:00
zhangshuyan
c35f31640e
HADOOP-18807. Close child file systems in ViewFileSystem when cache is disabled. (#5847) 
Contributed by Shuyan Zhang
 2023-07-20 11:39:13 +01:00
Steve Loughran
b3130056f5
HADOOP-18808. LogExactlyOnce to add a debug() method (#5850) 
Contributed by Steve Loughran
 2023-07-18 14:23:19 +01:00
Viraj Jasani
38ac2f7349
HADOOP-18809. S3A prefetch read/write file operations should guard channel close (#5853) 
Contributed by Viraj Jasani
 2023-07-18 14:16:12 +01:00
hfutatzhanghb
b95595158f
HADOOP-18801. Delete path directly when it can not be parsed in trash. (#5744). Contributed by farmmamba. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2023-07-16 12:20:46 +08:00
Viraj Jasani
e7d74f3d59
HADOOP-18291. S3A prefetch - Implement thread-safe LRU cache for SingleFilePerBlockCache (#5754) 
Contributed by Viraj Jasani
 2023-07-14 10:21:01 +01:00
Mehakmeet Singh
fac7d26c5d
HADOOP-18781. ABFS backReference passed down to streams to avoid GC closing the FS. (#5780) 
To avoid the ABFS instance getting closed due to GC while the streams are working, attach the ABFS instance to a backReference opaque object and passing down to the streams so that we have a hard reference while the streams are working. 

Contributed by: Mehakmeet Singh
 2023-07-11 17:57:05 +05:30
WangYuanben
6843f8e4e0
HADOOP-18794. ipc.server.handler.queue.size missing from core-default.xml (#5819). Contributed by WangYuanben. 
Reviewed-by: Hualong Zhang <hualong.z@hotmail.com>
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-07-11 16:39:50 +05:30
slfan1989
e8590adb7b
HADOOP-18779. Improve hadoop-function.sh#status script. (#5762) 2023-07-03 08:46:57 -07:00
slfan1989
8a52990150
YARN-11519. [Federation] Add RouterAuditLog to log4j.properties. (#5785) 2023-06-27 10:52:59 -07:00
Mehakmeet Singh
5db7107b77
HADOOP-18764. fs.azure.buffer.dir to be under Yarn container path on yarn applications (#5737) 
Changing fs.azure.buffer.dir for azure so things clean up better in long-lived yarn clusters.

Contributed by: Mehakmeet Singh
 2023-06-27 20:22:00 +05:30
Wei-Chiu Chuang
e239d40ab1 Post release update 
* Add jdiff xml files from 3.3.6 release.
* Declare 3.3.6 as the latest stable release.
* Copy release notes.

(cherry picked from commit 7db9895000)
(cherry picked from commit cc121e2124aa01458dc296a060edc5e21a295268)
 2023-06-26 16:08:24 +00:00
Xing Lin
427366b73b
HDFS-17042 Add rpcCallSuccesses and OverallRpcProcessingTime to RpcMetrics for Namenode (#5730) 2023-06-15 13:59:58 -07:00
Viraj Jasani
a75e378868
HADOOP-18756. S3A prefetch - CachingBlockManager to use AtomicBoolean for closed flag (#5718) 
Contributed by Viraj Jasani
 2023-06-14 12:51:54 +01:00
Dongjoon Hyun
fb16e00da0
HADOOP-18718. Fix several maven build warnings (#5592). Contributed by Dongjoon Hyun. 
Reviewed-by: Gautham B A <gautham.bangalore@gmail.com>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-06-11 11:38:13 +05:30
Steve Loughran
7a45ef4164
MAPREDUCE-7435. Manifest Committer OOM on abfs (#5519) 
This modifies the manifest committer so that the list of files
to rename is passed between stages as a file of
writeable entries on the local filesystem.

The map of directories to create is still passed in memory;
this map is built across all tasks, so even if many tasks
created files, if they all write into the same set of directories
the memory needed is O(directories) with the
task count not a factor.

The _SUCCESS file reports on heap size through gauges.
This should give a warning if there are problems.

Contributed by Steve Loughran
 2023-06-09 17:00:59 +01:00
Viraj Jasani
1dbaba8e70
HADOOP-18740. S3A prefetch cache blocks should be accessed by RW locks (#5675) 
Contributed by Viraj Jasani
 2023-06-07 14:05:52 +01:00
Ayush Saxena
1d0c9ab433
Revert "HADOOP-18207. Introduce hadoop-logging module (#5503)" 
This reverts commit 03a499821c.
 2023-06-05 09:34:40 +05:30
Szilard Nemeth
e0a339223a HADOOP-18709. Add curator based ZooKeeper communication support over SSL/TLS into the common library. Contributed by Ferenc Erdelyi 2023-06-04 14:40:41 -04:00
Viraj Jasani
03a499821c
HADOOP-18207. Introduce hadoop-logging module (#5503) 
Reviewed-by: Duo Zhang <zhangduo@apache.org>
 2023-06-02 18:07:34 -07:00
Steve Loughran
160b9fc3c9
HADOOP-18755. openFile builder new optLong() methods break hbase-filesystem (#5704) 
This is a followup to 
HADOOP-18724. Open file fails with NumberFormatException for S3AFileSystem

Contributed by Steve Loughran
 2023-06-01 14:31:08 +01:00
Patrick GRANDJEAN
4627242c44
HADOOP-18652. Path.suffix raises NullPointerException (#5653). Contributed by Patrick Grandjean. 
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-05-19 05:16:55 +05:30
LiuGuH
f6770dee47
HDFS-16979. RBF: Add proxyuser port in hdfsauditlog (#5552). Contributed by liuguanghua. 
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: Simbarashe Dzinamarira <sdzinamarira@linkedin.com>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-05-19 05:02:16 +05:30
Steve Loughran
a90c722143
HADOOP-18724. [FOLLOW-UP] cherrypick changes from branch-3.3 backport (#5662) 
* move FileContext.copy() onto optLong()
* move FileUtil onto optLong()

This brings trunk into sync with the branch-3.3 changes
 2023-05-16 18:16:24 +01:00
Viraj Jasani
bef40e9427
HADOOP-18688. S3A audit header to include count of items in delete ops (#5621) 
The auditor-generated http referrer URL now includes the count of keys
to delete in the "ks" query parameter

Contributed by Viraj Jasani
 2023-05-16 10:40:16 +01:00
Steve Loughran
ad1e3a0f5b
HADOOP-18724. (followup) remove deprecation on optLong/optDouble methods (#5650) 
Somehow @Deprecated crept in to the declaration of the
new FSBuilder optLong/optDouble methods.
 2023-05-12 15:22:37 +01:00
WangYuanben
905bfa84a8
HDFS-16965. Add switch to decide whether to enable native codec. (#5520). Contributed by WangYuanben. 
Reviewed-by: Tao Li <tomscut@apache.org>
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-05-12 04:12:02 +05:30
Steve Loughran
e76c09ac3b
HADOOP-18724. Open file fails with NumberFormatException for S3AFileSystem (#5611) 
This:

1. Adds optLong, optDouble, mustLong and mustDouble
   methods to the FSBuilder interface to let callers explicitly
   passin long and double arguments.
2. The opt() and must() builder calls which take float/double values
   now only set long values instead, so as to avoid problems
   related to overloaded methods resulting in a ".0" being appended
   to a long value.
3. All of the relevant opt/must calls in the hadoop codebase move to
   the new methods
4. And the s3a code is resilient to parse errors in is numeric options
   -it will downgrade to the default.

This is nominally incompatible, but the floating-point builder methods
were never used: nothing currently expects floating point numbers.

For anyone who wants to safely set numeric builder options across all compatible
releases, convert the number to a string and then use the opt(String, String)
and must(String, String) methods.

Contributed by Steve Loughran
 2023-05-11 17:57:25 +01:00
slfan1989
a2dda0ce03
HADOOP-18359. Update commons-cli from 1.2 to 1.5. (#5095). Contributed by Shilun Fan. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-05-10 01:42:12 +05:30
Gautham B A
c974710d8e
HADOOP-18729. Fix mvnsite on Windows 10 (#5618) 2023-05-05 13:08:58 -07:00
Tak Lon (Stephen) Wu
0e46388474
HADOOP-18671. Add recoverLease(), setSafeMode(), isFileClosed() as interfaces to hadoop-common (#5553) 
The HDFS lease APIs have been replicated as interfaces in hadoop-common so other filesystems can
also implement them.  Applications which use the leasing APIs should migrate to the new
interface where possible.

Contributed by Stephen Wu
 2023-05-03 11:05:55 +01:00
zhangshuyan
fddc9769a5
HADOOP-18726. Set the locale to avoid printing useless logs. (#5612). Contributed by Shuyan Zhang. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2023-05-03 00:09:36 +08:00
Viraj Jasani
bfcf5dd03b
HADOOP-18697. S3A prefetch: failure of ITestS3APrefetchingInputStream#testRandomReadLargeFile (#5580) 
Contributed by Viraj Jasani
 2023-05-02 15:21:46 +01:00
Szilard Nemeth
73ca64a3ba
YARN-11450. Improvements for TestYarnConfigurationFields and TestConfigurationFieldsBase (#5455) 2023-05-02 15:52:57 +02:00
Pralabh Kumar
d75c6d9d57
HADOOP-18715. Add debug log for getting details of tokenKindMap (#5608). Contributed by Pralabh Kumar. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-04-29 17:28:49 +05:30
Sebastian Baunsgaard
6aac6cb212
HADOOP-18660. Filesystem Spelling Mistake (#5475). Contributed by Sebastian Baunsgaard. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-04-25 21:44:04 +05:30
cxzl25
2f66f0b83a
HADOOP-18694. Client.Connection#updateAddress needs to ensure that address is resolved before updating (#5542). Contributed by dzcxzl. 
Reviewed-by: Steve Vaughan <email@stevevaughan.me>
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org
 2023-04-25 03:52:49 +05:30
Doroszlai, Attila
5b23224970
HADOOP-18714. Wrong StringUtils.join() called in AbstractContractRootDirectoryTest (#5578) 2023-04-24 09:17:12 +02:00
LiuGuH
742e07d9c3
HADOOP-18710. Add RPC metrics for response time (#5545). Contributed by liuguanghua. 
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-04-22 01:06:08 +05:30
Christos Bisias
9e24ed2196
HADOOP-18691. Add a CallerContext getter on the Schedulable interface (#5540) 2023-04-20 10:11:25 -07:00
Nikita Eshkeev
d07356e60e
HADOOP-18597. Simplify single node instructions for creating directories for Map Reduce. (#5305) 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2023-04-20 16:12:44 +05:30
rdingankar
5119d0c72f
HDFS-16982 Use the right Quantiles Array for Inverse Quantiles snapshot (#5556) 2023-04-18 10:47:37 -07:00
Viraj Jasani
0e3aafe6c0
HADOOP-18399. S3A Prefetch - SingleFilePerBlockCache to use LocalDirAllocator (#5054) 
Contributed by Viraj Jasani
 2023-04-18 16:37:48 +01:00
Steve Loughran
405ed1dde6
HADOOP-18470. Hadoop 3.3.5 release wrap-up (#5558) 
Post-release updates of the branches

* Add jdiff xml files from 3.3.5 release.
* Declare 3.3.5 as the latest stable release.
* Copy release notes.
 2023-04-18 10:12:07 +01:00
Melissa You
2b60d0c1f4
[HDFS-16971] Add read metrics for remote reads in FileSystem Statistics #5534 (#5536) 2023-04-13 09:07:42 -07:00
rdingankar
3e2ae1da00
HDFS-16949 Introduce inverse quantiles for metrics where higher numer… (#5495) 2023-04-10 08:56:00 -07:00
mjwiq
e45451f9c7
HADOOP-18687. hadoop-auth: remove unnecessary dependency on json-smart (#5524) 
Contributed by Michiel de Jong
 2023-04-06 16:00:33 +01:00
Viraj Jasani
b4bcbb9515
HDFS-16959. RBF: State store cache loading metrics (#5497) 2023-03-29 10:43:13 -07:00
Andras Katona
ee01c64c6c
HADOOP-18676. jettison dependency override in hadoop-common lib (#5513) 2023-03-27 09:59:02 +02:00
Ayush Saxena
b82bcbd8ad
Revert "HADOOP-18676. Fixing jettison vulnerability of hadoop-common lib (#5507)" 
This reverts commit 72b0122706.
 2023-03-25 12:04:28 +05:30
Andras Katona
72b0122706
HADOOP-18676. Fixing jettison vulnerability of hadoop-common lib (#5507) 
* HADOOP-18587. Fixing jettison vulnerability of hadoop-common lib

* no need for excluding, let it come

Change-Id: Ia6e4ad351158dd4b0510dec34bbde531a60e7654
 2023-03-24 16:31:45 +01:00
Ayush Saxena
e3cb9573e1
HADOOP-18662. ListFiles with recursive fails with FNF. (#5477). Contributed by Ayush Saxena. 
Reviewed-by: Steve Loughran <stevel@apache.org
 2023-03-23 08:30:08 +05:30
Yubi Lee
67e02a92e0
HADOOP-18666. A whitelist of endpoints to skip Kerberos authentication doesn't work for ResourceManager and Job History Server (#5480) 2023-03-22 10:54:41 +09:00
Viraj Jasani
9a8287c36f
HADOOP-18669. Remove Log4Json Layout (#5493) 2023-03-21 10:07:06 +08:00
Viraj Jasani
405bfa2800
HADOOP-18654. Remove unused custom appender TaskLogAppender (#5457) 2023-03-16 00:45:37 +08:00
Viraj Jasani
aff840c59c
HADOOP-18653. LogLevel servlet to determine log impl before using setLevel (#5456) 
The log level can only be set on Log4J log implementations;
probes are used to downgrade to a warning when other
logging back ends are used

Contributed by Viraj Jasani
 2023-03-13 12:30:12 +00:00
Steve Loughran
09469bf47d
HADOOP-18661. Fix bin/hadoop usage script terminology. (#5473) 
Followup to HADOOP-13209: s/slaves/r/workers in
the usage message you get when you type "bin/hadoop"

Contributed by Steve Loughran
 2023-03-13 12:24:36 +00:00
Viraj Jasani
e1ca466bdb
HADOOP-18648. Avoid loading kms log4j properties dynamically by KMSWebServer (#5441) 2023-03-02 08:02:07 +08:00
Viraj Jasani
28d2753d2f
HADOOP-18645. Provide keytab file key name with ServiceStateException (#5433) 
Signed-off-by: Tao Li <tomscut@apache.org>
 2023-03-01 09:34:12 +08:00
rdingankar
0ca5686034
HDFS-16917 Add transfer rate quantile metrics for DataNode reads (#5397) 
Co-authored-by: Ravindra Dingankar <rdingankar@linkedin.com>
 2023-02-27 18:26:32 +00:00
Simbarashe Dzinamarira
4cc33e5e37
HDFS-16901: RBF: Propagates real user's username via the caller context, when a proxy user is being used. (#5346) 2023-02-22 21:58:44 +00:00
hchaverr
fb31393b65
HADOOP-18535. Implement token storage solution based on MySQL 
Fixes #1240

Signed-off-by: Owen O'Malley <oomalley@linkedin.com>
 2023-02-22 10:38:50 -08:00
Steve Loughran
11a220c6e7
HADOOP-18636 LocalDirAllocator cannot recover from directory tree deletion (#5412) 
Even though DiskChecker.mkdirsWithExistsCheck() will create the directory tree,
it is only called *after* the enumeration of directories with available
space has completed.

Directories which don't exist are reported as having 0 space, therefore
the mkdirs code is never reached.

Adding a simple mkdirs() -without bothering to check the outcome-
ensures that if a dir has been deleted then it will be reconstructed
if possible. If it can't it will still have 0 bytes of space
reported and so be excluded from the allocation.

Contributed by Steve Loughran
 2023-02-22 11:48:12 +00:00
Arnout Engelen
02fd87a4d8
HADOOP-18627. Add stronger wording in 'secure mode' introduction (#5406) 
Make it more clear that when deploying Hadoop 'secure mode' is generally not optional.

Contributed by Arnout Engelen
 2023-02-17 16:30:41 +00:00
Bryan Beaudreault
7e19bc31b6
HADOOP-18215. Enhance WritableName to be able to return aliases for classes that use serializers (#4215) 2023-02-16 18:13:25 +00:00
slfan1989
c3706597a3
YARN-11349. [Federation] Router Support DelegationToken With SQL. (#5244) 2023-02-15 14:38:41 -08:00
Ankit Saurabh
f4f2793f3b
HADOOP-18351. Reduce excess logging of errors during S3A prefetching reads (#5274) 
Contributed by Ankit Saurabh
 2023-02-15 18:28:42 +00:00
Steve Loughran
d56977e909
HADOOP-18470. More in the 3.3.5 index.html about security (#5383) 
Expands on the comments in cluster config to tell people
they shouldn't be running a cluster without a private VLAN
in cloud, that Knox is good here, and unsecured clusters
without a VLAN are just computation-as-a-service to crypto miners

Contributed by Steve Loughran
 2023-02-14 17:22:59 +00:00
Viraj Jasani
021fcc6c5e
HADOOP-18628. IPC Server Connection should log host name before returning VersionMismatch error (#5385) 
Contributed by Viraj Jasani
 2023-02-14 11:48:48 +00:00
Viraj Jasani
90de1ff151
HADOOP-18206 Cleanup the commons-logging references and restrict its usage in future (#5315) 2023-02-14 03:24:06 +08:00
Owen O'Malley
26fba8701c
HDFS-18324. Fix race condition in closing IPC connections. (#5371) 2023-02-10 17:51:03 +00:00
huhaiyang
113a9e40cb
HADOOP-18625. Fix method name of RPC.Builder#setnumReaders (#5301) 
Changes method name of RPC.Builder#setnumReaders to setNumReaders()

The original method is still there, just marked deprecated.
It is the one which should be used when working with older branches.

Contributed by Haiyang Hu
 2023-02-09 13:28:34 +00:00
Viraj Jasani
4fcceff535
HADOOP-18620 Avoid using grizzly-http-* APIs (#5356) 2023-02-09 10:45:07 +08:00
gardenia
8714403dc7
HADOOP-18621. Resource leak in CryptoOutputStream.close() (#5347) 
When closing we need to wrap the flush() in a try .. finally, otherwise
when flush throws it will stop completion of the remainder of the
close activities and in particular the close of the underlying wrapped
stream object resulting in a resource leak.

Contributed by Colm Dougan
 2023-02-07 12:01:57 +00:00
Steve Vaughan
5f5157ac53
HADOOP-18612. Avoid mixing canonical and non-canonical when performing comparisons (#5339) 
Contributed by Steve Vaughan Jr
 2023-02-06 18:28:29 +00:00
Steve Vaughan
aed6fcee5b
HADOOP-18576. Java 11 JavaDoc fails due to missing package comments (#5344) 
Add JavaDoc comments to package-info.java to avoid errors resulting from the use of Hadoop annotations.

Contributed by Steve Vaughan Jr
 2023-02-06 18:17:57 +00:00
hfutatzhanghb
be564f5c20
[HDFS-16903]. Fix javadoc of LightWeightResizableGSet class (#5338) 2023-02-06 13:21:28 +09:00
Viraj Jasani
ad0cff2f97
HADOOP-18592. Sasl connection failure should log remote address. (#5294) 
Contributed by Viraj Jasani <vjasani@apache.org>

Signed-off-by: Chris Nauroth <cnauroth@apache.org>
Signed-off-by: Steve Loughran <stevel@apache.org>
Signed-off-by: Mingliang Liu <liuml07@apache.org>
 2023-02-01 10:15:20 -08:00
Wei-Chiu Chuang
9d47108b50
HADOOP-18584. [NFS GW] Fix regression after netty4 migration. (#5252) 
Reviewed-by: Tsz-Wo Nicholas Sze <szetszwo@apache.org>
 2023-01-31 01:17:04 +08:00
Ayush Saxena
952d707240
HADOOP-18604. Add compile platform in the hadoop version output. (#5327). Contributed by Ayush Saxena. 
Signed-off-by: Chris Nauroth <cnauroth@apache.org>
 2023-01-28 14:19:19 +05:30
Nikita Eshkeev
4de31123ce
Fix "the the" and friends typos (#5267) 
Signed-off-by: Nikita Eshkeev <neshkeev@yandex.ru>
 2023-01-17 03:33:59 +08:00
PJ Fanning
d81d98388c
HADOOP-18575: followup: try to avoid repeatedly hitting exceptions when transformer factories do not support attributes (#5253) 
Part of HADOOP-18469 and the hardening of XML/XSL parsers.
Followup to the main HADOOP-18575 patch, to improve performance when
working with xml/xsl engines which don't support the relevant attributes.

Include this change when backporting.

Contributed by PJ Fanning.
 2023-01-16 13:15:37 +00:00
huangxiaoping
a90e424d9f
HADOOP-18591. Fix a typo in Trash (#5291) 
Signed-off-by: Tao Li <tomscut@apache.org>
Signed-off-by: Chris Nauroth <cnauroth@apache.org>
 2023-01-12 13:21:21 -08:00
slfan1989
3d21cff263
YARN-11413. Fix Junit Test ERROR Introduced By YARN-6412. (#5289) 
* YARN-11413. Fix Junit Test ERROR Introduced By YARN-6412.

* YARN-11413. Fix CheckStyle.

* YARN-11413. Fix CheckStyle.

Co-authored-by: slfan1989 <louj1988@@>
 2023-01-12 14:29:05 +01:00
Chengbing Liu
4cf304de45
HDFS-16872. Fix log throttling by declaring LogThrottlingHelper as static members (#5246) 
Co-authored-by: Chengbing Liu <liuchengbing@qiyi.com>
Signed-off-by: Erik Krogen <xkrogen@apache.org>
 2023-01-10 10:03:25 -08:00
Surendra Singh Lilhore
a65d24488a
HADOOP-18581 : Handle Server KDC re-login when Server and Client run … (#5248) 
* HADOOP-18581 : Handle Server KDC re-login when Server and Client run in same JVM.
 2023-01-08 23:55:06 +05:30
David Dillon
b63b777c84
HDFS-16873 FileStatus compareTo specify ordering by path (#5219) 2022-12-21 10:11:55 +08:00
PJ Fanning
6a07b5dc10
HADOOP-18575. Make XML transformer factory more lenient (#5224) 
Due diligence followup to
HADOOP-18469. Add secure XML parser factories to XMLUtils (#4940)

Contributed by P J Fanning
 2022-12-18 12:25:10 +00:00
Chengbing Liu
ca3526da92
HADOOP-18567. LogThrottlingHelper: properly trigger dependent recorders in cases of infrequent logging (#5215) 
Signed-off-by: Erik Krogen <xkrogen@apache.org>
Co-authored-by: Chengbing Liu <liuchengbing@qiyi.com>
 2022-12-16 09:15:11 -08:00