Go to file
Steve Loughran a5806a9e7b
HADOOP-19161. S3A: option "fs.s3a.performance.flags" to take list of performance flags (#6789)
1. Configuration adds new method `getEnumSet()` to get a set of enum values from
   a configuration string.
   <E extends Enum<E>> EnumSet<E> getEnumSet(String key, Class<E> enumClass, boolean ignoreUnknown)

   Whitespace is ignored, case is ignored and the value "*" is mapped to "all values of the enum".
   If "ignoreUnknown" is true then when parsing, unknown values are ignored.
   This is recommended for forward compatiblity with later versions.

2. This support is implemented in org.apache.hadoop.fs.s3a.impl.ConfigurationHelper -it can be used
    elsewhere in the hadoop codebase.

3. A new private FlagSet class in hadoop common manages a set of enum flags.

     It implements StreamCapabilities and can be probed for a specific option being set
    (with a prefix)


S3A adds an option fs.s3a.performance.flags which builds a FlagSet with enum
type PerformanceFlagEnum

* which initially contains {Create, Delete, Mkdir, Open}
* the existing fs.s3a.create.performance option sets the flag "Create".
* tests which configure fs.s3a.create.performance MUST clear
  fs.s3a.performance.flags in test setup.

Future performance flags are planned, with different levels of safety
and/or backwards compatibility.

Contributed by Steve Loughran
2024-07-29 11:33:51 +01:00
.github HADOOP-19193. Create orphan commit for website deployment (#6864) 2024-06-05 15:25:48 +01:00
.yetus Add .yetus/excludes.txt (#4984) 2022-10-11 09:23:34 -07:00
dev-support HADOOP-19222. Switch yum repo baseurl due to CentOS 7 sunset (#6932) Contributed by Cheng Pan. 2024-07-12 06:16:44 +08:00
hadoop-assemblies HADOOP-19107. Drop support for HBase v1 & upgrade HBase v2 (#6629). Contributed by Ayush Saxena 2024-04-22 21:55:58 +05:30
hadoop-build-tools Preparing for 3.5.0 development (#6411) 2024-01-19 15:05:22 +08:00
hadoop-client-modules Revert "HADOOP-16822. Provide source artifacts for hadoop-client-api. Contributed by Karel Kolman." (#6458) 2024-04-10 12:03:59 +01:00
hadoop-cloud-storage-project HADOOP-19154. Upgrade bouncycastle to 1.78.1 due to CVEs (#6755) 2024-06-05 15:31:23 +01:00
hadoop-common-project HADOOP-19161. S3A: option "fs.s3a.performance.flags" to take list of performance flags (#6789) 2024-07-29 11:33:51 +01:00
hadoop-dist Preparing for 3.5.0 development (#6411) 2024-01-19 15:05:22 +08:00
hadoop-hdfs-project HDFS-16690. Automatically format unformatted JNs with JournalNodeSyncer (#6925). Contributed by Aswin M Prabhu. 2024-07-23 20:55:57 +08:00
hadoop-mapreduce-project MAPREDUCE-7475. Fix non-idempotent unit tests (#6785) 2024-05-17 14:51:47 +01:00
hadoop-maven-plugins HADOOP-19041. Use StandardCharsets in more places (#6449) 2024-03-28 23:17:18 -04:00
hadoop-minicluster Preparing for 3.5.0 development (#6411) 2024-01-19 15:05:22 +08:00
hadoop-project HADOOP-19195. S3A: Upgrade aws sdk v2 to 2.25.53 (#6900) 2024-07-08 10:18:53 +01:00
hadoop-project-dist HADOOP-19112. Hadoop 3.4.0 release wrap-up. (#6640) Contributed by Shilun Fan. 2024-03-19 20:08:03 +08:00
hadoop-tools HADOOP-19161. S3A: option "fs.s3a.performance.flags" to take list of performance flags (#6789) 2024-07-29 11:33:51 +01:00
hadoop-yarn-project YARN-11705. Turn off Node Manager working directories validation by default (#6948) 2024-07-18 16:55:40 +02:00
licenses HADOOP-17144. Update Hadoop's lz4 to v1.9.2. Contributed by Hemanth Boyina. 2020-10-18 18:37:46 +05:30
licenses-binary HADOOP-15993. Upgrade Kafka to 2.4.0 in hadoop-kafka module. (#1796) 2020-01-09 16:24:58 +09:00
.asf.yaml HADOOP-18630. Add gh-pages in asf.yaml to deploy the current trunk doc (#5393). Contributed by Simhadri Govindappa. 2023-02-14 18:13:29 +05:30
.gitattributes HADOOP-13598. Add eol=lf for unix format files in .gitattributes. Contributed by Yiqun Lin. 2016-09-14 11:14:31 +09:00
.gitignore HADOOP-18963. Fix typos in .gitignore (#6243) 2023-11-04 05:12:39 +05:30
BUILDING.txt HADOOP-19107. Drop support for HBase v1 & upgrade HBase v2 (#6629). Contributed by Ayush Saxena 2024-04-22 21:55:58 +05:30
LICENSE-binary HADOOP-19195. S3A: Upgrade aws sdk v2 to 2.25.53 (#6900) 2024-07-08 10:18:53 +01:00
LICENSE.txt YARN-11356. Upgrade DataTables to 1.11.5 to fix CVEs. Contributed by Bence Kosztolnik. 2022-10-26 22:29:01 +02:00
NOTICE-binary HADOOP-19046. S3A: update AWS V2 SDK to 2.23.5; v1 to 1.12.599 (#6467) 2024-01-21 19:00:34 +00:00
NOTICE.txt HADOOP-15958. Revisiting LICENSE and NOTICE files. 2019-08-27 13:47:12 +09:00
pom.xml Preparing for 3.5.0 development (#6411) 2024-01-19 15:05:22 +08:00
README.txt HADOOP-15958. Revisiting LICENSE and NOTICE files. 2019-08-27 13:47:12 +09:00
start-build-env.sh Minor, fix cpu arch compare to use correct Dockerfile (#6852) 2024-06-13 00:37:28 +05:30

For the latest information about Hadoop, please visit our website at:

   http://hadoop.apache.org/

and our wiki, at:

   https://cwiki.apache.org/confluence/display/HADOOP/