Go to file
Anuj Modi dbe2d61258
HADOOP-19096. [ABFS] [CST Optimization] Enhance Client-Side Throttling Metrics Logic (#6276)
ABFS has a client-side throttling mechanism which works on the metrics collected
from past requests

When requests are fail due to server-side throttling it updates its
metrics and recalculates any client side backoff.

The choice of which requests should be used to compute client side
backoff interval is based on the http status code:

- Status code in 2xx range: Successful Operations should contribute.
- Status code in 3xx range: Redirection Operations should not contribute.
- Status code in 4xx range: User Errors should not contribute.
- Status code is 503: Throttling Error should contribute only if they
  are due to client limits breach as follows:
  * 503, Ingress Over Account Limit: Should Contribute
  * 503, Egress Over Account Limit: Should Contribute
  * 503, TPS Over Account Limit: Should Contribute
  * 503, Other Server Throttling: Should not Contribute.
- Status code in 5xx range other than 503: Should not Contribute.
- IOException and UnknownHostExceptions: Should not Contribute.

Contributed by Anuj Modi
2024-04-10 14:46:23 +01:00
.github HADOOP-18823. Add Labeler Github Action. (#5874). Contributed by Ayush Saxena. 2023-07-25 03:04:49 +05:30
.yetus Add .yetus/excludes.txt (#4984) 2022-10-11 09:23:34 -07:00
dev-support HADOOP-18135. Produce Windows binaries of Hadoop (#6673) 2024-04-09 22:15:05 +05:30
hadoop-assemblies HADOOP-18088. Replace log4j 1.x with reload4j. (#4052) 2024-02-13 16:33:51 +00:00
hadoop-build-tools Preparing for 3.5.0 development (#6411) 2024-01-19 15:05:22 +08:00
hadoop-client-modules Revert "HADOOP-16822. Provide source artifacts for hadoop-client-api. Contributed by Karel Kolman." (#6458) 2024-04-10 12:03:59 +01:00
hadoop-cloud-storage-project HADOOP-19024. Use bouncycastle jdk18 1.77 (#6410). Contributed 2024-03-30 19:58:12 +05:30
hadoop-common-project HADOOP-18135. Produce Windows binaries of Hadoop (#6673) 2024-04-09 22:15:05 +05:30
hadoop-dist Preparing for 3.5.0 development (#6411) 2024-01-19 15:05:22 +08:00
hadoop-hdfs-project HDFS-17449. Fix ill-formed decommission host name and port pair triggers IndexOutOfBound error (#6691). Contributed by ConfX 2024-04-06 13:38:09 +05:30
hadoop-mapreduce-project HADOOP-19114. Upgrade to commons-compress 1.26.1 due to CVEs. (#6636) 2024-04-03 19:32:15 +01:00
hadoop-maven-plugins HADOOP-19041. Use StandardCharsets in more places (#6449) 2024-03-28 23:17:18 -04:00
hadoop-minicluster Preparing for 3.5.0 development (#6411) 2024-01-19 15:05:22 +08:00
hadoop-project HADOOP-19135. Remove Jcache 1.0-alpha. (#6695) Contributed by Shilun Fan. 2024-04-05 22:09:15 +08:00
hadoop-project-dist HADOOP-19112. Hadoop 3.4.0 release wrap-up. (#6640) Contributed by Shilun Fan. 2024-03-19 20:08:03 +08:00
hadoop-tools HADOOP-19096. [ABFS] [CST Optimization] Enhance Client-Side Throttling Metrics Logic (#6276) 2024-04-10 14:46:23 +01:00
hadoop-yarn-project YARN-11670. Add CallerContext in NodeManager (#6688) 2024-04-08 22:50:41 -04:00
licenses HADOOP-17144. Update Hadoop's lz4 to v1.9.2. Contributed by Hemanth Boyina. 2020-10-18 18:37:46 +05:30
licenses-binary HADOOP-15993. Upgrade Kafka to 2.4.0 in hadoop-kafka module. (#1796) 2020-01-09 16:24:58 +09:00
.asf.yaml HADOOP-18630. Add gh-pages in asf.yaml to deploy the current trunk doc (#5393). Contributed by Simhadri Govindappa. 2023-02-14 18:13:29 +05:30
.gitattributes HADOOP-13598. Add eol=lf for unix format files in .gitattributes. Contributed by Yiqun Lin. 2016-09-14 11:14:31 +09:00
.gitignore HADOOP-18963. Fix typos in .gitignore (#6243) 2023-11-04 05:12:39 +05:30
BUILDING.txt HADOOP-18135. Produce Windows binaries of Hadoop (#6673) 2024-04-09 22:15:05 +05:30
LICENSE-binary HADOOP-19135. Remove Jcache 1.0-alpha. (#6695) Contributed by Shilun Fan. 2024-04-05 22:09:15 +08:00
LICENSE.txt YARN-11356. Upgrade DataTables to 1.11.5 to fix CVEs. Contributed by Bence Kosztolnik. 2022-10-26 22:29:01 +02:00
NOTICE-binary HADOOP-19046. S3A: update AWS V2 SDK to 2.23.5; v1 to 1.12.599 (#6467) 2024-01-21 19:00:34 +00:00
NOTICE.txt HADOOP-15958. Revisiting LICENSE and NOTICE files. 2019-08-27 13:47:12 +09:00
pom.xml Preparing for 3.5.0 development (#6411) 2024-01-19 15:05:22 +08:00
README.txt HADOOP-15958. Revisiting LICENSE and NOTICE files. 2019-08-27 13:47:12 +09:00
start-build-env.sh HADOOP-18052. Support Apple Silicon in start-build-env.sh (#3817) 2021-12-23 18:13:18 +09:00

For the latest information about Hadoop, please visit our website at:

   http://hadoop.apache.org/

and our wiki, at:

   https://cwiki.apache.org/confluence/display/HADOOP/