hadoop/hadoop-tools
Steve Loughran 7651afd3db
HADOOP-19057. S3A: Landsat bucket used in tests no longer accessible (#6515)
The AWS landsat data previously used in some S3A tests is no
longer accessible

This PR moves to the new external file
s3a://noaa-cors-pds/raw/2024/001/akse/AKSE001x.24_.gz

* Large enough file for scale tests
* Bucket supports anonymous access
* Ends in .gz to keep codec tests happy
* No spaces in path to keep bucket-info happy

Test Code Changes
* Leaves the test key name alone: fs.s3a.scale.test.csvfile
* Rename all methods and fields move remove "csv" from their names and
  move to "external file" we no longer require it to be CSV.
* Path definition and helper methods have been moved to PublicDatasetTestUtils
* Improve error reporting in ITestS3AInputStreamPerformance if the file
  is too short
  
With S3 Select removed, there is no need for the file to be
a CSV file; there is a test which tries to unzip it; other
tests have a minimum file size.

Consult the JIRA for the settings to add to auth-keys.xml
to switch earlier builds to this same file.

Contributed by Steve Loughran
2024-02-13 10:46:36 +00:00
..
hadoop-aliyun Preparing for 3.5.0 development (#6411) 2024-01-19 15:05:22 +08:00
hadoop-archive-logs Preparing for 3.5.0 development (#6411) 2024-01-19 15:05:22 +08:00
hadoop-archives Preparing for 3.5.0 development (#6411) 2024-01-19 15:05:22 +08:00
hadoop-aws HADOOP-19057. S3A: Landsat bucket used in tests no longer accessible (#6515) 2024-02-13 10:46:36 +00:00
hadoop-azure HADOOP-18883. [ABFS]: Expect-100 JDK bug resolution: prevent multiple server calls (#6022) 2024-01-21 19:14:54 +00:00
hadoop-azure-datalake Preparing for 3.5.0 development (#6411) 2024-01-19 15:05:22 +08:00
hadoop-benchmark Preparing for 3.5.0 development (#6411) 2024-01-19 15:05:22 +08:00
hadoop-datajoin Preparing for 3.5.0 development (#6411) 2024-01-19 15:05:22 +08:00
hadoop-distcp HDFS-17376. Distcp creates Factor 1 replication file on target if Source is EC. (#6540) 2024-02-09 17:26:32 +00:00
hadoop-dynamometer Preparing for 3.5.0 development (#6411) 2024-01-19 15:05:22 +08:00
hadoop-extras Preparing for 3.5.0 development (#6411) 2024-01-19 15:05:22 +08:00
hadoop-federation-balance Preparing for 3.5.0 development (#6411) 2024-01-19 15:05:22 +08:00
hadoop-fs2img Preparing for 3.5.0 development (#6411) 2024-01-19 15:05:22 +08:00
hadoop-gridmix Preparing for 3.5.0 development (#6411) 2024-01-19 15:05:22 +08:00
hadoop-kafka Preparing for 3.5.0 development (#6411) 2024-01-19 15:05:22 +08:00
hadoop-openstack Preparing for 3.5.0 development (#6411) 2024-01-19 15:05:22 +08:00
hadoop-pipes Preparing for 3.5.0 development (#6411) 2024-01-19 15:05:22 +08:00
hadoop-resourceestimator Preparing for 3.5.0 development (#6411) 2024-01-19 15:05:22 +08:00
hadoop-rumen Preparing for 3.5.0 development (#6411) 2024-01-19 15:05:22 +08:00
hadoop-sls Preparing for 3.5.0 development (#6411) 2024-01-19 15:05:22 +08:00
hadoop-streaming Preparing for 3.5.0 development (#6411) 2024-01-19 15:05:22 +08:00
hadoop-tools-dist Preparing for 3.5.0 development (#6411) 2024-01-19 15:05:22 +08:00
pom.xml Preparing for 3.5.0 development (#6411) 2024-01-19 15:05:22 +08:00