big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
20,023 Commits 2 Branches 2 Tags 264 MiB
b54b0c1b67
Commit Graph

112 Commits

Author SHA1 Message Date
Thomas Marquardt
b54b0c1b67 HADOOP-15659. Code changes for bug fix and new tests. 
Contributed by Da Zhou.
 2018-09-17 19:54:01 +00:00
Thomas Marquardt
ce03a93f78 HADOOP-15446. ABFS: tune imports & javadocs; stabilise tests. 
Contributed by Steve Loughran and Da Zhou.
 2018-09-17 19:54:01 +00:00
Steve Loughran
a271fd0eca HADOOP-15560. ABFS: removed dependency injection and unnecessary dependencies. 
Contributed by Da Zhou.
 2018-09-17 19:54:01 +00:00
Steve Loughran
f044deedbb HADOOP-15407. HADOOP-15540. Support Windows Azure Storage - Blob file system "ABFS" in Hadoop: Core Commit. 
Contributed by Shane Mainali, Thomas Marquardt, Zichen Sun, Georgi Chalakov, Esfandiar Manii, Amit Singh, Dana Kaban, Da Zhou, Junhua Gu, Saher Ahwal, Saurabh Pant, James Baker, Shaoyu Zhang, Lawrence Chen, Kevin Chen and Steve Loughran
 2018-09-17 19:54:01 +00:00
Steve Loughran
1eef0acfe5
HADOOP-15677. WASB: Add support for StreamCapabilities. 
Contributed by Thomas Marquardt.
 2018-09-10 14:24:41 +01:00
Steve Loughran
45d9568aaa
HADOOP-15547/ WASB: improve listStatus performance. 
Contributed by Thomas Marquardt.

(cherry picked from commit 749fff577ed9afb4ef8a54b8948f74be083cc620)
 2018-07-19 12:31:19 -07:00
Akira Ajisaka
2b2399d623
HADOOP-15495. Upgrade commons-lang version to 3.7 in hadoop-common-project and hadoop-tools. Contributed by Takanobu Asanuma. 2018-06-28 14:37:22 +09:00
Todd Lipcon
7a3c6e9c3c HADOOP-15550. Avoid static initialization of ObjectMappers 2018-06-25 15:36:45 -07:00
Chris Douglas
f34744603e HADOOP-15533. Make WASB listStatus messages consistent. Contributed by Esfandiar Manii 2018-06-17 23:12:18 -07:00
Steve Loughran
d901be6795
HADOOP-15506. Upgrade Azure Storage Sdk version to 7.0.0 and update corresponding code blocks. 
Contributed by Esfandiar Manii.
 2018-06-06 18:28:14 +01:00
Steve Loughran
ba842847c9 HADOOP-15478. WASB: hflush() and hsync() regression. 
Contributed by Thomas Marquardt.
 2018-05-21 11:12:34 +01:00
Steve Loughran
5b11b9fd41 HADOOP-15446. WASB: PageBlobInputStream.skip breaks HBASE replication. 
Contributed by Thomas Marquardt
 2018-05-07 11:54:08 +01:00
Chris Douglas
081c350188 HADOOP-15320. Remove customized getFileBlockLocations for hadoop-azure and hadoop-azure-datalake. Contributed by Shanyu Zhao 2018-03-28 11:58:59 -07:00
Steve Loughran
52babbb4a0 HADOOP-15086. NativeAzureFileSystem file rename is not atomic. 
Contributed by Thomas Marquardt
 2017-12-22 11:39:55 +00:00
Steve Loughran
0ea182d0fa HADOOP-15042. Azure PageBlobInputStream.skip() can return negative value when numberOfPagesRemaining is 0. 
Contributed by Rajesh Balamohan
 2017-11-28 11:52:59 +00:00
John Zhuge
bf6a660232 HADOOP-15012. Add readahead, dropbehind, and unbuffer to StreamCapabilities. Contributed by John Zhuge. 2017-11-09 10:16:12 -08:00
Steve Loughran
9fcc3a1fc8 HADOOP-14935. Azure: POSIX permissions are taking effect in access() method even when authorization is enabled. 
Contributed by Santhosh G Nayak
 2017-10-16 16:01:47 +01:00
Steve Loughran
3de574413c HADOOP-14913. Sticky bit implementation for rename() operation in Azure WASB. 
Contributed by Varada Hemeswari.
 2017-10-11 18:06:43 +01:00
Steve Loughran
572cdb5463 HADOOP-14899. Restrict Access to setPermission operation when authorization is enabled in WASB 
Contributed by Kannapiran Srinivasan.
 2017-10-06 17:43:38 +01:00
Steve Loughran
9288206cb3
HADOOP-14845. Azure wasb: getFileStatus not making any auth check. 
Contributed by Sivaguru Sankaridurg
 2017-10-05 15:05:23 +01:00
Steve Loughran
a530e7ab3b
HADOOP-14768. Honoring sticky bit during Deletion when authorization is enabled in WASB 
Contributed by Varada Hemeswari
 2017-09-28 19:52:56 +01:00
Steve Loughran
2d2d97fa7d
HADOOP-14553. Add (parallelized) integration tests to hadoop-azure 
Contributed by Steve Loughran
 2017-09-15 17:03:01 +01:00
Steve Loughran
13eda50003
HADOOP-14520. WASB: Block compaction for Azure Block Blobs. 
Contributed by Georgi Chalakov
 2017-09-07 18:35:03 +01:00
Steve Loughran
792eff9ea7
HADOOP-14820 Wasb mkdirs security checks inconsistent with HDFS. 
Contributed by Sivaguru Sankaridurg
 2017-09-05 14:16:57 +01:00
Steve Loughran
9374f38820
HADOOP-14583. wasb throws an exception if you try to create a file and there's no parent directory 
Contributed by Esfandiar Manii and Thomas Marquardt.
 2017-08-29 19:03:44 +01:00
Steve Loughran
021974f4cb
HADOOP-14802. Add support for using container saskeys for all accesses. 
Contributed by Sivaguru Sankaridurg
 2017-08-29 19:02:43 +01:00
Steve Loughran
c6b4e656b7
HADOOP-14769. WASB: delete recursive should not fail if a file is deleted. 
Contributed by Thomas Marquardt
 2017-08-18 14:13:40 +01:00
Steve Loughran
54356b1e83
HADOOP-14183. Remove service loader config file for wasb fs. 
Contributed by Esfandiar Manii.
 2017-08-10 16:46:33 +01:00
Steve Loughran
d91b7a8451
HADOOP-14722. Azure: BlockBlobInputStream position incorrect after seek. 
Contributed by Thomas Marquardt
 2017-08-06 20:19:23 +01:00
Steve Loughran
778d4edd9a
HADOOP-14660 wasb: improve throughput by 34% when account limit exceeded. 
Contributed by Thomas Marquardt.
 2017-08-01 21:36:28 +01:00
Jitendra Pandey
a92bf39e23 HADOOP-14680. Azure: IndexOutOfBoundsException in BlockBlobInputStream. Contributed by Thomas Marquardt. 2017-07-25 16:26:48 -07:00
Jitendra Pandey
f2921e51f0 HADOOP-14518. Customize User-Agent header sent in HTTP/HTTPS requests by WASB. Contributed by Georgi Chalakov. 2017-07-24 23:01:01 -07:00
Jitendra Pandey
2843c688bc HADOOP-14642. wasb: add support for caching Authorization and SASKeys. Contributed by Sivaguru Sankaridurg. 2017-07-19 00:13:06 -07:00
Jitendra Pandey
b0e78ae085 HADOOP-14640. Azure: Support affinity for service running on localhost and reuse SPNEGO hadoop.auth cookie for authorization, SASKey and delegation token generation. Contributed by Santhosh G Nayak. 2017-07-17 02:27:55 -07:00
Steve Loughran
7d272ea124
HADOOP-14581. Restrict setOwner to list of user when security is enabled in wasb. 
Contributed by Varada Hemeswari

(cherry picked from commit 1e69e5260351effc8077d1bdc397cec57cf1ff1b)
 2017-07-12 10:37:39 +01:00
Steve Loughran
d670c3a4da
HADOOP-14535 wasb: implement high-performance random access and seek of block blobs. 
Contributed by Thomas Marquardt
 2017-07-11 21:34:27 +01:00
Mingliang Liu
38996fdcf0 HADOOP-14443. Azure: Support retry and client side failover for authorization, SASKey and delegation token generation. Contributed by Santhosh G Nayak 2017-06-30 16:53:48 -07:00
Mingliang Liu
b08cc97396 HADOOP-14601. Azure: Reuse ObjectMapper. Contributed by Mingliang Liu 2017-06-29 09:22:00 -07:00
Mingliang Liu
990aa34de2 HADOOP-14609. NPE in AzureNativeFileSystemStore.checkContainer() if StorageException lacks an error code. Contributed by Steve Loughran 2017-06-28 14:18:59 -07:00
Mingliang Liu
7e031c2c18 HADOOP-14546. Azure: Concurrent I/O does not work when secure.mode is enabled. Contributed by Thomas 2017-06-27 17:32:07 -07:00
Steve Loughran
48f4a229a9
HADOOP-14461 Azure: handle failure gracefully in case of missing account access key. 
Contributed by Mingliang Liu.
 2017-06-26 13:39:47 +01:00
Steve Loughran
c6bd73c6c5
HADOOP-14547. [WASB] the configured retry policy is not used for all storage operations. 
Contributed by Thomas.
 2017-06-23 21:38:41 +01:00
Mingliang Liu
325163f23f HADOOP-14512. WASB atomic rename should not throw exception if the file is neither in src nor in dst when doing the rename. Contributed by Duo Xu 2017-06-09 13:07:11 -07:00
Mingliang Liu
5fd9742c83 HADOOP-14478. Optimize NativeAzureFsInputStream for positional reads. Contributed by Rajesh Balamohan 2017-06-05 15:56:43 -07:00
Mingliang Liu
89bb8bfe58 HADOOP-14442. Owner support for ranger-wasb integration. Contributed by Varada Hemeswari 2017-05-26 17:54:00 -07:00
Mingliang Liu
b415c6fe74 HADOOP-14416. Path starting with 'wasb:///' not resolved correctly while authorizing with WASB-Ranger. Contributed by Sivaguru Sankaridurg 2017-05-16 11:22:32 -07:00
Mingliang Liu
eed731496f HADOOP-14361. Azure: NativeAzureFileSystem.getDelegationToken() call fails sometimes when invoked concurrently. Contributed by Santhosh G Nayak 2017-05-10 15:05:11 -07:00
Akira Ajisaka
2ba9903932
HADOOP-14400. Fix warnings from spotbugs in hadoop-tools. Contributed by Weiwei Yang. 2017-05-10 05:57:12 -05:00
Andrew Wang
54fd0e44b7 HDFS-11644. Support for querying outputstream capabilities. Contributed by Manoj Govindassamy. 2017-05-08 21:59:49 -07:00
Mingliang Liu
8b5f2c372e HADOOP-14351. Azure: RemoteWasbAuthorizerImpl and RemoteSASKeyGeneratorImpl should not use Kerberos interactive user cache. Contributed by Santhosh G Nayak 2017-04-26 13:47:18 -07:00