Sunil G
40e6a85d25
YARN-6560. SLS doesn't honor node total resource specified in sls-runner.xml. Contributed by Wangda Tan.
2017-05-18 22:49:42 +05:30
Steve Loughran
ba70225cf6
HADOOP-11572. s3a delete() operation fails during a concurrent delete of child entries.
...
Contributed by Steve Loughran.
(cherry picked from commit 2ac5aab8d725f761a9f9723471a4426f6b5d78c4)
2017-05-18 15:44:39 +01:00
Akira Ajisaka
d4aa9e3c69
HADOOP-14415. Use java.lang.AssertionError instead of junit.framework.AssertionFailedError. Contributed by Chen Liang.
2017-05-17 15:38:28 -04:00
Robert Kanter
101852ca11
YARN-6535. Program needs to exit when SLS finishes. (yufeigu via rkanter)
2017-05-16 17:52:17 -07:00
Mingliang Liu
b415c6fe74
HADOOP-14416. Path starting with 'wasb:///' not resolved correctly while authorizing with WASB-Ranger. Contributed by Sivaguru Sankaridurg
2017-05-16 11:22:32 -07:00
Akira Ajisaka
0d5c8ed8e0
HADOOP-14401. maven-project-info-reports-plugin can be removed. Contributed by Andras Bokor.
2017-05-11 16:37:32 -05:00
Mingliang Liu
eed731496f
HADOOP-14361. Azure: NativeAzureFileSystem.getDelegationToken() call fails sometimes when invoked concurrently. Contributed by Santhosh G Nayak
2017-05-10 15:05:11 -07:00
Carlo Curino
5cb6e3e082
YARN-6473. Create ReservationInvariantChecker to validate ReservationSystem + Scheduler operations.
2017-05-10 14:49:25 -07:00
Akira Ajisaka
2ba9903932
HADOOP-14400. Fix warnings from spotbugs in hadoop-tools. Contributed by Weiwei Yang.
2017-05-10 05:57:12 -05:00
Akira Ajisaka
c60164fb60
HADOOP-14374. License error in GridmixTestUtils.java. Contributed by lixinglong.
2017-05-09 05:37:43 -05:00
Andrew Wang
54fd0e44b7
HDFS-11644. Support for querying outputstream capabilities. Contributed by Manoj Govindassamy.
2017-05-08 21:59:49 -07:00
Andrew Wang
8065129d87
HADOOP-14298. TestHadoopArchiveLogsRunner fails. Contribute dby Akira Ajisaka.
2017-05-07 13:59:15 -07:00
Steve Loughran
4e6bbd049d
HADOOP-14382 Remove usages of MoreObjects.toStringHelper.
...
Contributed by Andrew Wang
2017-05-05 13:03:09 +01:00
Robert Kanter
3082552b3b
YARN-6522. Make SLS JSON input file format simple and scalable (yufeigu via rkanter)
2017-05-04 17:21:46 -07:00
Mingliang Liu
83dded556d
HDFS-11739. Fix regression in tests caused by YARN-679. Contributed by Steve Loughran
2017-05-03 11:22:44 -07:00
Andrew Wang
b062b323b7
HADOOP-14281. Fix TestKafkaMetrics#testPutMetrics. Contributed by Alison Yu.
2017-05-02 11:49:19 -07:00
Akira Ajisaka
62579b69a0
HADOOP-11614. Remove httpclient dependency from hadoop-openstack. Contributed by Akira Ajisaka, Brahma Reddy Battula, and Steve Loughran.
2017-04-27 16:57:08 +09:00
Mingliang Liu
8b5f2c372e
HADOOP-14351. Azure: RemoteWasbAuthorizerImpl and RemoteSASKeyGeneratorImpl should not use Kerberos interactive user cache. Contributed by Santhosh G Nayak
2017-04-26 13:47:18 -07:00
Mingliang Liu
e4c32bccde
HADOOP-14349. Rename ADLS CONTRACT_ENABLE_KEY. Contributed by Mingliang Liu
2017-04-25 20:54:56 -07:00
Robert Kanter
475f933b41
YARN-6423. Queue metrics doesn't work for Fair Scheduler in SLS (yufeigu via rkanter)
2017-04-25 16:26:09 -07:00
Steve Loughran
5f934f8386
HADOOP-14305 S3A SSE tests won't run in parallel: Bad request in directory GetFileStatus.
...
Contributed by Steve Moist.
2017-04-24 20:33:19 +01:00
Wangda Tan
de69d6e811
YARN-6363. Extending SLS: Synthetic Load Generator. (Carlo Curino via wangda)
2017-04-20 21:54:30 -07:00
Mingliang Liu
667966c13c
HADOOP-14324. Refine S3 server-side-encryption key as encryption secret; improve error reporting and diagnostics. Contributed by Steve Loughran
2017-04-20 17:13:36 -07:00
Andrew Wang
7e075a50e3
YARN-6365. Get static SLS html resources from classpath. Contributed by Yufei Gu.
2017-04-19 11:08:59 -07:00
Steve Loughran
6b015d00c9
HADOOP-14321. explicitly exclude s3a root dir ITests from parallel runs.
...
Contributed by Steve Loughran
2017-04-19 10:21:44 +01:00
Mingliang Liu
0cab57223e
HADOOP-14274. Azure: Simplify Ranger-WASB policy model. Contributed by Sivaguru Sankaridurg
2017-04-12 16:07:10 -07:00
Mingliang Liu
b053fdc547
HADOOP-14255. S3A to delete unnecessary fake directory objects in mkdirs(). Contributed by Mingliang Liu
2017-04-12 14:30:34 -07:00
Chris Nauroth
b8305e6d06
HADOOP-14248. Retire SharedInstanceProfileCredentialsProvider in trunk. Contributed by Mingliang Liu.
2017-04-12 10:02:13 -07:00
John Zhuge
56e81f2a20
HADOOP-14174. Set default ADLS access token provider type to ClientCredential. Contributed by John Zhuge.
...
Signed-off-by: John Zhuge <jzhuge@apache.org>
2017-04-07 12:04:44 -07:00
Andrew Wang
7eea8d83fa
HADOOP-14280. Fix compilation of TestKafkaMetrics.
2017-04-05 15:12:03 -07:00
Mingliang Liu
56ab02eed9
HADOOP-14273. Azure: NativeAzureFileSystem should respect config for kerberosSupportEnabled flag. Contributed by Santhosh G Nayak
2017-04-04 11:10:28 -07:00
Mingliang Liu
27a44b60c1
HADOOP-14272. Azure: WasbRemoteCallHelper should use String equals for comparison. Contributed by Santhosh G Nayak
2017-04-04 11:03:59 -07:00
Chris Douglas
6eba79232f
HADOOP-14271. Correct spelling of 'occurred' and variants. Contributed by Yeliang Cang
2017-04-03 20:13:14 -07:00
Mingliang Liu
5faa949b78
HADOOP-14268. Fix markdown itemization in hadoop-aws documents. Contributed by Akira Ajisaka
2017-04-03 11:07:14 -07:00
Mingliang Liu
26172a94d6
HADOOP-14267. Make DistCpOptions immutable. Contributed by Mingliang Liu
2017-03-31 20:04:26 -07:00
Yongjun Zhang
bf3fb585aa
HADOOP-11794. Enable distcp to copy blocks in parallel. Contributed by Yongjun Zhang, Wei-Chiu Chuang, Xiao Chen, Rosie Li.
2017-03-30 17:38:56 -07:00
Yongjun Zhang
144f1cf765
Revert "HADOOP-11794. Enable distcp to copy blocks in parallel. Contributed by Yongjun Zhang, Wei-Chiu Chuang, Xiao Chen."
...
This reverts commit 064c8b25ec
2017-03-30 17:38:18 -07:00
Yongjun Zhang
064c8b25ec
HADOOP-11794. Enable distcp to copy blocks in parallel. Contributed by Yongjun Zhang, Wei-Chiu Chuang, Xiao Chen.
2017-03-30 17:01:15 -07:00
Wei-Chiu Chuang
8c591b8d19
HDFS-10974. Document replication factor for EC files. Contributed by Yiqun Lin.
2017-03-30 11:16:05 -07:00
Akira Ajisaka
0d053eeb30
HADOOP-14256. [S3A DOC] Correct the format for "Seoul" example. Contributed by Brahma Reddy Battula.
2017-03-30 18:11:50 +09:00
Robert Kanter
6a5516c238
YARN-5654. Not be able to run SLS with FairScheduler (yufeigu via rkanter)
2017-03-29 16:18:13 -07:00
Andrew Wang
4966a6e26e
HADOOP-14223. Extend FileStatus#toString() to include details like Erasure Coding and Encryption. Contributed by Manoj Govindassamy.
2017-03-29 14:37:21 -07:00
Andrew Wang
0e6f8e4bc6
HDFS-10971. Distcp should not copy replication factor if source file is erasure coded. Contributed by Manoj Govindassamy.
2017-03-28 22:14:03 -07:00
John Zhuge
253e3e78ab
HADOOP-14038. Rename ADLS credential properties. Contributed by John Zhuge.
2017-03-27 23:05:57 -07:00
John Zhuge
d1b7439b48
HADOOP-14230. TestAdlFileSystemContractLive fails to clean up. Contributed by John Zhuge.
2017-03-24 13:37:04 -07:00
Andrew Wang
52b00600df
HADOOP-13715. Add isErasureCoded() API to FileStatus class. Contributed by Manoj Govindassamy.
2017-03-24 11:44:46 -07:00
Chris Douglas
c55195588f
HADOOP-14197. Fix ADLS doc for credential provider. Contributed by John Zhuge
2017-03-23 12:17:27 -07:00
Chris Douglas
03955612b7
HADOOP-14196. Azure Data Lake doc is missing required config entry. Contributed by Atul Sikaria
2017-03-23 12:13:23 -07:00
Mingliang Liu
2e30aa72e0
HADOOP-14135. Remove URI parameter in AWSCredentialProvider constructors. Contributed by Mingliang Liu
2017-03-23 11:33:29 -07:00
Steve Loughran
a5a4867f3b
HADOOP-14120 needless S3AFileSystem.setOptionalPutRequestParameters in S3ABlockOutputStream putObject().
...
Contributed by Yuanbo Liu
(cherry picked from commit 20878d052cebc715c2494a97362fdff08885a77f)
2017-03-23 12:54:54 +00:00
Robert Kanter
cc938e99ec
YARN-6326. Shouldn't use AppAttemptIds to fetch applications while AM Simulator tracks app in SLS (yufeigu via rkanter)
2017-03-21 15:21:11 -07:00
Steve Loughran
2841666f1f
HADOOP-14204 S3A multipart commit failing,
...
"UnsupportedOperationException at java.util.Collections$UnmodifiableList.sort".
Contributed by Steve Loughran
(cherry picked from commit 595501c9ddacd5fdb740ac1bf9bfeb0ffbbd6c0e)
2017-03-21 13:15:15 +00:00
Mingliang Liu
8e15e24059
HADOOP-13945. Azure: Add Kerberos and Delegation token support to WASB client. Contributed by Santhosh G Nayak
...
Change-Id: I39fb0a3b0491bf2160571366939a0502a0045429
2017-03-20 16:31:24 -07:00
Arpit Agarwal
6c399a88e9
HADOOP-14059. typo in s3a rename(self, subdir) error message. Contributed by Steve Loughran.
2017-03-20 11:29:23 -07:00
Mingliang Liu
86035c1644
HADOOP-14192. Aliyun OSS FileSystem contract test should implement getTestBaseDir(). Contributed by Mingliang Liu
2017-03-17 11:01:00 -07:00
Mingliang Liu
fa67a96d7b
HADOOP-14170. FileSystemContractBaseTest is not cleaning up test directory clearly. Contributed by Mingliang Liu
2017-03-14 14:38:21 -07:00
Mingliang Liu
e6cda5819b
Revert "HADOOP-14170. FileSystemContractBaseTest is not cleaning up test directory clearly. Contributed by Mingliang Liu"
...
This reverts commit b8c69557b7
2017-03-14 13:03:42 -07:00
Mingliang Liu
b8c69557b7
HADOOP-14170. FileSystemContractBaseTest is not cleaning up test directory clearly. Contributed by Mingliang Liu
2017-03-13 14:15:02 -07:00
John Zhuge
5a40bafdae
HADOOP-14173. Remove unused AdlConfKeys#ADL_EVENTS_TRACKING_SOURCE. Contributed by John Zhuge.
...
Change-Id: I5dc6f885816b8834f718874542dfa373458b0333
2017-03-13 08:11:25 -07:00
Steve Loughran
092ec39fb9
HADOOP-14111 cut some obsolete, ignored s3 tests in TestS3Credentials.
...
Contributed by Yuanbo Liu
2017-03-10 17:43:22 +00:00
Mingliang Liu
881ec4d97b
HADOOP-14153. ADL module has messed doc structure. Contributed by Mingliang Liu
2017-03-10 00:16:09 -08:00
John Zhuge
c5ee7fded4
HADOOP-14123. Remove misplaced ADL service provider config file for FileSystem. Contributed by John Zhuge.
...
Change-Id: Ic956e2eb8189625916442eaffdc69163d32f730e
2017-03-09 18:30:17 -08:00
John Zhuge
98142d2f72
Revert "HADOOP-13606 swift FS to add a service load metadata file. Contributed by Steve Loughran"
...
This reverts commit 53a12fa721
2017-03-08 09:54:22 -08:00
Mingliang Liu
28daaf0eb2
HADOOP-14150. Implement getHomeDirectory() method in NativeAzureFileSystem. Contributed by Santhosh G Nayak
2017-03-07 14:55:52 -08:00
Robert Kanter
1598fd3b79
YARN-6275. Fail to show real-time tracking charts in SLS (yufeigu via rkanter)
2017-03-07 13:47:52 -08:00
Akira Ajisaka
f597f4c43e
HADOOP-14087. S3A typo in pom.xml test exclusions. Contributed by Aaron Fabbri.
2017-03-07 15:14:55 +09:00
Mingliang Liu
686823529b
HADOOP-13930. Azure: Add Authorization support to WASB. Contributed by Sivaguru Sankaridurg and Dushyanth
2017-03-06 17:16:36 -08:00
Mingliang Liu
52d7d5aa1a
Revert "HADOOP-13930. Azure: Add Authorization support to WASB. Contributed by Sivaguru Sankaridurg and Dushyanth"
...
This reverts commit 6b7cd62b8c
2017-03-06 17:10:11 -08:00
Mingliang Liu
c571cda5c7
HADOOP-14048. REDO operation of WASB#AtomicRename should create placeholder blob for destination folder. Contributed by NITIN VERMA
2017-03-06 17:00:13 -08:00
Mingliang Liu
6b7cd62b8c
HADOOP-13930. Azure: Add Authorization support to WASB. Contributed by Sivaguru Sankaridurg and Dushyanth
2017-03-03 17:09:12 -08:00
Akira Ajisaka
490abfb10f
HADOOP-14057. Fix package.html to compile with Java 9.
2017-03-04 00:25:22 +09:00
Steve Loughran
a97833e0ed
HADOOP-14138. Remove S3A ref from META-INF service discovery, rely on existing core-default entry.
...
Contributed by Steve Loughran
(cherry picked from commit 51fefbe38bf40373d2855163fa6444821b0f994c)
2017-03-02 21:44:30 +00:00
Allen Wittenauer
989bd56b9f
HADOOP-12556. KafkaSink jar files are created but not copied to target dist (Babak Behzad via aw)
2017-02-28 16:15:40 -08:00
Wangda Tan
eac6b4c35c
YARN-6216. Unify Container Resizing code paths with Container Updates making it scheduler agnostic. (Arun Suresh via wangda)
2017-02-28 10:35:50 -08:00
Akira Ajisaka
258342e76c
HADOOP-14118. move jets3t into a dependency on hadoop-aws JAR.
2017-02-28 13:47:44 +09:00
Yongjun Zhang
d235dcdf0b
HADOOP-14127. Add log4j configuration to enable logging in hadoop-distcp's tests. (Xiao Chen via Yongjun Zhang)
2017-02-27 20:42:13 -08:00
Steve Loughran
dab00da19f
HADOOP-14028. S3A BlockOutputStreams doesn't delete temporary files in multipart uploads or handle part upload failures.
...
Contributed by Steve Loughran.
(cherry picked from commit 29fe5af017b945d8750c074ca39031b5b777eddd)
2017-02-25 15:35:19 +00:00
Sunil G
b32ffa2753
YARN-4779. Fix AM container allocation logic in SLS. Contributed by Wangda Tan.
2017-02-24 21:39:25 +05:30
Steve Loughran
e60c6543d5
HADOOP-14113. Review ADL Docs. Contributed by Steve Loughran
2017-02-24 13:24:59 +00:00
Steve Loughran
9c22a91662
HADOOP-14114 S3A can no longer handle unencoded + in URIs. Contributed by Sean Mackrory.
...
(cherry picked from commit ff87ca84418a710c6dc884fe8c70947fcc6489d5)
2017-02-24 10:41:36 +00:00
Mingliang Liu
82607fce39
HADOOP-14091. AbstractFileSystem implementaion for 'wasbs' scheme. Contributed Varada Hemeswari.
2017-02-23 13:48:44 -08:00
Mingliang Liu
a4d4a23785
HADOOP-13321. Deprecate FileSystem APIs that promote inefficient call patterns. Contributed by Chris Nauroth and Mingliang Liu
2017-02-23 12:55:40 -08:00
Mingliang Liu
0013090fb4
HADOOP-14102. Relax error message assertion in S3A test ITestS3AEncryptionSSEC. Contributed by Mingliang Liu
2017-02-22 13:34:20 -08:00
Steve Loughran
4f4250fbcc
HADOOP-14099 Split S3 testing documentation out into its own file. Contributed by Steve Loughran.
2017-02-22 11:43:48 +00:00
Mingliang Liu
924def7854
HADOOP-14017. User friendly name for ADLS user and group. Contributed by Vishwajeet Dusane
2017-02-21 13:44:42 -08:00
Steve Loughran
2158496f6b
HADOOP-13826. S3A Deadlock in multipart copy due to thread pool limits. Contributed by Sean Mackrory.
...
(cherry picked from commit e3a74e0369e6e2217d1280179b390227fe1b1684)
2017-02-21 18:28:49 +00:00
Steve Loughran
8035749c26
HADOOP-14081. S3A: Consider avoiding array copy in S3ABlockOutputStream (ByteArrayBlock). Contributed by Rajesh Balamohan
2017-02-20 16:21:46 +00:00
Steve Loughran
3a2e30fa9f
HADOOP-14092. Typo in hadoop-aws index.md. Contributed by John Zhuge
...
(cherry picked from commit b1c1f05b1dc997906390d653dfafb4f0d7e193c4)
2017-02-18 18:17:11 +00:00
Mingliang Liu
658702efff
HADOOP-14040. Use shaded aws-sdk uber-JAR 1.11.86. Contributed by Steve Loughran and Sean Mackrory
2017-02-16 16:51:03 -08:00
Mingliang Liu
bdad8b7b97
HADOOP-14019. Fix some typos in the s3a docs. Contributed by Steve Loughran
2017-02-16 16:41:31 -08:00
Mingliang Liu
f432999025
HADOOP-14049. Honour AclBit flag associated to file/folder permission for Azure datalake account. Contributed by Vishwajeet Dusane
2017-02-16 15:14:25 -08:00
Kai Zheng
cd3e59a3dc
HADOOP-14072. AliyunOSS: Failed to read from stream when seek beyond the download size. Contributed by Genmao Yu
2017-02-15 16:34:30 +08:00
Akira Ajisaka
b9f8491252
HADOOP-14058. Fix NativeS3FileSystemContractBaseTest#testDirWithDifferentMarkersWorks. Contributed by Yiqun Lin.
2017-02-15 01:45:56 +09:00
Lei Xu
71c23c9fc9
HADOOP-13929. ADLS connector should not check in contract-test-options.xml. (John Zhuge via lei)
2017-02-13 13:33:13 -08:00
Kai Zheng
01be4503c3
HADOOP-14069. AliyunOSS: listStatus returns wrong file info. Contributed by Fei Hui
2017-02-13 13:18:20 +08:00
Lei Xu
839b690ed5
HADOOP-13075. Add support for SSE-KMS and SSE-C in s3a filesystem. (Steve Moist via lei)
2017-02-11 13:59:03 -08:00
Arpit Agarwal
2b7a7bbe0f
HADOOP-14055. SwiftRestClient includes pass length in exception if auth fails. Contributed by Marcell Hegedus.
2017-02-10 10:09:31 -08:00
Kai Zheng
5b151290ae
HADOOP-13768. AliyunOSS: handle the failure in the batch delete operation deleteDirs. Contributed by Genmao Yu
2017-02-10 14:53:18 +08:00
Kai Zheng
a8a594b4c8
HADOOP-14065. AliyunOSS: oss directory filestatus should use meta time. Contributed by Fei Hui
2017-02-09 18:00:22 +08:00
Akira Ajisaka
98c35bb1a4
HADOOP-13858. TestGridmixMemoryEmulation and TestResourceUsageEmulators fail on the environment other than Linux or Windows.
2017-01-21 17:31:20 +09:00
Lei Xu
ccf2d66244
HADOOP-13962. Update ADLS SDK to 2.1.4. (John Zhuge via lei)
2017-01-21 10:52:40 +08:00
Lei Xu
e015b56319
HADOOP-13956. Read ADLS credentials from Credential Provider. (John Zhuge via lei)
2017-01-20 14:48:04 +08:00
Andrew Wang
5d8b80ea9b
Preparing for 3.0.0-alpha3 development
2017-01-19 15:50:07 -08:00
Steve Loughran
ed33ce11dd
HADOOP-13496. Include file lengths in Mismatch in length error for distcp. Contributed by Ted Yu
...
(cherry picked from commit 77401bd5fcca5127c9908156971eeec468371f47)
2017-01-19 11:25:40 +00:00
Steve Loughran
e648b6e138
HADOOP-13336 S3A to support per-bucket configuration. Contributed by Steve Loughran
2017-01-11 17:25:15 +00:00
Wangda Tan
287d3d6804
YARN-3955. Support for application priority ACLs in queues of CapacityScheduler. (Sunil G via wangda)
2017-01-09 08:40:39 -08:00
Chris Nauroth
4c8f9e1302
HDFS-9483. Documentation does not cover use of "swebhdfs" as URL scheme for SSL-secured WebHDFS. Contributed by Surendra Singh Lilhore.
2017-01-05 15:04:47 -08:00
Wangda Tan
0a55bd841e
YARN-5959. RM changes to support change of container ExecutionType. (Arun Suresh via wangda)
2017-01-05 10:31:05 -08:00
Wangda Tan
5ed63e3e9d
YARN-4899. Queue metrics of SLS capacity scheduler only activated after app submit to the queue. (Jonathan Hung via wangda)
2017-01-04 19:17:11 -08:00
Akira Ajisaka
01d31fe938
HADOOP-12733. Remove references to obsolete io.seqfile configuration variables. Contributed by Ray Chiang.
2017-01-04 14:10:36 +09:00
Mingliang Liu
c6a5b689db
HADOOP-13942. Build failure due to errors of javadoc build in hadoop-azure. Contributed by Kai Sasaki
2016-12-29 16:24:06 -08:00
Mingliang Liu
e92a77099b
HADOOP-13863. Azure: Add a new SAS key mode for WASB. Contributed by Dushyanth
2016-12-22 20:15:18 -08:00
Akira Ajisaka
ae401539ea
YARN-4994. Use MiniYARNCluster with try-with-resources in tests. Contributed by Andras Bokor.
2016-12-22 14:32:24 +09:00
Wangda Tan
3bcfe3a9de
YARN-5706. Fail to launch SLSRunner due to NPE. (Kai Sasaki via wangda)
2016-12-20 11:03:29 -08:00
Mingliang Liu
f92913c35b
HADOOP-13831. Correct check for error code to detect Azure Storage Throttling and provide retries. Contributed by Gaurav Kanade
2016-12-15 12:35:08 -08:00
Mingliang Liu
ef34bf2bb9
HADOOP-13900. Remove snapshot version of SDK dependency from Azure Data Lake Store File System. Contributed by Vishwajeet Dusane
2016-12-13 10:34:32 -08:00
Mingliang Liu
c6a3923245
HADOOP-13871. ITestS3AInputStreamPerformance.testTimeToOpenAndReadWholeFileBlocks performance awful. Contributed by Steve Loughran
2016-12-12 14:55:34 -08:00
Mingliang Liu
15dd1f3381
HADOOP-13675. Bug in return value for delete() calls in WASB. Contributed by Dushyanth
2016-12-05 12:04:07 -08:00
Mingliang Liu
4113ec5fa5
HADOOP-13257. Improve Azure Data Lake contract tests. Contributed by Vishwajeet Dusane
2016-12-02 15:54:57 -08:00
Mingliang Liu
51211a7d7a
HADOOP-13855. Fix a couple of the s3a statistic names to be consistent with the rest. Contributed by Steve Loughran
2016-12-02 14:01:42 -08:00
Mingliang Liu
2ff84a0040
HADOOP-13857. S3AUtils.translateException to map (wrapped) InterruptedExceptions to InterruptedIOEs. Contributed by Steve Loughran
2016-12-02 13:36:04 -08:00
Mingliang Liu
3fd844b99f
HADOOP-13830. Intermittent failure of ITestS3NContractRootDir#testRecursiveRootListing: "Can not create a Path from an empty string". Contributed by Steve Loughran
2016-11-30 13:01:19 -08:00
Mingliang Liu
d60a60be8a
HADOOP-13823. s3a rename: fail if dest file exists. Contributed by Steve Loughran
2016-11-28 16:30:29 -08:00
Akira Ajisaka
209e805430
HADOOP-13506. Redundant groupid warning in child projects. Contributed by Kai Sasaki.
2016-11-28 14:34:57 +09:00
Akira Ajisaka
9e173d2bac
HADOOP-13816. Ambiguous plugin version warning from maven build. Contributed by Kai Sasaki.
2016-11-28 13:10:31 +09:00
Steve Loughran
0de0c32ddd
HADOOP-13801 regression: ITestS3AMiniYarnCluster failing. Contributed by Steve Loughran
2016-11-23 21:37:39 +00:00
Mingliang Liu
af791b774b
HADOOP-13050. Upgrade to AWS SDK 10.11+. Contributed by Chris Nauroth and Steve Loughran
2016-11-22 14:00:35 -08:00
Mingliang Liu
beb70fed4f
HADOOP-13655. document object store use with fs shell and distcp. Contributed by Steve Loughran
...
This closes #131
2016-11-22 13:12:23 -08:00
Wei-Chiu Chuang
c0b1a44f6c
HADOOP-13660. Upgrade commons-configuration version. Contributed by Sean Mackrory.
2016-11-17 22:48:35 -06:00
Mingliang Liu
5af572b644
HADOOP-13427. Eliminate needless uses of FileSystem#{exists(), isFile(), isDirectory()}. Contributed by Steve Loughran and Mingliang Liu
2016-11-15 10:57:00 -08:00
Chris Douglas
5c61ad2488
HADOOP-13037. Refactor Azure Data Lake Store as an independent FileSystem. Contributed by Vishwajeet Dusane
2016-11-11 11:15:07 -08:00
Mingliang Liu
89354f0475
HADOOP-13687. Provide a unified dependency artifact that transitively includes the cloud storage modules shipped with Hadoop. Contributed by Chris Nauroth
2016-11-10 08:58:37 -08:00
Jason Lowe
3f93ac0733
YARN-5356. NodeManager should communicate physical resource capability to ResourceManager. Contributed by Inigo Goiri
2016-11-08 22:01:26 +00:00
Xiao Chen
acd509dc57
HADOOP-13795. Skip testGlobStatusThrowsExceptionForUnreadableDir in TestFSMainOperationsSwift. Contributed by John Zhuge.
2016-11-07 09:21:01 -08:00
Akira Ajisaka
b970446b2c
HADOOP-13798. TestHadoopArchives times out.
2016-11-07 19:53:43 +09:00
Akira Ajisaka
de0132748d
MAPREDUCE-6796. Remove unused properties from JTConfig.java. Contributed by Haibo Chen.
2016-11-05 03:49:19 +09:00
Arun Suresh
0aafc122d4
YARN-2995. Enhance UI to show cluster resource utilization of various container Execution types. (Konstantinos Karanasos via asuresh)
2016-11-04 07:31:54 -07:00
Xiao Chen
20c4d8efab
HADOOP-13787. Azure testGlobStatusThrowsExceptionForUnreadableDir fails. Contributed by John Zhuge.
2016-11-03 11:16:07 -07:00
Steve Loughran
a1761a841e
HADOOP-13680. fs.s3a.readahead.range to use getLongBytes. Contributed by Abhishek Modi.
2016-10-31 20:54:46 +00:00
Robert Kanter
5877f20f9c
HADOOP-10075. Update jetty dependency to version 9 (rkanter)
2016-10-27 16:09:00 -07:00
Masatake Iwasaki
0bdd263d82
HADOOP-13017. Implementations of InputStream.read(buffer, offset, bytes) to exit 0 if bytes==0. Contributed by Steve Loughran.
2016-10-27 15:46:59 +09:00
Chris Nauroth
1f8490a5ba
HADOOP-13502. Split fs.contract.is-blobstore flag into more descriptive flags for use by contract tests. Contributed by Chris Nauroth.
2016-10-26 08:55:42 -07:00
Chris Nauroth
9cad3e2350
HADOOP-13614. Purge some superfluous/obsolete S3 FS tests that are slowing test runs down. Contributed by Steve Loughran.
2016-10-26 08:27:26 -07:00
Akira Ajisaka
d3bb69a667
YARN-5575. Many classes use bare yarn. properties instead of the defined constants. Contributed by Daniel Templeton.
2016-10-26 15:32:07 +09:00
Yongjun Zhang
0f0c15f7a5
HDFS-11040. Add documentation for HDFS-9820 distcp improvement. Contributed by Yongjun Zhang.
2016-10-25 12:25:40 -07:00
Yongjun Zhang
3a60573039
Revert "Fix HDFS-11040"
...
This reverts commit 54c1815790
2016-10-25 12:25:02 -07:00
Yongjun Zhang
54c1815790
Fix HDFS-11040
2016-10-25 12:19:34 -07:00
Chris Nauroth
309a43925c
HADOOP-13309. Document S3A known limitations in file ownership and permission model. Contributed by Chris Nauroth.
2016-10-25 09:03:03 -07:00
Chris Nauroth
3372e94030
HADOOP-12774. s3a should use UGI.getCurrentUser.getShortname() for username. Contributed by Steve Loughran.
2016-10-24 21:54:06 -07:00
Chris Nauroth
d8fa1cfa67
HADOOP-13727. S3A: Reduce high number of connections to EC2 Instance Metadata Service caused by InstanceProfileCredentialsProvider. Contributed by Chris Nauroth.
2016-10-24 21:22:34 -07:00
Chris Douglas
a1a0281e12
HADOOP-13626. Remove distcp dependency on FileStatus serialization
2016-10-24 12:46:54 -07:00
Anu Engineer
3fbf4cd5da
HADOOP-13716. Add LambdaTestUtils class for tests; fix eventual consistency problem in contract test setup. Contributed by Steve Loughran.
2016-10-20 12:33:58 -07:00
Steve Loughran
9ae270af02
HADOOP-13735 ITestS3AFileContextStatistics.testStatistics() failing. Contributed by Pieter Reuse
2016-10-20 14:50:59 +01:00
Yongjun Zhang
8650cc84f2
HDFS-9820. Improve distcp to support efficient restore to an earlier snapshot. Contributed by Yongjun Zhang.
2016-10-19 17:37:54 -07:00
Xiao Chen
efdf810cf9
HADOOP-7352. FileSystem#listStatus should throw IOE upon access error. Contributed by John Zhuge.
2016-10-18 18:18:43 -07:00
Steve Loughran
6c348c5691
HADOOP-13560. S3ABlockOutputStream to support huge (many GB) file writes. Contributed by Steve Loughran
2016-10-18 21:16:02 +01:00
Yongjun Zhang
0bc6d37f3c
Revert "HDFS-9820. Improve distcp to support efficient restore to an earlier snapshot. Contributed by Yongjun Zhang."
...
This reverts commit 412c4c9a34
2016-10-17 22:47:37 -07:00
Andrew Wang
987ee51141
HADOOP-13724. Fix a few typos in site markdown documents. Contributed by Ding Fei.
2016-10-17 13:25:58 -07:00
Yongjun Zhang
412c4c9a34
HDFS-9820. Improve distcp to support efficient restore to an earlier snapshot. Contributed by Yongjun Zhang.
2016-10-17 11:04:42 -07:00
Mingliang Liu
d9f73f1b7c
HADOOP-13723. AliyunOSSInputStream#read() should update read bytes stat correctly. Contributed by Mingliang Liu
2016-10-13 22:33:55 -07:00
Jing Zhao
0a85d07983
HADOOP-13024. Distcp with -delete feature on raw data not implemented. Contributed by Mavin Martin.
2016-10-13 13:24:54 -07:00
Kai Zheng
c31b5e61b1
HADOOP-13701. AbstractContractRootDirectoryTest can fail when handling delete "/". Contributed by Genmao Yu
2016-10-08 17:35:59 +06:00
Kai Zheng
a57bba470b
Merge branch 'trunk' into HADOOP-12756
2016-10-09 10:29:40 +08:00
Chris Nauroth
69620f9559
HADOOP-13692. hadoop-aws should declare explicit dependency on Jackson 2 jars to prevent classpath conflicts. Contributed by Chris Nauroth.
2016-10-07 11:41:19 -07:00
Steve Loughran
ebd4f39a39
HADOOP-12977 s3a to handle delete("/", true) robustly. Contributed by Steve Loughran.
2016-10-07 12:51:40 +01:00
Steve Loughran
2cc841f16e
HADOOP-13678 Update jackson from 1.9.13 to 2.x in hadoop-tools. Contributed by Akira Ajisaka.
2016-10-06 16:31:00 +01:00
Steve Loughran
d6be1e75d8
HADOOP-12667 s3a to support createNonRecursive API. Contributed by Sean Mackrory
2016-10-05 15:02:11 +01:00
Akira Ajisaka
6cfe649375
HADOOP-13681. Reduce Kafka dependencies in hadoop-kafka module. Contributed by Grant Henke.
...
This closes #133
2016-10-05 20:44:39 +09:00
Chris Nauroth
88b9444a81
HADOOP-13674. S3A can provide a more detailed error message when accessing a bucket through an incorrect S3 endpoint. Contributed by Chris Nauroth.
2016-10-04 10:36:58 -07:00
Steve Loughran
ee0c722dc8
HADOOP-13164 Optimize S3AFileSystem::deleteUnnecessaryFakeDirectories. Contributed by Rajesh Balamohan.
2016-09-29 17:01:00 +01:00
Chris Nauroth
47f80922dc
HADOOP-13599. s3a close() to be non-synchronized, so avoid risk of deadlock on shutdown. Contributed by Steve Loughran.
2016-09-28 16:19:32 -07:00
Kai Zheng
d1443988f8
Revert "HADOOP-13584. hdoop-aliyun: merge HADOOP-12756 branch back"
...
This reverts commit 5707f88d85
2016-09-29 09:18:27 +08:00
Brahma Reddy Battula
e17a4970be
HDFS-9885. Correct the distcp counters name while displaying counters. Contributed by Surendra Singh Lilhore
2016-09-27 10:45:12 +05:30
Kai Zheng
5707f88d85
HADOOP-13584. hdoop-aliyun: merge HADOOP-12756 branch back.
...
HADOOP-12756 branch: Incorporate Aliyun OSS file system implementation. Contributors:
Mingfei Shi (mingfei.shi@intel.com )
Genmao Yu (genmao.ygm@alibaba-inc.com )
2016-09-26 20:42:22 +08:00
Steve Loughran
e5ef51e717
HADOOP-13643. Math error in AbstractContractDistCpTest. Contributed by Aaron Fabbri.
2016-09-23 10:01:30 +01:00
Kai Zheng
26d5df390c
HADOOP-13634. Some configuration in doc has been outdated. Contributed by Genmao Yu
2016-09-23 08:44:28 +08:00
Kai Zheng
a49b3be38e
Merge branch 'trunk' into HADOOP-12756
2016-09-21 14:03:21 +08:00
Kai Zheng
22af6f8db3
HADOOP-13624. Rename TestAliyunOSSContractDispCp. Contributed by Genmao Yu
2016-09-21 14:02:44 +08:00
Kai Zheng
08b37603d9
HADOOP-13591. Unit test failure in TestOSSContractGetFileStatus and TestOSSContractRootDir. Contributed by Genmao Yu
2016-09-20 15:12:02 +08:00
Kai Zheng
9cd4760257
HADOOP-13609. Refine credential provider related codes for AliyunOss integration. Contributed by Genmao Yu
2016-09-20 11:39:02 +08:00
Kai Zheng
005f4528c7
Merge branch 'trunk' into HADOOP-12756
2016-09-20 08:09:29 +08:00
Chris Nauroth
98bdb51397
HADOOP-13169. Randomize file list in SimpleCopyListing. Contributed by Rajesh Balamohan.
2016-09-19 15:16:47 -07:00
Kai Zheng
a1940464a4
HADOOP-13610. Clean up AliyunOss integration tests. Contributed by Genmao Yu
2016-09-18 19:10:48 +08:00
Kai Zheng
dbb28eb1f2
Merge branch 'trunk' into HADOOP-12756
2016-09-18 19:03:39 +08:00
Mingliang Liu
96142efa2d
HADOOP-13621. s3:// should have been fully cut off from trunk. Contributed by Mingliang Liu.
2016-09-17 22:07:46 -07:00
Steve Loughran
53a12fa721
HADOOP-13606 swift FS to add a service load metadata file. Contributed by Steve Loughran
2016-09-14 15:45:04 +01:00
Kai Zheng
60f66a9306
Merge branch 'trunk' into HADOOP-12756
2016-09-14 16:12:40 +08:00
Allen Wittenauer
58ed4fa544
HADOOP-13341. Deprecate HADOOP_SERVERNAME_OPTS; replace with (command)_(subcommand)_OPTS
...
This commit includes the following changes:
HADOOP-13356. Add a function to handle command_subcommand_OPTS
HADOOP-13355. Handle HADOOP_CLIENT_OPTS in a function
HADOOP-13554. Add an equivalent of hadoop_subcmd_opts for secure opts
HADOOP-13562. Change hadoop_subcommand_opts to use only uppercase
HADOOP-13358. Modify HDFS to use hadoop_subcommand_opts
HADOOP-13357. Modify common to use hadoop_subcommand_opts
HADOOP-13359. Modify YARN to use hadoop_subcommand_opts
HADOOP-13361. Modify hadoop_verify_user to be consistent with hadoop_subcommand_opts (ie more granularity)
HADOOP-13564. modify mapred to use hadoop_subcommand_opts
HADOOP-13563. hadoop_subcommand_opts should print name not actual content during debug
HADOOP-13360. Documentation for HADOOP_subcommand_OPTS
This closes apache/hadoop#126
2016-09-12 11:10:00 -07:00
Ravi Prakash
9faccd1046
HADOOP-13587. distcp.map.bandwidth.mb is overwritten even when -bandwidth flag isn't set. Contributed by Zoran Dimitrijevic
2016-09-12 08:26:08 -07:00
Steve Loughran
4b6d795f28
HADOOP-13540 improve section on troubleshooting s3a auth problems. Contributed by Steve Loughran
2016-09-09 18:55:32 +01:00
Steve Loughran
7fdfcd8a6c
HADOOP-13541 explicitly declare the Joda time version S3A depends on. Contributed by Stevel Loughran
2016-09-07 12:25:23 +01:00
Mingfei
e671a0f52b
HADOOP-13481. User documents for Aliyun OSS FileSystem. Contributed by Genmao Yu.
2016-09-07 11:17:43 +08:00
Mingfei
d33e928fbe
HADOOP-13529. Do some code refactoring. Contributed by Genmao Yu.
2016-09-07 11:15:48 +08:00
Mingfei
cdb77110e7
HADOOP-13498. The number of multi-part upload part should not bigger than 10000. Contributed by Genmao Yu.
2016-09-07 11:15:48 +08:00
Mingfei
6bb741b9f8
HADOOP-13499. Support session credentials for authenticating with Aliyun. Contributed by Genmao Yu.
2016-09-07 11:15:48 +08:00
Mingfei
bd2d97adee
HADOOP-13483. File create should throw error rather than overwrite directories. Contributed by Genmao Yu.
2016-09-07 11:15:47 +08:00
Mingfei
4d84c814fc
HADOOP-13491. Fix several warnings from findbugs. Contributed by Genmao Yu.
2016-09-07 11:15:47 +08:00
Kai Zheng
a5d5342228
HADOOP-12756. Incorporate Aliyun OSS file system implementation. Contributed by Mingfei Shi and Lin Zhou
2016-09-07 11:15:47 +08:00
Chris Nauroth
d152557cf7
HADOOP-13447. Refactor S3AFileSystem to support introduction of separate metadata repository and tests. Contributed by Chris Nauroth.
2016-09-06 09:36:21 -07:00
Arun Suresh
d6d9cff21b
YARN-5221. Expose UpdateResourceRequest API to allow AM to request for change in container properties. (asuresh)
2016-08-30 15:52:29 -07:00
Xiaoyu Yao
8b57be108f
HADOOP-12608. Fix exception message in WASB when connecting with anonymous credential. Contributed by Dushyanth.
...
(cherry picked from commit 007a8decc61750720033e06b6d08861ac3788e41)
(cherry picked from commit a65e159c59de41d26e12212dd1b7c28c28265cc3)
2016-08-29 15:30:49 -07:00
Chris Douglas
c37346d0e3
HADOOP-12726. Unsupported FS operations should throw UnsupportedOperationException. Contributed by Daniel Templeton.
2016-08-23 14:12:52 -07:00
Chris Nauroth
6f9c346e57
HADOOP-13446. Support running isolated unit tests separate from AWS integration tests. Contributed by Chris Nauroth.
2016-08-23 07:18:49 -07:00
Chris Nauroth
763f0497bb
HADOOP-13252. Tune S3A provider plugin mechanism. Contributed by Steve Loughran.
2016-08-19 10:48:10 -07:00
Chris Nauroth
ae4db25443
HADOOP-13513. Java 1.7 support for org.apache.hadoop.fs.azure testcases. Contributed by Tibor Kiss.
2016-08-18 10:44:36 -07:00
Junping Du
0da69c324d
YARN-4676. Automatic and Asynchronous Decommissioning Nodes Status Tracking. Contributed by Diniel Zhi.
...
(cherry picked from commit d464483bf7f0b3e3be3ba32cd6c3eee546747ab5)
Conflicts:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceTrackerService.java
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNode.java
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
2016-08-18 07:27:23 -07:00
Steve Loughran
040c185d62
HADOOP-13405 doc for fs.s3a.acl.default indicates incorrect values. Contributed by Shen Yinjie
2016-08-18 14:36:55 +01:00
Chris Nauroth
822d661b8f
HADOOP-13208. S3A listFiles(recursive=true) to do a bulk listObjects instead of walking the pseudo-tree of directories. Contributed by Steve Loughran.
2016-08-17 14:54:54 -07:00
Chris Nauroth
3808876c73
HADOOP-13324. s3a tests don't authenticate with S3 frankfurt (or other V4 auth only endpoints). Contributed by Steve Loughran.
2016-08-16 17:05:52 -07:00
Chris Nauroth
2ed58c40e5
HADOOP-13403. AzureNativeFileSystem rename/delete performance improvements. Contributed by Subramanyam Pattipaka.
2016-08-08 12:28:40 -07:00
Chris Nauroth
4ef1bc0bdb
HADOOP-13459. hadoop-azure runs several test cases repeatedly, causing unnecessarily long running time. Contributed by Chris Nauroth.
2016-08-02 22:32:21 -07:00
Chris Nauroth
b43de80031
HADOOP-13354. Update WASB driver to use the latest version (4.2.0) of SDK for Microsoft Azure Storage Clients. Contributed by Sivaguru Sankaridurg.
2016-07-27 15:50:38 -07:00
Akira Ajisaka
54fe17a607
HADOOP-9427. Use JUnit assumptions to skip platform-specific tests. Contributed by Gergely Novák.
2016-07-27 19:41:09 +09:00
Steve Loughran
86ae218893
HADOOP-13188 S3A file-create should throw error rather than overwrite directories. Contributed by Steve Loughran
2016-07-25 15:50:23 +01:00
Steve Loughran
be9e46b42d
HADOOP-13406 S3AFileSystem: Consider reusing filestatus in delete() and mkdirs(). Contributed by Rajesh Balamohan
2016-07-25 14:45:03 +01:00
Steve Loughran
7052ca8804
HADOOP-13389 TestS3ATemporaryCredentials.testSTS error when using IAM credentials. Contributed by Steven K. Wong
2016-07-23 18:00:08 +01:00
Chris Nauroth
c60a68ce1d
HADOOP-13392. [Azure Data Lake] OAuth2 configuration should be default set to true when AdlFileSystem is configured. Contributed by Vishwajeet Dusane and Chris Douglas.
2016-07-22 09:16:38 -07:00
Chris Douglas
dca6335dbc
HADOOP-13393. Omit unsupported fs.defaultFS setting in ADLS documentation
2016-07-21 23:21:47 -07:00
mattf-horton
12aa184479
HADOOP-13382. Remove unneeded commons-httpclient dependencies from POM files in Hadoop and sub-projects, while adding it in to the only place it is still needed, hadoop-openstack. Contributed by Matt Foley.
2016-07-21 14:43:57 -07:00
Steve Loughran
37362c2f92
HADOOP-13212 Provide an option to set the socket buffers in S3AFileSystem (Rajesh Balamohan)
2016-07-20 13:42:51 +01:00
Andrew Wang
da456ffd62
Preparing for 3.0.0-alpha2 development
2016-07-15 19:04:17 -07:00
Jitendra Pandey
c2f9cd584c
HADOOP-13368. DFSOpsCountStatistics$OpType#fromSymbol and s3a.Statistic#fromSymbol should be O(1) operation. Contributed by Mingliang Liu.
2016-07-15 14:28:53 -07:00
Jian He
819224dcf9
YARN-5270. Solve miscellaneous issues caused by YARN-4844. Contributed by Wangda Tan
2016-07-11 22:36:20 -07:00
Andrew Wang
f292624bd8
HDFS-10300. TestDistCpSystem should share MiniDFSCluster. Contributed by John Zhuge.
2016-07-11 18:06:28 -07:00
Jitendra Pandey
6e597600f7
HADOOP-13283. Support reset operation for new global storage statistics and per FS storage stats. Contributed by Mingliang Liu.
2016-07-07 14:58:19 -07:00
Jitendra Pandey
aa42c7a6dd
HADOOP-13305. Define common statistics names across schemes. Contributed by Mingliang Liu.
2016-07-01 15:34:03 -07:00
Steve Loughran
96fa0f848b
HADOOP-12709 Cut s3:// from trunk. Contributed by Mingliang Liu.
2016-06-29 16:04:50 +01:00
Akira Ajisaka
8d202f1258
HADOOP-13235. Use Date and Time API in KafkaSink.
2016-06-29 07:40:12 -07:00
Yongjun Zhang
8113855b3a
HDFS-10396. Using -diff option with DistCp may get "Comparison method violates its general contract" exception. Contributed by Yongjun Zhang.
2016-06-28 23:15:13 -07:00
Allen Wittenauer
422c73a865
HADOOP-13034. Log message about input options in distcp lacks some items (Takashi Ohnishi via aw)
2016-06-28 07:21:04 -07:00
Steve Loughran
4ee3543625
HADOOP-13203 S3A: Support fadvise "random" mode for high performance readPositioned() reads. Contributed by Rajesh Balamohan and stevel.
2016-06-22 15:45:25 +01:00
Chris Nauroth
b2c596cdda
HADOOP-13287. TestS3ACredentials#testInstantiateFromURL fails if AWS secret key contains +. Contributed by Chris Nauroth.
2016-06-21 11:28:52 -07:00
Jitendra Pandey
b7c4cf7129
HADOOP-13291. Probing stats in DFSOpsCountStatistics/S3AStorageStatistics should be correctly implemented. Contributed by Mingliang Liu.
2016-06-20 16:25:30 -07:00
Chris Nauroth
51d16e7b38
HADOOP-13242. Authenticate to Azure Data Lake using client ID and keys. Contributed by Atul Sikaria.
2016-06-16 23:35:20 -07:00
Chris Nauroth
127d2c7281
HADOOP-13241. document s3a better. Contributed by Steve Loughran.
2016-06-16 11:18:02 -07:00
Ravi Prakash
4aefe119a0
HADOOP-3733. "s3x:" URLs break when Secret Key contains a slash, even if encoded. Contributed by Steve Loughran.
2016-06-16 11:13:35 -07:00
Chris Nauroth
c9e71382a5
HADOOP-12875. [Azure Data Lake] Support for contract test and unit test cases. Contributed by Vishwajeet Dusane.
2016-06-16 09:46:05 -07:00
Wangda Tan
c77a1095dc
YARN-1942. Deprecate toString/fromString methods from ConverterUtils and move them to records classes like ContainerId/ApplicationId, etc. (wangda)
2016-06-14 15:06:38 -07:00
Chris Nauroth
9581fb715c
HADOOP-12666. Support Microsoft Azure Data Lake - as a file system in Hadoop. Contributed by Vishwajeet Dusane.
2016-06-09 14:33:31 -07:00
Andrew Wang
58c3486850
HADOOP-13175. Remove hadoop-ant from hadoop-tools. Contributed by Chris Douglas.
2016-06-09 13:49:52 -07:00
Steve Loughran
31ffaf76f2
HADOOP-12537 S3A to support Amazon STS temporary credentials. Contributed by Sean Mackrory.
2016-06-09 21:00:47 +01:00
Steve Loughran
656c460c0e
HADOOP-13237: s3a initialization against public bucket fails if caller lacks any credentials. Contributed by Chris Nauroth
2016-06-09 17:28:49 +01:00
Steve Loughran
a3f78d8fa8
HADOOP-12807 S3AFileSystem should read AWS credentials from environment variables. Contributed by Tobin Baker.
2016-06-06 23:42:36 +02:00
Chris Nauroth
c58a59f708
HADOOP-13171. Add StorageStatistics to S3A; instrument some more operations. Contributed by Steve Loughran.
2016-06-03 08:55:33 -07:00
Chris Nauroth
16b1cc7af9
HADOOP-13131. Add tests to verify that S3A supports SSE-S3 encryption. Contributed by Steve Loughran.
2016-06-01 14:49:22 -07:00
Steve Loughran
5870611030
HADOOP-13162. Consider reducing number of getFileStatus calls in S3AFileSystem.mkdirs. (Rajesh Balamohan via stevel)
2016-06-01 14:18:20 +01:00
Varun Vasudev
42f90ab885
YARN-4844. Add getMemorySize/getVirtualCoresSize to o.a.h.y.api.records.Resource. Contributed by Wangda Tan.
2016-05-29 21:24:16 +05:30
Yongjun Zhang
cfb860dee7
HADOOP-13199. Add doc for distcp -filters. (John Zhuge via Yongjun Zhang)
2016-05-26 23:30:31 -07:00
Steve Loughran
39ec1515a2
HADOOP-13130. s3a failures can surface as RTEs, not IOEs. (Steve Loughran)
2016-05-21 16:39:31 +01:00
Steve Loughran
757050ff35
HADOOP-12723 S3A: Add ability to plug in any AWSCredentialsProvider. Contributed by Steven Wong.
2016-05-20 13:52:15 +01:00
Steve Loughran
c918286b17
HADOOP-13145 In DistCp, prevent unnecessary getFileStatus call when not preserving metadata. Contributed by Chris Nauroth.
2016-05-20 12:21:59 +01:00
Chris Nauroth
93258459fa
HADOOP-13183. S3A proxy tests fail after httpclient/httpcore upgrade. Contributed by Steve Loughran.
2016-05-19 22:00:21 -07:00
Jing Zhao
03788d3015
HDFS-10397. Distcp should ignore -delete option if -diff option is provided instead of exiting. Contributed by Mingliang Liu.
2016-05-17 15:46:30 -07:00
Steve Loughran
08ea07f1b8
HADOOP-13158 S3AFileSystem#toString might throw NullPointerException due to null cannedACL. (Chris Nauroth via stevel)
2016-05-17 13:19:29 +01:00
Steve Loughran
c69a649257
HADOOP-13163 Reuse pre-computed filestatus in Distcp-CopyMapper (Rajesh Balamohan via stevel)
2016-05-17 13:00:18 +01:00
Allen Wittenauer
730bc746f9
HADOOP-12930. Dynamic subcommands for hadoop shell scripts (aw)
...
This commit contains the following JIRA issues:
HADOOP-12931. bin/hadoop work for dynamic subcommands
HADOOP-12932. bin/yarn work for dynamic subcommands
HADOOP-12933. bin/hdfs work for dynamic subcommands
HADOOP-12934. bin/mapred work for dynamic subcommands
HADOOP-12935. API documentation for dynamic subcommands
HADOOP-12936. modify hadoop-tools to take advantage of dynamic subcommands
HADOOP-13086. enable daemonization of dynamic commands
HADOOP-13087. env var doc update for dynamic commands
HADOOP-13088. fix shellprofiles in hadoop-tools to allow replacement
HADOOP-13089. hadoop distcp adds client opts twice when dynamic
HADOOP-13094. hadoop-common unit tests for dynamic commands
HADOOP-13095. hadoop-hdfs unit tests for dynamic commands
HADOOP-13107. clean up how rumen is executed
HADOOP-13108. dynamic subcommands need a way to manipulate arguments
HADOOP-13110. add a streaming subcommand to mapred
HADOOP-13111. convert hadoop gridmix to be dynamic
HADOOP-13115. dynamic subcommand docs should talk about exit vs. continue program flow
HADOOP-13117. clarify daemonization and security vars for dynamic commands
HADOOP-13120. add a --debug message when dynamic commands have been used
HADOOP-13121. rename sub-project shellprofiles to match the rest of Hadoop
HADOOP-13129. fix typo in dynamic subcommand docs
HADOOP-13151. Underscores should be escaped in dynamic subcommands document
HADOOP-13153. fix typo in debug statement for dynamic subcommands
2016-05-16 17:54:45 -07:00
Chris Nauroth
b9685e85d5
HADOOP-13148. TestDistCpViewFs to include IOExceptions in test error reports. Contributed by Steve Loughran.
2016-05-16 11:53:17 -07:00
Steve Loughran
dd3a8bed0a
HADOOP-13113 Enable parallel test execution for hadoop-aws. Chris Nauroth via stevel
2016-05-13 10:47:12 +01:00
Andrew Wang
fa440a39b7
HADOOP-12868. Fix hadoop-openstack undeclared and unused dependencies. Contributed by Masatake Iwasaki.
2016-05-12 23:58:18 -07:00
Andrew Wang
3c5c57af28
HADOOP-13142. Change project version from 3.0.0 to 3.0.0-alpha1.
2016-05-12 18:27:28 -07:00
Andrew Wang
ca5613af91
Revert "Update project version to 3.0.0-alpha1-SNAPSHOT."
...
This reverts commit 6b53802cba
2016-05-12 15:32:45 -07:00
Steve Loughran
27c4e90efc
HADOOP-13028 add low level counter metrics for S3A; use in read performance tests. contributed by: stevel
...
patch includes
HADOOP-12844 Recover when S3A fails on IOException in read()
HADOOP-13058 S3A FS fails during init against a read-only FS if multipart purge
HADOOP-13047 S3a Forward seek in stream length to be configurable
2016-05-12 19:24:20 +01:00
Andrew Wang
6b53802cba
Update project version to 3.0.0-alpha1-SNAPSHOT.
2016-05-12 11:05:05 -07:00
Steve Loughran
3f2816ab5a
HADOOP-13116 Jets3tNativeS3FileSystemContractTest does not run.
2016-05-12 16:35:57 +01:00
Steve Loughran
def2a6d385
HADOOP-13122 Customize User-Agent header sent in HTTP requests by S3A. Chris Nauroth via stevel.
2016-05-12 13:57:35 +01:00
Chris Nauroth
27242f211e
HADOOP-12801. Suppress obsolete S3FileSystem tests. Contributed by Chris Nauroth.
2016-05-10 15:21:06 -07:00
Steve Loughran
025219b12f
HADOOP-12982 Document missing S3A and S3 properties. (Wei-Chiu Chuang via stevel)
2016-05-10 21:37:22 +01:00
Jian He
996a210ab0
YARN-4778. Support specifying resources for task containers in SLS. Contributed by Wangda Tan
2016-05-09 14:49:41 -07:00
Allen Wittenauer
1c5bbf6499
HADOOP-12866. add a subcommand for gridmix (Kai Sasaki via aw)
2016-05-06 14:07:02 -07:00
Jason Lowe
d0da13229c
YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla
2016-05-05 14:07:54 +00:00
Jing Zhao
af942585a1
HADOOP-12469. distcp should not ignore the ignoreFailures option. Contributed by Mingliang Liu.
2016-05-04 10:23:04 -07:00
Masatake Iwasaki
ef0870ad03
HADOOP-12378. Fix findbugs warnings in hadoop-tools module. Contributed by Akira AJISAKA.
2016-04-29 20:25:52 +09:00
Arun Suresh
341888a0aa
YARN-4412. Create ClusterMonitor to compute ordered list of preferred NMs for OPPORTUNITIC containers. (asuresh)
2016-04-26 20:12:12 -07:00
Yongjun Zhang
959a28dd12
HDFS-10313. Distcp need to enforce the order of snapshot names passed to -diff. (Lin Yiqun via Yongjun Zhang)
2016-04-26 16:08:03 -07:00
Akira Ajisaka
02c51c27d9
HDFS-10298. Document the usage of distcp -diff option. Contributed by Takashi Ohnishi.
2016-04-25 22:33:09 +09:00
Jing Zhao
63e5412f1a
HDFS-9427. HDFS should not default to ephemeral ports. Contributed by Xiaobing Zhou.
2016-04-22 15:14:40 -07:00
Steve Loughran
19f0f9608e
HADOOP-12891. S3AFileSystem should configure Multipart Copy threshold and chunk size. (Andrew Olson via stevel)
2016-04-22 11:25:03 +01:00
Yongjun Zhang
a749ba0cea
HDFS-9670. DistCp throws NPE when source is root. (John Zhuge via Yongjun Zhang)
2016-04-21 12:17:17 -07:00
Jing Zhao
404f57f328
HDFS-10216. Distcp -diff throws exception when handling relative path. Contributed by Takashi Ohnishi.
2016-04-14 10:35:22 -07:00
Steve Loughran
df18b6e984
HADOOP-12963 Allow using path style addressing for accessing the s3 endpoint. (Stephen Montgomery via stevel)
2016-04-14 12:44:55 +01:00
Jason Lowe
2a5da97f81
Revert "YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla"
...
This reverts commit 1cbcd4a491
2016-04-11 15:51:01 +00:00
Steve Loughran
b9e3eff62a
HADOOP-12444 Support lazy seek in S3AInputStream. Rajesh Balamohan via stevel
2016-04-09 11:25:31 +01:00
Chris Nauroth
843ee8d59d
HADOOP-12994. Specify PositionedReadable, add contract tests, fix problems. Contributed by Steve Loughran.
2016-04-08 13:36:58 -07:00
Steve Loughran
ce5b4812f0
HADOOP-12753 S3A JUnit tests failing if using HTTP proxy. (Zoran Rajic via stevel)
2016-04-08 17:10:00 +01:00
Akira Ajisaka
9c32f8785e
HDFS-10266. Remove unused properties dfs.client.file-block-storage-locations.num-threads and dfs.client.file-block-storage-locations.timeout.millis. Contributed by Ray Chiang.
2016-04-08 15:32:13 +09:00
Jason Lowe
1cbcd4a491
YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla
2016-04-05 13:40:19 +00:00
Steve Loughran
89c9347511
HADOOP-12169 ListStatus on empty dir in S3A lists itself instead of returning an empty list. author: Pieter Reuse. - omitted new S3A subclass
2016-04-04 17:02:04 +01:00
Steve Loughran
0ecdd4cffa
HADOOP-12169 ListStatus on empty dir in S3A lists itself instead of returning an empty list. author: Pieter Reuse.
2016-04-03 16:40:19 +01:00
Harsh J
256c82fe29
HADOOP-11687. Ignore x-* and response headers when copying an Amazon S3 object. Contributed by Aaron Peterson and harsh.
2016-04-01 14:18:10 +05:30
Allen Wittenauer
0a74610d1c
HADOOP-11393. Revert HADOOP_PREFIX, go back to HADOOP_HOME (aw)
2016-03-31 07:51:05 -07:00
Akira Ajisaka
0050fa5f1c
MAPREDUCE-6543. Migrate MR client test cases part 2. Contributed by Dustin Cote.
2016-03-29 18:17:52 +09:00
Akira Ajisaka
115be193df
Revert "MAPREDUCE-6543. Migrate MR client test cases part 2. Contributed by Dustin Cote."
...
This reverts commit 2c268cc936
2016-03-28 16:05:35 +09:00
Akira Ajisaka
18c7e58283
HDFS-9640. Remove hsftp from DistCp in trunk. Contributed by Wei-Chiu Chuang.
2016-03-28 15:32:38 +09:00
Akira Ajisaka
2c268cc936
MAPREDUCE-6543. Migrate MR client test cases part 2. Contributed by Dustin Cote.
2016-03-25 15:17:27 +09:00
Allen Wittenauer
738155063e
HADOOP-12857. rework hadoop-tools (aw)
2016-03-23 13:46:38 -07:00
Chris Nauroth
c50aad0f85
HADOOP-12717. NPE when trying to rename a directory in Windows Azure Storage FileSystem. Contributed by Robert Yokota and Gaurav Kanade.
2016-03-04 22:57:43 -08:00
Masatake Iwasaki
cbd31328a6
HADOOP-12470. In-page TOC of documentation should be automatically generated by doxia macro (iwasakims)
2016-03-04 14:11:36 +09:00
Masatake Iwasaki
33a412e8a4
HDFS-9048. DistCp documentation is out-of-dated (Daisuke Kobayashi via iwasakims)
2016-03-03 18:57:23 +09:00
Chris Nauroth
f98dff329b
HADOOP-12853. Change WASB documentation regarding page blob support. Contributed by Madhumita Chakraborty.
2016-03-01 13:18:42 -08:00
Chris Nauroth
d251e55415
HADOOP-12851. S3AFileSystem Uptake of ProviderUtils.excludeIncompatibleCredentialProviders. Contributed by Larry McCay.
2016-02-29 21:59:52 -08:00
Masatake Iwasaki
809ebc0b14
HADOOP-12552. Fix undeclared/unused dependency to httpclient (iwasakims)
2016-02-29 18:20:14 +09:00
Chris Nauroth
7634d404b7
HADOOP-12846. Credential Provider Recursive Dependencies. Contributed by Larry McCay.
2016-02-28 11:22:55 -08:00
cnauroth
3369a4f691
HADOOP-12535. Run FileSystem contract tests with hadoop-azure. Contributed by Madhumita Chakraborty.
2016-02-23 21:37:50 -08:00
cnauroth
27b77751c1
HADOOP-12555. WASB to read credentials from a credential provider. Contributed by Larry McCay.
2016-02-22 17:30:13 -08:00
cnauroth
d4f5fc23b2
HADOOP-11613. Remove commons-httpclient dependency from hadoop-azure. Contributed by Masatake Iwasaki.
2016-02-22 13:39:38 -08:00
Yongjun Zhang
ba1c9d484a
HDFS-9764. DistCp doesn't print value for several arguments including -numListstatusThreads. (Wei-Chiu Chuang via Yongjun Zhang)
2016-02-19 10:17:37 -08:00
cnauroth
8ab7658025
HADOOP-11031. Design Document for Credential Provider API. Contributed by Larry McCay.
2016-02-18 14:06:38 -08:00
cnauroth
76fab26c5c
HADOOP-12548. Read s3a creds from a Credential Provider. Contributed by Larry McCay.
2016-02-17 12:19:59 -08:00
cnauroth
91a96eaa53
HADOOP-12780. During WASB atomic rename handle crash when one directory has been renamed but not file under it. Contributed by Madhumita Chakraborty.
2016-02-12 15:50:10 -08:00
Steve Loughran
29ae258013
HADOOP-12292. Make use of DeleteObjects optional. (Thomas Demoor via stevel)
2016-02-06 15:05:16 +00:00
Akira Ajisaka
700a176fa7
HADOOP-12757. Findbug compilation fails for 'Kafka Library support'. (aajisaka)
2016-02-02 23:15:39 +09:00
Yongjun Zhang
eddd823cd6
HDFS-9638. Improve DistCp Help and documentation. (Wei-Chiu Chuang via Yongjun Zhang)
2016-01-29 12:11:55 -08:00
cnauroth
41da9a0773
MAPREDUCE-6563. Streaming documentation contains a stray % character. Contributed by Chris Nauroth.
2016-01-28 14:46:01 -08:00
Xuan
4efdf3a979
YARN-4612. Fix rumen and scheduler load simulator handle killed tasks
...
properly. Contributed by Ming Ma.
2016-01-26 18:17:12 -08:00
Akira Ajisaka
736eb17a79
HADOOP-12731. Remove useless boxing/unboxing code. Contributed by Kousuke Saruta.
2016-01-25 13:47:29 +09:00
Akira Ajisaka
0bae506c22
HADOOP-12730. Hadoop streaming -mapper and -reducer options are wrongly documented as required. Contributed by Kengo Seki.
2016-01-22 21:08:31 +09:00
Xuan
1708a4cd23
YARN-4611. Fix scheduler load simulator to support multi-layer network
...
location. Contributed by Ming Ma.
2016-01-20 15:41:50 -08:00
Ravi Prakash
1acc509b45
HADOOP-12696. Add tests for S3FileSystem Contract. Contributed by Matt Paduano
2016-01-19 13:57:08 -08:00
Wangda Tan
89d1fd5dac
HADOOP-12356. Fix computing CPU usage statistics on Windows. (Inigo Goiri via wangda)
2016-01-19 21:27:38 +08:00
Wangda Tan
a44ce3f14f
YARN-4502. Fix two AM containers get allocated when AM restart. (Vinod Kumar Vavilapalli via wangda)
2016-01-19 09:30:04 +08:00
Wangda Tan
150f5ae034
Revert "YARN-4502. Fix two AM containers get allocated when AM restart. (Vinod Kumar Vavilapalli via wangda)"
...
This reverts commit 3fe5728563
2016-01-19 09:27:36 +08:00
cnauroth
8bc93db2e7
HADOOP-12635. Adding Append API support for WASB. Contributed by Dushyanth.
2016-01-18 09:08:58 -08:00
Wangda Tan
3fe5728563
YARN-4502. Fix two AM containers get allocated when AM restart. (Vinod Kumar Vavilapalli via wangda)
...
(cherry picked from commit 805a9ed85e
2016-01-18 17:06:05 +08:00
Wangda Tan
adf260a728
Revert "YARN-4502. Fix two AM containers get allocated when AM restart. (Vinod Kumar Vavilapalli via wangda)"
...
This reverts commit 805a9ed85e
2016-01-18 16:50:45 +08:00
Wangda Tan
805a9ed85e
YARN-4502. Fix two AM containers get allocated when AM restart. (Vinod Kumar Vavilapalli via wangda)
2016-01-18 11:04:25 +08:00
Yongjun Zhang
a9c69ebeb7
HDFS-9612. DistCp worker threads are not terminated after jobs are done. (Wei-Chiu Chuang via Yongjun Zhang)
2016-01-15 10:03:09 -08:00
Colin Patrick Mccabe
b1ed28fa77
HADOOP-8887. Use a Maven plugin to build the native code using CMake (cmccabe)
2016-01-14 11:02:34 -08:00
Colin Patrick Mccabe
8315582c4f
HDFS-9517. Fix missing @Test annotation on TestDistCpUtils.testUnpackAttributes (Wei-Chiu Chuang via cmccabe)
2016-01-13 16:28:06 -08:00
Lei Xu
126705f67e
HADOOP-11262. Enable YARN to use S3A. (Pieter Reuse via lei)
2016-01-12 12:19:53 -08:00
Xiaoyu Yao
c2e2e13455
HDFS-8584. NPE in distcp when ssl configuration file does not exist in class path. Contributed by Surendra Singh Lilhore.
2016-01-11 17:08:26 -08:00
Zhe Zhang
95f32015ad
HDFS-9630. DistCp minor refactoring and clean up. Contributed by Kai Zheng.
...
Change-Id: I363c4ffcac32116ddcdc0a22fac3db92f14a0db0
2016-01-11 09:46:56 -08:00
cnauroth
0e76f1fcea
HADOOP-12551. Introduce FileNotFoundException for WASB FileSystem API. Contributed by Dushyanth.
2016-01-09 22:18:11 -08:00
cnauroth
f0fa6d869b
HADOOP-12678. Handle empty rename pending metadata file during atomic rename in redo path. Contributed by Madhumita Chakraborty.
2016-01-08 20:18:59 -08:00
cnauroth
978bbdfeb2
HADOOP-12634. Change Lazy Rename Pending Operation Completion of WASB to address case of potential data loss due to partial copy. Contributed by Gaurav Kanade.
2016-01-06 11:15:59 -08:00
Ravi Prakash
2d16f40dab
HADOOP-12689. S3 filesystem operations stopped working correctly
2016-01-05 23:26:03 -08:00
Wangda Tan
a9594c61bb
YARN-4510. Fix SLS startup failure caused by NPE. (Bibin A Chundatt via wangda)
...
(cherry picked from commit 6eefae1b33928486e7a8572e14eaf00b83f2acfc)
2015-12-30 15:39:27 -08:00
Wangda Tan
8310b2e9ff
YARN-4522. Queue acl can be checked at app submission. (Jian He via wangda)
2015-12-30 15:30:12 -08:00
Wangda Tan
79c41b1d83
YARN-4293. ResourceUtilization should be a part of yarn node CLI. (Sunil G via wangda)
2015-12-16 13:18:19 -08:00
Robert Kanter
6d84cc16b3
MAPREDUCE-6550. archive-logs tool changes log ownership to the Yarn user when using DefaultContainerExecutor (rkanter)
2015-11-25 17:12:40 -08:00
Akira Ajisaka
bab03f3ee6
HADOOP-12598. Add XML namespace declarations for some hadoop/tools modules. Contributed by Xin Wang.
2015-11-26 01:21:20 +09:00
Karthik Kambatla
52948bb20b
YARN-3980. Plumb resource-utilization info in node heartbeat through to the scheduler. (Inigo Goiri via kasha)
2015-11-24 13:47:17 +05:30
Haohui Mai
dc46c46b91
HADOOP-10465. Fix use of generics within SortedMapWritable. Contributed by Bertrand Dechoux.
2015-11-22 18:10:08 -08:00
Haohui Mai
0d27917042
HADOOP-12098. Remove redundant test dependencies in Hadoop Archives. Contributed by Varun Saxena.
2015-11-22 17:25:22 -08:00
Haohui Mai
176ff5ce90
HDFS-9402. Switch DataNode.LOG to use slf4j. Contributed by Walter Su.
2015-11-22 15:54:41 -08:00
Akira Ajisaka
f94d892704
HADOOP-12545. Hadoop javadoc has broken links for AccessControlList, ImpersonationProvider, DefaultImpersonationProvider, and DistCp. Contributed by Arshad Mohammad.
2015-11-13 21:52:57 +09:00
Akira Ajisaka
3ae82863f0
HADOOP-12553. [JDK8] Fix javadoc error caused by illegal tag. (aajisaka)
2015-11-08 19:17:57 +09:00
Lei Xu
bff7c90a56
HADOOP-11684. S3a to use thread pool that blocks clients. (Thomas Demoor and Aaron Fabbri via lei)
2015-11-05 18:35:15 -08:00
Allen Wittenauer
19a0c2660c
MAPREDUCE-1125. SerialUtils.cc: deserializeFloat is out of sync with SerialUtils.hh (Simone Leo via aw)
2015-11-05 16:57:22 -08:00
Allen Wittenauer
fc7cd46faf
MAPREDUCE-3914. Mismatched free() / delete / delete [] in HadoopPipes (Joe Mudd via aw)
2015-11-05 16:53:32 -08:00
cnauroth
0fb1867fd6
HADOOP-12540. TestAzureFileSystemInstrumentation#testClientErrorMetrics fails intermittently due to assumption that a lease error will be thrown. Contributed by Gaurav Kanade.
2015-11-04 10:28:44 -08:00
Varun Vasudev
73b9c7b82b
HADOOP-10787. Rename/remove non-HADOOP_*, etc from the shell scripts. Contributed by Allen Wittenauer.
2015-11-04 15:56:17 +05:30
cnauroth
9e7dcab185
HADOOP-12508. delete fails with exception when lease is held on blob. Contributed by Gaurav Kanade.
2015-11-02 10:21:39 -08:00
cnauroth
3ce0a6502e
HADOOP-12533. Introduce FileNotFoundException in WASB for read and seek API. Contributed by Dushyanth.
2015-11-02 10:17:41 -08:00
Kihwal Lee
b24fe06483
Addendum to MAPREDUCE-6451
2015-10-30 16:05:23 -05:00
Kihwal Lee
2868ca0328
MAPREDUCE-6451. DistCp has incorrect chunkFilePath for multiple jobs when strategy is dynamic. Contributed by Kuhu Shukla.
2015-10-30 14:56:41 -05:00
Gábor Lipták
18727c63da
HADOOP-12133 Add schemas to Maven Assembly XMLs
...
Signed-off-by: Allen Wittenauer <aw@apache.org>
2015-10-30 11:36:52 -07:00
cnauroth
2d10cb8e00
HADOOP-12519. hadoop-azure tests should avoid creating a metrics configuration fiale in the module root directory. Contributed by Chris Nauroth.
2015-10-28 09:44:10 -07:00
cnauroth
1f7ecb0c84
HADOOP-11685. StorageException complaining " no lease ID" during HBase distributed log splitting. Contributed by Duo Xu.
2015-10-27 22:56:22 -07:00
cnauroth
73822de7c3
HADOOP-12520. Use XInclude in hadoop-azure test configuration to isolate Azure Storage account keys for service integration tests. Contributed by Chris Nauroth.
2015-10-27 22:48:56 -07:00
Varun Vasudev
c28e16b40c
YARN-4302. SLS not able start due to NPE in SchedulerApplicationAttempt. Contributed by Bibin A Chundatt.
2015-10-27 20:56:00 +05:30
Akira Ajisaka
bcb2386e39
HADOOP-12515. Mockito dependency is missing in hadoop-kafka module. Contributed by Kai Zheng.
2015-10-27 20:17:16 +09:00
cnauroth
cb282d5b89
HADOOP-12484. Single File Rename Throws Incorrectly In Potential Race Condition Scenarios. Contributed by Gaurav Kanade.
2015-10-22 14:29:57 -07:00
cnauroth
47641fcbc9
HADOOP-12334. Change Mode Of Copy Operation of HBase WAL Archiving to bypass Azure Storage Throttling after retries. Contributed by Gaurav Kanade.
2015-10-22 12:21:32 -07:00
Robert Kanter
0c4af0f998
MAPREDUCE-6495. Docs for archive-logs tool (rkanter)
2015-10-20 17:34:34 -07:00
Haohui Mai
da8441d0fe
HDFS-9167. Update pom.xml in other modules to depend on hdfs-client instead of hdfs. Contributed by Mingliang Liu.
2015-10-13 15:24:24 -07:00
Haohui Mai
7e2c971fed
Revert "HADOOP-12469. distcp shout not ignore the ignoreFailures option. Contributed by Mingliang Liu."
...
This reverts commit fa5a54cac7
2015-10-10 03:24:59 -07:00
Haohui Mai
fa5a54cac7
HADOOP-12469. distcp shout not ignore the ignoreFailures option. Contributed by Mingliang Liu.
2015-10-09 22:38:06 -07:00
Robert Kanter
1bca1bb517
MAPREDUCE-6503. archive-logs tool should use HADOOP_PREFIX instead of HADOOP_HOME (rkanter)
2015-10-06 16:49:50 -07:00
cnauroth
5f6edb30c2
HADOOP-12350. WASB Logging: Improve WASB Logging around deletes, reads and writes. Contributed by Dushyanth.
2015-10-05 20:11:41 -07:00
Allen Wittenauer
b59e434116
HADOOP-10949. metrics2 sink plugin for Apache Kafka (Babak Behzad via aw)
2015-10-05 14:02:07 -07:00
Akira Ajisaka
ecbfd68974
HADOOP-10296. Incorrect null check in SwiftRestClient#buildException(). Contributed by Rahul Palamuttam and Kanaka Kumar Avvaru.
2015-10-01 18:25:31 +02:00
Robert Kanter
5db371f52f
MAPREDUCE-6494. Permission issue when running archive-logs tool as different users (rkanter)
2015-09-30 17:33:53 -07:00
Kihwal Lee
3abbdc929b
HADOOP-12446. Undeprecate createNonRecursive(). Contributed by Ted Yu.
2015-09-28 17:31:51 -05:00
Harsh J
66dad854c0
MAPREDUCE-6471. Document distcp incremental copy. Contributed by Neelesh Srinivas Salian.
2015-09-28 13:12:43 +05:30
cnauroth
7fe521b1dd
HADOOP-11918. Listing an empty s3a root directory throws FileNotFound. Contributed by Lei (Eddy) Xu.
2015-09-25 22:33:53 -07:00
Robert Kanter
d3c49e7662
MAPREDUCE-6480. archive-logs tool may miss applications (rkanter)
2015-09-25 15:02:42 -07:00
Jian He
89cab1ba5f
YARN-1651. CapacityScheduler side changes to support container resize. Contributed by Wangda Tan
2015-09-23 13:29:38 -07:00
Harsh J
cc2b473990
MAPREDUCE-5045. UtilTest#isCygwin method appears to be unused. Contributed by Neelesh Srinivas Salian.
2015-09-22 21:38:06 +05:30
Tsuyoshi Ozawa
dfd807afab
HADOOP-12428. Fix inconsistency between log-level guards and statements. Contributed by Jagadesh Kiran N and Jackie Chang.
2015-09-22 12:54:29 +09:00
Steve Loughran
7269906254
HADOOP-12087. [JDK8] Fix javadoc errors caused by incorrect or illegal tags. (Akira AJISAKA via stevel).
2015-09-13 14:25:26 +01:00
Karthik Kambatla
119cc75e7e
MAPREDUCE-6415. Create a tool to combine aggregated logs into HAR files. (Robert Kanter via kasha)
2015-09-09 17:45:19 -07:00
Tsuyoshi Ozawa
355eaaa33d
HADOOP-10318. Incorrect reference to nodeFile in RumenToSLSConverter error message. Contributed by Wei Yan.
2015-09-03 22:48:53 +09:00
Lei Xu
6ab2d19f5c
HADOOP-12346. Increase some default timeouts / retries for S3a connector. (Sean Mackrory via Lei (Eddy) Xu)
2015-08-29 09:59:30 -07:00
Rohith Sharma K S
feaf034994
YARN-3896. RMNode transitioned from RUNNING to REBOOTED because its response id has not been reset synchronously. (Jun Gong via rohithsharmaks)
2015-08-24 11:25:07 +05:30
Yongjun Zhang
0bc15cb6e6
HDFS-8828. Utilize Snapshot diff report to build diff copy list in distcp. (Yufei Gu via Yongjun Zhang)
2015-08-20 08:02:54 -07:00
cnauroth
3da0bedaec
HADOOP-12304. Applications using FileContext fail with the default file system configured to be wasb/s3/etc. Contributed by Chris Nauroth.
2015-08-05 14:56:41 -07:00
Lei Xu
d5403747b5
HADOOP-12269. Update aws-sdk dependency to 1.10.6 (Thomas Demoor via Lei (Eddy) Xu)
2015-08-04 18:51:52 -07:00
cnauroth
efa97243ec
HADOOP-12239. StorageException complaining " no lease ID" when updating FolderLastModifiedTime in WASB. Contributed by Duo Xu.
2015-07-22 11:16:49 -07:00
Vinayakumar B
94c6a4aa85
HADOOP-12017. Hadoop archives command should use configurable replication factor when closing (Contributed by Bibin A Chundatt)
2015-07-22 10:25:49 +05:30
Allen Wittenauer
68d1f4bfe8
HADOOP-11762. Enable swift distcp to secure HDFS (Chen He via aw)
2015-07-21 11:19:29 -07:00
Wangda Tan
c39ca541f4
YARN-2003. Support for Application priority : Changes in RM and Capacity Scheduler. (Sunil G via wangda)
2015-07-21 09:57:23 -07:00
Vinayakumar B
87f29c6b8a
HADOOP-12214. Parse 'HadoopArchive' commandline using cli Options. (Contributed by Vinayakumar B)
2015-07-21 13:12:46 +05:30
Steve Loughran
05130e94c5
HADOOP-12235 hadoop-openstack junit & mockito dependencies should be "provided". (Ted Yu via stevel)
2015-07-20 11:22:39 +01:00
Chris Douglas
a431ed9075
HADOOP-12211. Collect disk usage on the node. Contributed by Robert Grandl
2015-07-13 15:36:11 -07:00
Ming Ma
08244264c0
YARN-3445. Cache runningApps in RMNode for getting running apps on given NodeId. (Junping Du via mingma)
2015-07-10 08:30:10 -07:00
Chris Douglas
1a0752d85a
HADOOP-12210. Collect network usage on the node. Contributed by Robert Grandl
2015-07-09 17:48:43 -07:00
Robert Kanter
aaafa0b2ee
HADOOP-12159. Move DistCpUtils#compareFs() to org.apache.hadoop.fs.FileUtil and fix for HA namespaces (rchiang via rkanter)
2015-06-30 16:49:27 -07:00
cnauroth
460e98f7b3
HADOOP-12089. StorageException complaining " no lease ID" when updating FolderLastModifiedTime in WASB. Contributed by Duo Xu.
2015-06-29 13:48:02 -07:00
Colin Patrick Mccabe
5a27c3fd76
HADOOP-12104. Migrate Hadoop Pipes native build to new CMake framework (Alan Burlison via Colin P. McCabe)
2015-06-29 12:31:26 -07:00
Amareshwari Sriramadasu
8ef07f767f
HADOOP-11203. Allow ditscp to accept bandwitdh in fraction MegaBytes. Contributed by Raju Bairishetti
2015-06-26 09:52:06 +05:30
cnauroth
c45784bc90
HADOOP-12073. Azure FileSystem PageBlobInputStream does not return -1 on EOF. Contributed by Ivan Mitic.
2015-06-08 22:42:14 -07:00
Andrew Wang
2dbc40e608
HADOOP-12059. S3Credentials should support use of CredentialProvider. Contributed by Sean Busbey.
2015-06-05 13:11:01 -07:00
Robert Kanter
cc70df98e7
MAPREDUCE-5965. Hadoop streaming throws error if list of input files is high. Error is: "error=7, Argument list too long at if number of input file is high" (wilfreds via rkanter)
2015-06-03 18:41:45 -07:00
cnauroth
94e7d49a6d
HADOOP-11959. WASB should configure client side socket timeout in storage client blob request options. Contributed by Ivan Mitic.
2015-05-28 12:31:06 -07:00
Colin Patrick Mccabe
7dba7005b7
HADOOP-11969. ThreadLocal initialization in several classes is not thread safe (Sean Busbey via Colin P. McCabe)
2015-05-26 12:15:46 -07:00
Vinod Kumar Vavilapalli
500a1d9c76
YARN-160. Enhanced NodeManager to automatically obtain cpu/memory values from underlying OS when configured to do so. Contributed by Varun Vasudev.
2015-05-26 11:38:35 -07:00
Colin Patrick Mccabe
f34638399a
HADOOP-11927. Fix "undefined reference to dlopen" error when compiling libhadooppipes (Xianyin Xin via Colin P. McCabe)
2015-05-22 12:51:03 -07:00
Akira Ajisaka
4aa730ce85
HADOOP-11698. Remove DistCpV1 and Logalyzer. Contributed by Brahma Reddy Battula.
2015-05-20 19:25:45 +09:00
Jing Zhao
0790275f05
HADOOP-1540. Support file exclusion list in distcp. Contributed by Rich Haase.
2015-05-18 13:24:35 -07:00
Akira Ajisaka
f5c48238d5
HADOOP-11939. Deprecate DistCpV1 and Logalyzer. Contributed by Brahma Reddy Battula.
2015-05-18 16:13:16 +09:00
cnauroth
cb8e69a80c
HADOOP-11960. Enable Azure-Storage Client Side logging. Contributed by Dushyanth.
2015-05-14 22:22:24 -07:00
Akira Ajisaka
92c38e41e1
HADOOP-9723. Improve error message when hadoop archive output path already exists. Contributed by Jean-Baptiste Onofré and Yongjun Zhang.
2015-05-13 17:28:57 +09:00
Allen Wittenauer
ed0f4db0dd
HADOOP-11775. Fix Javadoc typos in hadoop-openstack module (Yanjun Wang via aw)
2015-05-08 17:01:08 -07:00
Akira Ajisaka
c5afe444df
HDFS-8349. Remove .xml and documentation references to dfs.webhdfs.enabled. Contributed by Ray Chiang.
2015-05-08 15:57:13 +09:00
Vinayakumar B
f33efbc827
HADOOP-11922. Misspelling of threshold in log4j.properties for tests in hadoop-tools (Contributed by Gabor Liptak)
2015-05-08 10:22:11 +05:30
Jonathan Eagles
2d7363b273
HADOOP-11889. Make checkstyle runnable from root project (Gera Shegalov via jeagles)
2015-05-01 13:11:50 -05:00
Akira Ajisaka
f384a063a6
HADOOP-11821. Fix findbugs warnings in hadoop-sls. Contributed by Brahma Reddy Battula.
2015-04-30 19:34:41 +09:00
Allen Wittenauer
618ba707f0
MAPREDUCE-6057. Remove obsolete entries from mapred-default.xml (Ray Chiang via aw)
2015-04-26 20:31:40 -07:00
Ravi Prakash
a287d2fb77
HADOOP-11876. Refactor code to make it more readable, minor maybePrintStats bug (Zoran Dimitrijevic via raviprak)
2015-04-24 13:39:07 -07:00
Steve Loughran
19262d99eb
HADOOP-11730. Regression: s3n read failure recovery broken. (Takenori Sato via stevel)
2015-04-23 21:41:51 +01:00
Ravi Prakash
cfba355052
HADOOP-11827. Speed-up distcp buildListing() using threadpool (Zoran Dimitrijevic via raviprak)
2015-04-21 16:49:37 -07:00
Jing Zhao
4c097e473b
HDFS-8151. Always use snapshot path as source when invalid snapshot names are used for diff based distcp. Contributed by Jing Zhao.
2015-04-15 12:37:20 -07:00
Akira Ajisaka
83979e61a4
MAPREDUCE-6307. Remove property mapreduce.tasktracker.taskmemorymanager.monitoringinterval. Contributed by J.Andreina.
2015-04-10 20:21:00 +09:00
Colin Patrick Mccabe
932730df7d
HADOOP-11785. Reduce the number of listStatus operation in distcp buildListing (Zoran Dimitrijevic via Colin P. McCabe)
2015-04-03 14:08:25 -07:00
Haohui Mai
75cb1d42ab
HDFS-8036. Use snapshot path as source when using snapshot diff report in DistCp. Contributed by Jing Zhao.
2015-04-01 16:50:59 -07:00
Harsh J
27d49e6714
MAPREDUCE-6291. Correct mapred queue usage command. Contributed by Brahma Reddy Battula.
2015-03-28 11:58:17 +05:30
Tsuyoshi Ozawa
e074952bd6
HADOOP-11760. Fix typo of javadoc in DistCp. Contributed by Brahma Reddy Battula.
2015-03-27 23:15:51 +09:00
Yongjun Zhang
44809b8081
HADOOP-11724. DistCp throws NPE when the target directory is root. (Lei Eddy Xu via Yongjun Zhang)
2015-03-25 15:45:45 -07:00
Harsh J
b6dea9776b
MAPREDUCE-579. Streaming slowmatch documentation.
2015-03-25 14:39:00 +05:30
Harsh J
e5370477c2
MAPREDUCE-4653. TestRandomAlgorithm has an unused import statement. Contributed by Amir Sanjar.
2015-03-17 14:11:54 +05:30
Tsuyoshi Ozawa
ef9946cd52
HADOOP-11720. [JDK8] Fix javadoc errors caused by incorrect or illegal tags in hadoop-tools. Contributed by Akira AJISAKA.
2015-03-17 16:09:21 +09:00
Tsuyoshi Ozawa
79426f3334
HADOOP-11558. Fix dead links to doc of hadoop-tools. Contributed by Jean-Pierre Matsumoto.
2015-03-15 14:29:49 +09:00
Tsuyoshi Ozawa
b308a8d181
Revert "HADOOP-11558. Fix dead links to doc of hadoop-tools. Contributed by Masatake Iwasaki."
...
This reverts commit 7da136ecca
2015-03-15 14:27:38 +09:00
Tsuyoshi Ozawa
7da136ecca
HADOOP-11558. Fix dead links to doc of hadoop-tools. Contributed by Masatake Iwasaki.
2015-03-15 14:17:35 +09:00
cnauroth
7a346bcb4f
HADOOP-11693. Azure Storage FileSystem rename operations are throttled too aggressively to complete HBase WAL archiving. Contributed by Duo Xu.
2015-03-11 14:36:51 -07:00
Steve Loughran
64443490d7
HADOOP-11670. Regression: s3a auth setup broken. (Adam Budde via stevel)
2015-03-08 11:22:16 -07:00
cnauroth
608ebd52ba
HADOOP-11642. Upgrade azure sdk version from 0.6.0 to 2.0.0. Contributed by Shashank Khandelwal and Ivan Mitic.
2015-03-06 15:25:01 -08:00
Jing Zhao
ed70fa142c
HDFS-7535. Utilize Snapshot diff report for distcp. Contributed by Jing Zhao.
2015-03-04 10:30:53 -08:00
Vinod Kumar Vavilapalli
5af693fde2
MAPREDUCE-6248. Exposed the internal MapReduce job's information as a public API in DistCp. Contributed by Jing Zhao.
2015-03-03 16:28:41 -08:00
Steve Loughran
15b7076ad5
HADOOP-11183. Memory-based S3AOutputstream. (Thomas Demoor via stevel)
2015-03-03 16:18:51 -08:00
Tsuyoshi Ozawa
9ae7f9eb7b
MAPREDUCE-5657. Fix Javadoc errors caused by incorrect or illegal tags in doc comments. Contributed by Akira AJISAKA.
2015-03-03 18:06:26 +09:00
Tsuyoshi Ozawa
d1c6accb6f
HADOOP-11602. Fix toUpperCase/toLowerCase to use Locale.ENGLISH. (ozawa)
2015-03-03 14:17:52 +09:00
Allen Wittenauer
039366e3b4
MAPREDUCE-5653. DistCp does not honour config-overrides for mapreduce.[map,reduce].memory.mb (Ratandeep Ratti via aw)
2015-02-28 22:53:38 -08:00
Allen Wittenauer
dce8b9c4d0
HADOOP-11637. bash location hard-coded in shell scripts (aw)
2015-02-26 09:29:16 -08:00
cnauroth
5731c0e0d0
HADOOP-11629. WASB filesystem should not start BandwidthGaugeUpdater if fs.azure.skip.metrics set to true. Contributed by Shanyu Zhao.
2015-02-25 09:08:55 -08:00
Akira Ajisaka
1a625b8158
HADOOP-11480. Typo in hadoop-aws/index.md uses wrong scheme for test.fs.s3.name. Contributed by Ted Yu.
2015-02-24 17:11:46 -08:00
Tsuyoshi Ozawa
9cedad11d8
Revert "HADOOP-11602. Fix toUpperCase/toLowerCase to use Locale.ENGLISH. (ozawa)"
...
This reverts commit 946456c6d8
2015-02-25 00:32:04 +09:00
Steve Loughran
709ff99cff
HADOOP-11584 s3a file block size set to 0 in getFileStatus. (Brahma Reddy Battula via stevel)
2015-02-21 12:03:03 +00:00
Steve Loughran
aa1c437b6a
HADOOP-11607. Reduce log spew in S3AFileSystem. (Lei (Eddy) Xu via stevel)
2015-02-20 20:51:24 +00:00
Tsuyoshi Ozawa
946456c6d8
HADOOP-11602. Fix toUpperCase/toLowerCase to use Locale.ENGLISH. (ozawa)
2015-02-19 13:06:53 +09:00
Allen Wittenauer
b6fc1f3e43
HADOOP-11593. Convert site documentation from apt to markdown (stragglers) (Masatake Iwasaki via aw)
2015-02-17 21:30:24 -10:00
Karthik Kambatla
409113d8f9
MAPREDUCE-6234. TestHighRamJob fails due to the change in MAPREDUCE-5785. (Masatake Iwasaki via kasha)
2015-02-17 14:38:00 -08:00
Steve Loughran
00b80958d8
HADOOP-11521. Make connection timeout configurable in s3a. (Thomas Demoor via stevel)
2015-02-17 20:06:27 +00:00
Steve Loughran
78a7e8d3a6
HADOOP-11522. Update S3A Documentation. (Thomas Demoor via stevel)
2015-02-17 18:15:02 +00:00
Steve Loughran
826267f789
HADOOP-11570. S3AInputStream.close() downloads the remaining bytes of the object from S3. (Dan Hecht via stevel).
2015-02-17 16:36:44 +00:00
Allen Wittenauer
7776847a59
HADOOP-11437. Remove the version and author information from distcp's README file (Brahma Reddy Battula via aw)
2015-02-11 15:47:36 -08:00
Steve Loughran
17165d3df9
HADOOP-11520. Clean incomplete multi-part uploads in S3A tests. (Thomas Demoor via stevel)
2015-02-06 21:01:41 +00:00
Steve Loughran
4e7ad4f0a8
HADOOP-11463 Replace method-local TransferManager object with S3AFileSystem#transfers. (Ted Yu via stevel)
2015-02-05 12:20:23 +00:00
Allen Wittenauer
9112f093cd
HADOOP-10976. moving the source code of hadoop-tools docs to the directory under hadoop-tools (Masatake Iwasaki via aw)
2015-02-04 17:57:34 -08:00
Allen Wittenauer
f990e9d229
HADOOP-11346. Rewrite sls/rumen to use new shell framework (John Smith via aw)
2015-02-04 17:27:45 -08:00
Akira Ajisaka
34fe11c987
MAPREDUCE-6243. Fix findbugs warnings in hadoop-rumen. Contributed by Masatake Iwasaki.
2015-02-04 09:25:44 -08:00
Tsuyoshi Ozawa
ba58da29f4
HADOOP-11546. Checkstyle failing: Unable to instantiate DoubleCheckedLockingCheck. (ozawa)
2015-02-05 00:42:14 +09:00
Akira Ajisaka
bd69fb2d44
MAPREDUCE-5800. Use Job#getInstance instead of deprecated constructors. (aajisaka)
2015-02-03 14:30:09 -08:00
Karthik Kambatla
c559df2219
YARN-1393. SLS: Add how-to-use instructions. (Wei Yan via kasha)
2015-02-03 13:20:41 -08:00
Haohui Mai
c89977f89c
HADOOP-11529. Fix findbugs warnings in hadoop-archives. Contributed by Masatake Iwasaki.
2015-02-03 10:53:17 -08:00
cnauroth
1c09ca2ba4
HADOOP-11442. hadoop-azure: Create test jar. Contributed by Shashank Khandelwal.
2015-02-02 11:50:43 -08:00
Harsh J
ffc75d6ebe
HADOOP-11488. Difference in default connection timeout for S3A FS. Contributed by Daisuke Kobayashi.
2015-02-01 00:17:04 +05:30
Allen Wittenauer
12e883007c
MAPREDUCE-6151. Update document of GridMix (Masatake Iwasaki via aw)
2015-01-30 11:08:20 -08:00
cnauroth
f2c91098c4
HADOOP-11523. StorageException complaining " no lease ID" when updating FolderLastModifiedTime in WASB. Contributed by Duo Xu.
2015-01-29 17:02:38 -08:00
Allen Wittenauer
3f982c5c26
MAPREDUCE-6150. Update document of Rumen (Masatake Iwasaki via aw)
2015-01-29 14:17:44 -08:00
cnauroth
ad55083f75
HADOOP-11441. Hadoop-azure: Change few methods scope to public. Contributed by Shashank Khandelwal.
2015-01-29 13:07:51 -08:00
Allen Wittenauer
0a05ae1782
HDFS-7566. Remove obsolete entries from hdfs-default.xml (Ray Chiang via aw)
2015-01-27 14:31:27 -08:00
Tsuyoshi Ozawa
7b82c4ab4e
HADOOP-11450. Cleanup DistCpV1 not to use deprecated methods and fix javadocs. Contributed by Varun Saxena.
2015-01-26 12:58:38 +09:00
Allen Wittenauer
42715547f0
HADOOP-11256. Some site docs have inconsistent appearance (Masatake Iwasaki via aw)
2015-01-21 08:44:22 -08:00
Steve Loughran
2908fe4ec5
HADOOP-11171 Enable using a proxy server to connect to S3a. (Thomas Demoor via stevel)
2015-01-17 18:40:50 +00:00
Steve Loughran
c6c0f4eb25
HADOOP-10542 Potential null pointer dereference in Jets3tFileSystemStore retrieveBlock(). (Ted Yu via stevel)
2015-01-17 18:26:26 +00:00
Steve Loughran
000ca83ea3
HADOOP-11261 Set custom endpoint for S3A. (Thomas Demoor via stevel)
2015-01-16 10:15:51 +00:00
Arpit Agarwal
c4cba6165a
HADOOP-11465. Fix findbugs warnings in hadoop-gridmix. (Contributed by Varun Saxena)
2015-01-12 21:22:58 -08:00
Steve Loughran
27d8395867
HADOOP-11446 S3AOutputStream should use shared thread pool to avoid OutOfMemoryError
2015-01-05 13:00:01 +00:00
Karthik Kambatla
e7257acd8a
YARN-2998. Abstract out scheduler independent PlanFollower components. (Anubhav Dhoot via kasha)
2014-12-30 19:55:24 -08:00
Tsuyoshi Ozawa
a164ce2ac9
HADOOP-11283. SequenceFile.Writer can leak file descriptors in DistCpV1#setup(). (Contributed by Varun Saxena via ozawa)
2014-12-25 23:30:45 +09:00
Jian He
fdf042dfff
YARN-2920. Changed CapacityScheduler to kill containers on nodes where node labels are changed. Contributed by Wangda Tan
2014-12-22 16:51:15 -08:00
Haohui Mai
02b21b7131
HADOOP-11429. Findbugs warnings in hadoop extras. Contributed by Varun Saxena.
2014-12-21 14:57:34 -08:00
arp
c1f857b0b4
HADOOP-11395. Add site documentation for Azure Storage FileSystem integration. (Contributed by Chris Nauroth)
2014-12-19 18:54:22 -08:00
Haohui Mai
efe6357748
HADOOP-11431. Clean up redundant maven-site-plugin configuration. Contributed by Herve Boutemy.
2014-12-18 16:23:35 -08:00
Steve Loughran
07bb0b0bbb
HADOOP-11412 POMs mention "The Apache Software License" rather than "Apache License". (Herve Boutemy via stevel)
2014-12-16 11:04:36 +00:00
cnauroth
9458cd5bce
HADOOP-11394. hadoop-aws documentation missing. Contributed by Chris Nauroth.
2014-12-12 23:29:11 -08:00
Haohui Mai
2e98ad34ce
HADOOP-11381. Fix findbugs warnings in hadoop-distcp, hadoop-aws, hadoop-azure, and hadoop-openstack. Contributed by Li Lu.
2014-12-09 20:45:21 -08:00
Haohui Mai
7bceb13ba9
HADOOP-11367. Fix warnings from findbugs 3.0 in hadoop-streaming. Contributed by Li Lu.
2014-12-09 10:41:35 -08:00
Jing Zhao
57cb43be50
HADOOP-11354. ThrottledInputStream doesn't perform effective throttling. Contributed by Ted Yu.
2014-12-08 11:08:39 -08:00
cnauroth
79301e80d7
HADOOP-11201. Hadoop Archives should support globs resolving to files. Contributed by Gera Shegalov.
2014-11-18 17:05:48 -08:00
Aaron T. Myers
6ba52d88ec
HADOOP-10714. AmazonS3Client.deleteObjects() need to be limited to 1000 entries per call. Contributed by Juan Yu.
2014-11-05 17:17:04 -08:00
cnauroth
734eeb4f35
HADOOP-11248. Add hadoop configuration to disable Azure Filesystem metrics collection. Contributed by Shanyu Zhao.
2014-11-03 14:29:18 -08:00
Zhijie Shen
260ab6d5f4
HADOOP-11241. Fixed intermittent TestNMSimulator failure due to timing issue. Contributed by Varun Vasudev.
2014-10-31 18:02:39 -07:00
Haohui Mai
f1a149e916
HADOOP-11253. Hadoop streaming test TestStreamXmlMultipleRecords fails on Windows. Contributed by Varun Vasudev.
2014-10-31 11:54:33 -07:00
Kihwal Lee
b056048114
MAPREDUCE-6022. map_input_file is missing from streaming job
...
environment. Contributed by Jason Lowe.
2014-10-29 12:29:07 -05:00
Jian He
b8f7966c7a
MAPREDUCE-6126. Fixed Rumen JobBuilder to ignore NormalizedResourceEvent. Contributed by Junping Du
2014-10-22 10:43:20 -07:00
Vinod Kumar Vavilapalli
f2ea555ac6
YARN-2496. Enhanced Capacity Scheduler to have basic support for allocating resources based on node-labels. Contributed by Wangda Tan.
...
YARN-2500. Ehnaced ResourceManager to support schedulers allocating resources based on node-labels. Contributed by Wangda Tan.
2014-10-15 18:33:06 -07:00
cnauroth
f4b7e99f4e
HADOOP-11188. hadoop-azure: automatically expand page blobs when they become full. Contributed by Eric Hanson.
2014-10-10 15:05:52 -07:00
cnauroth
2217e2f8ff
HADOOP-10809. hadoop-azure: page blob support. Contributed by Dexter Bradshaw, Mostafa Elhemali, Eric Hanson, and Mike Liddell.
2014-10-08 14:20:23 -07:00
Kihwal Lee
c8617ff136
Missing files from the previous commit of HADOOP-11007
2014-10-07 16:30:53 -05:00
Kihwal Lee
99d7a452be
HADOOP-11007. Reinstate building of ant tasks support. Contributed by
...
Jason Lowe.
2014-10-07 16:20:16 -05:00
Allen Wittenauer
83264cf457
HADOOP-10954. Adding site documents of hadoop-tools (Masatake Iwasaki via aw)
2014-09-26 11:59:45 -07:00
Andrew Wang
4ea77efa3a
HADOOP-11140. hadoop-aws only need test-scoped dependency on hadoop-common's tests jar. Contributed by Juan Yu.
2014-09-25 21:56:26 -07:00
Allen Wittenauer
f614cb71d4
MAPREDUCE-6109. Fix minor typo in distcp -p usage text (Charles Lamb via aw)
2014-09-25 11:27:33 -07:00
Allen Wittenauer
3cde37c991
HADOOP-11009. Add Timestamp Preservation to DistCp (Gary Steelman via aw)
2014-09-24 15:38:30 -07:00
Allen Wittenauer
aa1052c34b
YARN-2460. Remove obsolete entries from yarn-default.xml (Ray Chiang via aw)
2014-09-19 14:07:38 -07:00
Allen Wittenauer
9f03a7c018
HADOOP-10946. Fix a bunch of typos in log messages (Ray Chiang via aw)
2014-09-19 11:33:07 -07:00
Colin Patrick Mccabe
0ac760a58d
HADOOP-11091. Eliminate old configuration parameter names from s3a (dsw via cmccabe)
2014-09-15 16:49:15 -07:00
Aaron T. Myers
24d920b80e
HADOOP-10400. Incorporate new S3A FileSystem implementation. Contributed by Jordan Mendelson and Dave Wang.
2014-09-15 08:27:07 -07:00
Colin Patrick Mccabe
5ec7fcd9dd
HADOOP-11074. Move s3-related FS connector code to hadoop-aws. (David S. Wang via Colin Patrick McCabe)
2014-09-10 16:14:53 -07:00
Karthik Kambatla
28d99db992
YARN-2526. SLS can deadlock when all the threads are taken by AMSimulators. (Wei Yan via kasha)
2014-09-09 15:13:16 -07:00
Allen Wittenauer
d989ac0444
MAPREDUCE-5972. Fix typo 'programatically' in job.xml (and a few other places) (Akira AJISAKA via aw)
2014-09-08 13:15:46 -07:00
Steve Loughran
59384dfb71
HADOOP-10373 create tools/hadoop-amazon for aws/EMR support
2014-09-02 20:11:13 +01:00
Jian He
5c14bc426b
YARN-1506. Changed RMNode/SchedulerNode to update resource with event notification. Contributed by Junping Du
2014-08-29 23:05:51 -07:00
Andrew Wang
ea1c6f31c2
HADOOP-11021. Configurable replication factor in the hadoop archive command. Contributed by Zhe Zhang.
2014-08-29 14:44:37 -07:00
Sanford Ryza
5109157ed1
MAPREDUCE-5130. Add missing job config options to mapred-default.xml (Ray Chiang via Sandy Ryza)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1619626 13f79535-47bb-0310-9956-ffa450edef68
2014-08-21 23:28:44 +00:00
Andrew Wang
d2a39b61aa
Merge from trunk to branch.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1619018 13f79535-47bb-0310-9956-ffa450edef68
2014-08-20 01:22:42 +00:00
Charles Lamb
14b01dd046
HDFS-6872. Fix TestOptionsParser. (clamb)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1618974 13f79535-47bb-0310-9956-ffa450edef68
2014-08-19 20:41:15 +00:00
Zhijie Shen
f6a778c372
YARN-2249. Avoided AM release requests being lost on work preserving RM restart. Contributed by Jian He.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618972 13f79535-47bb-0310-9956-ffa450edef68
2014-08-19 20:33:49 +00:00
Andrew Wang
0cc08f6da4
Merge from trunk to branch.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1618700 13f79535-47bb-0310-9956-ffa450edef68
2014-08-18 18:41:31 +00:00
Allen Wittenauer
399e428deb
HADOOP-10121. Fix javadoc spelling for HadoopArchives#writeTopLevelDirs (Akira AJISAKA via aw)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618044 13f79535-47bb-0310-9956-ffa450edef68
2014-08-14 20:28:36 +00:00
Colin McCabe
e90d940699
HADOOP-10843. TestGridmixRecord unit tests failure on PowerPC (Jinghui Wang via Colin Patrick McCabe)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617860 13f79535-47bb-0310-9956-ffa450edef68
2014-08-14 01:36:12 +00:00
Karthik Kambatla
486e718fc1
YARN-2399. FairScheduler: Merge AppSchedulable and FSSchedulerApp into FSAppAttempt. (kasha)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617600 13f79535-47bb-0310-9956-ffa450edef68
2014-08-12 21:43:27 +00:00
Charles Lamb
041b8326a1
MAPREDUCE-6007. Add support to distcp to preserve raw.* namespace extended attributes. (clamb)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1616657 13f79535-47bb-0310-9956-ffa450edef68
2014-08-08 01:32:18 +00:00
Steve Loughran
96fc3b3ecd
HADOOP-10931 compile error on tools/hadoop-openstack
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616482 13f79535-47bb-0310-9956-ffa450edef68
2014-08-07 12:38:35 +00:00
Karthik Kambatla
4625792c20
YARN-1726. Add missing files. ResourceSchedulerWrapper broken due to AbstractYarnScheduler. (Wei Yan via kasha)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1613552 13f79535-47bb-0310-9956-ffa450edef68
2014-07-26 02:03:18 +00:00
Karthik Kambatla
b0c2c85619
YARN-1726. ResourceSchedulerWrapper broken due to AbstractYarnScheduler. (Wei Yan via kasha)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1613547 13f79535-47bb-0310-9956-ffa450edef68
2014-07-26 01:29:25 +00:00
Karthik Kambatla
77363b9d83
YARN-2335. Annotate all hadoop-sls APIs as @Private. (Wei Yan via kasha)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1613478 13f79535-47bb-0310-9956-ffa450edef68
2014-07-25 17:12:22 +00:00
Chris Nauroth
0a02b5a19b
HADOOP-10840. Fix OutOfMemoryError caused by metrics system in Azure File System. Contributed by Shanyu Zhao.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1611247 13f79535-47bb-0310-9956-ffa450edef68
2014-07-17 05:04:53 +00:00
Andrew Wang
e8a1ff7efe
MAPREDUCE-5971. Move the default options for distcp -p to DistCpOptionSwitch. Contributed by Charles Lamb.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1611217 13f79535-47bb-0310-9956-ffa450edef68
2014-07-16 23:42:34 +00:00
Steve Loughran
a10a4f72ab
HADOOP-9361: Strictly define FileSystem APIs - OpenStack portion
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1607600 13f79535-47bb-0310-9956-ffa450edef68
2014-07-03 12:12:52 +00:00
Steve Loughran
becc23fb65
HADOOP-9361: Strictly define FileSystem APIs - OpenStack portion
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1607599 13f79535-47bb-0310-9956-ffa450edef68
2014-07-03 12:10:52 +00:00
Chris Nauroth
0d91576ec3
HADOOP-10728. Metrics system for Windows Azure Storage Filesystem. Contributed by Dexter Bradshaw, Mostafa Elhemali, Xi Fang, Johannes Klein, David Lao, Mike Liddell, Chuan Liu, Lengning Liu, Ivan Mitic, Michael Rys, Alexander Stojanovic, Brian Swan, and Min Wei.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1605187 13f79535-47bb-0310-9956-ffa450edef68
2014-06-24 20:52:44 +00:00
Chris Nauroth
61bf9f7799
HADOOP-10690. Lack of synchronization on access to InputStream in NativeAzureFileSystem#NativeAzureFsInputStream#close(). Contributed by Chen He.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1604236 13f79535-47bb-0310-9956-ffa450edef68
2014-06-20 18:36:11 +00:00
Chris Nauroth
8300b9fb38
HADOOP-10689. InputStream is not closed in AzureNativeFileSystemStore#retrieve(). Contributed by Chen He.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1604233 13f79535-47bb-0310-9956-ffa450edef68
2014-06-20 18:30:49 +00:00
Haohui Mai
22cbcd16cb
HADOOP-10717. Revert r1603571.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1603620 13f79535-47bb-0310-9956-ffa450edef68
2014-06-18 20:37:25 +00:00
Steve Loughran
bae5837d3b
HADOOP-10717. Missing JSP support in Jetty
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1603534 13f79535-47bb-0310-9956-ffa450edef68
2014-06-18 17:08:45 +00:00
Andrew Wang
50cd2a6bed
HADOOP-10666. Remove Copyright /d/d/d/d Apache Software Foundation from the source files license header. Contributed by Henry Saputra.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1603025 13f79535-47bb-0310-9956-ffa450edef68
2014-06-16 23:23:33 +00:00
Chris Nauroth
81bc395deb
HADOOP-9629. Support Windows Azure Storage - Blob as a file system in Hadoop. Contributed by Dexter Bradshaw, Mostafa Elhemali, Xi Fang, Johannes Klein, David Lao, Mike Liddell, Chuan Liu, Lengning Liu, Ivan Mitic, Michael Rys, Alexander Stojanovic, Brian Swan, and Min Wei.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1601781 13f79535-47bb-0310-9956-ffa450edef68
2014-06-10 22:26:45 +00:00
Vinod Kumar Vavilapalli
6480fdfd33
MAPREDUCE-5834. Increased test-timeouts in TestGridMixClasses to avoid occassional failures. Contributed by Mit Desai.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1601764 13f79535-47bb-0310-9956-ffa450edef68
2014-06-10 20:24:57 +00:00
Steve Loughran
3d077726d0
HADOOP-10647. String Format Exception in SwiftNativeFileSystemStore.java.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1600971 13f79535-47bb-0310-9956-ffa450edef68
2014-06-06 18:00:52 +00:00
Uma Maheswara Rao G
f81c7b0252
MAPREDUCE-5898. distcp to support preserving HDFS extended attributes(XAttrs). Contributed by Yi Liu.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1600900 13f79535-47bb-0310-9956-ffa450edef68
2014-06-06 14:45:39 +00:00
Karthik Kambatla
a4ba451802
YARN-1474. Make schedulers services. (Tsuyoshi Ozawa via kasha)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598908 13f79535-47bb-0310-9956-ffa450edef68
2014-05-31 19:33:09 +00:00
Chris Nauroth
806f4d11f2
HADOOP-10602. Documentation has broken "Go Back" hyperlinks. Contributed by Akira AJISAKA.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598337 13f79535-47bb-0310-9956-ffa450edef68
2014-05-29 17:26:50 +00:00
Jing Zhao
3671a5e16f
MAPREDUCE-5899. Support incremental data copy in DistCp. Contributed by Jing Zhao.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1596931 13f79535-47bb-0310-9956-ffa450edef68
2014-05-22 18:17:11 +00:00
Vinod Kumar Vavilapalli
82f3454f5a
YARN-2017. Merged some of the common scheduler code. Contributed by Jian He.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1596753 13f79535-47bb-0310-9956-ffa450edef68
2014-05-22 05:32:26 +00:00
Chris Nauroth
11be7334c4
MAPREDUCE-5809. Enhance distcp to support preserving HDFS ACLs. Contributed by Chris Nauroth.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1595283 13f79535-47bb-0310-9956-ffa450edef68
2014-05-16 18:25:23 +00:00
Tsz-wo Sze
03db13206f
MAPREDUCE-5402. In DynamicInputFormat, change MAX_CHUNKS_TOLERABLE, MAX_CHUNKS_IDEAL, MIN_RECORDS_PER_CHUNK and SPLIT_RATIO to be configurable. Contributed by Tsuyoshi OZAWA
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1592703 13f79535-47bb-0310-9956-ffa450edef68
2014-05-06 10:24:10 +00:00
Jonathan Turner Eagles
a059eadbe9
MAPREDUCE-5639. Port DistCp2 document to trunk (Akira AJISAKA via jeagles)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1590058 13f79535-47bb-0310-9956-ffa450edef68
2014-04-25 15:23:42 +00:00
Ravi Prakash
a93be7b46c
MAPREDUCE-5846. Rumen doesn't understand JobQueueChangedEvent (Nathan Roberts via raviprak)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1589595 13f79535-47bb-0310-9956-ffa450edef68
2014-04-24 05:59:28 +00:00
Chris Nauroth
06c15b6a3e
MAPREDUCE-5852. Prepare MapReduce codebase for JUnit 4.11. Contributed by Chris Nauroth.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1589006 13f79535-47bb-0310-9956-ffa450edef68
2014-04-21 23:44:41 +00:00
Chris Nauroth
84388525a3
YARN-1970. Prepare YARN codebase for JUnit 4.11. Contributed by Chris Nauroth.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1589001 13f79535-47bb-0310-9956-ffa450edef68
2014-04-21 23:31:18 +00:00
Haohui Mai
0862ee6520
HADOOP-10485. Remove dead classes in hadoop-streaming. Contributed by Haohui Mai.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1586059 13f79535-47bb-0310-9956-ffa450edef68
2014-04-09 18:10:36 +00:00
Steve Loughran
b8f250a99d
HADOOP-10104. Update jackson to 1.9.13 (Akira Ajisaka via stevel)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1585932 13f79535-47bb-0310-9956-ffa450edef68
2014-04-09 11:11:20 +00:00
Haohui Mai
8ca32df08e
HADOOP-10474. Move o.a.h.record to hadoop-streaming. Contributed by Haohui Mai.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1585886 13f79535-47bb-0310-9956-ffa450edef68
2014-04-09 04:02:46 +00:00
Aaron Myers
640a097533
HADOOP-10459. distcp V2 doesn't preserve root dir's attributes when -p is specified. Contributed by Yongjun Zhang.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1584227 13f79535-47bb-0310-9956-ffa450edef68
2014-04-03 00:32:25 +00:00
Haohui Mai
fb1d7fb596
HDFS-5570. Deprecate hftp / hsftp and replace them with webhdfs / swebhdfs. Contributed by Haohui Mai.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1584100 13f79535-47bb-0310-9956-ffa450edef68
2014-04-02 17:28:08 +00:00
Vinod Kumar Vavilapalli
86ef362a7d
MAPREDUCE-5810. Removed the faulty and failing streaming test TestStreamingTaskLog. Contributed by Akira Ajisaka.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1582896 13f79535-47bb-0310-9956-ffa450edef68
2014-03-28 22:16:55 +00:00
Jing Zhao
0b3ee72af8
HDFS-6090. Use MiniDFSCluster.Builder instead of deprecated constructors. Contributed by Akira AJISAKA.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1578669 13f79535-47bb-0310-9956-ffa450edef68
2014-03-17 23:53:48 +00:00
Jonathan Turner Eagles
a8f302a1e6
MAPREDUCE-5765. Update hadoop-pipes examples README (Mit Desai via jeagles)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1577188 13f79535-47bb-0310-9956-ffa450edef68
2014-03-13 15:13:07 +00:00
Sanford Ryza
1079c37128
YARN-1504. RM changes for moving apps between queues (Sandy Ryza)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1563371 13f79535-47bb-0310-9956-ffa450edef68
2014-02-01 04:03:36 +00:00
Alejandro Abdelnur
5a2428d39f
HADOOP-10311. Cleanup vendor names from the code base. (tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1563239 13f79535-47bb-0310-9956-ffa450edef68
2014-01-31 20:11:29 +00:00
Jing Zhao
067d52b98c
HADOOP-10295. Allow distcp to automatically identify the checksum type of source files and use it for the target. Contributed by Jing Zhao and Laurent Goujon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1563019 13f79535-47bb-0310-9956-ffa450edef68
2014-01-30 23:53:23 +00:00
Konstantin Boudnik
4c197b5d56
HADOOP-10167. Mark hadoop-common source as UTF-8 in Maven pom files / refactoring. Contributed by Mikhail Antonov.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1560831 13f79535-47bb-0310-9956-ffa450edef68
2014-01-23 22:29:23 +00:00
Vinod Kumar Vavilapalli
1393581bce
YARN-1490. Introduced the ability to make ResourceManager optionally not kill all containers when an ApplicationMaster exits. Contributed by Jian He.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1557143 13f79535-47bb-0310-9956-ffa450edef68
2014-01-10 15:04:39 +00:00
Vinod Kumar Vavilapalli
a45017a5f1
YARN-1493. Changed ResourceManager and Scheduler interfacing to recognize app-attempts separately from apps. Contributed by Jian He.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1554896 13f79535-47bb-0310-9956-ffa450edef68
2014-01-02 20:19:45 +00:00
Christopher Douglas
e07f3c87cd
YARN-1471. Preserve scheduler typeinfo in simulator to work with resource monitors.
...
Contributed by Carlo Curino.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1552173 13f79535-47bb-0310-9956-ffa450edef68
2013-12-19 00:07:59 +00:00
Steve Loughran
af367d1409
HADOOP-9611 mvn-rpmbuild against google-guice > 3.0 yields missing cglib dependency
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1551916 13f79535-47bb-0310-9956-ffa450edef68
2013-12-18 11:16:57 +00:00
Jian He
28db28d491
YARN-1311. Fixed app specific scheduler-events' names to be app-attempt based. Contributed by Vinod Kumar Vavilapalli
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1550613 13f79535-47bb-0310-9956-ffa450edef68
2013-12-13 01:44:16 +00:00
Sanford Ryza
e1d3670f4c
YARN-807. When querying apps by queue, iterating over all apps is inefficient and limiting (Sandy Ryza)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1548983 13f79535-47bb-0310-9956-ffa450edef68
2013-12-08 03:09:34 +00:00
Daryn Sharp
9ea61e4415
HADOOP-10129. Distcp may succeed when it fails (daryn)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1548175 13f79535-47bb-0310-9956-ffa450edef68
2013-12-05 15:47:55 +00:00
Steve Loughran
7b60e94c09
HADOOP-10135 writes to swift fs over partition size leave temp files and empty output file
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1546959 13f79535-47bb-0310-9956-ffa450edef68
2013-12-02 11:03:00 +00:00
Steve Loughran
01ccd4d3ab
MAPREDUCE-5624 Move grizzly-test and junit dependencies to test scope
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1541702 13f79535-47bb-0310-9956-ffa450edef68
2013-11-13 20:46:54 +00:00
Jason Darrell Lowe
744506f34d
MAPREDUCE-5610. TestSleepJob fails in jdk7. Contributed by Jonathan Eagles
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1541566 13f79535-47bb-0310-9956-ffa450edef68
2013-11-13 15:49:29 +00:00
Luke Lu
6990355e57
YARN-311. RM/scheduler support for dynamic resource configuration. (Junping Du via llu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1539134 13f79535-47bb-0310-9956-ffa450edef68
2013-11-05 21:23:53 +00:00
Colin McCabe
7abf7e8151
HADOOP-9478. Fix race conditions during the initialization of Configuration related to deprecatedKeyMap (cmccabe)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1538248 13f79535-47bb-0310-9956-ffa450edef68
2013-11-02 18:42:27 +00:00
Sanford Ryza
da317f2ea5
YARN-786. Expose application resource usage in RM REST API (Sandy Ryza)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1537809 13f79535-47bb-0310-9956-ffa450edef68
2013-11-01 02:54:28 +00:00
Sanford Ryza
87adffe877
YARN-1306. Clean up hadoop-sls sample-conf according to YARN-1228 (Wei Yan via Sandy Ryza)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1536982 13f79535-47bb-0310-9956-ffa450edef68
2013-10-30 06:32:53 +00:00
Arpit Agarwal
68a79b0d3f
HDFS-5436. Move HsFtpFileSystem and HFtpFileSystem into org.apache.hdfs.web. (Contributed by Haohui Mai)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1536921 13f79535-47bb-0310-9956-ffa450edef68
2013-10-29 22:44:34 +00:00
Jonathan Turner Eagles
f09c52bb7c
HADOOP-9016. HarFsInputStream.skip(long) must never return negative value. (Ivan A. Veselovsky via jeagles)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1535086 13f79535-47bb-0310-9956-ffa450edef68
2013-10-23 18:03:08 +00:00
Sanford Ryza
7d637a3a99
MAPREDUCE-5457. Add a KeyOnlyTextOutputReader to enable streaming to write out text files without separators (Sandy Ryza)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1533624 13f79535-47bb-0310-9956-ffa450edef68
2013-10-18 20:43:53 +00:00
Suresh Srinivas
dea1b2e84a
HADOOP-10005. No need to check INFO severity level is enabled or not. Contributed by Jackie Chang.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1532907 13f79535-47bb-0310-9956-ffa450edef68
2013-10-16 21:00:07 +00:00
Jonathan Turner Eagles
74a2191aae
MAPREDUCE-5586. TestCopyMapper#testCopyFailOnBlockSizeDifference fails when run from hadoop-tools/hadoop-distcp directory (jeagles)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1532887 13f79535-47bb-0310-9956-ffa450edef68
2013-10-16 20:21:21 +00:00
Jonathan Turner Eagles
034039b917
MAPREDUCE-5585. TestCopyCommitter#testNoCommitAction Fails on JDK7 (jeagles)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1532875 13f79535-47bb-0310-9956-ffa450edef68
2013-10-16 19:57:55 +00:00
Chris Nauroth
11716cf390
HADOOP-10040. Revert svn propset for CRLF line endings on Windows files. Contributed by Chris Nauroth.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1531977 13f79535-47bb-0310-9956-ffa450edef68
2013-10-14 17:01:33 +00:00
Chris Nauroth
bbac0cf05e
HADOOP-10040. hadoop.cmd in UNIX format and would not run by default on Windows. Contributed by Chris Nauroth.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1531491 13f79535-47bb-0310-9956-ffa450edef68
2013-10-12 03:14:24 +00:00
Sanford Ryza
335923be6f
Delete fair-scheduler-allocation.xml for YARN-1300
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1531425 13f79535-47bb-0310-9956-ffa450edef68
2013-10-11 21:25:59 +00:00
Sanford Ryza
156b9f2e88
YARN-1300. SLS tests fail because conf puts YARN properties in fair-scheduler.xml (Ted Yu via Sandy Ryza)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1531423 13f79535-47bb-0310-9956-ffa450edef68
2013-10-11 21:18:48 +00:00
Jonathan Turner Eagles
7b687dda09
YARN-1199. Make NM/RM Versions Available (Mit Desai via jeagles)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529003 13f79535-47bb-0310-9956-ffa450edef68
2013-10-03 21:04:12 +00:00
Vinod Kumar Vavilapalli
1c5b49eeaf
YARN-899. Added back queue level administrator-acls so that there is no regression w.r.t 1.x. Contributed by Xuan Gong.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1527282 13f79535-47bb-0310-9956-ffa450edef68
2013-09-29 08:21:36 +00:00
Steve Loughran
990e494556
HADOOP-8545 findbugs patches from the final -037 patch
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1527189 13f79535-47bb-0310-9956-ffa450edef68
2013-09-28 13:59:13 +00:00
Steve Loughran
42b6cadd76
HADOOP-10006. Compilation failure in trunk for o.a.h.fs.swift.util.JSONUtil
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1527187 13f79535-47bb-0310-9956-ffa450edef68
2013-09-28 13:31:14 +00:00
Alejandro Abdelnur
58b08e11b9
YARN-1021. Yarn Scheduler Load Simulator. (ywskycn via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1527059 13f79535-47bb-0310-9956-ffa450edef68
2013-09-27 20:23:19 +00:00
Steve Loughran
3caca924bc
HADOOP-8545. Filesystem Implementation for OpenStack Swift
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1526854 13f79535-47bb-0310-9956-ffa450edef68
2013-09-27 11:32:53 +00:00
Jason Darrell Lowe
e1d398b2c9
MAPREDUCE-5440. TestCopyCommitter Fails on JDK7. Contributed by Robert Parker
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1509900 13f79535-47bb-0310-9956-ffa450edef68
2013-08-02 22:51:34 +00:00
Kihwal Lee
1b6324265d
HADOOP-9738. TestDistCh fails. Contributed by Jing Zhao.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1503976 13f79535-47bb-0310-9956-ffa450edef68
2013-07-17 02:22:14 +00:00
Chris Nauroth
8df9a8e6d2
HADOOP-9656. Gridmix unit tests fail on Windows and Linux. Contributed by Chuan Liu.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1496597 13f79535-47bb-0310-9956-ffa450edef68
2013-06-25 19:20:15 +00:00
Daryn Sharp
723ef662a9
MAPREDUCE-5315. DistCp reports success even on failure. (mithun and jlowe via daryn)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1491914 13f79535-47bb-0310-9956-ffa450edef68
2013-06-11 19:22:53 +00:00
Siddharth Seth
a2205a3b72
YARN-720 and MAPREDUCE-5291. container-log4j.properties should not refer to mapreduce properties. Update MRApp to use YARN properties for log setup. Contributed by Zhijie Shen.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488829 13f79535-47bb-0310-9956-ffa450edef68
2013-06-03 02:35:34 +00:00
Jason Darrell Lowe
85623a2d75
HADOOP-9287. Parallel-testing hadoop-common. Contributed by Andrey Klochkov
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488040 13f79535-47bb-0310-9956-ffa450edef68
2013-05-30 22:43:16 +00:00
Alejandro Abdelnur
d303f01297
MAPREDUCE-4942. mapreduce.Job has a bunch of methods that throw InterruptedException so its incompatible with MR1. (rkanter via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480748 13f79535-47bb-0310-9956-ffa450edef68
2013-05-09 20:04:32 +00:00
Suresh Srinivas
8f7ce62085
MAPREDUCE-5177. Use common utils FileUtil#setReadable/Writable/Executable & FileUtil#canRead/Write/Execute. Contributed by Ivan Mitic.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1477403 13f79535-47bb-0310-9956-ffa450edef68
2013-04-29 23:00:39 +00:00
Kihwal Lee
2e789dd9a6
MAPREDUCE-5065. DistCp should skip checksum comparisons if block-sizes are different on source/target. Contributed by Mithun Radhakrishnan.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1468629 13f79535-47bb-0310-9956-ffa450edef68
2013-04-16 22:01:18 +00:00
Bikas Saha
41c4cd08a0
MAPREDUCE-4885. Streaming tests have multiple failures on Windows. (Chris Nauroth via bikas)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1467158 13f79535-47bb-0310-9956-ffa450edef68
2013-04-12 03:00:29 +00:00
Arun Murthy
7f13207ed1
MAPREDUCE-5129. Allow tags to JobHistory for deeper analytics. Contributed by Billie Rinaldi.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1465188 13f79535-47bb-0310-9956-ffa450edef68
2013-04-06 05:30:56 +00:00
Alejandro Abdelnur
070916130a
MAPREDUCE-5098. Fix findbugs warnings in gridmix. (kkambatl via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1464209 13f79535-47bb-0310-9956-ffa450edef68
2013-04-03 21:22:02 +00:00
Thomas Graves
75ef1b4845
MAPREDUCE-4991. coverage for gridmix (Aleksey Gorshkov via tgraves)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1463806 13f79535-47bb-0310-9956-ffa450edef68
2013-04-03 01:45:24 +00:00
Alejandro Abdelnur
806073867e
MAPREDUCE-5113. Streaming input/output types are ignored with java mapper/reducer. (sandyr via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1463307 13f79535-47bb-0310-9956-ffa450edef68
2013-04-01 21:42:12 +00:00
Thomas White
375584ae0f
MAPREDUCE-5006. Fix failing streaming tests due to MAPREDUCE-4994. Contributed by Sandy Ryza.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1461541 13f79535-47bb-0310-9956-ffa450edef68
2013-03-27 11:47:06 +00:00
Siddharth Seth
d41e67b966
MAPREDUCE-5077. Remove mapreduce.util.ResourceCalculatorPlugin and related code. Contributed by Karthik Kambatla.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1461251 13f79535-47bb-0310-9956-ffa450edef68
2013-03-26 18:15:40 +00:00
Amareshwari Sri Ramadasu
6f92ad9868
MAPREDUCE-5014. Extend Distcp to accept a custom CopyListing. Contributed by Srikanth Sundarrajan
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1459690 13f79535-47bb-0310-9956-ffa450edef68
2013-03-22 10:09:06 +00:00
Tsz-wo Sze
718f0f92a9
MAPREDUCE-5075. DistCp leaks input file handles since ThrottledInputStream does not close the wrapped InputStream. Contributed by Chris Nauroth
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1458741 13f79535-47bb-0310-9956-ffa450edef68
2013-03-20 10:40:14 +00:00
Aaron Myers
4008b37b74
HADOOP-9405. TestGridmixSummary#testExecutionSummarizer is broken. Contributed by Andrew Wang.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1456639 13f79535-47bb-0310-9956-ffa450edef68
2013-03-14 20:11:49 +00:00
Suresh Srinivas
638801cce1
HADOOP-8952. Enhancements to support Hadoop on Windows Server and Windows Azure environments. Contributed by Ivan Mitic, Chuan Liu, Ramya Sunil, Bikas Saha, Kanna Karanam, John Gordon, Brandon Li, Chris Nauroth, David Lao, Sumadhur Reddy Bolli, Arpit Agarwal, Ahmed El Baz, Mike Liddell, Jing Zhao, Thejas Nair, Steve Maine, Ganeshan Iyer, Raja Aluri, Giridharan Kesavan, Ramya Bharathi Nimmagadda.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1453486 13f79535-47bb-0310-9956-ffa450edef68
2013-03-06 19:15:18 +00:00
Alejandro Abdelnur
6401d5d2ec
HADOOP-9230. TestUniformSizeInputFormat fails intermittently. (kkambatl via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1451291 13f79535-47bb-0310-9956-ffa450edef68
2013-02-28 18:51:39 +00:00
Siddharth Seth
979fb054f8
MAPREDUCE-4693. Historyserver should provide counters for failed tasks. Contributed by Xuan Gong.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1450956 13f79535-47bb-0310-9956-ffa450edef68
2013-02-27 21:03:19 +00:00
Aaron Myers
63522b3224
HADOOP-8569. CMakeLists.txt: define _GNU_SOURCE and _LARGEFILE_SOURCE. Contributed by Colin Patrick McCabe.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1449922 13f79535-47bb-0310-9956-ffa450edef68
2013-02-25 22:07:45 +00:00
Alejandro Abdelnur
4ad1279b2e
HADOOP-9297. remove old record IO generation and tests. (tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1446044 13f79535-47bb-0310-9956-ffa450edef68
2013-02-14 04:27:07 +00:00
Aaron Myers
bdb5342b2d
MAPREDUCE-4953. HadoopPipes misuses fprintf. Contributed by Andy Isaacson.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1442471 13f79535-47bb-0310-9956-ffa450edef68
2013-02-05 04:53:32 +00:00
Siddharth Seth
c163dc2fce
MAPREDUCE-4838. Add additional fields like Locality, Avataar to the JobHistory logs. Contributed by Zhijie Shen
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1439714 13f79535-47bb-0310-9956-ffa450edef68
2013-01-29 00:21:58 +00:00
Todd Lipcon
e322e0a05f
HADOOP-9215. when using cmake-2.6, libhadoop.so doesn't get created (only libhadoop.so.1.0.0). Contributed by Colin Patrick McCabe.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1434530 13f79535-47bb-0310-9956-ffa450edef68
2013-01-17 01:37:41 +00:00
Thomas Graves
ab0e94d2a5
HADOOP-9097. Maven RAT plugin is not checking all source files (tgraves)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1432934 13f79535-47bb-0310-9956-ffa450edef68
2013-01-14 14:37:12 +00:00
Suresh Srinivas
1f41061132
MAPREDUCE-4884. Streaming tests fail to start MiniMRCluster due to missing queue configuration. Contributed by Chris Nauroth.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1427945 13f79535-47bb-0310-9956-ffa450edef68
2013-01-02 19:31:11 +00:00
Jason Darrell Lowe
40c3b7f0b2
MAPREDUCE-4793. Problem with adding resources when using both -files and -file to hadoop streaming. Contributed by Jason Lowe
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1425177 13f79535-47bb-0310-9956-ffa450edef68
2012-12-21 23:05:54 +00:00
Robert Joseph Evans
966e20fa3d
YARN-204. test coverage for org.apache.hadoop.tools (Aleksey Gorshkov via bobby)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1414239 13f79535-47bb-0310-9956-ffa450edef68
2012-11-27 16:05:16 +00:00
Jason Darrell Lowe
1ebe1d8303
HADOOP-9025. org.apache.hadoop.tools.TestCopyListing failing. Contributed by Jonathan Eagles
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1408604 13f79535-47bb-0310-9956-ffa450edef68
2012-11-13 05:53:26 +00:00
Robert Joseph Evans
416c7ded54
HADOOP-9022. Hadoop distcp tool fails to copy file if -m 0 specified (Jonathan Eagles vai bobby)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1408488 13f79535-47bb-0310-9956-ffa450edef68
2012-11-12 22:43:07 +00:00
Alejandro Abdelnur
a4f30ff53c
reverting commit 1399898 (HADOOP-8887)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1399972 13f79535-47bb-0310-9956-ffa450edef68
2012-10-19 05:38:26 +00:00
Alejandro Abdelnur
53c598ab11
HADOOP-8887. Use a Maven plugin to build the native code using CMake. (cmccabe via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1399898 13f79535-47bb-0310-9956-ffa450edef68
2012-10-18 22:40:41 +00:00
Thomas White
aa934616dd
MAPREDUCE-4654. TestDistCp is ignored. Contributed by Sandy Ryza.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1396047 13f79535-47bb-0310-9956-ffa450edef68
2012-10-09 14:37:46 +00:00
Eli Collins
aa4c3fc8aa
HADOOP-8789. Tests setLevel(Level.OFF) should be Level.ERROR. Contributed by Andy Isaacson
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1383494 13f79535-47bb-0310-9956-ffa450edef68
2012-09-11 17:56:19 +00:00
Aaron Myers
6ba9ff9177
HADOOP-8775. MR2 distcp permits non-positive value to -bandwidth option which causes job never to complete. Contributed by Sandy Ryza.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1382119 13f79535-47bb-0310-9956-ffa450edef68
2012-09-07 18:40:30 +00:00
Eli Collins
404ab8ec66
HADOOP-8431. Running distcp wo args throws IllegalArgumentException. Contributed by Sandy Ryza
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1381840 13f79535-47bb-0310-9956-ffa450edef68
2012-09-07 01:25:30 +00:00
Todd Lipcon
e57843e02a
HDFS-3054. distcp -skipcrccheck has no effect. Contributed by Colin Patrick McCabe.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1381296 13f79535-47bb-0310-9956-ffa450edef68
2012-09-05 18:54:45 +00:00
Robert Joseph Evans
aeb364d8b5
HADOOP-8225. DistCp fails when invoked by Oozie (daryn via bobby)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1376618 13f79535-47bb-0310-9956-ffa450edef68
2012-08-23 18:06:56 +00:00
Robert Joseph Evans
b05c70bd96
HADOOP-8703: Fix formatting issue.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1373599 13f79535-47bb-0310-9956-ffa450edef68
2012-08-15 19:50:07 +00:00
Robert Joseph Evans
6167123bcb
HADOOP-8703. distcpV2: turn CRC checking off for 0 byte size (Dave Thompson via bobby)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1373581 13f79535-47bb-0310-9956-ffa450edef68
2012-08-15 19:16:13 +00:00
Todd Lipcon
0733adf670
HADOOP-8659. Native libraries must build with soft-float ABI for Oracle JVM on ARM. Contributed by Trevor Robinson.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1371507 13f79535-47bb-0310-9956-ffa450edef68
2012-08-09 21:58:21 +00:00
Thomas Graves
735b50e8bd
MAPREDUCE-4493. Distibuted Cache Compatability Issues (Robert Evans via tgraves)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1367713 13f79535-47bb-0310-9956-ffa450edef68
2012-07-31 19:20:03 +00:00
Robert Joseph Evans
9c87911c4a
HADOOP-8521. Port StreamInputFormat to new Map Reduce API (madhukara phatak via bobby)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1360238 13f79535-47bb-0310-9956-ffa450edef68
2012-07-11 15:44:43 +00:00
Eli Collins
8188c1cb41
HADOOP-8547. Package hadoop-pipes examples/bin directory (again). Contributed by Colin Patrick McCabe
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1356947 13f79535-47bb-0310-9956-ffa450edef68
2012-07-03 21:12:04 +00:00
Eli Collins
dcc1d3b7f0
HADOOP-8538. CMake builds fail on ARM. Contributed by Trevor Robinson
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1356041 13f79535-47bb-0310-9956-ffa450edef68
2012-07-01 22:47:48 +00:00
Eli Collins
cd9ce99740
MAPREDUCE-4383. HadoopPipes.cc needs to include unistd.h. Contributed by Andy Isaacson
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1355555 13f79535-47bb-0310-9956-ffa450edef68
2012-06-29 20:57:20 +00:00
Thomas Graves
b91910dd3e
MAPREDUCE-4320. gridmix mainClass wrong in pom.xml (tgraves)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1352550 13f79535-47bb-0310-9956-ffa450edef68
2012-06-21 15:01:35 +00:00
Ravi Gummadi
b685486fbb
MAPREDUCE-4356. [Rumen] Provide access to the method ParsedTask.obtainTaskAttempts(). (ravigummadi)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1352415 13f79535-47bb-0310-9956-ffa450edef68
2012-06-21 05:44:20 +00:00
Siddharth Seth
10bc39ea02
MAPREDUCE-4270. Move the data_join test classes to the correct path. (Contributed by Thomas Graves)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1351869 13f79535-47bb-0310-9956-ffa450edef68
2012-06-19 21:51:15 +00:00
Robert Joseph Evans
49feb6a039
MAPREDUCE-4267. mavenize pipes (tgraves via bobby)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1351818 13f79535-47bb-0310-9956-ffa450edef68
2012-06-19 19:40:44 +00:00
Robert Joseph Evans
8bbacfaa31
MAPREDUCE-4297. Usersmap file in gridmix should not fail on empty lines (Ravi Prakash via bobby)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1344763 13f79535-47bb-0310-9956-ffa450edef68
2012-05-31 15:16:48 +00:00
Harsh J
1fab229f30
HADOOP-8268. A few pom.xml across Hadoop project may fail XML validation. Contributed by Radim Kolar. (harsh)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1343272 13f79535-47bb-0310-9956-ffa450edef68
2012-05-28 14:51:05 +00:00
