big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
22,639 Commits 2 Branches 2 Tags 264 MiB
7653ebdbb2
Commit Graph

22639 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Bharat Viswanadham
3dc256ef81
HDDS-1689. Implement S3 Create Bucket request to use Cache and DoubleBuffer. (#1088) 2019-07-17 21:46:27 -07:00
Bharat Viswanadham
73e6ffce69
HDDS-1802. Add Eviction policy for table cache. (#1100) 2019-07-17 18:19:06 -07:00
Takanobu Asanuma
9792f58662 HDFS-14653. RBF: Correct the default value for dfs.federation.router.namenode.heartbeat.enable. Contributed by Ayush Saxena. 2019-07-18 10:16:20 +09:00
Chen Liang
5e6cc6fe8a HDFS-12979. [SBN read] StandbyNode should upload FsImage to ObserverNode after checkpointing. Contributed by Chen Liang. 2019-07-17 14:37:16 -07:00
Bharat Viswanadham
0541322c9a
HDDS-1675. Cleanup Volume Request 2 phase old code. (#964) 2019-07-17 14:13:06 -07:00
avijayanhwx
303a7f8a39 HDDS-1721 : Client Metrics are not being pushed to the configured sink while running a hadoop command to write to Ozone. (#1034) 2019-07-17 12:05:43 -07:00
Ayush Saxena
af0665c7f5 HDFS-13647. Fix the description of storageType option for space quota. Contributed by Takanobu Asanuma. 2019-07-17 23:22:32 +05:30
Xiaoyu Yao
eb0849b547 HDDS-1801. Make Topology Aware Replication/Read non-default for ozone 0.4.1. Contributed by Xiaoyu Yao. 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-07-17 10:32:59 -07:00
Gabor Bota
c58e11bf52
HADOOP-16383. Pass ITtlTimeProvider instance in initialize method in MetadataStore interface. Contributed by Gabor Bota. (#1009) 2019-07-17 16:24:39 +02:00
Doroszlai, Attila
256fcc160e
HDDS-1785. OOM error in Freon due to the concurrency handling 
Closes #1085
 2019-07-17 15:02:15 +02:00
Gopal V
b4466a3b0a
HADOOP-16341. ShutDownHookManager: Regressed performance on Hook removals after HADOOP-15679 
Contributed by Gopal V and Atilla Magyar.

Change-Id: I066d5eece332a1673594de0f9b484443f95530ec
 2019-07-17 13:50:02 +01:00
Steve Loughran
19a001826f
Revert "HDFS-9913. DistCp to add -useTrash to move deleted files to Trash." 
Reverting due to test failures if ~/.Trash not present during test setup.

This reverts commit ee3115f488.

Change-Id: Icbeeb261570b9131ff99d765ac0945c335b26658
 2019-07-17 13:13:24 +01:00
Shen Yinjie
ee3115f488
HDFS-9913. DistCp to add -useTrash to move deleted files to Trash. 
Contributed by Shen Yinjie.

Change-Id: I03ac7d22ab1054f8e5de4aa7552909c734438f4a
 2019-07-17 11:50:46 +01:00
Weiwei Yang
85d9111a88 YARN-9682. Wrong log message when finalizing the upgrade. Contributed by kyungwan nam. 2019-07-17 10:28:20 +08:00
Haibo Chen
5915c902aa YARN-9646. DistributedShell tests failed to bind to a local host name. (Contributed by Ray Yang) 2019-07-16 17:36:49 -07:00
Sean Mackrory
5672efa5c7
HADOOP-15729. [s3a] Allow core threads to time out. (#1075) 2019-07-16 18:14:23 -06:00
Hanisha Koneru
7adc4a327a
HDDS-1775. Make OM KeyDeletingService compatible with HA model (#1063) 2019-07-16 16:59:57 -07:00
Xiaoyu Yao
71c21f0a9b
HDDS-1813. Fix false warning from ozones3 acceptance test. Contributed by Xiaoyu Yao. (#1107) 2019-07-16 13:47:51 -07:00
Xiaoyu Yao
d429d742f0 HDDS-1544. Support default Acls for volume, bucket, keys and prefix. Contributed by Ajay Kumar, Xiaoyu Yao. 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-07-16 12:58:59 -07:00
Sammi Chen
493b0b5760 HDDS-1787. NPE thrown while trying to find DN closest to client. Contributed by Sammi Chen. (#1094) 2019-07-16 12:52:29 -07:00
Doroszlai, Attila
c5e3ab5a4d
HDDS-1793. Acceptance test of ozone-topology cluster is failing 
Closes #1096
 2019-07-16 16:52:14 +02:00
Shashikant Banerjee
d21eccf8ba HDDS-1492. Generated chunk size name too long. Contributed by Shashikannt Banerjee. (#1084) 2019-07-16 18:01:13 +05:30
Nanda kumar
1502e6a22f HDDS-1756. DeleteContainerCommandHandler fails with NPE. Contributed by Nanda kumar(#1095). 2019-07-16 15:06:41 +05:30
bibinchundatt
7a93be0f60 YARN-9645. Fix Invalid event FINISHED_CONTAINERS_PULLED_BY_AM at NEW on NM restart. Contributed by Bilwa S T. 2019-07-16 14:03:22 +05:30
Bharat Viswanadham
395cb3cfd7
HDDS-1736. Cleanup 2phase old HA code for Key requests. (#1038) 2019-07-15 21:51:59 -07:00
Ayush Saxena
f77d54c243 HDFS-14642. processMisReplicatedBlocks does not return correct processed count. Contributed by Stephen O'Donnell. 2019-07-16 08:14:27 +05:30
Bharat Viswanadham
ef66e4999f HDDS-1666. Issue in openKey when allocating block. (#943) 2019-07-15 17:54:41 -07:00
Bharat Viswanadham
585f4d5c64
HDDS-1761. Fix class hierarchy for KeyRequest and FileRequest classes. (#1052) 2019-07-15 17:53:19 -07:00
Anu Engineer
141151325b
Merge pull request #944 from elek/HDDS-1668 
HDDS-1668. Add liveness probe to the example k8s resources files
 2019-07-15 13:19:32 -07:00
Anu Engineer
47345e598e
Merge pull request #1058 from elek/HDDS-1763 
HDDS-1763. Use vendor neutral s3 logo in ozone doc
 2019-07-15 13:17:21 -07:00
Ayush Saxena
64d4abf489 HDFS-14593. RBF: Implement deletion feature for expired records in State Store. Contributed by Takanobu Asanuma. 2019-07-15 22:38:00 +05:30
Doroszlai, Attila
61bbdeee19
HDDS-1800. Result of author check is inverted 
Closes #1092
 2019-07-15 18:00:10 +02:00
Szilard Nemeth
5446308360 YARN-9326. Fair Scheduler configuration defaults are not documented in case of min and maxResources. Contributed by Adam Antal 2019-07-15 13:28:01 +02:00
Szilard Nemeth
18ee1092b4 YARN-9127. Create more tests to verify GpuDeviceInformationParser. Contributed by Peter Bacsko 2019-07-15 11:59:11 +02:00
Szilard Nemeth
be784de2d4 SUBMARINE-62. PS_LAUNCH_CMD CLI description is wrong in RunJobCli. Contributed by Adam Antal 2019-07-15 11:17:16 +02:00
Szilard Nemeth
91ce09e706 YARN-9360. Do not expose innards of QueueMetrics object into FSLeafQueue#computeMaxAMResource. Contributed by Peter Bacsko 2019-07-15 10:47:20 +02:00
Rakesh Radhakrishnan
30a8f840f1 HDFS-14357. Update documentation for HDFS cache on SCM support. Contributed by Feilong He. 2019-07-15 13:18:23 +05:30
Rakesh Radhakrishnan
e98adb00b7 HDFS-14458. Report pmem stats to namenode. Contributed by Feilong He. 2019-07-15 13:02:37 +05:30
Mukul Kumar Singh
0976f6fc30
HDDS-1766. ContainerStateMachine is unable to increment lastAppliedTermIndex. Contributed by Mukul Kumar Singh. (#1072) 2019-07-14 10:53:51 +05:30
Anu Engineer
4a70a0d816
HDDS-1735. Create separate unit and integration test executor dev-support script. Contributed by Elek, Marton. 2019-07-12 09:25:00 -07:00
Szilard Nemeth
557056e18e YARN-9626. UI2 - Fair scheduler queue apps page issues. Contributed by Zoltan Siegl 2019-07-12 17:40:14 +02:00
Szilard Nemeth
61b0c2bb7c YARN-9337. GPU auto-discovery script runs even when the resource is given by hand. Contributed by Adam Antal 2019-07-12 17:28:14 +02:00
Szilard Nemeth
8b3c6791b1 YARN-9135. NM State store ResourceMappings serialization are tested with Strings instead of real Device objects. Contributed by Peter Bacsko 2019-07-12 17:20:42 +02:00
Szilard Nemeth
c416284bb7 YARN-9235. If linux container executor is not set for a GPU cluster GpuResourceHandlerImpl is not initialized and NPE is thrown. Contributed by Antal Balint Steinbach, Adam Antal 2019-07-12 16:51:58 +02:00
Nanda kumar
190e4349d7
HDDS-1790. Fix checkstyle issues in TestDataScrubber 
Closes #1082
 2019-07-12 14:42:29 +02:00
Nanda kumar
7b8177ba0f
HDDS-1791. Update network-tests/src/test/blockade/README.md file 
Closes #1083
 2019-07-12 14:38:22 +02:00
Steve Loughran
b15ef7dc3d
HADOOP-16384: S3A: Avoid inconsistencies between DDB and S3. 
Contributed by Steve Loughran

Contains

- HADOOP-16397. Hadoop S3Guard Prune command to support a -tombstone option.
- HADOOP-16406. ITestDynamoDBMetadataStore.testProvisionTable times out intermittently

This patch doesn't fix the underlying problem but it

* changes some tests to clean up better
* does a lot more in logging operations in against DDB, if enabled
* adds an entry point to dump the state of the metastore and s3 tables (precursor to fsck)
* adds a purge entry point to help clean up after a test run has got a store into a mess
* s3guard prune command adds -tombstone option to only clear tombstones

The outcome is that tests should pass consistently and if problems occur we have better diagnostics.

Change-Id: I3eca3f5529d7f6fec398c0ff0472919f08f054eb
 2019-07-12 13:02:25 +01:00
Shashikant Banerjee
f9fab9f22a HDFS-14499. Misleading REM_QUOTA value with snapshot and trash feature enabled for a directory. Contributed by Shashikant Banerjee. 2019-07-12 15:42:06 +05:30
Márton Elek
51e7dc3221
make author check yetus safe 2019-07-12 11:59:24 +02:00
Márton Elek
ca7b3e8310
reenable unit test execution 2019-07-12 11:59:20 +02:00