big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
22,652 Commits 2 Branches 2 Tags 264 MiB
2ae7f444bd
Commit Graph

22652 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Ayush Saxena
6282c02d7d HDFS-14577. RBF: FederationUtil#newInstance should allow constructor without context. Contributed by CR Hota. 2019-07-19 17:44:13 +05:30
Xiaoyu Yao
4e66cb9333 HDDS-1653. Add option to "ozone scmcli printTopology" to order the output acccording to topology layer. Contributed by Xiaoyu Yao. (#1067) 
* HDDS-1653. Add option to "ozone scmcli printTopology" to order the output acccording to topology layer. Contributed by Xiaoyu Yao.

* use ip/hostname instead of network name for -o output and add smoke test
 2019-07-19 11:00:49 +08:00
Josh Rosen
d545f9c290 HADOOP-16437 documentation typo fix: fs.s3a.experimental.input.fadvise 
Fix fs.s3a.experimental.fadvise to fs.s3a.experimental.input.fadvise 

Contributed by: Josh Rosen
 2019-07-18 23:19:38 +01:00
Doroszlai, Attila
d5ef38b093 HDDS-1822. NPE in SCMCommonPolicy.chooseDatanodes (#1120) 2019-07-18 13:28:03 -07:00
Eric Yang
9838a47d44 YARN-6046. Fixed documentation error in YarnApplicationSecurity. 
Contributed by Yousef Abu-Salah
 2019-07-18 12:39:18 -04:00
Bharat Viswanadham
f22818d0d6
HDDS-1820. Fix numKeys metrics in OM HA. (#1116) 2019-07-18 09:36:15 -07:00
Eric Yang
c34ceb5fde YARN-9568. Fixed NPE in MiniYarnCluster during FileSystemNodeAttributeStore.recover. 
Contributed by Steve Loughran
 2019-07-18 12:30:53 -04:00
Shashikant Banerjee
8ef2365ffd HDDS-1779. TestWatchForCommit tests are flaky.Contributed by Shashikant Banerjee. (#1071) 2019-07-18 19:57:12 +05:30
Masatake Iwasaki
79f6118dcc
MAPREDUCE-6521. MiniMRYarnCluster should not create /tmp/hadoop-yarn/staging on local filesystem in unit test. 
Contributed by Masatake Iwasaki.

Change-Id: Id74b90eb7cfb9d676188fc5aa47249ee8904a3d5
 2019-07-18 13:15:18 +01:00
Shashikant Banerjee
20cf50c6d0 HDDS-1654. Ensure container state on datanode gets synced to disk whennever state change happens. Cotributed by Shashikant Banerjee. (#923) 2019-07-18 17:09:05 +05:30
Shashikant Banerjee
ccceedb432 HDDS-1780. TestFailureHandlingByClient tests are flaky. Contributed by Shashikant Banerjee. (#1073) 2019-07-18 16:01:58 +05:30
Lokesh Jain
23e9bebe13
HDDS-1767: ContainerStateMachine should have its own executors for executing applyTransaction calls (#1087) 2019-07-18 14:48:13 +05:30
Lokesh Jain
53a4c22b40
HDDS-1481: Cleanup BasicOzoneFileSystem#mkdir (#1114) 2019-07-18 14:40:45 +05:30
Bharat Viswanadham
3dc256ef81
HDDS-1689. Implement S3 Create Bucket request to use Cache and DoubleBuffer. (#1088) 2019-07-17 21:46:27 -07:00
Bharat Viswanadham
73e6ffce69
HDDS-1802. Add Eviction policy for table cache. (#1100) 2019-07-17 18:19:06 -07:00
Takanobu Asanuma
9792f58662 HDFS-14653. RBF: Correct the default value for dfs.federation.router.namenode.heartbeat.enable. Contributed by Ayush Saxena. 2019-07-18 10:16:20 +09:00
Chen Liang
5e6cc6fe8a HDFS-12979. [SBN read] StandbyNode should upload FsImage to ObserverNode after checkpointing. Contributed by Chen Liang. 2019-07-17 14:37:16 -07:00
Bharat Viswanadham
0541322c9a
HDDS-1675. Cleanup Volume Request 2 phase old code. (#964) 2019-07-17 14:13:06 -07:00
avijayanhwx
303a7f8a39 HDDS-1721 : Client Metrics are not being pushed to the configured sink while running a hadoop command to write to Ozone. (#1034) 2019-07-17 12:05:43 -07:00
Ayush Saxena
af0665c7f5 HDFS-13647. Fix the description of storageType option for space quota. Contributed by Takanobu Asanuma. 2019-07-17 23:22:32 +05:30
Xiaoyu Yao
eb0849b547 HDDS-1801. Make Topology Aware Replication/Read non-default for ozone 0.4.1. Contributed by Xiaoyu Yao. 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-07-17 10:32:59 -07:00
Gabor Bota
c58e11bf52
HADOOP-16383. Pass ITtlTimeProvider instance in initialize method in MetadataStore interface. Contributed by Gabor Bota. (#1009) 2019-07-17 16:24:39 +02:00
Doroszlai, Attila
256fcc160e
HDDS-1785. OOM error in Freon due to the concurrency handling 
Closes #1085
 2019-07-17 15:02:15 +02:00
Gopal V
b4466a3b0a
HADOOP-16341. ShutDownHookManager: Regressed performance on Hook removals after HADOOP-15679 
Contributed by Gopal V and Atilla Magyar.

Change-Id: I066d5eece332a1673594de0f9b484443f95530ec
 2019-07-17 13:50:02 +01:00
Steve Loughran
19a001826f
Revert "HDFS-9913. DistCp to add -useTrash to move deleted files to Trash." 
Reverting due to test failures if ~/.Trash not present during test setup.

This reverts commit ee3115f488.

Change-Id: Icbeeb261570b9131ff99d765ac0945c335b26658
 2019-07-17 13:13:24 +01:00
Shen Yinjie
ee3115f488
HDFS-9913. DistCp to add -useTrash to move deleted files to Trash. 
Contributed by Shen Yinjie.

Change-Id: I03ac7d22ab1054f8e5de4aa7552909c734438f4a
 2019-07-17 11:50:46 +01:00
Weiwei Yang
85d9111a88 YARN-9682. Wrong log message when finalizing the upgrade. Contributed by kyungwan nam. 2019-07-17 10:28:20 +08:00
Haibo Chen
5915c902aa YARN-9646. DistributedShell tests failed to bind to a local host name. (Contributed by Ray Yang) 2019-07-16 17:36:49 -07:00
Sean Mackrory
5672efa5c7
HADOOP-15729. [s3a] Allow core threads to time out. (#1075) 2019-07-16 18:14:23 -06:00
Hanisha Koneru
7adc4a327a
HDDS-1775. Make OM KeyDeletingService compatible with HA model (#1063) 2019-07-16 16:59:57 -07:00
Xiaoyu Yao
71c21f0a9b
HDDS-1813. Fix false warning from ozones3 acceptance test. Contributed by Xiaoyu Yao. (#1107) 2019-07-16 13:47:51 -07:00
Xiaoyu Yao
d429d742f0 HDDS-1544. Support default Acls for volume, bucket, keys and prefix. Contributed by Ajay Kumar, Xiaoyu Yao. 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-07-16 12:58:59 -07:00
Sammi Chen
493b0b5760 HDDS-1787. NPE thrown while trying to find DN closest to client. Contributed by Sammi Chen. (#1094) 2019-07-16 12:52:29 -07:00
Doroszlai, Attila
c5e3ab5a4d
HDDS-1793. Acceptance test of ozone-topology cluster is failing 
Closes #1096
 2019-07-16 16:52:14 +02:00
Shashikant Banerjee
d21eccf8ba HDDS-1492. Generated chunk size name too long. Contributed by Shashikannt Banerjee. (#1084) 2019-07-16 18:01:13 +05:30
Nanda kumar
1502e6a22f HDDS-1756. DeleteContainerCommandHandler fails with NPE. Contributed by Nanda kumar(#1095). 2019-07-16 15:06:41 +05:30
bibinchundatt
7a93be0f60 YARN-9645. Fix Invalid event FINISHED_CONTAINERS_PULLED_BY_AM at NEW on NM restart. Contributed by Bilwa S T. 2019-07-16 14:03:22 +05:30
Bharat Viswanadham
395cb3cfd7
HDDS-1736. Cleanup 2phase old HA code for Key requests. (#1038) 2019-07-15 21:51:59 -07:00
Ayush Saxena
f77d54c243 HDFS-14642. processMisReplicatedBlocks does not return correct processed count. Contributed by Stephen O'Donnell. 2019-07-16 08:14:27 +05:30
Bharat Viswanadham
ef66e4999f HDDS-1666. Issue in openKey when allocating block. (#943) 2019-07-15 17:54:41 -07:00
Bharat Viswanadham
585f4d5c64
HDDS-1761. Fix class hierarchy for KeyRequest and FileRequest classes. (#1052) 2019-07-15 17:53:19 -07:00
Anu Engineer
141151325b
Merge pull request #944 from elek/HDDS-1668 
HDDS-1668. Add liveness probe to the example k8s resources files
 2019-07-15 13:19:32 -07:00
Anu Engineer
47345e598e
Merge pull request #1058 from elek/HDDS-1763 
HDDS-1763. Use vendor neutral s3 logo in ozone doc
 2019-07-15 13:17:21 -07:00
Ayush Saxena
64d4abf489 HDFS-14593. RBF: Implement deletion feature for expired records in State Store. Contributed by Takanobu Asanuma. 2019-07-15 22:38:00 +05:30
Doroszlai, Attila
61bbdeee19
HDDS-1800. Result of author check is inverted 
Closes #1092
 2019-07-15 18:00:10 +02:00
Szilard Nemeth
5446308360 YARN-9326. Fair Scheduler configuration defaults are not documented in case of min and maxResources. Contributed by Adam Antal 2019-07-15 13:28:01 +02:00
Szilard Nemeth
18ee1092b4 YARN-9127. Create more tests to verify GpuDeviceInformationParser. Contributed by Peter Bacsko 2019-07-15 11:59:11 +02:00
Szilard Nemeth
be784de2d4 SUBMARINE-62. PS_LAUNCH_CMD CLI description is wrong in RunJobCli. Contributed by Adam Antal 2019-07-15 11:17:16 +02:00
Szilard Nemeth
91ce09e706 YARN-9360. Do not expose innards of QueueMetrics object into FSLeafQueue#computeMaxAMResource. Contributed by Peter Bacsko 2019-07-15 10:47:20 +02:00
Rakesh Radhakrishnan
30a8f840f1 HDFS-14357. Update documentation for HDFS cache on SCM support. Contributed by Feilong He. 2019-07-15 13:18:23 +05:30