big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
23,033 Commits 2 Branches 2 Tags 264 MiB
ec8f691201
Commit Graph

23033 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Doroszlai, Attila
ec8f691201
HDDS-2225. SCM fails to start in most unsecure environments due to leftover secure config 
Closes #1568
 2019-10-04 08:50:13 +02:00
Stephen O'Donnell
c99a12167f HDFS-14637. Namenode may not replicate blocks to meet the policy after enabling upgradeDomain. Contributed by Stephen O'Donnell. 
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-10-03 22:13:50 -07:00
Virajith Jalaparti
844b766da5 HDFS-14889. Ability to check if a block has a replica on provided storage. Contributed by Ashvin Agrawal. (#1573)" 2019-10-03 21:32:48 -07:00
Nanda kumar
9700e2003a
HDDS-2223. Support ReadWrite lock in LockManager. (#1564) 2019-10-04 08:32:43 +05:30
Nanda kumar
cdaa480dbf
HDDS-2198. SCM should not consider containers in CLOSING state to come out of safemode. (#1540) 2019-10-04 08:20:58 +05:30
pingsutw
1dde3efb91
HADOOP-16624. Upgrade hugo to the latest version in Dockerfile 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2019-10-04 10:01:07 +09:00
Aravindan Vijayan
b7cb8fe07c HDDS-2200 : Recon does not handle the NULL snapshot from OM DB cleanly. 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-10-03 15:38:19 -07:00
Aravindan Vijayan
76605f17dd HDDS-1720 : Add ability to configure RocksDB logs for Ozone Manager. 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-10-03 15:17:09 -07:00
Doroszlai, Attila
944668674b HDDS-2231. test-single.sh cannot copy results (#1575) 2019-10-03 14:50:04 -07:00
Doroszlai, Attila
47d721d7dd HDDS-2234. rat.sh fails due to ozone-recon-web/build files (#1580) 2019-10-03 14:46:50 -07:00
Doroszlai, Attila
51eaecab20 HDDS-2211. Collect docker logs if env fails to start (#1553) 2019-10-03 12:44:04 -07:00
Ayush Saxena
a3fe404244 HDFS-14881. Safemode 'forceExit' option, doesn’t shown in help message. Contributed by Renukaprasad C. 2019-10-03 22:34:55 +05:30
Anu Engineer
d59bcbfa0f
HDDS-2226. S3 Secrets should use a strong RNG. (#1572) 2019-10-03 09:28:41 -07:00
Ayush Saxena
5a7483ca5c HDFS-14888. RBF: Enable Parallel Test Profile for builds. Contributed by Ayush Saxena. 2019-10-03 19:21:27 +05:30
Sneha Vijayarajan
c19fa3d4fe HADOOP-16605. Fix testcase testSSLChannelModeConfig 
Contributed by Sneha Vijayarajan.
 2019-10-03 11:13:55 +01:00
Akira Ajisaka
0e026cb0ce
HADOOP-16620. [pb-upgrade] Remove protocol buffers 3.7.1 from requirements in BUILDING.txt (#1558) 
* Removed protobuf 3.7.1 from Dockerfile
 2019-10-03 15:24:47 +09:00
Xiaoyu Yao
c5665b23ca
HDDS-2228. Fix NPE in OzoneDelegationTokenManager#addPersistedDelegat… (#1571) 2019-10-02 23:09:06 -07:00
Anu Engineer
4c24f2434d HDDS-2072. Make StorageContainerLocationProtocolService message based 
Contributed by Elek, Marton.
 2019-10-02 17:00:53 -07:00
Chen Liang
1303255aee HDFS-14858. [SBN read] Allow configurably enable/disable AlignmentContext on NameNode. Contributed by Chen Liang. 2019-10-02 16:38:20 -07:00
Siddharth Seth
559ee277f5
HADOOP-16599. Allow a SignerInitializer to be specified along with a Custom Signer 2019-10-02 16:03:48 -07:00
Bharat Viswanadham
b09d389001
HDDS-2019. Handle Set DtService of token in S3Gateway for OM HA. (#1489) 2019-10-02 15:41:54 -07:00
Bharat Viswanadham
53ed78bcdb
HDDS-2224. Fix loadup cache for cache cleanup policy NEVER. (#1567) 2019-10-02 15:18:43 -07:00
Bharat Viswanadham
169cef758d
HDDS-2162. Make OM Generic related configuration support HA style config. (#1511) 2019-10-02 15:09:33 -07:00
Anu Engineer
685918ef41
HDDS-2227. GDPR key generation could benefit from secureRandom. (#1574) 2019-10-02 12:34:53 -07:00
Anu Engineer
ffd4e52725 HDDS-2073. Make SCMSecurityProtocol message based. 
Contributed by Elek, Marton.
 2019-10-02 12:19:58 -07:00
Márton Elek
e8ae632d4c HDDS-2068. Make StorageContainerDatanodeProtocolService message based 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-10-02 11:18:41 -07:00
Sunil G
0d2d6f9267 YARN-9792. Document examples of SchedulerConf with Node Labels. Contributed by Prabhu Joseph 2019-10-02 23:43:47 +05:30
Anu Engineer
2e1fd44596
HDDS-2201. Rename VolumeList to UserVolumeInfo. (#1566) 2019-10-02 09:25:47 -07:00
Abhishek Modi
61a8436004 YARN-9870. Remove unused function from OpportunisticContainerAllocatorAMService. Contributed by Abhishek Modi. 2019-10-02 17:55:07 +05:30
Doroszlai, Attila
f1ba9bfad7
HDDS-2187. ozone-mr test fails with No FileSystem for scheme "o3fs" 
Closes #1537
 2019-10-02 12:57:23 +02:00
bshashikant
41440ec890
HDDS-2210. ContainerStateMachine should not be marked unhealthy if applyTransaction fails with closed container exception(#1552). 2019-10-02 10:11:01 +05:30
Akira Ajisaka
3df733c250
HDFS-14885. UI: Fix a typo on WebUI of DataNode. Contributed by Xieming Li. 2019-10-02 11:03:16 +09:00
Eric Yang
d1ddba60e5 YARN-9862. Increase yarn-services-core test timeout value. 
Contributed by Prabhu Joseph
 2019-10-01 21:15:26 -04:00
Sneha Vijayarajan
770adc5d4a
HADOOP-16578 : Avoid FileSystem API calls when FileSystem already exists 2019-10-01 17:38:11 -07:00
Siyao Meng
d947ded053 HADOOP-16619. Upgrade jackson and jackson-databind to 2.9.10 (#1554). Contributed by Siyao Meng. 2019-10-01 12:46:40 -07:00
Steve Loughran
1921e94292
HADOOP-16458. LocatedFileStatusFetcher.getFileStatuses failing intermittently with S3 
Contributed by Steve Loughran.

Includes
-S3A glob scans don't bother trying to resolve symlinks
-stack traces don't get lost in getFileStatuses() when exceptions are wrapped
-debug level logging of what is up in Globber
-Contains HADOOP-13373. Add S3A implementation of FSMainOperationsBaseTest.
-ITestRestrictedReadAccess tests incomplete read access to files.

This adds a builder API for constructing globbers which other stores can use
so that they too can skip symlink resolution when not needed.

Change-Id: I23bcdb2783d6bd77cf168fdc165b1b4b334d91c7
 2019-10-01 18:11:05 +01:00
Márton Elek
918b470deb
HDDS-2166. Some RPC metrics are missing from SCM prometheus endpoint 
Closes #1505
 2019-10-01 18:01:51 +02:00
Wei-Chiu Chuang
6ef6594c7e HDFS-14492. Snapshot memory leak. Contributed by Wei-Chiu Chuang. (#1370) 
* HDFS-14492. Snapshot memory leak. Contributed by Wei-Chiu Chuang.

Change-Id: I9e5e450c07ad70aa1905973896c4f627042dbd37

* Fix checkstyle

Change-Id: I16d4bd4f03a971e1ed36cf57d89dc42357ef8fbf
 2019-10-01 21:16:46 +05:30
Sunil G
99cd7572f1 YARN-9801. SchedConfCli does not work wiwith https mode. Contributed by Prabhu Joseph 2019-10-01 20:06:21 +05:30
bibinchundatt
425a6c89ae YARN-9858. Optimize RMContext getExclusiveEnforcedPartitions. Contributed by Jonathan Hung. 2019-10-01 16:02:26 +05:30
Mukul Kumar Singh
8efd25b33a
HDDS-1615. ManagedChannel references are being leaked in ReplicationSupervisor.java. Contributed by Mukul Kumar Singh. (#1547) 2019-10-01 14:31:15 +05:30
Sunil G
137546a78a YARN-9864. Format CS Configuration present in Configuration Store. Contributeed by Prabhu Joseph 2019-10-01 09:08:41 +05:30
Konstantin V Shvachko
b3275ab1f2 HDFS-14305. Fix serial number calculation in BlockTokenSecretManager to avoid token key ID overlap between NameNodes. Contributed by Konstantin V Shvachko. 2019-09-30 16:48:10 -07:00
Doroszlai, Attila
e5bba592a8 HDDS-2205. checkstyle.sh reports wrong failure count 
Signed-off-by: Anu Engineer <aengineer@apache.org>
 2019-09-30 16:32:38 -07:00
Abhishek Modi
4d3c580b03 YARN-9859. Refactoring of OpportunisticContainerAllocator. Contributed by Abhishek Modi. 2019-09-30 23:40:15 +05:30
bshashikant
98ca07ebed HDDS-2207. Update Ratis to latest snapshot. Contributed by Shashikant Bannerjee. (#1550) 2019-09-30 20:36:27 +05:30
dchitlangia
b46d82339f
HDDS-2202. Remove unused import in OmUtils 
Closes #1543
 2019-09-30 16:02:30 +02:00
Shashikant Banerjee
a530ac3f50
HDDS-2153. Add a config to tune max pending requests in Ratis leader 
Closes #1474
 2019-09-30 15:42:04 +02:00
cxorm
d6b0a8da77
HDDS-2183. Container and pipline subcommands of scmcli should be grouped 
Closes #1532
 2019-09-30 14:38:01 +02:00
Konstantin V Shvachko
760b523e58 Revert "HDFS-14305. Fix serial number calculation in BlockTokenSecretManager to avoid token key ID overlap between NameNodes. Contributed by He Xiaoqiao." 
This reverts commit 0feba4396f.
 2019-09-29 13:11:31 -07:00