big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
23,475 Commits 2 Branches 2 Tags 264 MiB
ca29768035
Commit Graph

23475 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Yiqun Lin
c3659f8f94 HDFS-15019. Refactor the unit test of TestDeadNodeDetection. Contributed by Lisheng Sun. 2019-11-28 14:41:49 +08:00
prabhujoseph
013c6e51b6 MAPREDUCE-7249. Fix Invalid event TA_TOO_MANY_FETCH_FAILURE at SUCCESS_CONTAINER_CLEANUP causes job failure. 
Contributed by Wilfred Spiegelenburg.
 2019-11-28 10:57:15 +05:30
Yiqun Lin
2b452b4e60 HDFS-14986. ReplicaCachingGetSpaceUsed throws ConcurrentModificationException. Contributed by Aiphago. 2019-11-28 10:43:35 +08:00
Sneha Vijayarajan
82ad9b549f
HADOOP-16660. ABFS: Make RetryCount in ExponentialRetryPolicy Configurable. 
Contributed by Sneha Vijayarajan.
 2019-11-27 15:10:21 -08:00
bilaharith
9e69628f55 HADOOP-16455. ABFS: Implement FileSystem.access() method. 
Contributed by Bilahari T H.
 2019-11-27 15:56:38 +00:00
prabhujoseph
7f2ea2ac46 MAPREDUCE-7240. Fix Invalid event: TA_TOO_MANY_FETCH_FAILURE at SUCCESS_FINISHING_CONTAINER. 
Contributed by Huachao and Peter Bacsko. Reviewed by Wilfred Spiegelenburg.
 2019-11-27 14:27:27 +05:30
Yiqun Lin
c8bef4d6a6 HDFS-14649. Add suspect probe for DeadNodeDetector. Contributed by Lisheng Sun. 2019-11-27 10:57:20 +08:00
Szilard Nemeth
ef950b0863 YARN-9290. Invalid SchedulingRequest not rejected in Scheduler PlacementConstraintsHandler. Contributed by Prabhu Joseph 2019-11-26 22:04:07 +01:00
Szilard Nemeth
828ab400ee YARN-9362. Code cleanup in TestNMLeveldbStateStoreService. Contributed by Denes Gerencser 2019-11-26 21:45:12 +01:00
prabhujoseph
aa7ab2719f YARN-9991. Fix Application Tag prefix to userid. Contributed by Szilard Nemeth. 2019-11-27 01:59:12 +05:30
Szilard Nemeth
8c9018d5c7 YARN-9899. Migration tool that help to generate CS config based on FS config [Phase 2]. Contributed by Peter Bacsko 2019-11-26 21:22:35 +01:00
Sahil Takiar
3161813482 HADOOP-16685: FileSystem#listStatusIterator does not check if given path exists (#1695) 2019-11-26 11:11:26 -08:00
Szilard Nemeth
52e9ee39a1 YARN-9444. YARN API ResourceUtils's getRequestedResourcesFromConfig doesn't recognize yarn.io/gpu as a valid resource. Contributed by Gergely Pollak 2019-11-26 16:42:59 +01:00
Gabor Bota
ea25f4de23
HADOOP-16709. S3Guard: Make authoritative mode exclusive for metadata - don't check for expiry for authoritative paths (#1721). Contributed by Gabor Bota. 2019-11-26 16:36:19 +01:00
Szilard Nemeth
448ffb12ec YARN-9937. addendum: Add missing queue configs in RMWebService#CapacitySchedulerQueueInfo. Contributed by Prabhu Joseph 2019-11-26 13:41:41 +01:00
Eric Badger
6c20512b68 Revert "YARN-9561. Add C changes for the new RuncContainerRuntime. Contributed by Eric Badger" 
This reverts commit 289bbca870.
 2019-11-25 17:46:42 +00:00
HUAN-PING SU
f8e36e03b4 YARN-9966. Code duplication in UserGroupMappingPlacementRule (#1709) 2019-11-25 15:28:53 +09:00
Akira Ajisaka
da9435182a
HADOOP-16717. Remove GenericsUtil isLog4jLogger dependency on Log4jLoggerAdapter. Contributed by Xieming Li. 2019-11-25 14:36:27 +09:00
Ayush Saxena
049940e77b HDFS-14519. NameQuota is not update after concat operation, so namequota is wrong. Contributed by Ranith Sardar. 2019-11-23 02:30:02 +05:30
Szilard Nemeth
4c1a1287bc YARN-9968. Public Localizer is exiting in NodeManager due to NullPointerException. Contributed by Tarun Parimi 2019-11-22 12:58:10 +01:00
Tao Yang
8afabe41e4 YARN-9838. Fix resource inconsistency for queues when moving app with reserved container to another queue. Contributed by jiulongzhu. 2019-11-22 16:06:14 +08:00
Ayush Saxena
c422e36397 HDFS-13842. RBF: Exceptions are conflicting when creating the same mount entry twice. Contributed by Ranith Sardar. 2019-11-22 11:42:19 +05:30
Takanobu Asanuma
b25e94ce29 HDFS-14924. RenameSnapshot not updating new modification time. Contributed by hemanthboyina 2019-11-22 13:48:02 +09:00
Yiqun Lin
9b6906fe91 HDFS-14651. DeadNodeDetector checks dead node periodically. Contributed by Lisheng Sun. 2019-11-22 10:53:55 +08:00
Ayush Saxena
b89fd4dfe9 HDFS-15002. RBF: Fix annotation in RouterAdmin. Contributed by Jinglun. 2019-11-22 01:08:11 +05:30
Surendra Singh Lilhore
26270196a2 HDFS-14940. HDFS Balancer : Do not allow to set balancer maximum network bandwidth more than 1TB. Contributed by hemanthboyina. 2019-11-22 00:31:25 +05:30
Ayush Saxena
98d249dcda HDFS-14996. RBF: GetFileStatus fails for directory with EC policy set in case of multiple destinations. Contributed by Ayush Saxena. 2019-11-21 12:08:42 +05:30
Sneha Vijayarajan
de38045021 HADOOP-16687. ABFS: Fix testcase added for HADOOP-16138 for namespace enabled account. (#1701) 2019-11-21 11:24:12 +09:00
Siyao Meng
1a0c0e42e7 HADOOP-16276. Fix jsvc startup command in hadoop-functions.sh due to jsvc >= 1.0.11 changed default current working directory (#1272) 2019-11-20 16:44:44 -08:00
David Mollitor
6f899e9030 HADOOP-15852. Refactor QuotaUsage. Contributed by David Mollitor. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-11-20 14:40:10 -08:00
Wei-Chiu Chuang
9664b9c7a6 HADOOP-15686. Supress bogus AbstractWadlGeneratorGrammarGenerator in KMS stderr. Contributed by Wei-Chiu Chuang. 
Reviewed-by: Xiaoyu Yao <xyao@apache.org>
 2019-11-20 13:07:43 -08:00
Inigo Goiri
3037762b2c HDFS-14949. Add getServerDefaults() support to HttpFS. Contributed by hemanthboyina. 2019-11-20 10:51:58 -08:00
Surendra Singh Lilhore
fd264b8265 HDFS-14995. Use log variable directly instead of passing as argument in InvalidateBlocks#printBlockDeletionTime(). Contributed by Lisheng Sun. 2019-11-21 00:00:42 +05:30
Erik Krogen
be77231452 HADOOP-16700. Track response time separately in Server.Call to avoid negative RpcQueueTime values. Contributed by xuzq. 2019-11-20 07:56:33 -08:00
Steve Loughran
1415ad3800
HADOOP-16707. NPE in UGI.getCurrentUser in ITestAbfsIdentityTransformer setup. 
Contributed by Steve Loughran.

Change-Id: I38fdba2fa70e534d78b15e61de19368912588b0c
 2019-11-20 14:32:12 +00:00
Wanqiang Ji
dfc61d8ea5 HADOOP-16719. Remove the disallowed element config within maven-checkstyle-plugin (#1724) 2019-11-20 16:18:56 +09:00
He Xiaoqiao
0b50aa29fd HDFS-14952. Skip safemode if blockTotal is 0 in new NN. Contributed by Xiaoqiao He. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Mukul Kumar Singh <msingh@apache.org>
 2019-11-19 12:16:57 -08:00
Szilard Nemeth
dfdc6d6dd9 YARN-9886. Queue mapping based on userid passed through application tag. Contributed by Julia Kinga Marton 2019-11-19 21:14:15 +01:00
Eric Yang
ea8ffac121 YARN-9983. Fixed typo in YARN Service overview. 
Contributed by Denes Gerencser
 2019-11-19 14:16:51 -05:00
Surendra Singh Lilhore
c870514740 HDFS-14992. TestOfflineEditsViewer is failing in Trunk. Contributed by hemanthboyina. 2019-11-20 00:07:43 +05:30
Jeetesh Mangwani
b033c681e4
HADOOP-16612. Track Azure Blob File System client-perceived latency 
Contributed by Jeetesh Mangwani.

This add the ability to track the end-to-end performance of ADLS Gen 2 REST APIs by measuring latency in the Hadoop ABFS driver.
The latency information is sent back to the ADLS Gen 2 REST API endpoints in the subsequent requests.
 2019-11-19 09:00:24 -08:00
Steve Loughran
9fbfe6c8f9
HADOOP-16632 Speculating & Partitioned S3A magic committers can leave pending files under __magic (#1599) 
Contributed by Steve Loughran.

This downgrade the checks for leftover __magic entries from fail to warn now the parallel
test runs make speculation more likely. 

Change-Id: Ia4df2e90f82a06dbae69f3fdaadcbb0e0d713b38
 2019-11-19 13:54:33 +00:00
Prabhu Joseph
30b2096933 YARN-9986. Allow app owner to signal container via RMWebServices. Contributed by kyungwan nam. 2019-11-19 11:35:00 +00:00
Duo Zhang
3cecb2a469 HADOOP-16110 Upgrade to yetus 0.11.1 and use emoji vote on github pre commit (#1527). Contributed by Duo Zhang. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2019-11-19 14:21:49 +05:30
Sunil G
ea68756c0c YARN-8373. RM Received RMFatalEvent of type CRITICAL_THREAD_CRASH. Contributed by Wilfred Spiegelenburg. 2019-11-19 14:10:41 +05:30
Sunil G
215f2052fc YARN-9984. FSPreemptionThread can cause NullPointerException while app is unregistered with containers running on a node. Contributed by Wilfred Spiegelenburg. 2019-11-19 14:03:02 +05:30
Abhishek Modi
dc3f4fc2f4 YARN-9965. Fix NodeManager failing to start on subsequent times when Hdfs Auxillary Jar is set (addendum). Contributed by Prabhu Joseph. 2019-11-19 08:08:40 +05:30
Abhishek Modi
4050471b45 HADOOP-16657. Move remaining log4j APIs over to slf4j in hadoop-common. Contributed by Minni Mittal. 2019-11-19 08:01:17 +05:30
Eric Badger
0e22e9ab83 YARN-9562. Add Java changes for the new RuncContainerRuntime. Contributed by Eric Badger 2019-11-18 21:56:31 +00:00
Eric Badger
289bbca870 YARN-9561. Add C changes for the new RuncContainerRuntime. Contributed by Eric Badger 2019-11-18 21:54:16 +00:00