big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
22,903 Commits 2 Branches 2 Tags 264 MiB
56248f9d87
Commit Graph

573 Commits

Author SHA1 Message Date
Inigo Goiri
99948565cb YARN-8412. Move ResourceRequest.clone logic everywhere into a proper API. Contributed by Botong Huang. 2018-06-21 18:24:10 -07:00
Miklos Szegedi
bbbc7cc426 YARN-7449. Split up class TestYarnClient to TestYarnClient and TestYarnClientImpl. Contributed by Szilard Nemeth. 2018-06-20 11:40:56 -07:00
Akira Ajisaka
652bcbb3e4
YARN-8363. Upgrade commons-lang version to 3.7 in hadoop-yarn-project. Contributed by Takanobu Asanuma. 2018-06-13 00:35:37 +09:00
Wangda Tan
ff583d3fa3 YARN-8349. Remove YARN registry entries when a service is killed by the RM. (Billie Rinaldi via wangda) 
Change-Id: Ia58db3637789a8921482f564aa9bdf99c45cc36c
 2018-06-01 14:49:18 -07:00
Eric Yang
7ff5a40218 YARN-8316. Improved diagnostic message for ATS unavailability for YARN Service. 
Contributed by Billie Rinaldi
 2018-05-24 16:26:02 -04:00
Inigo Goiri
e30938af12 YARN-8336. Fix potential connection leak in SchedConfCLI and YarnWebServiceUtils. Contributed by Giovanni Matteo Fumarola. 2018-05-23 11:55:31 -07:00
Arun Suresh
3159bffce2 YARN-7900. [AMRMProxy] AMRMClientRelayer for stateful FederationInterceptor. (Botong Huang via asuresh) 2018-05-17 20:00:52 -07:00
Eric Yang
8d3b39de89 YARN-8081. Add support to upgrade a component. 
Contributed by Chandni Singh
 2018-05-15 20:40:39 -04:00
Wangda Tan
b1833d9ba2 YARN-8210. AMRMClient logging on every heartbeat to track updation of AM RM token causes too many log lines to be generated in AM logs. (Suma Shivaprasad via wangda) 
Change-Id: I70edd6e301fd5e78d479e1882aedc9332a0827aa
 2018-04-27 13:07:38 -07:00
Eric Yang
4a7369b095 YARN-7939. Added support to upgrade a component instance. 
Contributed by Chandni Singh
 2018-04-26 13:54:40 -07:00
Haibo Chen
bb92bfb4ef YARN-7088. Add application launch time to Resource Manager REST API. (Kanwaljeet Sachdev via Haibo Chen) 2018-04-17 11:15:20 -07:00
Haibo Chen
86dfb70875 Revert "YARN-7088. Add application launch time to Resource Manager REST API. (Kanwaljeet Sachdev via Haibo Chen)" 
This reverts commit 0c14b6f0d8.
 2018-04-17 06:04:38 -07:00
Haibo Chen
0c14b6f0d8 YARN-7088. Add application launch time to Resource Manager REST API. (Kanwaljeet Sachdev via Haibo Chen) 2018-04-17 05:59:03 -07:00
Subru Krishnan
09999d7e01 YARN-8010. Add config in FederationRMFailoverProxy to not bypass facade cache when failing over. (Botong Huang via Subru). 2018-03-28 11:33:19 -07:00
Eric Yang
27d60a1634 YARN-8018. Added support for initiating yarn service upgrade. 
Contributed by Chandni Singh
 2018-03-26 18:46:31 -04:00
Jason Lowe
9497215084 YARN-7813. Capacity Scheduler Intra-queue Preemption should be configurable for each queue. Contributed by Eric Payne 2018-02-19 14:06:28 -06:00
Sunil G
aae629913c YARN-7292. Retrospect Resource Profile Behavior for overriding capability. Contributed by Wangda Tan. 2018-02-16 12:37:40 +05:30
Konstantinos Karanasos
0b489e564c YARN-7920. Simplify configuration for PlacementConstraints. Contributed by Wangda Tan. 2018-02-15 14:23:38 -08:00
Eric Payne
bddfe42e2c Revert "YARN-7813: Capacity Scheduler Intra-queue Preemption should be configurable for each queue" 
This reverts commit c5e6e3de1c.
 2018-02-14 14:29:20 -06:00
Eric Payne
c5e6e3de1c YARN-7813: Capacity Scheduler Intra-queue Preemption should be configurable for each queue 2018-02-13 10:11:02 -06:00
Wangda Tan
60f9e60b3b Preparing for 3.2.0 development 
Change-Id: I6d0e01f3d665d26573ef2b957add1cf0cddf7938
 2018-02-11 11:17:38 +08:00
Arun Suresh
38af237969 YARN-6599. Support anti-affinity constraint via AppPlacementAllocator. (Wangda Tan via asuresh) 2018-01-31 01:30:17 -08:00
Wangda Tan
29d9e4d581 YARN-6619. AMRMClient Changes to use the PlacementConstraint and SchcedulingRequest objects. (Arun Suresh via wangda) 2018-01-31 01:30:17 -08:00
Billie Rinaldi
e307edcb47 YARN-7540 and YARN-7605. Convert yarn app cli to call yarn api services and implement doAs for Api Service REST API. Contributed by Eric Yang and Jian He 2018-01-23 17:54:39 -08:00
Jian He
a72cdcc47a YARN-7766. Introduce a new config property for YARN Service dependency tarball location. Contributed by Gour Saha 2018-01-23 10:53:27 -08:00
Rohith Sharma K S
d09058b2fd YARN-6736. Consider writing to both ats v1 & v2 from RM for smoother upgrades. Contributed by Aaron Gresch. 2018-01-16 07:58:29 +05:30
Billie Rinaldi
53f2768926 YARN-7724. yarn application status should support application name. Contributed by Jian He 2018-01-12 15:33:46 -08:00
Jian He
836e3c45e8 Revert "YARN-7540. Route YARN service CLI function through YARN Service API. (Contributed by Eric Yang)" 
This reverts commit 438c1d333e.
 2018-01-05 18:25:00 -08:00
Weiwei Yang
89b6c482c1 YARN-7642. Add test case to verify context update after container promotion or demotion with or without auto update. Contributed by Weiwei Yang. 2017-12-15 11:58:52 +08:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
95d4ec7fc0 YARN-7565. Addendum to fix an incompatible change. Contributed by Wangda Tan. 2017-12-14 17:10:37 -08:00
Sunil G
2564b4d07f YARN-7119. Support multiple resource types in rmadmin updateNodeResource command. Contributed by Manikandan R. 2017-12-14 16:11:57 +05:30
Eric Yang
438c1d333e YARN-7540. Route YARN service CLI function through YARN Service API. (Contributed by Eric Yang) 2017-12-13 19:29:04 -05:00
Jian He
3ebe6a7819 YARN-7565. Yarn service pre-maturely releases the container after AM restart. Contributed by Chandni Singh 2017-12-12 13:35:56 -08:00
Junping Du
a409425986 YARN-7558. yarn logs command fails to get logs for running containers if UI authentication is enabled. Contributed by Xuan Gong. 2017-11-30 13:47:47 -08:00
Arun Suresh
b46ca7e73b YARN-6483. Add nodes transitioning to DECOMMISSIONING state to the list of updated nodes returned to the AM. (Juan Rodriguez Hortala via asuresh) 2017-11-22 19:18:30 -08:00
Haibo Chen
28d0fcbef4 YARN-7390. All reservation related test cases failed when TestYarnClient runs against Fair Scheduler. (Yufei Gu via Haibo Chen) 2017-11-16 10:48:24 -08:00
Jian He
ba7ed7b66a YARN-7384. Remove apiserver cmd and merge service cmd into application cmd. Contributed by Billie Rinaldi 2017-11-06 13:30:19 -08:00
Akira Ajisaka
c02d2ba50d
YARN-7379. Moving logging APIs over to slf4j in hadoop-yarn-client. Contributed by Yeliang Cang. 2017-10-31 17:09:45 +09:00
Wangda Tan
36e158ae98 YARN-7307. Allow client/AM update supported resource types via YARN APIs. (Sunil G via wangda) 
Change-Id: I14c5ea7252b7c17e86ab38f692b5f9d43196dbe0
 2017-10-26 20:15:19 -07:00
Haibo Chen
3fb4718886 YARN-7310. TestAMRMProxy#testAMRMProxyE2E fails with FairScheduler. (Robert Kanter via Haibo Chen) 2017-10-13 15:49:21 -07:00
Wangda Tan
5cfbad68c0 YARN-7237. Cleanup usages of ResourceProfiles. (Wangda Tan) 
Change-Id: I959c8531bd48231956068cb7d2312f3235549f9c
(cherry picked from commit 8536db5a7549fa44d9f9e444b41bd5978b5c6ae1)
 2017-10-09 20:36:57 -07:00
Jonathan Hung
09c5dfe937 YARN-7251. Misc changes to YARN-5734 2017-10-09 11:11:23 -07:00
Xuan
916bdbd6be YARN-5953:Create CLI for changing YARN configurations. (Jonathan Hung via xgong) 2017-10-09 11:11:23 -07:00
Haibo Chen
c071aad5da YARN-7009. TestNMClient.testNMClientNoCleanupOnStop is flaky by design. (Miklos Szegedi via Haibo Chen) 2017-10-05 15:50:13 -07:00
Arun Suresh
b733348dde YARN-7258. Add Node and Rack Hints to Opportunistic Scheduler. (Kartheek Muthyala via asuresh). 2017-10-05 09:58:04 -07:00
Akira Ajisaka
c9b525bd94
YARN-6943. Update Yarn to YARN in documentation. Contributed by Chetna Chaudhari. 2017-10-04 07:01:54 +09:00
Wangda Tan
ec2ae3060a YARN-6509. Add a size threshold beyond which yarn logs will require a force option. (Xuan Gong via wangda) 
Change-Id: I755fe903337d4ff9ec35dae5b9cce638794e1d0f
 2017-09-29 16:42:56 -07:00
Chris Trezzo
c114da5e64 YARN-7250. Update Shared cache client api to use URLs. 2017-09-28 15:28:06 -07:00
Akira Ajisaka
e928ee583c
YARN-7153. Remove duplicated code in AMRMClientAsyncImpl.java. Contributed by Sen Zhao. 2017-09-26 02:03:30 +09:00
Arun Suresh
56ef5279c1 YARN-7203. Add container ExecutionType into ContainerReport. (Botong Huang via asuresh) 2017-09-18 15:49:31 -07:00
Arun Suresh
29dd55153e YARN-7199. Fix TestAMRMClientContainerRequest.testOpportunisticAndGuaranteedRequests. (Botong Huang via asuresh) 2017-09-18 11:26:44 -07:00
Junping Du
1a84c24b0c YARN-7174. Add retry logic in LogsCLI when fetch running application logs. Contributed by Xuan Gong. 2017-09-15 15:33:24 -07:00
Robert Kanter
bb34ae9554 YARN-7146. Many RM unit tests failing with FairScheduler (rkanter) 2017-09-13 15:44:54 -07:00
Wangda Tan
e490602e9b YARN-7039. Fix javac and javadoc errors in YARN-3926 branch. (Sunil G via wangda) 
Change-Id: I442bf6d838b3aba83f1f6779cf9dcf8596a2102d
 2017-09-12 09:19:12 -07:00
Wangda Tan
39240b61a1 YARN-7043. Cleanup ResourceProfileManager. (wangda) 
Change-Id: I463356f37bf1f6a3f1fc3c594c79916e8c0ab913
 2017-09-12 09:19:12 -07:00
Wangda Tan
df3855541a YARN-6789. Add Client API to get all supported resource types from RM. (Sunil G via wangda) 
Change-Id: I366d8db6f6700acd087db5acb7a1be7e41b2b68d
 2017-09-12 09:19:12 -07:00
Daniel Templeton
3aeaafecb8 YARN-6788. [YARN-3926] Improve performance of resource profile branch 
(Contributed by Sunil Govindan via Daniel Templeton)
 2017-09-12 09:19:11 -07:00
Sunil G
dae65f3bef YARN-6232. Update resource usage and preempted resource calculations to take into account all resource types. Contributed by Varun Vasudev. 2017-09-12 09:19:11 -07:00
Sunil G
7805deed48 YARN-5588. [YARN-3926] Add support for resource profiles in distributed shell. Contributed by Varun Vasudev. 2017-09-12 09:19:11 -07:00
Arun Suresh
6708ac3301 YARN-5587. Add support for resource profiles. (vvasudev via asuresh) 2017-09-12 09:19:10 -07:00
Varun Vasudev
c2032e251e YARN-5708. Implement APIs to get resource profiles from the RM. Contributed by Varun Vasudev. 2017-09-12 09:19:10 -07:00
Junping Du
3fddabc2fe YARN-7072. Add a new log aggregation file format controller. Contributed by Xuan Gong. 2017-09-08 15:16:19 -07:00
Haibo Chen
8edc60531f HADOOP-14847. Remove Guava Supplier and change to java Supplier in AMRMClient and AMRMClientAysnc. (Bharat Viswanadham via Haibo Chen) 2017-09-08 14:33:11 -07:00
Sunil G
56d93d2e39 YARN-6600. Introduce default and max lifetime of application at LeafQueue level. Contributed by Rohith Sharma K S. 2017-09-08 07:15:17 +05:30
Arun Suresh
c41118a7f8 YARN-6978. Add updateContainer API to NMClient. (Kartheek Muthyala via asuresh) 2017-09-07 10:23:12 -07:00
Jason Lowe
704267cb49 YARN-7164. TestAMRMClientOnRMRestart fails sporadically with bind address in use. Contributed by Jason Lowe 2017-09-06 15:23:49 -05:00
Junping Du
63231a2a30 YARN-7148. TestLogsCLI fails in trunk and branch-2 and javadoc error. Contributed by Xuan Gong. 2017-09-06 13:19:34 -07:00
Varun Saxena
ef87d34a0e YARN-7152. [ATSv2] Registering timeline client before AMRMClient service init throw exception (Rohith Sharma K S via Varun Saxena) 2017-09-04 20:11:02 +05:30
Andrew Wang
0d419c984f Preparing for 3.1.0 development 2017-09-01 11:53:48 -07:00
Junping Du
91cc070d67 YARN-6877. Create an abstract log reader for extendability. Contributed by Xuan Gong. 2017-08-31 16:41:43 -07:00
Robert Kanter
ce79f7be29 YARN-7094. Document the current known issue with server-side NM graceful decom (rkanter) 2017-08-30 18:04:55 -07:00
Jian He
08f40bcc7f MAPREDUCE-6838. [ATSv2 Security] Add timeline delegation token received in allocate response to UGI. Contributed by Varun Saxena 2017-08-30 11:29:54 +05:30
Rohith Sharma K S
7594d1de7b YARN-6130. [ATSv2 Security] Generate a delegation token for AM when app collector is created and pass it to AM via NM and RM. Contributed by Varun Saxena. 2017-08-30 11:29:53 +05:30
Jian He
8201ed8009 YARN-6756. ContainerRequest#executionTypeRequest causes NPE. Contributed by Jian He 2017-08-29 20:35:53 -07:00
Wangda Tan
51881a863c YARN-7112. TestAMRMProxy is failing with invalid request. (Jason Lowe via wangda) 
Change-Id: I432f4523921666ac218fcf9936f826a0d8d635e0
 2017-08-28 11:37:35 -07:00
Junping Du
c2cb7ea1ef YARN-6876. Create an abstract log writer for extendability. Contributed by Xuan Gong. 2017-08-24 13:36:49 -07:00
Jason Lowe
14553061be YARN-7020. TestAMRMProxy#testAMRMProxyTokenRenewal is flakey. Contributed by Robert Kanter 2017-08-16 13:04:36 -05:00
Arun Suresh
4d7be1d857 YARN-5978. ContainerScheduler and ContainerManager changes to support ExecType update. (Kartheek Muthyala via asuresh) 2017-08-14 19:46:17 -07:00
Arun Suresh
8d3fd81980 YARN-6920. Fix resource leak that happens during container re-initialization. (asuresh) 2017-08-07 18:59:25 -07:00
Arun Suresh
35dc782923 YARN-5977. ContainerManagementProtocol changes to support change of container ExecutionType. (Kartheek Muthyala via asuresh) 2017-08-03 21:15:40 -07:00
Yufei Gu
c617fe02b3 YARN-6832. Tests use assertTrue(....equals(...)) instead of assertEquals()). (Daniel Templeton via Yufei Gu) 2017-08-03 11:33:37 -07:00
Subru Krishnan
98b45b0ed3 YARN-6093. Minor bugs with AMRMtoken renewal and state store availability when using FederationRMFailoverProxyProvider during RM failover. (Botong Huang via Subru). 
(cherry picked from commit 66500f4fa6155d29435d7c92fd6d68079c4cab86)
 2017-08-01 17:28:22 -07:00
Jian He
bdfad4523f YARN-3673. Create a FailoverProxy for Federation services. Contributed by Subru Krishnan 
(cherry picked from commit 3307564a5f8c8abc5fe84efcd05ee0f7dfdd921c)
 2017-08-01 17:28:20 -07:00
Daniel Templeton
4c40cd451c YARN-6844. AMRMClientImpl.checkNodeLabelExpression() has wrong error message 
(Contributed by Manikandan R via Daniel Templeton)
 2017-07-24 12:03:02 -07:00
Andrew Wang
af2773f609 Updating version for 3.0.0-beta1 development 2017-06-29 17:57:40 -07:00
Daniel Templeton
036a24bab3 MAPREDUCE-6896. Document wrong spelling in usage of MapredTestDriver tools. 
(Contributed by LiXin Ge via Daniel Templeton)
 2017-06-13 13:21:23 -07:00
Brahma Reddy Battula
19ef3a81f8 HADOOP-14485. Redundant 'final' modifier in try-with-resources statement. Contributed by wenxin he. 2017-06-06 23:11:47 +08:00
Andrew Wang
16ad896d5c Update maven version for 3.0.0-alpha4 development 2017-05-26 14:09:44 -07:00
Daniel Templeton
d81372dfad YARN-6646. Modifier 'static' is redundant for inner enums 
(Contributed by ZhangBing Lin via Daniel Templeton)
 2017-05-26 12:05:48 -07:00
Jian He
8236130b2c YARN-6306. NMClient API change for container upgrade. Contributed by Arun Suresh 2017-05-16 10:48:46 -07:00
Jason Lowe
6099deebcb YARN-6552. Increase YARN test timeouts from 1 second to 10 seconds. Contributed by Eric Badger 2017-05-10 08:47:48 -05:00
Karthik Kambatla
166be0ee95 YARN-3742. YARN RM will shut down if ZKClient creation times out. (Daniel Templeton via kasha) 2017-05-09 14:44:16 -07:00
Naganarasimha
64f68cb0b8 YARN-6520. Fix warnings from Spotbugs in hadoop-yarn-client. Contributed by Weiwei Yang. 2017-05-01 18:38:22 +05:30
Eric Payne
fdf5192bbb YARN-6536. TestAMRMClient.testAMRMClientWithSaslEncryption fails intermittently. Contributed by Jason Lowe. 2017-04-28 15:20:35 -05:00
Akira Ajisaka
8a99eba96d
Revert "YARN-5007. Remove deprecated constructors of MiniYARNCluster and MiniMRYarnCluster. Contributed by Andras Bokor." 
This reverts commit 34ab8e73d4.
 2017-04-26 15:34:23 +09:00
Sunil G
9ff25408bc YARN-6164. Expose Queue Configurations per Node Label through YARN client api. Contributed by Benson Qiu. 2017-04-19 20:39:09 +05:30
Jason Lowe
90d97372ed YARN-6461. TestRMAdminCLI has very low test timeouts. Contributed by Eric Badger 2017-04-11 09:16:11 -05:00
Jason Lowe
1b081ca27e YARN-6288. Exceptions during aggregated log writes are mishandled. Contributed by Akira Ajisaka 2017-04-06 16:24:36 -05:00
Akira Ajisaka
34ab8e73d4
YARN-5007. Remove deprecated constructors of MiniYARNCluster and MiniMRYarnCluster. Contributed by Andras Bokor. 2017-04-05 17:33:07 +09:00
Jason Lowe
4d1fac5df2 YARN-6411. Clean up the overwrite of createDispatcher() in subclass of MockRM. Contributed by Yufei Gu 2017-03-31 10:05:34 -05:00