big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
20,886 Commits 2 Branches 2 Tags 264 MiB
dfceffa70d
Commit Graph

532 Commits

Author SHA1 Message Date
Jian He
a72cdcc47a YARN-7766. Introduce a new config property for YARN Service dependency tarball location. Contributed by Gour Saha 2018-01-23 10:53:27 -08:00
Rohith Sharma K S
d09058b2fd YARN-6736. Consider writing to both ats v1 & v2 from RM for smoother upgrades. Contributed by Aaron Gresch. 2018-01-16 07:58:29 +05:30
Billie Rinaldi
53f2768926 YARN-7724. yarn application status should support application name. Contributed by Jian He 2018-01-12 15:33:46 -08:00
Jian He
836e3c45e8 Revert "YARN-7540. Route YARN service CLI function through YARN Service API. (Contributed by Eric Yang)" 
This reverts commit 438c1d333e.
 2018-01-05 18:25:00 -08:00
Weiwei Yang
89b6c482c1 YARN-7642. Add test case to verify context update after container promotion or demotion with or without auto update. Contributed by Weiwei Yang. 2017-12-15 11:58:52 +08:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
95d4ec7fc0 YARN-7565. Addendum to fix an incompatible change. Contributed by Wangda Tan. 2017-12-14 17:10:37 -08:00
Sunil G
2564b4d07f YARN-7119. Support multiple resource types in rmadmin updateNodeResource command. Contributed by Manikandan R. 2017-12-14 16:11:57 +05:30
Eric Yang
438c1d333e YARN-7540. Route YARN service CLI function through YARN Service API. (Contributed by Eric Yang) 2017-12-13 19:29:04 -05:00
Jian He
3ebe6a7819 YARN-7565. Yarn service pre-maturely releases the container after AM restart. Contributed by Chandni Singh 2017-12-12 13:35:56 -08:00
Junping Du
a409425986 YARN-7558. yarn logs command fails to get logs for running containers if UI authentication is enabled. Contributed by Xuan Gong. 2017-11-30 13:47:47 -08:00
Arun Suresh
b46ca7e73b YARN-6483. Add nodes transitioning to DECOMMISSIONING state to the list of updated nodes returned to the AM. (Juan Rodriguez Hortala via asuresh) 2017-11-22 19:18:30 -08:00
Haibo Chen
28d0fcbef4 YARN-7390. All reservation related test cases failed when TestYarnClient runs against Fair Scheduler. (Yufei Gu via Haibo Chen) 2017-11-16 10:48:24 -08:00
Jian He
ba7ed7b66a YARN-7384. Remove apiserver cmd and merge service cmd into application cmd. Contributed by Billie Rinaldi 2017-11-06 13:30:19 -08:00
Akira Ajisaka
c02d2ba50d
YARN-7379. Moving logging APIs over to slf4j in hadoop-yarn-client. Contributed by Yeliang Cang. 2017-10-31 17:09:45 +09:00
Wangda Tan
36e158ae98 YARN-7307. Allow client/AM update supported resource types via YARN APIs. (Sunil G via wangda) 
Change-Id: I14c5ea7252b7c17e86ab38f692b5f9d43196dbe0
 2017-10-26 20:15:19 -07:00
Haibo Chen
3fb4718886 YARN-7310. TestAMRMProxy#testAMRMProxyE2E fails with FairScheduler. (Robert Kanter via Haibo Chen) 2017-10-13 15:49:21 -07:00
Wangda Tan
5cfbad68c0 YARN-7237. Cleanup usages of ResourceProfiles. (Wangda Tan) 
Change-Id: I959c8531bd48231956068cb7d2312f3235549f9c
(cherry picked from commit 8536db5a7549fa44d9f9e444b41bd5978b5c6ae1)
 2017-10-09 20:36:57 -07:00
Jonathan Hung
09c5dfe937 YARN-7251. Misc changes to YARN-5734 2017-10-09 11:11:23 -07:00
Xuan
916bdbd6be YARN-5953:Create CLI for changing YARN configurations. (Jonathan Hung via xgong) 2017-10-09 11:11:23 -07:00
Haibo Chen
c071aad5da YARN-7009. TestNMClient.testNMClientNoCleanupOnStop is flaky by design. (Miklos Szegedi via Haibo Chen) 2017-10-05 15:50:13 -07:00
Arun Suresh
b733348dde YARN-7258. Add Node and Rack Hints to Opportunistic Scheduler. (Kartheek Muthyala via asuresh). 2017-10-05 09:58:04 -07:00
Akira Ajisaka
c9b525bd94
YARN-6943. Update Yarn to YARN in documentation. Contributed by Chetna Chaudhari. 2017-10-04 07:01:54 +09:00
Wangda Tan
ec2ae3060a YARN-6509. Add a size threshold beyond which yarn logs will require a force option. (Xuan Gong via wangda) 
Change-Id: I755fe903337d4ff9ec35dae5b9cce638794e1d0f
 2017-09-29 16:42:56 -07:00
Chris Trezzo
c114da5e64 YARN-7250. Update Shared cache client api to use URLs. 2017-09-28 15:28:06 -07:00
Akira Ajisaka
e928ee583c
YARN-7153. Remove duplicated code in AMRMClientAsyncImpl.java. Contributed by Sen Zhao. 2017-09-26 02:03:30 +09:00
Arun Suresh
56ef5279c1 YARN-7203. Add container ExecutionType into ContainerReport. (Botong Huang via asuresh) 2017-09-18 15:49:31 -07:00
Arun Suresh
29dd55153e YARN-7199. Fix TestAMRMClientContainerRequest.testOpportunisticAndGuaranteedRequests. (Botong Huang via asuresh) 2017-09-18 11:26:44 -07:00
Junping Du
1a84c24b0c YARN-7174. Add retry logic in LogsCLI when fetch running application logs. Contributed by Xuan Gong. 2017-09-15 15:33:24 -07:00
Robert Kanter
bb34ae9554 YARN-7146. Many RM unit tests failing with FairScheduler (rkanter) 2017-09-13 15:44:54 -07:00
Wangda Tan
e490602e9b YARN-7039. Fix javac and javadoc errors in YARN-3926 branch. (Sunil G via wangda) 
Change-Id: I442bf6d838b3aba83f1f6779cf9dcf8596a2102d
 2017-09-12 09:19:12 -07:00
Wangda Tan
39240b61a1 YARN-7043. Cleanup ResourceProfileManager. (wangda) 
Change-Id: I463356f37bf1f6a3f1fc3c594c79916e8c0ab913
 2017-09-12 09:19:12 -07:00
Wangda Tan
df3855541a YARN-6789. Add Client API to get all supported resource types from RM. (Sunil G via wangda) 
Change-Id: I366d8db6f6700acd087db5acb7a1be7e41b2b68d
 2017-09-12 09:19:12 -07:00
Daniel Templeton
3aeaafecb8 YARN-6788. [YARN-3926] Improve performance of resource profile branch 
(Contributed by Sunil Govindan via Daniel Templeton)
 2017-09-12 09:19:11 -07:00
Sunil G
dae65f3bef YARN-6232. Update resource usage and preempted resource calculations to take into account all resource types. Contributed by Varun Vasudev. 2017-09-12 09:19:11 -07:00
Sunil G
7805deed48 YARN-5588. [YARN-3926] Add support for resource profiles in distributed shell. Contributed by Varun Vasudev. 2017-09-12 09:19:11 -07:00
Arun Suresh
6708ac3301 YARN-5587. Add support for resource profiles. (vvasudev via asuresh) 2017-09-12 09:19:10 -07:00
Varun Vasudev
c2032e251e YARN-5708. Implement APIs to get resource profiles from the RM. Contributed by Varun Vasudev. 2017-09-12 09:19:10 -07:00
Junping Du
3fddabc2fe YARN-7072. Add a new log aggregation file format controller. Contributed by Xuan Gong. 2017-09-08 15:16:19 -07:00
Haibo Chen
8edc60531f HADOOP-14847. Remove Guava Supplier and change to java Supplier in AMRMClient and AMRMClientAysnc. (Bharat Viswanadham via Haibo Chen) 2017-09-08 14:33:11 -07:00
Sunil G
56d93d2e39 YARN-6600. Introduce default and max lifetime of application at LeafQueue level. Contributed by Rohith Sharma K S. 2017-09-08 07:15:17 +05:30
Arun Suresh
c41118a7f8 YARN-6978. Add updateContainer API to NMClient. (Kartheek Muthyala via asuresh) 2017-09-07 10:23:12 -07:00
Jason Lowe
704267cb49 YARN-7164. TestAMRMClientOnRMRestart fails sporadically with bind address in use. Contributed by Jason Lowe 2017-09-06 15:23:49 -05:00
Junping Du
63231a2a30 YARN-7148. TestLogsCLI fails in trunk and branch-2 and javadoc error. Contributed by Xuan Gong. 2017-09-06 13:19:34 -07:00
Varun Saxena
ef87d34a0e YARN-7152. [ATSv2] Registering timeline client before AMRMClient service init throw exception (Rohith Sharma K S via Varun Saxena) 2017-09-04 20:11:02 +05:30
Junping Du
91cc070d67 YARN-6877. Create an abstract log reader for extendability. Contributed by Xuan Gong. 2017-08-31 16:41:43 -07:00
Robert Kanter
ce79f7be29 YARN-7094. Document the current known issue with server-side NM graceful decom (rkanter) 2017-08-30 18:04:55 -07:00
Jian He
08f40bcc7f MAPREDUCE-6838. [ATSv2 Security] Add timeline delegation token received in allocate response to UGI. Contributed by Varun Saxena 2017-08-30 11:29:54 +05:30
Rohith Sharma K S
7594d1de7b YARN-6130. [ATSv2 Security] Generate a delegation token for AM when app collector is created and pass it to AM via NM and RM. Contributed by Varun Saxena. 2017-08-30 11:29:53 +05:30
Jian He
8201ed8009 YARN-6756. ContainerRequest#executionTypeRequest causes NPE. Contributed by Jian He 2017-08-29 20:35:53 -07:00
Wangda Tan
51881a863c YARN-7112. TestAMRMProxy is failing with invalid request. (Jason Lowe via wangda) 
Change-Id: I432f4523921666ac218fcf9936f826a0d8d635e0
 2017-08-28 11:37:35 -07:00
Junping Du
c2cb7ea1ef YARN-6876. Create an abstract log writer for extendability. Contributed by Xuan Gong. 2017-08-24 13:36:49 -07:00
Jason Lowe
14553061be YARN-7020. TestAMRMProxy#testAMRMProxyTokenRenewal is flakey. Contributed by Robert Kanter 2017-08-16 13:04:36 -05:00
Arun Suresh
4d7be1d857 YARN-5978. ContainerScheduler and ContainerManager changes to support ExecType update. (Kartheek Muthyala via asuresh) 2017-08-14 19:46:17 -07:00
Arun Suresh
8d3fd81980 YARN-6920. Fix resource leak that happens during container re-initialization. (asuresh) 2017-08-07 18:59:25 -07:00
Arun Suresh
35dc782923 YARN-5977. ContainerManagementProtocol changes to support change of container ExecutionType. (Kartheek Muthyala via asuresh) 2017-08-03 21:15:40 -07:00
Yufei Gu
c617fe02b3 YARN-6832. Tests use assertTrue(....equals(...)) instead of assertEquals()). (Daniel Templeton via Yufei Gu) 2017-08-03 11:33:37 -07:00
Subru Krishnan
98b45b0ed3 YARN-6093. Minor bugs with AMRMtoken renewal and state store availability when using FederationRMFailoverProxyProvider during RM failover. (Botong Huang via Subru). 
(cherry picked from commit 66500f4fa6155d29435d7c92fd6d68079c4cab86)
 2017-08-01 17:28:22 -07:00
Jian He
bdfad4523f YARN-3673. Create a FailoverProxy for Federation services. Contributed by Subru Krishnan 
(cherry picked from commit 3307564a5f8c8abc5fe84efcd05ee0f7dfdd921c)
 2017-08-01 17:28:20 -07:00
Daniel Templeton
4c40cd451c YARN-6844. AMRMClientImpl.checkNodeLabelExpression() has wrong error message 
(Contributed by Manikandan R via Daniel Templeton)
 2017-07-24 12:03:02 -07:00
Daniel Templeton
036a24bab3 MAPREDUCE-6896. Document wrong spelling in usage of MapredTestDriver tools. 
(Contributed by LiXin Ge via Daniel Templeton)
 2017-06-13 13:21:23 -07:00
Brahma Reddy Battula
19ef3a81f8 HADOOP-14485. Redundant 'final' modifier in try-with-resources statement. Contributed by wenxin he. 2017-06-06 23:11:47 +08:00
Daniel Templeton
d81372dfad YARN-6646. Modifier 'static' is redundant for inner enums 
(Contributed by ZhangBing Lin via Daniel Templeton)
 2017-05-26 12:05:48 -07:00
Jian He
8236130b2c YARN-6306. NMClient API change for container upgrade. Contributed by Arun Suresh 2017-05-16 10:48:46 -07:00
Jason Lowe
6099deebcb YARN-6552. Increase YARN test timeouts from 1 second to 10 seconds. Contributed by Eric Badger 2017-05-10 08:47:48 -05:00
Karthik Kambatla
166be0ee95 YARN-3742. YARN RM will shut down if ZKClient creation times out. (Daniel Templeton via kasha) 2017-05-09 14:44:16 -07:00
Naganarasimha
64f68cb0b8 YARN-6520. Fix warnings from Spotbugs in hadoop-yarn-client. Contributed by Weiwei Yang. 2017-05-01 18:38:22 +05:30
Eric Payne
fdf5192bbb YARN-6536. TestAMRMClient.testAMRMClientWithSaslEncryption fails intermittently. Contributed by Jason Lowe. 2017-04-28 15:20:35 -05:00
Akira Ajisaka
8a99eba96d
Revert "YARN-5007. Remove deprecated constructors of MiniYARNCluster and MiniMRYarnCluster. Contributed by Andras Bokor." 
This reverts commit 34ab8e73d4.
 2017-04-26 15:34:23 +09:00
Sunil G
9ff25408bc YARN-6164. Expose Queue Configurations per Node Label through YARN client api. Contributed by Benson Qiu. 2017-04-19 20:39:09 +05:30
Jason Lowe
90d97372ed YARN-6461. TestRMAdminCLI has very low test timeouts. Contributed by Eric Badger 2017-04-11 09:16:11 -05:00
Jason Lowe
1b081ca27e YARN-6288. Exceptions during aggregated log writes are mishandled. Contributed by Akira Ajisaka 2017-04-06 16:24:36 -05:00
Akira Ajisaka
34ab8e73d4
YARN-5007. Remove deprecated constructors of MiniYARNCluster and MiniMRYarnCluster. Contributed by Andras Bokor. 2017-04-05 17:33:07 +09:00
Jason Lowe
4d1fac5df2 YARN-6411. Clean up the overwrite of createDispatcher() in subclass of MockRM. Contributed by Yufei Gu 2017-03-31 10:05:34 -05:00
Daniel Templeton
33815af424 YARN-6334. TestRMFailover#testAutomaticFailover always passes even when it should fail 
(Contributed by Yufei Gu via Daniel Templeton)
 2017-03-24 11:42:54 -07:00
Junping Du
c01d15ab27 YARN-6367. YARN logs CLI needs alway check containerLogsInfo/containerLogInfo before parse the JSON object from NMWebService. Contributed by Xuan Gong. 2017-03-21 10:53:27 -07:00
Junping Du
871dc420f8 YARN-6313. YARN logs cli should provide logs for a completed container even when application is still running. Contributed by Xuan Gong. 
(cherry picked from commit b88f5e0f7858d1d89b79dfd325b767c34416052d)
 2017-03-14 12:58:12 -07:00
Jian He
4478273e5f HADOOP-14062. ApplicationMasterProtocolPBClientImpl.allocate fails with EOFException when RPC privacy is enabled. Contributed by Steven Rand 2017-03-10 09:25:58 -08:00
Jian He
2be8947d12 Revert "HADOOP-14062. ApplicationMasterProtocolPBClientImpl.allocate fails with EOFException when RPC privacy is enabled. Contributed by Steven Rand" 
This reverts commit 241c1cc05b.
 2017-03-08 13:20:01 -08:00
Jian He
241c1cc05b HADOOP-14062. ApplicationMasterProtocolPBClientImpl.allocate fails with EOFException when RPC privacy is enabled. Contributed by Steven Rand 2017-03-08 10:48:27 -08:00
Ray Chiang
2148b83993 YARN-6218. Fix TestAMRMClient when using FairScheduler. (Miklos Szegedi via rchiang) 2017-03-03 13:04:51 -08:00
Wangda Tan
1a6ca75f38 YARN-6143. Fix incompatible issue caused by YARN-3583. (Sunil G via wangda) 2017-02-22 11:17:09 -08:00
Li Lu
5d339c46f5 YARN-6177. Yarn client should exit with an informative error message if an incompatible Jersey library is used at client. Contributed by Weiwei Yang. 2017-02-16 13:41:42 -08:00
Sangjin Lee
4fa1afdb88 YARN-4675. Reorganize TimelineClient and TimelineClientImpl into separate classes for ATSv1.x and ATSv2. Contributed by Naganarasimha G R. 2017-02-16 11:41:04 -08:00
Karthik Kambatla
652679aa8a YARN-6061. Add an UncaughtExceptionHandler for critical threads in RM. (Yufei Gu via kasha) 2017-02-14 13:39:41 -08:00
Arun Suresh
aaf106fde3 YARN-5966. AMRMClient changes to support ExecutionType update. (asuresh) 2017-02-14 06:09:10 -08:00
Daniel Templeton
1238fa71f8 YARN-6166. Unnecessary INFO logs in AMRMClientAsyncImpl$CallbackHandlerThread.run (Contributed by Grant Whiteheart via Daniel Templeton) 2017-02-10 09:46:11 -08:00
Jason Lowe
37b4acf7ce YARN-6137. Yarn client implicitly invoke ATS client which accesses HDFS. Contributed by Li Lu 2017-02-08 14:59:09 -06:00
Junping Du
327c9980aa YARN-6100. Improve YARN webservice to output aggregated container logs. Contributed by Xuan Gong. 2017-02-02 00:41:18 -08:00
Junping Du
1a16431bd0 YARN-6126. Obtaining app logs for Running application fails with json parse error. Contributed by Xuan Gong. 2017-01-27 06:31:10 -08:00
Sangjin Lee
425a7e5028 YARN-3637. Handle localization sym-linking correctly at the YARN level. Contributed by Chris Trezzo. 2017-01-25 15:51:36 -08:00
Junping Du
8528d85a68 YARN-6099. Improve webservice to list aggregated log files. Contributed by Xuan Gong. 2017-01-24 15:26:53 -08:00
Subru Krishnan
4d1f3d9020 YARN-6016. Fix minor bugs in handling of local AMRMToken in AMRMProxy. (Botong Huang via Subru). 2017-01-17 14:48:03 -08:00
Akira Ajisaka
e407449ddb HADOOP-13933. Add haadmin -getAllServiceState option to get the HA state of all the NameNodes/ResourceManagers. Contributed by Surendra Singh Lilhore. 2017-01-17 10:10:23 +09:00
Junping Du
4db119b7b5 YARN-6079. Fix simple spelling errors in yarn test code. Contributed by vijay. 2017-01-10 18:20:27 -08:00
Daniel Templeton
6332a318bc YARN-6073. Misuse of format specifier in Preconditions.checkArgument (Contributed by Yuanbo Liu via Daniel Templeton) 2017-01-09 22:12:58 -08:00
Wangda Tan
0a55bd841e YARN-5959. RM changes to support change of container ExecutionType. (Arun Suresh via wangda) 2017-01-05 10:31:05 -08:00
Rohith Sharma K S
e297be74ed YARN-6001. Improve moveApplicationQueues command line. Contributed by Sunil G. 2016-12-28 12:16:11 +05:30
Robert Kanter
9262797e86 YARN-5257. Fix unreleased resources and null dereferences (yufeigu via rkanter) 2016-12-27 14:14:08 -08:00
Varun Saxena
38e66d4d64 YARN-5903. Fix race condition in TestResourceManagerAdministrationProtocolPBClientImpl beforeclass setup method (Haibo Chen via Varun Saxena) 2016-12-22 23:08:33 +05:30
Akira Ajisaka
ae401539ea YARN-4994. Use MiniYARNCluster with try-with-resources in tests. Contributed by Andras Bokor. 2016-12-22 14:32:24 +09:00