big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
17,509 Commits 2 Branches 2 Tags 264 MiB
5e81f32d11
Commit Graph

1041 Commits

Author SHA1 Message Date
Haibo Chen
4f3ca0396a YARN-6510. Fix profs stat file warning caused by process names that includes parenthesis. (Wilfred Spiegelenburg via Haibo Chen) 2017-04-26 11:46:55 -07:00
Daniel Templeton
dd43b895c2 YARN-6202. Configuration item Dispatcher.DISPATCHER_EXIT_ON_ERROR_KEY is disregarded 
(Contributed by Yufei Gu via Daniel Templeton)
 2017-04-19 11:44:55 -07:00
Daniel Templeton
15acdbdba2 YARN-3427. Remove deprecated methods from ResourceCalculatorProcessTree 
(Contributed by Miklos Szegedi via Daniel Templeton)
 2017-04-19 10:31:14 -07:00
Sunil G
9ff25408bc YARN-6164. Expose Queue Configurations per Node Label through YARN client api. Contributed by Benson Qiu. 2017-04-19 20:39:09 +05:30
Junping Du
9d9087a67a YARN-3760. FSDataOutputStream leak in AggregatedLogFormat.LogWriter.close(). Contributed by Haibo Chen. 
(cherry picked from commit c26ccf1adb3a72df3f68e1150b86b813c691203a)
 2017-04-12 13:43:18 -07:00
Jason Lowe
1b081ca27e YARN-6288. Exceptions during aggregated log writes are mishandled. Contributed by Akira Ajisaka 2017-04-06 16:24:36 -05:00
Jason Lowe
e8071aa249 YARN-6403. Invalid local resource request can raise NPE and make NM exit. Contributed by Tao Yang 2017-04-05 12:30:45 -05:00
Varun Saxena
7c2bc444b3 YARN-6342. Make TimelineV2Client's drain timeout after stop configurable (Haibo Chen via Varun Saxena) 2017-03-31 02:02:57 +05:30
Daniel Templeton
640ba1d23f YARN-5685. RM configuration allows all failover methods to disabled when automatic failover is enabled 2017-03-29 12:38:11 -07:00
Robert Kanter
9bae6720cb YARN-6050. AMs can't be scheduled on racks or nodes (rkanter) 2017-03-27 17:23:13 -07:00
Wangda Tan
cd014d57aa YARN-6339. Improve performance for createAndGetApplicationReport. (Yunjiong Zhao via wangda) 2017-03-27 13:29:09 -07:00
Tsuyoshi Ozawa
84ddedc0b2 HADOOP-10101. Update guava dependency to the latest version. (ozawa) 2017-03-25 10:06:52 +09:00
Junping Du
221783b35b YARN-6294. ATS client should better handle Socket closed case. Contributed by Li Lu. 2017-03-15 11:48:17 -07:00
Junping Du
871dc420f8 YARN-6313. YARN logs cli should provide logs for a completed container even when application is still running. Contributed by Xuan Gong. 
(cherry picked from commit b88f5e0f7858d1d89b79dfd325b767c34416052d)
 2017-03-14 12:58:12 -07:00
Varun Saxena
229c7c9f89 YARN-6069. CORS support in timeline v2 (Rohith Sharma K S via Varun Saxena) 2017-03-11 04:09:01 +05:30
Jason Lowe
9649c27864 YARN-6321. TestResources test timeouts are too aggressive. Contributed by Eric Badger 2017-03-10 13:06:54 -06:00
Jason Lowe
deb9f56946 YARN-6310. OutputStreams in AggregatedLogFormat.LogWriter can be left open upon exceptions. Contributed by Haibo Chen 2017-03-10 11:08:33 -06:00
Karthik Kambatla
a96afae125 YARN-6264. AM not launched when a single vcore is available on the cluster. (Yufei Gu via kasha) 2017-03-09 23:11:54 -08:00
Jian He
279d187f72 YARN-6270. WebUtils.getRMWebAppURLWithScheme() needs to honor RM HA setting. Contributed by Xuan Gong 2017-03-03 15:24:04 -08:00
Robert Kanter
6f6dfe0202 YARN-5280. Allow YARN containers to run with Java Security Manager (gphillips via rkanter) 2017-03-01 14:53:47 -08:00
Gera Shegalov
480b4dd574 YARN-1728. Regression test for guice-undecoded pathInfo in YARN WebApp. (Yuanbo Liu via gera) 2017-02-28 09:27:22 -08:00
Xuan
53d372a255 YARN-6228: EntityGroupFSTimelineStore should allow configurable cache 
stores. Contributed by Li Lu
 2017-02-24 10:59:35 -08:00
Karthik Kambatla
718ad9f6ee YARN-6210. FairScheduler: Node reservations can interfere with preemption. (kasha) 2017-02-22 15:46:07 -08:00
Wangda Tan
1a6ca75f38 YARN-6143. Fix incompatible issue caused by YARN-3583. (Sunil G via wangda) 2017-02-22 11:17:09 -08:00
Daniel Templeton
c7a36e6130 YARN-6125. The application attempt's diagnostic message should have a maximum size 
(Contributed by Andras Piros via Daniel Templeton)
 2017-02-17 13:40:58 -08:00
Sangjin Lee
4fa1afdb88 YARN-4675. Reorganize TimelineClient and TimelineClientImpl into separate classes for ATSv1.x and ATSv2. Contributed by Naganarasimha G R. 2017-02-16 11:41:04 -08:00
Karthik Kambatla
6c25dbcdc0 YARN-6163. FS Preemption is a trickle for severely starved applications. (kasha) 2017-02-15 23:16:12 -08:00
Junping Du
0fc6f38379 Addendum patch for YARN-6174 - remove deleted file. 2017-02-15 09:07:49 -08:00
Junping Du
ce2d5bfa5f YARN-6174. Log files pattern should be same for both running and finished container. Contributed by Xuan Gong. 2017-02-15 09:05:14 -08:00
Arun Suresh
aaf106fde3 YARN-5966. AMRMClient changes to support ExecutionType update. (asuresh) 2017-02-14 06:09:10 -08:00
Junping Du
464ff479ce YARN-6113. Re-direct NM Web Service to get container logs for finished applications. Contributed by Xuan Gong. 2017-02-13 06:12:54 -08:00
Sunil G
3a0a0a485a YARN-6118. Add javadoc for Resources.isNone. Contributed by Andres Perez. 2017-02-10 15:48:46 +05:30
Junping Du
eec52e158b YARN-6145. Improve log message on fail over. Contributed by Jian He. 2017-02-08 02:11:19 -08:00
Junping Du
9dbfab1284 YARN-6108. Improve AHS webservice to accept NM address as a parameter to get container logs. Contributed by Xuan Gong. 2017-02-07 02:13:14 -08:00
Junping Du
cce35c3815 HADOOP-12097. Allow port range to be specified while starting webapp. Contributed by Varun Saxena. 2017-02-05 19:42:11 -08:00
Junping Du
327c9980aa YARN-6100. Improve YARN webservice to output aggregated container logs. Contributed by Xuan Gong. 2017-02-02 00:41:18 -08:00
Junping Du
8528d85a68 YARN-6099. Improve webservice to list aggregated log files. Contributed by Xuan Gong. 2017-01-24 15:26:53 -08:00
Sangjin Lee
dc6ec97048 YARN-6117. SharedCacheManager does not start up. Contributed by Chris Trezzo. 2017-01-23 21:07:25 -08:00
Jason Lowe
69fa81679f YARN-5910. Support for multi-cluster delegation tokens. Contributed by Jian He 2017-01-23 17:12:51 +00:00
Andrew Wang
5d8b80ea9b Preparing for 3.0.0-alpha3 development 2017-01-19 15:50:07 -08:00
bibinchundatt
a9d946d4df YARN-6057. yarn.scheduler.minimum-allocation-* descriptions are incorrect about behavior when a request is out of bounds. Contributed by Julia Sommer. 2017-01-17 14:16:00 +05:30
Daniel Templeton
e6f13fe5d1 YARN-5849. Automatically create YARN control group for pre-mounted cgroups (Contributed by Miklos Szegedi via Daniel Templeton) 2017-01-11 16:00:01 -08:00
Junping Du
4db119b7b5 YARN-6079. Fix simple spelling errors in yarn test code. Contributed by vijay. 2017-01-10 18:20:27 -08:00
Wangda Tan
287d3d6804 YARN-3955. Support for application priority ACLs in queues of CapacityScheduler. (Sunil G via wangda) 2017-01-09 08:40:39 -08:00
Naganarasimha
a59df15757 YARN-6015. AsyncDispatcher thread name can be set to improved debugging. Contributed by Ajith S. 2017-01-07 08:11:58 +05:30
Daniel Templeton
165d01a73e YARN-5931. Document timeout interfaces CLI and REST APIs (Contributed by Rohith Sharma K S via Daniel Templeton) 2016-12-30 08:41:52 -08:00
Robert Kanter
9262797e86 YARN-5257. Fix unreleased resources and null dereferences (yufeigu via rkanter) 2016-12-27 14:14:08 -08:00
Daniel Templeton
e216e8e233 HADOOP-13932. Fix indefinite article in comments (Contributed by LiXin Ge via Daniel Templeton) 2016-12-27 13:03:16 -08:00
Arun Suresh
ac1e5d4f77 YARN-5938. Refactoring OpportunisticContainerAllocator to use SchedulerRequestKey instead of Priority and other misc fixes (asuresh) 2016-12-27 12:40:44 -08:00
Arun Suresh
2273a74c1f YARN-5646. Add documentation and update config parameter names for scheduling of OPPORTUNISTIC containers. (Konstantinos Karanasos via asuresh) 2016-12-16 08:14:34 -08:00
Steve Loughran
f66f61892a HADOOP-13852 hadoop build to allow hadoop version property to be explicitly set. Contriibuted by Steve Loughran 2016-12-12 13:58:27 +00:00
Robert Kanter
4b149a1e77 YARN-4457. Cleanup unchecked types for EventHandler (templedf via rkanter) 2016-12-09 17:28:25 -08:00
Arun Suresh
b0aace21b1 YARN-5982. Simplify opportunistic container parameters and metrics. (Konstantinos Karanasos via asuresh) 2016-12-09 16:41:25 -08:00
Akira Ajisaka
7d8e440eee Revert "HADOOP-13852 hadoop build to allow hadoop version property to be explicitly set. Contriibuted by Steve Loughran" 
This reverts commit c265515725.
 2016-12-09 12:49:27 +09:00
Steve Loughran
c265515725 HADOOP-13852 hadoop build to allow hadoop version property to be explicitly set. Contriibuted by Steve Loughran 2016-12-08 17:57:15 +00:00
Sunil G
74d0066d33 YARN-5970. Validate application update timeout request parameters. Contributed by Rohith Sharma K S. 2016-12-08 15:53:56 +05:30
Robert Kanter
72fe546841 YARN-5963. Spelling errors in logging and exceptions for node manager, client, web-proxy, common, and app history code (gsohn via rkanter) 2016-12-07 15:07:25 -08:00
Sunil
ab923a53fc YARN-5965. Retrospect ApplicationReport#getApplicationTimeouts. Contributed by Rohith Sharma K S 2016-12-08 00:27:25 +05:30
Jian He
43ebff2e35 YARN-5559. Analyse 2.8.0/3.0.0 jdiff reports and fix any issues. Contributed by Akira Ajisaka & Wangda Tan 2016-12-05 11:39:34 -08:00
Jason Lowe
f304ccae3c YARN-5915. ATS 1.5 FileSystemTimelineWriter causes flush() to be called after every event write. Contributed by Atul Sikaria 2016-12-02 16:54:15 +00:00
Robert Kanter
c87b3a448a MAPREDUCE-6787. Allow job_conf.xml to be downloadable on the job overview page in JHS (haibochen via rkanter) 2016-12-01 17:29:38 -08:00
Daniel Templeton
b3befc021b YARN-4997. Update fair scheduler to use pluggable auth provider (Contributed by Tao Jie via Daniel Templeton) 2016-11-30 09:50:33 -08:00
Daniel Templeton
25f9872be6 YARN-5774. MR Job stuck in ACCEPTED status without any progress in Fair Scheduler 
if set yarn.scheduler.minimum-allocation-mb to 0. (Contributed by Yufei Gu via Daniel Templeton)
 2016-11-29 09:40:49 -08:00
Akira Ajisaka
209e805430 HADOOP-13506. Redundant groupid warning in child projects. Contributed by Kai Sasaki. 2016-11-28 14:34:57 +09:00
Sunil
eb0a483ed0 YARN-4206. Add Application timeouts in Application report and CLI. Contributed by Rohith Sharma K S. 2016-11-24 18:18:42 +05:30
Daniel Templeton
10468529a9 YARN-4752. Improved preemption in FairScheduler. (kasha) 
Contains:
YARN-5605. Preempt containers (all on one node) to meet the requirement of starved applications
YARN-5821. Drop left-over preemption-related code and clean up method visibilities in the Schedulable hierarchy
YARN-5783. Verify identification of starved applications.
YARN-5819. Verify fairshare and minshare preemption
YARN-5885. Cleanup YARN-4752 branch for merge

Change-Id: Iee0962377d019dd64dc69a020725d2eaf360858c
 2016-11-23 19:48:59 -10:00
Naganarasimha
3541ed8068 YARN-4330. MiniYARNCluster is showing multiple Failed to instantiate default resource calculator warning messages. Contributed by Varun Saxena 2016-11-23 14:12:23 +05:30
Naganarasimha
4667564162 YARN-5911. DrainDispatcher does not drain all events on stop even if setDrainEventsOnStop is true. Contributed by Varun Saxena. 2016-11-23 08:49:48 +05:30
Akira Ajisaka
6f8074298d YARN-5713. Update jackson from 1.9.13 to 2.x in hadoop-yarn. 2016-11-22 10:20:31 +09:00
Rohith Sharma K S
d65603517e YARN-5375. invoke MockRM#drainEvents implicitly in MockRM methods to reduce test failures. Contributed by sandflee. 2016-11-16 15:14:00 +05:30
Mingliang Liu
5af572b644 HADOOP-13427. Eliminate needless uses of FileSystem#{exists(), isFile(), isDirectory()}. Contributed by Steve Loughran and Mingliang Liu 2016-11-15 10:57:00 -08:00
Arun Suresh
3219b7b4ac YARN-4597. Introduce ContainerScheduler and a SCHEDULED state to NodeManager container lifecycle. (asuresh) 2016-11-15 07:56:25 -08:00
Wangda Tan
ede1a473f5 YARN-5552. Add Builder methods for common yarn API records. (Tao Jie via wangda) 2016-11-11 13:34:56 -08:00
Eric Payne
93eeb13164 YARN-4218. Metric for resource*time that was preempted. Contributed by Chang Li. 2016-11-10 22:35:12 +00:00
Jian He
bcc15c6290 YARN-5611. Provide an API to update lifetime of an application. Contributed by Rohith Sharma K S 2016-11-09 16:08:05 -08:00
Arun Suresh
280357c29f YARN-5833. Addendum patch to include missing changes to yarn-default.xml 2016-11-09 07:15:11 -08:00
Andrew Wang
e1c6ef2efa HADOOP-13789. Hadoop Common includes generated test protos in both jar and test-jar. Contributed by Sean Busbey. 2016-11-08 16:33:55 -08:00
Jason Lowe
3f93ac0733 YARN-5356. NodeManager should communicate physical resource capability to ResourceManager. Contributed by Inigo Goiri 2016-11-08 22:01:26 +00:00
Wangda Tan
c00b5d1e51 YARN-5804. New UI2 is not able to launch with jetty 9 upgrade post HADOOP-10075. (Sunil G via wangda) 
(cherry picked from commit ddebaa6b7ae1031899139d153d57df5387d60542)
 2016-11-06 13:13:31 -08:00
Wangda Tan
3de0da2a76 YARN-5698. [YARN-3368] Launch new YARN UI under hadoop web app port. (Sunil G via wangda) 2016-11-06 13:13:31 -08:00
Wangda Tan
c85cc3b56e YARN-4515. [YARN-3368] Support hosting web UI framework inside YARN RM. (Sunil G via wangda) 
YARN-5000. [YARN-3368] App attempt page is not loading when timeline server is not started (Sunil G via wangda)
YARN-5038. [YARN-3368] Application and Container pages shows wrong values when RM is stopped. (Sunil G via wangda)
 2016-11-06 13:13:31 -08:00
Varun Saxena
07ab89e8bb YARN-5809. AsyncDispatcher possibly invokes multiple shutdown threads when handling exception (Jian He via Varun Saxena) 2016-11-01 04:37:03 +05:30
Daniel Templeton
773c60bd7b YARN-5800. Delete LinuxContainerExecutor comment from yarn-default.xml (Contributed by Jan Hentschel via Daniel Templeton) 2016-10-31 14:47:48 -07:00
Naganarasimha
3de130252d YARN-5794. Fix the asflicense warnings. Contributed by Yiqun Lin 2016-10-28 12:37:37 +05:30
Robert Kanter
5877f20f9c HADOOP-10075. Update jetty dependency to version 9 (rkanter) 2016-10-27 16:09:00 -07:00
Li Lu
221582c4ab YARN-4668. Reuse objectMapper instance in Yarn. (Yiqun Lin via gtcarrera9) 2016-10-27 15:20:17 -07:00
Varun Saxena
dd4ed6a587 YARN-4388. Cleanup mapreduce.job.hdfs-servers from yarn-default.xml (Junping Du via Varun Saxena) 2016-10-28 02:22:25 +05:30
Karthik Kambatla
de6faae97c YARN-5388. Deprecate and remove DockerContainerExecutor. (Daniel Templeton via kasha) 2016-10-25 13:35:47 -07:00
Xiao Chen
c88c1dc50c YARN-5777. TestLogsCLI#testFetchApplictionLogsAsAnotherUser fails. Contributed by Akira Ajisaka. 2016-10-25 12:15:09 -07:00
Subru Krishnan
0a166b1347 YARN-5711. Propogate exceptions back to client when using hedging RM failover provider. 2016-10-24 18:59:51 -07:00
Robert Kanter
23d7d53a41 YARN-5679. TestAHSWebServices is failing (ajisakaa via rkanter) 2016-10-21 13:31:03 -07:00
Xuan
b733a6f862 YARN-5718. TimelineClient (and other places in YARN) shouldn't over-write HDFS client retry settings which could cause unexpected behavior. Contributed by Junping Du. 2016-10-18 11:06:47 -07:00
Karthik Kambatla
6378845f9e YARN-4464. Lower the default max applications stored in the RM and store. (Daniel Templeton via kasha) 2016-10-11 21:42:08 -07:00
Jason Lowe
ecb51b857a YARN-5551. Ignore file backed pages from memory computation when smaps is enabled. Contributed by Rajesh Balamohan 2016-10-11 15:12:43 +00:00
Chris Douglas
d65b957776 YARN-5702. Refactor TestPBImplRecords for reuse in other YARN modules. Contributed by Subru Krishnan 2016-10-05 12:16:49 -07:00
Karthik Kambatla
736d33cddd YARN-4767. Network issues can cause persistent RM UI outage. (Daniel Templeton via kasha) 2016-10-03 14:35:57 -07:00
Naganarasimha
6e130c308c YARN-4855. Should check if node exists when replace nodelabels. Contributeed by Tao Jie 2016-10-03 02:02:26 -04:00
Subru Krishnan
3a3697deab YARN-5384. Expose priority in ReservationSystem submission APIs. (Sean Po via Subru). 2016-09-30 19:41:43 -07:00
Jian He
2ae5a3a5bf YARN-4205. Add a service for monitoring application life time out. Contributed by Rohith Sharma K S 2016-09-29 22:00:31 +08:00
Varun Saxena
9b0fd01d2e YARN-5599. Publish AM launch command to ATS (Rohith Sharma K S via Varun Saxena) 2016-09-28 16:10:10 +05:30
Varun Vasudev
bc2656f09f YARN-5662. Provide an option to enable ContainerMonitor. Contributed by Jian He. 2016-09-28 15:18:18 +05:30
Arun Suresh
4815d024c5 YARN-5609. Expose upgrade and restart API in ContainerManagementProtocol. Contributed by Arun Suresh 2016-09-26 08:46:54 -07:00
Arun Suresh
2f163cd5cf Revert "YARN-5609. Expose upgrade and restart API in ContainerManagementProtocol. Contributed by Arun Suresh" 
This reverts commit fe644bafe7.
 2016-09-26 08:36:59 -07:00
Jian He
fe644bafe7 YARN-5609. Expose upgrade and restart API in ContainerManagementProtocol. Contributed by Arun Suresh 2016-09-26 22:41:16 +08:00
Varun Saxena
b8a2d7b8fc YARN-5539. TimelineClient failed to retry on java.net.SocketTimeoutException: Read timed out (Junping Du via Varun Saxena) 2016-09-23 13:27:31 +05:30
Varun Saxena
3293a7d92d Revert "TimelineClient failed to retry on java.net.SocketTimeoutException: Read timed out" 
This reverts commit 2e6ee95716.
 2016-09-23 13:25:46 +05:30
Varun Saxena
2e6ee95716 TimelineClient failed to retry on java.net.SocketTimeoutException: Read timed out 2016-09-23 13:23:54 +05:30
Naganarasimha
d0372dc613 YARN-3692. Allow REST API to set a user generated message when killing an application. Contributed by Rohith Sharma K S 2016-09-23 06:30:49 +05:30
Robert Kanter
5ffd4b7c1e YARN-4973. YarnWebParams next.fresh.interval should be next.refresh.interval (templedf via rkanter) 2016-09-22 16:45:34 -07:00
Wangda Tan
5a58bfee30 YARN-4591. YARN Web UIs should provide a robots.txt. (Sidharta Seethana via wangda) 2016-09-20 17:20:50 -07:00
Naganarasimha
4174b9756c YARN-5642. Typos in 9 log messages. Contributed by Mehran Hassani 2016-09-17 10:35:39 +05:30
Naganarasimha
ade7c2bc9c YARN-4232. TopCLI console support for HA mode. Contributed by Bibin A Chundatt 2016-09-17 09:52:39 +05:30
Ray Chiang
378f624a39 YARN-5549. AMLauncher#createAMContainerLaunchContext() should not log the command to be launched indiscriminately. (Daniel Templeton via rchiang) 2016-09-02 11:14:35 -07:00
Arun Suresh
d6d9cff21b YARN-5221. Expose UpdateResourceRequest API to allow AM to request for change in container properties. (asuresh) 2016-08-30 15:52:29 -07:00
Subru Krishnan
b930dc3ec0 YARN-5327. API changes required to support recurring reservations in the YARN ReservationSystem. (Sangeetha Abdu Jyothi via Subru). 2016-08-26 16:58:47 -07:00
Junping Du
9ef632f3b0 YARN-5557. Add localize API to the ContainerManagementProtocol. Contributed by Jian He. 2016-08-26 09:04:44 -07:00
Varun Vasudev
bfb4d95059 YARN-5430. Return container's ip and host from NM ContainerStatus call. Contributed by Jian He. 2016-08-26 16:30:18 +05:30
Junping Du
0da69c324d YARN-4676. Automatic and Asynchronous Decommissioning Nodes Status Tracking. Contributed by Diniel Zhi. 
(cherry picked from commit d464483bf7f0b3e3be3ba32cd6c3eee546747ab5)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceTrackerService.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNode.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
 2016-08-18 07:27:23 -07:00
Varun Saxena
913a895136 YARN=5526. DrainDispacher#serviceStop blocked if setDrainEventsOnStop invoked (sandflee via Varun Saxena) 2016-08-18 11:38:13 +05:30
Varun Saxena
b427ce12bc YARN-5475. Fix test failure of TestAggregatedLogFormat#testReadAcontainerLogs1 (Jun Gong via Varun Saxena) 2016-08-16 20:24:53 +05:30
Ray Chiang
aea3e65749 YARN-5137. Make DiskChecker pluggable in NodeManager. (Yufei Gu via rchiang) 2016-08-10 19:23:29 -07:00
Arun Suresh
82c9e06101 YARN-5457. Refactor DistributedScheduling framework to pull out common functionality. (asuresh) 2016-08-09 00:42:29 -07:00
Junping Du
e605d47df0 YARN-5470. Differentiate exactly match with regex in yarn log CLI. Contributed by Xuan Gong. 2016-08-05 10:55:29 -07:00
Arun Suresh
e5766b1dbe YARN-5113. Refactoring and other clean-up for distributed scheduling. (Konstantinos Karanasos via asuresh) 2016-07-31 11:48:25 -07:00
Li Lu
7086fc72ee YARN-5436. Race in AsyncDispatcher can cause random test failures in Tez (probably YARN also). (Zhiyuan Yang via gtcarrera9) 2016-07-28 16:50:57 -07:00
Jason Lowe
a1890c32c5 YARN-5438. TimelineClientImpl leaking FileSystem Instances causing Long running services like HiverServer2 daemon going OOM. Contributed by Rohith Sharma K S 2016-07-28 21:47:13 +00:00
Junping Du
7f3c306e2e YARN-5432. Lock already held by another process while LevelDB cache store creation for dag. Contributed by Li Lu. 2016-07-28 06:35:24 -07:00
Subru Krishnan
d2cbfd7de3 YARN-5441. Fixing minor Scheduler test case failures 2016-07-27 18:21:58 -07:00
Akira Ajisaka
54fe17a607 HADOOP-9427. Use JUnit assumptions to skip platform-specific tests. Contributed by Gergely Novák. 2016-07-27 19:41:09 +09:00
Arun Suresh
2d8d183b19 YARN-5351. ResourceRequest should take ExecutionType into account during comparison. (Konstantinos Karanasos via asuresh) 2016-07-26 19:08:30 -07:00
Varun Vasudev
9ccf935b2c YARN-5309. Fix SSLFactory truststore reloader thread leak in TimelineClientImpl. Contributed by Weiwei Yang. 2016-07-20 12:48:32 +05:30
Junping Du
dc2f4b6ac8 YARN-5213. Fix a bug in LogCLIHelpers which cause TestLogsCLI#testFetchApplictionLogs fails intermittently. Contributed by Xuan Gong. 2016-07-19 07:17:58 -07:00
Andrew Wang
da456ffd62 Preparing for 3.0.0-alpha2 development 2016-07-15 19:04:17 -07:00
Junping Du
7e5355c14e YARN-5339. Passing file to -out for YARN log CLI doesnt give warning or error code. Contributed by Xuan Gong. 2016-07-15 09:23:11 -07:00
Vinod Kumar Vavilapalli
eb47163234 YARN-5200. Enhanced "yarn logs" to be able to get a list of containers whose logs are aggregated via a "show_container_log_info" option. Contributed by Xuan Gong. 2016-07-13 10:54:15 -07:00
Robert Kanter
bf6f4a3b98 YARN-4366. Fix Lint Warnings in YARN Common (templedf via rkanter) 2016-07-12 15:12:25 -07:00
Jian He
819224dcf9 YARN-5270. Solve miscellaneous issues caused by YARN-4844. Contributed by Wangda Tan 2016-07-11 22:36:20 -07:00
Sangjin Lee
6cf6ab7b78 Made a number of miscellaneous fixes for javac, javadoc, and checstyle warnings. 2016-07-10 08:46:05 -07:00
Varun Saxena
1ff6833bba YARN-5243. fix several rebase and other miscellaneous issues before merge. (Sangjin Lee via Varun Saxena) 2016-07-10 08:46:03 -07:00
Li Lu
0a9b085f05 YARN-5189. Make HBaseTimeline[Reader|Writer]Impl default and move FileSystemTimeline*Impl. (Joep Rottinghuis and Sangjin Lee via gtcarrera9) 2016-07-10 08:46:01 -07:00
Li Lu
c2055a97d5 YARN-3150. Documenting the timeline service v2. (Sangjin Lee and Vrushali C via gtcarrera9) 2016-07-10 08:45:57 -07:00
Sangjin Lee
39cce4e629 YARN-3816. [Aggregation] App-level aggregation and accumulation for YARN system metrics (Li Lu via sjlee) 2016-07-10 08:45:56 -07:00
Varun Saxena
a3cf40e532 YARN-3461. Consolidate flow name/version/run defaults. (Sangjin Lee via Varun Saxena) 2016-07-10 08:45:55 -07:00
Sangjin Lee
84c35ac6c4 YARN-4711. NM is going down with NPE's due to single thread processing of events by Timeline client (Naganarasimha G R via sjlee) 2016-07-10 08:45:55 -07:00
Sangjin Lee
9bdd455dce YARN-4062. Add the flush and compaction functionality via coprocessors and scanners for flow run table (Vrushali C via sjlee) 2016-07-10 08:45:54 -07:00
Sangjin Lee
0d02ab8729 YARN-3367. Replace starting a separate thread for post entity with event loop in TimelineClient (Naganarasimha G R via sjlee) 2016-07-10 08:45:53 -07:00
Sangjin Lee
960af7d471 YARN-4409. Fix javadoc and checkstyle issues in timelineservice code (Varun Saxena via sjlee) 2016-07-10 08:45:53 -07:00
Naganarasimha
6934b05c71 YARN-4238. createdTime and modifiedTime is not reported while publishing entities to ATSv2. (Varun Saxena via Naganarasimha G R) 2016-07-10 08:45:52 -07:00
Varun Saxena
f5380d850e YARN-4622. TestDistributedShell fails for v2 test cases after modifications for 1.5. (Naganarasimha G R via Varun Saxena) 2016-07-10 08:45:51 -07:00
Li Lu
34f02f07d5 Rebase to latest trunk 2016-07-10 08:45:51 -07:00
Sangjin Lee
cc16683cef YARN-3995. Some of the NM events are not getting published due race condition when AM container finishes in NM (Naganarasimha G R via sjlee) 2016-07-10 08:45:50 -07:00
Li Lu
8ef546c1ee YARN-4445. Unify the term flowId and flowName in timeline v2 codebase. 
Contributed by Zhan Zhang.
 2016-07-10 08:45:49 -07:00
Xuan
2e2dbf59d1 YARN-4392. ApplicationCreatedEvent event time resets after RM 
restart/failover. Contributed by Naganarasimha G R and Xuan Gong

(cherry picked from commit 4546c7582b)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java
 2016-07-10 08:45:49 -07:00
Li Lu
b7d484aead YARN-4450. TestTimelineAuthenticationFilter and 
TestYarnConfigurationFields fail. Contributed by Sangjin Lee.
 2016-07-10 08:45:49 -07:00
Li Lu
89e5c44f9e YARN-4356. Ensure the timeline service v.2 is disabled cleanly and has no 
impact when it's turned off. Contributed by Sangjin Lee.
 2016-07-10 08:45:48 -07:00
Junping Du
cf4666bb8c YARN-3623. Add a config to indicate the Timeline Service version. Contributed by Xuan Gong. 
(cherry picked from commit f910e4f639)
 2016-07-10 08:45:48 -07:00
Vrushali
10fa6da7d8 YARN-4074. [timeline reader] implement support for querying for flows and flow runs (sjlee via vrushali) 2016-07-10 08:45:44 -07:00
Junping Du
8603736ef2 YARN-3949. Ensure timely flush of timeline writes. Contributed by Sangjin Lee. 
(cherry picked from commit 967bef7e0396d857913caa2574afb103a5f0b81b)
 2016-07-10 08:45:39 -07:00
Sangjin Lee
57d8dc2fb7 YARN-3836. add equals and hashCode to TimelineEntity and other classes in the data model (Li Lu via sjlee) 
(cherry picked from commit 2d4a8f4563c06339717ca9410b2794754603fba3)
 2016-07-10 08:45:39 -07:00
Sangjin Lee
e27642abf4 YARN-3047. [Data Serving] Set up ATS reader with basic request serving structure and lifecycle (Varun Saxena via sjlee) 
(cherry picked from commit 4c5f88fb0f04b7919738d07598b0f006a9ff91f2)
 2016-07-10 08:45:39 -07:00
Sangjin Lee
22e7ae5771 YARN-3792. Test case failures in TestDistributedShell and some issue fixes related to ATSV2 (Naganarasimha G R via sjlee) 
(cherry picked from commit 84f37f1c7eefec6d139cbf091c50d6c06f734323)
 2016-07-10 08:45:38 -07:00
Zhijie Shen
42d7864b84 YARN-3276. Code cleanup for timeline service API records. Contributed by Junping Du. 
(cherry picked from commit d88f30ba5359f59fb71b93a55e1c1d9a1c0dff8e)
 2016-07-10 08:45:37 -07:00
Sangjin Lee
fc8485d893 YARN-3551. Consolidate data model change according to the backend implementation (Zhijie Shen via sale) 
(cherry picked from commit 557a3950bddc837469244835f5577899080115d8)
 2016-07-10 08:45:35 -07:00
Junping Du
2bdefbc4a0 YARN-3431. Sub resources of timeline entity needs to be passed to a separate endpoint. Contributed By Zhijie Shen. 
(cherry picked from commit fa5cc75245a6dba549620a8b26c7b4a8aed9838e)
 2016-07-10 08:45:34 -07:00
Junping Du
47f35a30bb YARN-3391. Clearly define flow ID/ flow run / flow version in API and storage. Contributed by Zhijie Shen 
(cherry picked from commit 68c6232f8423e55b4d152ef3d1d66aeb2d6a555e)
 2016-07-10 08:45:33 -07:00
Zhijie Shen
5712b8f9fd YARN-3334. NM uses timeline client to publish container metrics to new timeline service. Contributed by Junping Du. 2016-07-10 08:45:33 -07:00
Junping Du
d67c9bdb4d YARN-3040. Make putEntities operation be aware of the app's context. Contributed by Zhijie Shen 2016-07-10 08:45:32 -07:00
Junping Du
5e3d9a477b YARN-3034. Implement RM starting its timeline collector. Contributed by Naganarasimha G R 2016-07-10 08:45:32 -07:00
Junping Du
2188a07e5b YARN-3333. Rename TimelineAggregator etc. to TimelineCollector. Contributed by Sangjin Lee 2016-07-10 08:45:31 -07:00
Zhijie Shen
9b56364080 YARN-3039. Implemented the app-level timeline aggregator discovery service. Contributed by Junping Du. 2016-07-10 08:45:31 -07:00
Zhijie Shen
309592b9e5 YARN-3240. Added ".keep" file which was missed before. 
(cherry picked from commit 6b4dc96414b3f15537670c67a0dccd048862f4f4)
 2016-07-10 08:45:29 -07:00
Junping Du
4f0c7eaff3 YARN-3240. Implement client API to put generic entities. Contributed by Zhijie Shen 
(cherry picked from commit 4487da249f448d5c67b712cd0aa723e764eed77d)
 2016-07-10 08:45:29 -07:00
Zhijie Shen
c77f86bf63 YARN-3041. Added the overall data model of timeline service next gen. Contributed by Zhijie Shen. 
(cherry picked from commit fdff5d2625c129c20c44f90a0517b28cee1522bd)
 2016-07-10 08:45:29 -07:00
Jian He
8a9d293dd6 YARN-5233. Support for specifying a path for ATS plugin jars. Contributed by Li Lu 2016-07-06 16:44:46 -07:00
Wangda Tan
04f6ebb66a YARN-5294. Pass remote ip address down to YarnAuthorizationProvider. (Jian He via wangda) 2016-07-06 10:36:48 -07:00
Xuan
6ab5aa1c1f YARN-5266. Wrong exit code while trying to get app logs using regex via CLI. Contributed by Xuan Gong 2016-06-22 21:48:49 -07:00
Arun Suresh
99e5dd68d0 YARN-5171. Extend DistributedSchedulerProtocol to notify RM of containers allocated by the Node. (Inigo Goiri via asuresh) 2016-06-22 19:04:54 -07:00
Tsuyoshi Ozawa
5d58858bb6 HADOOP-9613. [JDK8] Update jersey version to latest 1.x release. 2016-06-21 08:05:32 +09:00
Junping Du
d0162f2040 YARN-5246. NMWebAppFilter web redirects drop query parameters. Contributed by Varun Vasudev. 2016-06-19 17:44:54 -07:00
Junping Du
c35fa4a0e5 YARN-5251. Yarn CLI to obtain App logs for last 'n' bytes fails. Contributed by Xuan Gong. 2016-06-17 08:24:24 -07:00
Junping Du
5dfc38ff57 YARN-5237. Fix missing log files issue in rolling log aggregation. Contributed by Xuan Gong. 2016-06-15 16:17:54 -07:00
Varun Vasudev
25064fb2fb YARN-5223. Container line in yarn logs output for a live application should include the hostname for the container. Contributed by Xuan Gong. 2016-06-15 12:42:44 +05:30
Wangda Tan
c77a1095dc YARN-1942. Deprecate toString/fromString methods from ConverterUtils and move them to records classes like ContainerId/ApplicationId, etc. (wangda) 2016-06-14 15:06:38 -07:00
Wangda Tan
eec835ec17 YARN-4887. Add allocation request ID to AM-RM protocol for identifying resource-requests explicitly. (Subru Krishnan via wangda) 2016-06-13 21:57:33 -07:00
Arun Suresh
5143277958 YARN-5124. Modify AMRMClient to set the ExecutionType in the ResourceRequest. (asuresh) 2016-06-12 09:42:38 -07:00
Wangda Tan
03fc6b1bb0 YARN-3426. Add jdiff support to YARN. (vinodkv via wangda) 2016-06-10 09:51:09 -07:00
Vinod Kumar Vavilapalli
9378d9428f YARN-5191. Renamed the newly added “download=true” option for getting logs via NMWebServices and AHSWebServices to be a better "format" option. (Xuan Gong via vinodkv) 2016-06-09 12:30:58 -07:00
Naganarasimha
1500a0a300 YARN-4308. ContainersAggregated CPU resource utilization reports negative usage in first few heartbeats. Contributed by Sunil G 2016-06-09 05:41:09 +05:30
Wangda Tan
620325e816 YARN-4837. User facing aspects of 'AM blacklisting' feature need fixing. (vinodkv via wangda) 2016-06-07 15:06:42 -07:00
Ming Ma
4a1cedc010 MAPREDUCE-5044. Have AM trigger jstack on task attempts that timeout before killing them. (Eric Payne and Gera Shegalov via mingma) 2016-06-06 14:30:51 -07:00
Arun Suresh
dc26601d8f YARN-5180. Allow ResourceRequest to specify an enforceExecutionType flag. (asuresh) 2016-06-02 09:01:02 -07:00
Xuan
0bc05e40fa YARN-5088. Improve "yarn log" command-line to read the last K bytes for the log files. Contributed by Xuan Gong 2016-06-01 13:44:21 -07:00
Varun Vasudev
86fb58b7dc YARN-4007. Add support for different network setups when launching the docker container. Contributed by Sidharta Seethana. 2016-05-29 21:32:51 +05:30
Varun Vasudev
42f90ab885 YARN-4844. Add getMemorySize/getVirtualCoresSize to o.a.h.y.api.records.Resource. Contributed by Wangda Tan. 2016-05-29 21:24:16 +05:30
Arun Suresh
aa975bc781 YARN-5127. Expose ExecutionType in Container api record. (Hitesh Sharma via asuresh) 2016-05-27 14:06:32 -07:00
Xuan
bde819abbb YARN-5089. Improve "yarn log" command-line "logFiles" option to support 
regex. Contributed by Xuan Gong
 2016-05-26 23:49:20 -07:00
Rohith Sharma K S
0a544f8a3e YARN-5005. TestRMWebServices#testDumpingSchedulerLogs fails randomly. Contributed by Bibin A Chundatt. 2016-05-27 10:44:35 +05:30
Arun Suresh
5b41b288d0 YARN-5162. Fix Exceptions thrown during in registerAM call when Distributed Scheduling is Enabled (Hitesh Sharma via asuresh) 2016-05-26 14:56:37 -07:00
Carlo Curino
013532a95e YARN-4957. Add getNewReservation in ApplicationClientProtocol (Sean Po via curino) 2016-05-25 16:55:49 -07:00
Karthik Kambatla
77d5ce9069 YARN-3344. Fix warning - procfs stat file is not in the expected format. (Ravindra Kumar Naik and Akira Ajisaka via kasha) 2016-05-25 10:50:54 -07:00