Bikas Saha
33b5a81509
YARN-513. Create common proxy client for communicating with RM (Xuan Gong & Jian He via bikas)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1503933 13f79535-47bb-0310-9956-ffa450edef68
2013-07-16 22:47:40 +00:00
Vinod Kumar Vavilapalli
4466d8653f
YARN-523. Modified a test-case to validate container diagnostics on localization failures. Contributed by Jian He.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1503532 13f79535-47bb-0310-9956-ffa450edef68
2013-07-16 00:57:41 +00:00
Chris Nauroth
1db4319072
YARN-909. Disable TestLinuxContainerExecutorWithMocks on Windows. Contributed by Chuan Liu.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1503357 13f79535-47bb-0310-9956-ffa450edef68
2013-07-15 17:27:31 +00:00
Alejandro Abdelnur
1714932158
YARN-333. Schedulers cannot control the queue-name of an application. (sandyr via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1502374 13f79535-47bb-0310-9956-ffa450edef68
2013-07-11 21:39:28 +00:00
Christopher Douglas
85f0efb68f
YARN-569. Add support for requesting and enforcing preemption requests via
...
a capacity monitor. Contributed by Carlo Curino, Chris Douglas
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1502083 13f79535-47bb-0310-9956-ffa450edef68
2013-07-11 01:20:23 +00:00
Vinod Kumar Vavilapalli
dfe70f6a03
YARN-295. Fixed a race condition in ResourceManager RMAppAttempt state machine. Contributed by Mayank Bansal.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1501856 13f79535-47bb-0310-9956-ffa450edef68
2013-07-10 17:08:43 +00:00
Vinod Kumar Vavilapalli
3ee5949912
YARN-368. Fixed a typo in error message in Auxiliary services. Contributed by Albert Chu.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1501852 13f79535-47bb-0310-9956-ffa450edef68
2013-07-10 17:02:08 +00:00
Bikas Saha
15ce82b9c5
YARN-369. Handle ( or throw a proper error when receiving) status updates from application masters that have not registered (Mayank Bansal & Abhishek Kapoor via bikas)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1501605 13f79535-47bb-0310-9956-ffa450edef68
2013-07-09 23:14:27 +00:00
Hitesh Shah
e2cce7bd26
YARN-727, MAPREDUCE-5325. ClientRMProtocol.getAllApplications should accept ApplicationType as a parameter. Contributed by Xuan Gong.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1501599 13f79535-47bb-0310-9956-ffa450edef68
2013-07-09 23:01:01 +00:00
Chris Nauroth
31fff48ec4
YARN-894. NodeHealthScriptRunner timeout checking is inaccurate on Windows. Contributed by Chuan Liu.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1501016 13f79535-47bb-0310-9956-ffa450edef68
2013-07-08 23:40:43 +00:00
Vinod Kumar Vavilapalli
5e4f6ad1d9
YARN-791. Changed RM APIs and web-services related to nodes to ensure that both are consistent with each other. Contributed by Sandy Ryza.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1500994 13f79535-47bb-0310-9956-ffa450edef68
2013-07-08 22:30:52 +00:00
Bikas Saha
eff5d9b17e
YARN-845. RM crash with NPE on NODE_UPDATE (Mayank Bansal via bikas)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1499886 13f79535-47bb-0310-9956-ffa450edef68
2013-07-04 23:31:26 +00:00
Luke Lu
8eb3be63f5
YARN-877. Support resource blacklisting for FifoScheduler. (Junping Du via llu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1498021 13f79535-47bb-0310-9956-ffa450edef68
2013-06-29 20:18:57 +00:00
Alejandro Abdelnur
c221204cca
YARN-883. Expose Fair Scheduler-specific queue metrics. (sandyr via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1497884 13f79535-47bb-0310-9956-ffa450edef68
2013-06-28 18:59:47 +00:00
Arun Murthy
f3a893b696
YARN-869. Move ResourceManagerAdministrationProtocol out of main YARN api. Contributed by Vinod K V.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1496751 13f79535-47bb-0310-9956-ffa450edef68
2013-06-26 05:57:05 +00:00
Vinod Kumar Vavilapalli
b3a8d99817
YARN-874. Making common RPC to switch to not switch to simple when other mechanisms are enabled and thus fix YARN/MR test failures after HADOOP-9421. Contributed by Daryn Sharp and Vinod Kumar Vavilapalli.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1496692 13f79535-47bb-0310-9956-ffa450edef68
2013-06-26 00:42:26 +00:00
Alejandro Abdelnur
e60fbbcc2e
YARN-736. Add a multi-resource fair sharing metric. (sandyr via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1496153 13f79535-47bb-0310-9956-ffa450edef68
2013-06-24 18:33:45 +00:00
Bikas Saha
87d67c8c55
MiniYARNCluster broken after YARN-827 (bikas)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1495684 13f79535-47bb-0310-9956-ffa450edef68
2013-06-22 08:08:21 +00:00
Bikas Saha
ba44df9166
Deleting files missed for YARN-827
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1495633 13f79535-47bb-0310-9956-ffa450edef68
2013-06-21 22:54:04 +00:00
Bikas Saha
37d7935a9d
YARN-827. Need to make Resource arithmetic methods accessible^CJian He via bikas)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1495533 13f79535-47bb-0310-9956-ffa450edef68
2013-06-21 18:28:57 +00:00
Alejandro Abdelnur
066f970457
YARN-866. Add test for class ResourceWeights. (ywskycn via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1495494 13f79535-47bb-0310-9956-ffa450edef68
2013-06-21 16:24:53 +00:00
Hitesh Shah
575495b5b1
YARN-861. TestContainerManager is failing. Contributed by Vinod Kumar Vavilapalli.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1495160 13f79535-47bb-0310-9956-ffa450edef68
2013-06-20 20:18:04 +00:00
Vinod Kumar Vavilapalli
3f809667bc
YARN-854. Fixing YARN bugs that are failing applications in secure environment. Contributed by Omkar Vinit Joshi.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494845 13f79535-47bb-0310-9956-ffa450edef68
2013-06-20 03:41:36 +00:00
Siddharth Seth
ab647cfcdb
YARN-848. Fix NodeManager to register with RM using the fully qualified hostname. Contributed by Hitesh Shah.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494385 13f79535-47bb-0310-9956-ffa450edef68
2013-06-18 23:49:23 +00:00
Vinod Kumar Vavilapalli
243bcd367f
YARN-694. Starting to use NMTokens to authenticate all communication with NodeManagers. Contributed by Omkar Vinit Joshi.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494369 13f79535-47bb-0310-9956-ffa450edef68
2013-06-18 23:19:49 +00:00
Arun Murthy
6451288704
YARN-799. Fix CgroupsLCEResourcesHandler to use /tasks instead of /cgroup.procs. Contributed by Chris Riccomini.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494035 13f79535-47bb-0310-9956-ffa450edef68
2013-06-18 06:34:10 +00:00
Vinod Kumar Vavilapalli
f4d80e91ae
YARN-841. Move Auxiliary service to yarn-api, annotate and document it. Contributed by Vinod Kumar Vavilapalli.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494031 13f79535-47bb-0310-9956-ffa450edef68
2013-06-18 06:20:37 +00:00
Arun Murthy
acc0d3eb52
YARN-840. Moved ProtoUtils to yarn.api.records.pb.impl. Contributed by Jian He.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494027 13f79535-47bb-0310-9956-ffa450edef68
2013-06-18 05:47:54 +00:00
Vinod Kumar Vavilapalli
5d1b453b85
YARN-610. ClientToken is no longer set in the environment of the Containers. Contributed by Omkar Vinit Joshi.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493968 13f79535-47bb-0310-9956-ffa450edef68
2013-06-17 23:13:17 +00:00
Chris Nauroth
b02dc5b464
YARN-839. TestContainerLaunch.testContainerEnvVariables fails on Windows. Contributed by Chuan Liu.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493937 13f79535-47bb-0310-9956-ffa450edef68
2013-06-17 21:05:00 +00:00
Vinod Kumar Vavilapalli
7d0c43d7cf
YARN-822. Renamed ApplicationToken to be AMRMToken, and similarly the corresponding TokenSelector and SecretManager. Contributed by Omkar Vinit Joshi.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493889 13f79535-47bb-0310-9956-ffa450edef68
2013-06-17 19:11:29 +00:00
Vinod Kumar Vavilapalli
4662b8b443
YARN-833. Moved Graph and VisualizeStateMachine into yarn.state package. Contributed by Zhijie Shen.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493654 13f79535-47bb-0310-9956-ffa450edef68
2013-06-17 06:51:51 +00:00
Vinod Kumar Vavilapalli
b9efe6bd4a
HADOOP-9649. Promoted YARN service life-cycle libraries into Hadoop Common for usage across all Hadoop projects. Contributed by Zhijie Shen.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493650 13f79535-47bb-0310-9956-ffa450edef68
2013-06-17 06:39:33 +00:00
Vinod Kumar Vavilapalli
d4a811edb2
YARN-826. Moved Clock and SystemClock into yarn.util package. Contributed by Zhijie Shen.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493639 13f79535-47bb-0310-9956-ffa450edef68
2013-06-17 04:49:13 +00:00
Vinod Kumar Vavilapalli
f5f8f3bca4
YARN-825. Fixed javadoc and annotations for yarn-common module. Contributed by Vinod Kumar Vavilapalli.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493634 13f79535-47bb-0310-9956-ffa450edef68
2013-06-17 03:32:49 +00:00
Arun Murthy
6b75a5c3b5
YARN-831. Removed minimum resource from GetNewApplicationResponse as a follow-up to YARN-787. Contributed Jian He.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493626 13f79535-47bb-0310-9956-ffa450edef68
2013-06-17 02:33:11 +00:00
Arun Murthy
77e134d13f
YARN-387. Renamed YARN protocols for consistency. Contributed by Vinod K V.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493623 13f79535-47bb-0310-9956-ffa450edef68
2013-06-17 02:27:18 +00:00
Arun Murthy
98d97d316c
YARN-787. Removed minimum resource from RegisterApplicationMasterResponse. Contributed by Alejandro Abdelnur.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493509 13f79535-47bb-0310-9956-ffa450edef68
2013-06-16 13:59:41 +00:00
Vinod Kumar Vavilapalli
769a0bd831
YARN-693. Modified RM to send NMTokens on allocate call so that AMs can then use them for authentication with NMs. Contributed by Omkar Vinit Joshi.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493448 13f79535-47bb-0310-9956-ffa450edef68
2013-06-16 03:11:33 +00:00
Vinod Kumar Vavilapalli
29a3708724
YARN-781. Exposing LOGDIR in all containers' environment which should be used by containers for logging purposes. Contributed by Jian He.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493428 13f79535-47bb-0310-9956-ffa450edef68
2013-06-15 23:09:24 +00:00
Vinod Kumar Vavilapalli
c2d0cf1b20
YARN-821. Renamed setFinishApplicationStatus to setFinalApplicationStatus in FinishApplicationMasterRequest for consistency. Contributed by Jian He.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493315 13f79535-47bb-0310-9956-ffa450edef68
2013-06-15 04:57:04 +00:00
Alejandro Abdelnur
b9753e509c
YARN-789. Enable zero capabilities resource requests in fair scheduler. (tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493219 13f79535-47bb-0310-9956-ffa450edef68
2013-06-14 19:16:40 +00:00
Alejandro Abdelnur
334de8d211
ARN-803. factor out scheduler config validation from the ResourceManager to each scheduler implementation. (tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493160 13f79535-47bb-0310-9956-ffa450edef68
2013-06-14 17:07:25 +00:00
Vinod Kumar Vavilapalli
fdc9412a81
YARN-806. Moved ContainerExitStatus from yarn.api to yarn.api.records. Contributed by Jian He.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493138 13f79535-47bb-0310-9956-ffa450edef68
2013-06-14 16:10:37 +00:00
Alejandro Abdelnur
710a5eea62
YARN-782. vcores-pcores ratio functions differently from vmem-pmem ratio in misleading way. (sandyr via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493064 13f79535-47bb-0310-9956-ffa450edef68
2013-06-14 13:41:45 +00:00
Vinod Kumar Vavilapalli
e9f753a11e
YARN-792. Moved NodeHealthStatus from yarn.api.record to yarn.server.api.record. Contributed by Jian He.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492943 13f79535-47bb-0310-9956-ffa450edef68
2013-06-14 03:17:03 +00:00
Vinod Kumar Vavilapalli
f0eb4bc342
YARN-692. Creating NMToken master key on RM and sharing it with NM as a part of RM-NM heartbeat. Contributed by Omkar Vinit Joshi.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492907 13f79535-47bb-0310-9956-ffa450edef68
2013-06-14 00:06:42 +00:00
Vinod Kumar Vavilapalli
c0cd68d8fb
YARN-773. Moved YarnRuntimeException from package api.yarn to api.yarn.exceptions. Contributed by Jian He.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492903 13f79535-47bb-0310-9956-ffa450edef68
2013-06-13 23:25:14 +00:00
Vinod Kumar Vavilapalli
686ac311c9
YARN-746. Renamed Service.register() and Service.unregister() to registerServiceListener() & unregisterServiceListener() respectively. Contributed by Steve Loughran.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492780 13f79535-47bb-0310-9956-ffa450edef68
2013-06-13 17:49:14 +00:00
Vinod Kumar Vavilapalli
0928502029
YARN-530. Defined Service model strictly, implemented AbstractService for robust subclassing and migrated yarn-common services. Contributed by Steve Loughran.
...
YARN-117. Migrated rest of YARN to the new service model. Contributed by Steve Louhran.
MAPREDUCE-5298. Moved MapReduce services to YARN-530 stricter lifecycle. Contributed by Steve Loughran.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492718 13f79535-47bb-0310-9956-ffa450edef68
2013-06-13 15:54:38 +00:00
Alejandro Abdelnur
80eb92aff0
YARN-600. Hook up cgroups CPU settings to the number of virtual cores allocated. (sandyr via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492365 13f79535-47bb-0310-9956-ffa450edef68
2013-06-12 18:57:31 +00:00
Jonathan Turner Eagles
b90755c296
YARN-427. Coverage fix for org.apache.hadoop.yarn.server.api.* (Aleksey Gorshkov via jeagles)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492282 13f79535-47bb-0310-9956-ffa450edef68
2013-06-12 16:52:15 +00:00
Alejandro Abdelnur
e75ee47fc9
YARN-795. Fair scheduler queue metrics should subtract allocated vCores from available vCores. (ywskycn via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492021 13f79535-47bb-0310-9956-ffa450edef68
2013-06-12 00:25:07 +00:00
Arun Murthy
25946da413
YARN-767. Initialize application metrics at RM bootup. Contributed by Jian He.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1491989 13f79535-47bb-0310-9956-ffa450edef68
2013-06-11 22:15:02 +00:00
Siddharth Seth
c6c41abf68
YARN-737. Throw some specific exceptions directly instead of wrapping them in YarnException. Contributed by Jian He.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1491896 13f79535-47bb-0310-9956-ffa450edef68
2013-06-11 18:47:16 +00:00
Vinod Kumar Vavilapalli
0b0be8d4dd
YARN-642. Removed health parameter from ResourceManager /nodes web-service and cleaned the behaviour of the status parameter. Contributed by Sandy Ryza.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1490890 13f79535-47bb-0310-9956-ffa450edef68
2013-06-08 00:54:22 +00:00
Alejandro Abdelnur
af8514eef2
YARN-686. Flatten NodeReport. (sandyr via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1490827 13f79535-47bb-0310-9956-ffa450edef68
2013-06-07 20:59:23 +00:00
Jason Darrell Lowe
a8c330fd33
YARN-760. NodeManager throws AvroRuntimeException on failed start. Contributed by Niranjan Singh
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1490704 13f79535-47bb-0310-9956-ffa450edef68
2013-06-07 16:20:20 +00:00
Bikas Saha
9fcfbf5f51
YARN-759. Create Command enum in AllocateResponse (bikas)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1490470 13f79535-47bb-0310-9956-ffa450edef68
2013-06-06 23:33:48 +00:00
Bikas Saha
2051fd5ee2
YARN-750. Allow for black-listing resources in YARN API and Impl in CS (acmurthy via bikas)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1490392 13f79535-47bb-0310-9956-ffa450edef68
2013-06-06 18:46:59 +00:00
Thomas Graves
98f55792ff
YARN-764. blank Used Resources on Capacity Scheduler page (Nemon Lou via tgraves)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1490387 13f79535-47bb-0310-9956-ffa450edef68
2013-06-06 18:25:36 +00:00
Vinod Kumar Vavilapalli
66de4ccf60
YARN-724. Moved ProtoBase from api.records to api.records.impl.pb. Contributed by Jian He.
...
MAPREDUCE-5303. Changed MR app after moving ProtoBase to package impl.pb via YARN-724. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489658 13f79535-47bb-0310-9956-ffa450edef68
2013-06-04 22:43:19 +00:00
Vinod Kumar Vavilapalli
af65d6f80e
YARN-756. Remnants from moving Preemption* records to yarn.api where they really belong. Contributed by Jian He.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489614 13f79535-47bb-0310-9956-ffa450edef68
2013-06-04 21:21:09 +00:00
Kihwal Lee
78063b3a76
YARN-742. Log aggregation causes a lot of redundant setPermission calls. Contributed by Jason Lowe.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489596 13f79535-47bb-0310-9956-ffa450edef68
2013-06-04 20:49:34 +00:00
Vinod Kumar Vavilapalli
978012b9b6
YARN-755. Renamed AllocateResponse.reboot to AllocateResponse.resync. Contributed by Bikas Saha.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489295 13f79535-47bb-0310-9956-ffa450edef68
2013-06-04 05:53:47 +00:00
Vinod Kumar Vavilapalli
28aabe0b2b
YARN-756. Move Preemption* records to yarn.api where they really belong. Contributed by Jian He.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489290 13f79535-47bb-0310-9956-ffa450edef68
2013-06-04 05:03:51 +00:00
Vinod Kumar Vavilapalli
a83fb61ac0
YARN-635. Renamed YarnRemoteException to YarnException. Contributed by Siddharth Seth.
...
MAPREDUCE-5301. Updated MR code to work with YARN-635 changes of renaming YarnRemoteException to YarnException. Contributed by Siddharth Seth
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489283 13f79535-47bb-0310-9956-ffa450edef68
2013-06-04 04:05:50 +00:00
Vinod Kumar Vavilapalli
c71fb196b9
YARN-757. Changed TestRMRestart to use the default scheduler to avoid test failures. Contributed by Bikas Saha.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489268 13f79535-47bb-0310-9956-ffa450edef68
2013-06-04 01:14:22 +00:00
Vinod Kumar Vavilapalli
d33534c4fb
YARN-748. Moved BuilderUtils from yarn-common to yarn-server-common for eventual retirement. Contributed by Jian He.
...
MAPREDUCE-5297. Updated MR App since BuilderUtils is no longer public after YARN-748. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489257 13f79535-47bb-0310-9956-ffa450edef68
2013-06-04 00:34:18 +00:00
Arun Murthy
5cf48b4612
YARN-726. Fix queue & finish time fields in web-ui for ResourceManager. Contributed by Mayank Bansal.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489234 13f79535-47bb-0310-9956-ffa450edef68
2013-06-03 23:50:19 +00:00
Arun Murthy
4a14264ddb
YARN-398. Make it possible to specify hard locality constraints in resource requests for CapacityScheduler. Contributed by Arun C. Murthy.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489087 13f79535-47bb-0310-9956-ffa450edef68
2013-06-03 18:13:53 +00:00
Alejandro Abdelnur
c1b635ed48
YARN-326. Add multi-resource scheduling to the fair scheduler. (sandyr via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489070 13f79535-47bb-0310-9956-ffa450edef68
2013-06-03 17:33:55 +00:00
Siddharth Seth
a2205a3b72
YARN-720 and MAPREDUCE-5291. container-log4j.properties should not refer to mapreduce properties. Update MRApp to use YARN properties for log setup. Contributed by Zhijie Shen.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488829 13f79535-47bb-0310-9956-ffa450edef68
2013-06-03 02:35:34 +00:00
Arun Murthy
f27e484636
YARN-749. Rename ResourceRequest.(get,set)HostName to ResourceRequest.(get,set)ResourceName. Contributed by Arun C. Murthy.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488806 13f79535-47bb-0310-9956-ffa450edef68
2013-06-02 22:53:58 +00:00
Vinod Kumar Vavilapalli
a2c4233004
YARN-717. Put object creation factories for Token in the class itself and remove useless derivations for specific tokens. Contributed by Jian He.
...
MAPREDUCE-5289. Updated MR App to use Token directly after YARN-717. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488616 13f79535-47bb-0310-9956-ffa450edef68
2013-06-01 21:43:17 +00:00
Siddharth Seth
39f019f413
YARN-735. Make ApplicationAttemptId, ContainerId and NodeId immutable. Contributed by Jian He.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488439 13f79535-47bb-0310-9956-ffa450edef68
2013-06-01 00:14:31 +00:00
Alejandro Abdelnur
781e82ca9a
YARN-392. Make it possible to specify hard locality constraints in resource requests. (sandyr via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488326 13f79535-47bb-0310-9956-ffa450edef68
2013-05-31 17:26:18 +00:00
Siddharth Seth
2692675fc3
YARN-684. ContainerManager.startContainer should use ContainerTokenIdentifier instead of the entire Container. Contributed by Vinod Kumar Vavilapalli.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488085 13f79535-47bb-0310-9956-ffa450edef68
2013-05-31 04:13:16 +00:00
Vinod Kumar Vavilapalli
982753dc8e
YARN-716. Making ApplicationID immutable. Contributed by Siddharth Seth.
...
MAPREDUCE-5282. Updating MR App to use immutable ApplicationID after YARN-716. Contributed by Siddharth Seth.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1487994 13f79535-47bb-0310-9956-ffa450edef68
2013-05-30 20:18:39 +00:00
Siddharth Seth
b16c5638b5
YARN-719. Move RMIdentifier from Container to ContainerTokenIdentifier. Contributed by Vinod Kumar Vavilapalli.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1487741 13f79535-47bb-0310-9956-ffa450edef68
2013-05-30 04:59:05 +00:00
Vinod Kumar Vavilapalli
29902cd53c
YARN-638. Modified ResourceManager to restore RMDelegationTokens after restarting. Contributed by Jian He.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1487720 13f79535-47bb-0310-9956-ffa450edef68
2013-05-30 04:14:26 +00:00
Vinod Kumar Vavilapalli
5420f287cc
YARN-578. Fixed NM to use SecureIOUtils for reading and aggregating logs. Contributed by Omkar Vinit Joshi.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1487672 13f79535-47bb-0310-9956-ffa450edef68
2013-05-29 23:14:59 +00:00
Jason Darrell Lowe
31c96829c7
YARN-512. Log aggregation root directory check is more expensive than it needs to be. Contributed by Maysam Yabandeh
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1487498 13f79535-47bb-0310-9956-ffa450edef68
2013-05-29 14:25:04 +00:00
Vinod Kumar Vavilapalli
259edf8dca
YARN-571. Remove user from ContainerLaunchContext. Contributed by Omkar Vinit Joshi.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1485928 13f79535-47bb-0310-9956-ffa450edef68
2013-05-24 03:22:54 +00:00
Vinod Kumar Vavilapalli
61885df2fa
YARN-711. Copied BuilderUtil methods in individual API records as BuilderUtils is going to be dismantled. Contributed by Jian He.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1485888 13f79535-47bb-0310-9956-ffa450edef68
2013-05-23 22:47:33 +00:00
Vinod Kumar Vavilapalli
43876770d9
YARN-563. Add the concept of an application-type for each application. Contributed by Mayank Bansal.
...
MAPREDUCE-5246. Specify application-type at the time of job submission after YARN-563. Contributed by Mayank Bansal.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1485790 13f79535-47bb-0310-9956-ffa450edef68
2013-05-23 18:02:04 +00:00
Vinod Kumar Vavilapalli
1fac1ac98c
YARN-708. Moved RecordFactory classes to hadoop-yarn-api, and put some miscellaneous fixes to the interfaces. Contributed by Siddharth Seth.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1485478 13f79535-47bb-0310-9956-ffa450edef68
2013-05-22 22:51:04 +00:00
Christopher Douglas
f9ab715880
YARN-568. Followup; remove an unused field added to Allocation
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1484375 13f79535-47bb-0310-9956-ffa450edef68
2013-05-20 05:46:43 +00:00
Vinod Kumar Vavilapalli
ca80246731
YARN-617. Made ContainerTokens to be used for validation at NodeManager also in unsecure mode to prevent AMs from faking resource requirements in unsecure mode. Contributed by Omkar Vinit Joshi.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1483667 13f79535-47bb-0310-9956-ffa450edef68
2013-05-17 06:36:33 +00:00
Robert Joseph Evans
3641f938f6
YARN-690. RM exits on token cancel/renew problems (daryn via bobby)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1483578 13f79535-47bb-0310-9956-ffa450edef68
2013-05-16 21:41:49 +00:00
Vinod Kumar Vavilapalli
065747efab
YARN-628. Fix the way YarnRemoteException is being unrolled to extract out the underlying exception. Contributed by Siddharth Seth.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1483207 13f79535-47bb-0310-9956-ffa450edef68
2013-05-16 06:58:34 +00:00
Siddharth Seth
1a119f87b4
YARN-615. Rename ContainerLaunchContext.containerTokens to tokens. Contributed by Vinod Kumar Vavilapalli.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1482199 13f79535-47bb-0310-9956-ffa450edef68
2013-05-14 04:11:55 +00:00
Vinod Kumar Vavilapalli
68148989bf
YARN-590. Added an optional mesage to be returned by ResourceMaanger when RM asks an RM to shutdown/resync etc so that NMs can log this message locally for better debuggability. Contributed by Mayank Bansal.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1481234 13f79535-47bb-0310-9956-ffa450edef68
2013-05-10 23:37:44 +00:00
Vinod Kumar Vavilapalli
a35c7fd80b
YARN-663. Changed ResourceTracker API and LocalizationProtocol API to throw YarnRemoteException and IOException. Contributed by Xuan Gong.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1481215 13f79535-47bb-0310-9956-ffa450edef68
2013-05-10 21:58:59 +00:00
Vinod Kumar Vavilapalli
2638bc67a4
YARN-634. Modified YarnRemoteException to be not backed by PB and introduced a separate SerializedException record. Contributed by Siddharth Seth.
...
MAPREDUCE-5239. Updated MR App to reflect YarnRemoteException changes after YARN-634. Contributed by Siddharth Seth.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1481205 13f79535-47bb-0310-9956-ffa450edef68
2013-05-10 21:49:28 +00:00
Alejandro Abdelnur
f19465f72f
YARN-598. Add virtual cores to queue metrics. (sandyr via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480816 13f79535-47bb-0310-9956-ffa450edef68
2013-05-09 22:23:50 +00:00
Alejandro Abdelnur
51ccb87031
YARN-655. Fair scheduler metrics should subtract allocated memory from available memory. (sandyr via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480809 13f79535-47bb-0310-9956-ffa450edef68
2013-05-09 22:15:58 +00:00
Alejandro Abdelnur
e0562e3d07
YARN-637. FS: maxAssign is not honored. (kkambatl via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480802 13f79535-47bb-0310-9956-ffa450edef68
2013-05-09 21:52:03 +00:00
Alejandro Abdelnur
c7bfd17ad1
YARN-507. Add interface visibility and stability annotations to FS interfaces/classes. (kkambatl via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480799 13f79535-47bb-0310-9956-ffa450edef68
2013-05-09 21:47:21 +00:00
Christopher Douglas
505fe26539
YARN-568. Add support for work preserving preemption to the FairScheduler.
...
Contributed by Carlo Curino and Sandy Ryza
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480778 13f79535-47bb-0310-9956-ffa450edef68
2013-05-09 21:21:42 +00:00
Vinod Kumar Vavilapalli
9c4f86879c
YARN-630. Changed AMRMProtocol api to throw IOException and YarnRemoteException. Contributed by Xuan Gong.
...
MAPREDUCE-5226. Handling YarnRemoteException separately from IOException in MR App's use of AMRMProtocol after YARN-630. Contributed by Xuan Gong.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480529 13f79535-47bb-0310-9956-ffa450edef68
2013-05-09 05:32:04 +00:00
Vinod Kumar Vavilapalli
0727ecaf50
YARN-631. Changed ClientRMProtocol api to throw IOException and YarnRemoteException. Contributed by Xuan Gong.
...
MAPREDUCE-5212. Handling YarnRemoteException separately from IOException in MR App after YARN-631. Contributed by Xuan Gong.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480494 13f79535-47bb-0310-9956-ffa450edef68
2013-05-08 22:53:01 +00:00
Vinod Kumar Vavilapalli
74231f0276
YARN-582. Changed ResourceManager to recover Application token and client tokens for app attempt so that RM can be restarted while preserving current applications. Contributed by Jian He.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480168 13f79535-47bb-0310-9956-ffa450edef68
2013-05-08 06:20:22 +00:00
Vinod Kumar Vavilapalli
9eb53b9c84
YARN-632. Changed ContainerManager api to throw IOException and YarnRemoteException. Contributed by Xuan Gong.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1479740 13f79535-47bb-0310-9956-ffa450edef68
2013-05-07 00:42:48 +00:00
Vinod Kumar Vavilapalli
92b7165a71
YARN-629. Make YarnRemoteException not be rooted at IOException. Contributed by Xuan Gong.
...
MAPREDUCE-5204. Handling YarnRemoteException separately from IOException in MR app after YARN-629. Contributed by Xuan Gong.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1479680 13f79535-47bb-0310-9956-ffa450edef68
2013-05-06 19:04:31 +00:00
Vinod Kumar Vavilapalli
c0fa9d2231
YARN-645. Moved RMDelegationTokenSecretManager from yarn-server-common to yarn-server-resourcemanager where it really belongs. Contributed by Jian He.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1479022 13f79535-47bb-0310-9956-ffa450edef68
2013-05-04 01:12:17 +00:00
Vinod Kumar Vavilapalli
fc275fed66
YARN-625. Move the utility method unwrapAndThrowException from YarnRemoteExceptionPBImpl to RPCUtil. Contributed by Siddharth Seth.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1478594 13f79535-47bb-0310-9956-ffa450edef68
2013-05-02 22:44:12 +00:00
Vinod Kumar Vavilapalli
46b2898a1a
YARN-618. Modified RM_INVALID_IDENTIFIER to be -1 instead of zero. Contributed by Jian He.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1478230 13f79535-47bb-0310-9956-ffa450edef68
2013-05-01 23:23:20 +00:00
Vinod Kumar Vavilapalli
6de09af244
YARN-599. Refactoring submitApplication in ClientRMService and RMAppManager to separate out various validation checks depending on whether they rely on RM configuration or not. Contributed by Zhijie Shen.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1477478 13f79535-47bb-0310-9956-ffa450edef68
2013-04-30 05:44:06 +00:00
Suresh Srinivas
27d4592771
YARN-506. Move to common utils FileUtil#setReadable/Writable/Executable and FileUtil#canRead/Write/Execute. Contributed by Ivan Mitic.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1477408 13f79535-47bb-0310-9956-ffa450edef68
2013-04-29 23:09:28 +00:00
Vinod Kumar Vavilapalli
d9ce42479e
YARN-576. Modified ResourceManager to reject NodeManagers that don't satisy minimum resource requirements. Contributed by Kenji Kikushima.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476824 13f79535-47bb-0310-9956-ffa450edef68
2013-04-28 20:03:37 +00:00
Suresh Srinivas
fe3853484e
HADOOP-9290. Some tests cannot load native library on windows. Contributed by Chris Nauroth.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476586 13f79535-47bb-0310-9956-ffa450edef68
2013-04-27 14:24:02 +00:00
Vinod Kumar Vavilapalli
95fa3e82e4
YARN-363. Add webapps/proxy directory without which YARN proxy-server fails when started in stand-alone mode. Contributed by Kenji Kikushima.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476330 13f79535-47bb-0310-9956-ffa450edef68
2013-04-26 18:24:30 +00:00
Vinod Kumar Vavilapalli
f0ac5a849c
YARN-591. Moved RM recovery related records out of public API as they do not belong there. Contributed by Vinod Kumar Vavilapalli.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476324 13f79535-47bb-0310-9956-ffa450edef68
2013-04-26 18:02:45 +00:00
Vinod Kumar Vavilapalli
635f96e74e
YARN-562. Missed files from previous commit.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476038 13f79535-47bb-0310-9956-ffa450edef68
2013-04-26 04:42:34 +00:00
Vinod Kumar Vavilapalli
fbb55784d9
YARN-562. Modified NM to reject any containers allocated by a previous ResourceManager. Contributed by Jian He.
...
MAPREDUCE-5167. Update MR App after YARN-562 to use the new builder API for the container. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476034 13f79535-47bb-0310-9956-ffa450edef68
2013-04-26 03:50:20 +00:00
Thomas White
d10428cab2
YARN-289. Fair scheduler allows reservations that won't fit on node. Contributed by Sandy Ryza.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1475681 13f79535-47bb-0310-9956-ffa450edef68
2013-04-25 09:15:51 +00:00
Thomas White
a43c344eee
YARN-595. Refactor fair scheduler to use common Resources. Contributed by Sandy Ryza.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1475670 13f79535-47bb-0310-9956-ffa450edef68
2013-04-25 08:33:43 +00:00
Vinod Kumar Vavilapalli
448df18646
YARN-577. Add application-progress also to ApplicationReport. Contributed by Hitesh Shah.
...
MAPREDUCE-5178. Update MR App to set progress in ApplicationReport after YARN-577. Contributed by Hitesh Shah.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1475636 13f79535-47bb-0310-9956-ffa450edef68
2013-04-25 05:42:22 +00:00
Siddharth Seth
be256a1819
YARN-579. Stop setting the Application Token in the AppMaster env, in favour of the copy present in the container token field. Contributed by Vinod Kumar Vavilapalli.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471814 13f79535-47bb-0310-9956-ffa450edef68
2013-04-25 02:46:47 +00:00
Vinod Kumar Vavilapalli
a2f97fd9cb
YARN-605. Fix failing unit test in TestNMWebServices when versionInfo has parantheses like when running on a git checkout. Contributed by Hitesh Shah.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471608 13f79535-47bb-0310-9956-ffa450edef68
2013-04-24 19:42:47 +00:00
Vinod Kumar Vavilapalli
cd975d4b64
YARN-581. Added a test to verify that app delegation tokens are restored after RM restart. Contributed by Jian He.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471187 13f79535-47bb-0310-9956-ffa450edef68
2013-04-23 22:49:44 +00:00
Vinod Kumar Vavilapalli
27e8c86999
YARN-561. Modified NodeManager to set key information into the environment of every container that it launches. Contributed by Xuan Gong.
...
MAPREDUCE-5175. Updated MR App to not set envs that will be set by NMs anyways after YARN-561. Contributed by Xuan Gong.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471156 13f79535-47bb-0310-9956-ffa450edef68
2013-04-23 21:39:51 +00:00
Vinod Kumar Vavilapalli
ffd7dbb34a
YARN-583. Moved application level local resources to be localized under the filecache sub-directory under application directory. Contributed by Omkar Vinit Joshi.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1470812 13f79535-47bb-0310-9956-ffa450edef68
2013-04-23 04:37:45 +00:00
Bikas Saha
e24a9b83f1
YARN-594. Update test and add comments in YARN-534 (Jian He via bikas)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1470243 13f79535-47bb-0310-9956-ffa450edef68
2013-04-20 22:04:08 +00:00
Vinod Kumar Vavilapalli
a91067fc5e
YARN-542. Changed the default global AM max-attempts value to be not one. Contributed by Zhijie Shen.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1470094 13f79535-47bb-0310-9956-ffa450edef68
2013-04-20 00:35:29 +00:00
Vinod Kumar Vavilapalli
c570309b07
YARN-547. Fixed race conditions in public and private resource localization which used to cause duplicate downloads. Contributed by Omkar Vinit Joshi.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1470076 13f79535-47bb-0310-9956-ffa450edef68
2013-04-19 22:35:43 +00:00
Bikas Saha
edcfd4527c
YARN-593. container launch on Windows does not correctly populate classpath with new process's environment variables and localized resources (Chris Nauroth via bikas)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1469998 13f79535-47bb-0310-9956-ffa450edef68
2013-04-19 19:29:22 +00:00
Vinod Kumar Vavilapalli
44bf8525a5
YARN-493. Fixed some shell related flaws in YARN on Windows. Contributed by Chris Nauroth.
...
HADOOP-9486. Promoted Windows and Shell related utils from YARN to Hadoop Common. Contributed by Chris Nauroth.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1469667 13f79535-47bb-0310-9956-ffa450edef68
2013-04-19 02:14:58 +00:00
Vinod Kumar Vavilapalli
8e1c2823fc
YARN-441. Removed unused utility methods for collections from two API records. Contributed by Xuan Gong.
...
MAPREDUCE-5163. Update MR App to not use API utility methods for collections after YARN-441. Contributed by Xuan Gong.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1469657 13f79535-47bb-0310-9956-ffa450edef68
2013-04-19 01:33:13 +00:00
Alejandro Abdelnur
df55edd5de
YARN-482. FS: Extend SchedulingMode to intermediate queues. (kkambatl via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1469506 13f79535-47bb-0310-9956-ffa450edef68
2013-04-18 18:11:21 +00:00
Vinod Kumar Vavilapalli
d219c904d3
YARN-585. Fix failure in TestFairScheduler#testNotAllowSubmitApplication caused by YARN-514. Contributed by Zhijie Shen.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1469492 13f79535-47bb-0310-9956-ffa450edef68
2013-04-18 17:49:22 +00:00
Bikas Saha
ef9f251679
YARN-514.Delayed store operations should not result in RM unavailability for app submission (Zhijie Shen via bikas)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1469059 13f79535-47bb-0310-9956-ffa450edef68
2013-04-17 20:19:43 +00:00
Vinod Kumar Vavilapalli
0e01f26821
YARN-444. Moved special container exit codes from YarnConfiguration to API where they belong. Contributed by Sandy Ryza.
...
MAPREDUCE-5151. Updated MR AM to use standard exit codes from the API after YARN-444. Contributed by Sandy Ryza.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1468276 13f79535-47bb-0310-9956-ffa450edef68
2013-04-15 23:54:09 +00:00
Arun Murthy
c2592021f3
YARN-412. Fixed FifoScheduler to check hostname of a NodeManager rather than its host:port during scheduling which caused incorrect locality for containers. Contributed by Roger Hoover.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1467244 13f79535-47bb-0310-9956-ffa450edef68
2013-04-12 12:12:08 +00:00
Vinod Kumar Vavilapalli
e4c55e17fe
YARN-486. Changed NM's startContainer API to accept Container record given by RM as a direct parameter instead of as part of the ContainerLaunchContext record. Contributed by Xuan Gong.
...
MAPREDUCE-5139. Update MR AM to use the modified startContainer API after YARN-486. Contributed by Xuan Gong.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1467063 13f79535-47bb-0310-9956-ffa450edef68
2013-04-11 19:28:51 +00:00
Vinod Kumar Vavilapalli
4234bc87b3
YARN-539. Addressed memory leak of LocalResource objects NM when a resource localization fails. Contributed by Omkar Vinit Joshi.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1466756 13f79535-47bb-0310-9956-ffa450edef68
2013-04-11 02:08:11 +00:00
Vinod Kumar Vavilapalli
2e3b56f6e9
YARN-495. Changed NM reboot behaviour to be a simple resync - kill all containers and re-register with RM. Contributed by Jian He.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1466752 13f79535-47bb-0310-9956-ffa450edef68
2013-04-11 02:00:47 +00:00
Vinod Kumar Vavilapalli
add00d6d74
YARN-487. Modify path manipulation in LocalDirsHandlerService to let TestDiskFailures pass on Windows. Contributed by Chris Nauroth.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1466746 13f79535-47bb-0310-9956-ffa450edef68
2013-04-11 01:00:38 +00:00
Vinod Kumar Vavilapalli
7d00d3d20f
YARN-534. Change RM restart recovery to also account for AM max-attempts configuration after the restart. Contributed by Jian He.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1466208 13f79535-47bb-0310-9956-ffa450edef68
2013-04-09 20:16:19 +00:00
Vinod Kumar Vavilapalli
3a54a5653b
YARN-112. Fixed a race condition during localization that fails containers. Contributed by Omkar Vinit Joshi.
...
MAPREDUCE-5138. Fix LocalDistributedCacheManager after YARN-112. Contributed by Omkar Vinit Joshi.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1466196 13f79535-47bb-0310-9956-ffa450edef68
2013-04-09 19:56:10 +00:00
Vinod Kumar Vavilapalli
b96d18bd23
YARN-99. Modify private distributed cache to localize files such that no local directory hits unix file count limits and thus prevent job failures. Contributed by Omkar Vinit Joshi.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1465853 13f79535-47bb-0310-9956-ffa450edef68
2013-04-09 01:33:38 +00:00
Bikas Saha
16fedf5473
YARN-479. NM retry behavior for connection to RM should be similar for lost heartbeats (Jian He via bikas)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1465731 13f79535-47bb-0310-9956-ffa450edef68
2013-04-08 19:17:16 +00:00
Bikas Saha
bc6777dd5b
YARN-193. Scheduler.normalizeRequest does not account for allocation requests that exceed maximumAllocation limits (Zhijie Shen via bikas)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1465067 13f79535-47bb-0310-9956-ffa450edef68
2013-04-05 17:45:44 +00:00
Jason Darrell Lowe
18757c1267
YARN-525. make CS node-locality-delay refreshable. Contributed by Thomas Graves
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1465009 13f79535-47bb-0310-9956-ffa450edef68
2013-04-05 15:30:24 +00:00
Vinod Kumar Vavilapalli
0836d68abf
YARN-532. Change RMAdmin and Localization client protocol PB implementations to implement closeable so that they can be stopped when needed via RPC.stopProxy(). Contributed by Siddharth Seth.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1464788 13f79535-47bb-0310-9956-ffa450edef68
2013-04-04 23:30:15 +00:00
Vinod Kumar Vavilapalli
1fd462b118
YARN-536. Removed the unused objects ContainerStatus and ContainerStatus from Container which also don't belong to the container. Contributed by Xuan Gong.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1464271 13f79535-47bb-0310-9956-ffa450edef68
2013-04-04 03:41:18 +00:00
Eli Collins
2d02b7741d
Revert YARN-516 per HADOOP-9357.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1464181 13f79535-47bb-0310-9956-ffa450edef68
2013-04-03 19:42:12 +00:00
Vinod Kumar Vavilapalli
3e9200ddde
YARN-101. Fix NodeManager heartbeat processing to not lose track of completed containers in case of dropped heartbeats. Contributed by Xuan Gong.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1464105 13f79535-47bb-0310-9956-ffa450edef68
2013-04-03 16:57:07 +00:00
Vinod Kumar Vavilapalli
e67e3ff05d
YARN-467. Modify public distributed cache to localize files such that no local directory hits unix file count limits and thus prevent job failures. Contributed by Omkar Vinit Joshi.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1463823 13f79535-47bb-0310-9956-ffa450edef68
2013-04-03 05:00:28 +00:00
Bikas Saha
d40b154d27
YARN-382. SchedulerUtils improve way normalizeRequest sets the resource capabilities (Zhijie Shen via bikas)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1463653 13f79535-47bb-0310-9956-ffa450edef68
2013-04-02 18:07:07 +00:00
Vinod Kumar Vavilapalli
bcce9354bc
YARN-447. Move ApplicationComparator in CapacityScheduler to use comparator in ApplicationId. Contributed by Nemon Lou.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1463405 13f79535-47bb-0310-9956-ffa450edef68
2013-04-02 06:59:13 +00:00
Vinod Kumar Vavilapalli
17be263e9b
YARN-516. Fix failure in TestContainerLocalizer caused by HADOOP-9357. Contributed by Andrew Wang.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1463362 13f79535-47bb-0310-9956-ffa450edef68
2013-04-02 02:16:35 +00:00
Vinod Kumar Vavilapalli
7ca9fe73a0
YARN-309. Changed NodeManager to obtain heart-beat interval from the ResourceManager. Contributed by Xuan Gong.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1463346 13f79535-47bb-0310-9956-ffa450edef68
2013-04-02 01:04:07 +00:00
Jason Darrell Lowe
2fd56e33b4
YARN-515. Node Manager not getting the master key. Contributed by Robert Joseph Evans
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1462632 13f79535-47bb-0310-9956-ffa450edef68
2013-03-29 20:20:19 +00:00
Thomas Graves
01aabf7363
YARN-460. CS user left in list of active users for the queue even when application finished (tgraves)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1462486 13f79535-47bb-0310-9956-ffa450edef68
2013-03-29 14:36:53 +00:00
Bikas Saha
520033b1cd
YARN-450. Define value for * in the scheduling protocol (Zhijie Shen via bikas)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1462271 13f79535-47bb-0310-9956-ffa450edef68
2013-03-28 19:44:28 +00:00
Alejandro Abdelnur
967b62f5d2
YARN-24. Nodemanager fails to start if log aggregation enabled and namenode unavailable. (sandyr via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1461891 13f79535-47bb-0310-9956-ffa450edef68
2013-03-27 23:14:05 +00:00
Vinod Kumar Vavilapalli
d0bbff6c32
YARN-209. Fix CapacityScheduler to trigger application-activation when the cluster capacity changes. Contributed by Zhijie Shen.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1461773 13f79535-47bb-0310-9956-ffa450edef68
2013-03-27 18:38:28 +00:00
Thomas White
414458ff9d
YARN-496. Fair scheduler configs are refreshed inconsistently in reinitialize. Contributed by Sandy Ryza
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1461614 13f79535-47bb-0310-9956-ffa450edef68
2013-03-27 14:42:41 +00:00
Vinod Kumar Vavilapalli
d60c2fa17f
YARN-474. Fix CapacityScheduler to trigger application-activation when am-resource-percent configuration is refreshed. Contributed by Zhijie Shen.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1461402 13f79535-47bb-0310-9956-ffa450edef68
2013-03-27 03:44:34 +00:00
Siddharth Seth
66e90b205a
YARN-440. Flatten RegisterNodeManagerResponse. Contributed by Xuan Gong.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1461256 13f79535-47bb-0310-9956-ffa450edef68
2013-03-26 18:20:36 +00:00
Alejandro Abdelnur
e74d1f0435
YARN-469. Make scheduling mode in FS pluggable. (kkambatl via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1460961 13f79535-47bb-0310-9956-ffa450edef68
2013-03-26 03:25:26 +00:00
Vinod Kumar Vavilapalli
0757364014
YARN-378. Fix RM to make the AM max attempts/retries to be configurable per application by clients. Contributed by Zhijie Shen.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1460895 13f79535-47bb-0310-9956-ffa450edef68
2013-03-25 21:11:33 +00:00
Siddharth Seth
18e08a8f51
YARN-439. Flatten NodeHeartbeatResponse. Contributed by Xuan Gong.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1460811 13f79535-47bb-0310-9956-ffa450edef68
2013-03-25 18:28:50 +00:00
Siddharth Seth
6a482a88b8
YARN-71. Fix the NodeManager to clean up local-dirs on restart. Contributed by Xuan Gong.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1460808 13f79535-47bb-0310-9956-ffa450edef68
2013-03-25 18:23:59 +00:00
Hitesh Shah
b15b44e722
YARN-470. Support a way to disable resource monitoring on the NodeManager. Contributed by Siddharth Seth.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1460001 13f79535-47bb-0310-9956-ffa450edef68
2013-03-22 21:28:31 +00:00
Hitesh Shah
0cda8267d5
YARN-491. TestContainerLogsPage fails on Windows. Contributed by Chris Nauroth.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1459526 13f79535-47bb-0310-9956-ffa450edef68
2013-03-21 21:04:59 +00:00
Hitesh Shah
92fb7cdcde
YARN-488. TestContainerManagerSecurity fails on Windows. Contributed by Chris Nauroth.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1459514 13f79535-47bb-0310-9956-ffa450edef68
2013-03-21 20:38:43 +00:00
Hitesh Shah
1bd345d6e3
YARN-396. Rationalize AllocateResponse in RM Scheduler API. Contributed by Zhijie Shen.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1459040 13f79535-47bb-0310-9956-ffa450edef68
2013-03-20 20:44:35 +00:00
Jason Darrell Lowe
67699c2d18
YARN-200. yarn log does not output all needed information, and is in a binary format. Contributed by Ravi Prakash
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1458466 13f79535-47bb-0310-9956-ffa450edef68
2013-03-19 19:29:15 +00:00
Hitesh Shah
2ba66a54f5
YARN-196. Nodemanager should be more robust in handling connection failure to ResourceManager when a cluster is started. Contributed by Xuan Gong.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1457038 13f79535-47bb-0310-9956-ffa450edef68
2013-03-15 17:58:58 +00:00
Robert Joseph Evans
9ec88e8e29
YARN-468. coverage fix for org.apache.hadoop.yarn.server.webproxy.amfilter (Aleksey Gorshkov via bobby)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1456458 13f79535-47bb-0310-9956-ffa450edef68
2013-03-14 14:52:33 +00:00
Vinod Kumar Vavilapalli
44eb4c31b8
YARN-198. Added a link to RM pages from the NodeManager web app. Contributed by Jian He.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1455800 13f79535-47bb-0310-9956-ffa450edef68
2013-03-13 04:24:35 +00:00
Thomas Graves
5f2c518c95
YARN-443. allow OS scheduling priority of NM to be different than the containers it launches (tgraves)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1454411 13f79535-47bb-0310-9956-ffa450edef68
2013-03-08 14:46:26 +00:00
Hitesh Shah
2e4382b7c3
YARN-429. capacity-scheduler config missing from yarn-test artifact. Contributed by Siddharth Seth.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1453501 13f79535-47bb-0310-9956-ffa450edef68
2013-03-06 19:26:57 +00:00
Suresh Srinivas
638801cce1
HADOOP-8952. Enhancements to support Hadoop on Windows Server and Windows Azure environments. Contributed by Ivan Mitic, Chuan Liu, Ramya Sunil, Bikas Saha, Kanna Karanam, John Gordon, Brandon Li, Chris Nauroth, David Lao, Sumadhur Reddy Bolli, Arpit Agarwal, Ahmed El Baz, Mike Liddell, Jing Zhao, Thejas Nair, Steve Maine, Ganeshan Iyer, Raja Aluri, Giridharan Kesavan, Ramya Bharathi Nimmagadda.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1453486 13f79535-47bb-0310-9956-ffa450edef68
2013-03-06 19:15:18 +00:00
Jonathan Turner Eagles
21066255da
YARN-227. Application expiration difficult to debug for end-users (Jason Lowe via jeagles)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1453080 13f79535-47bb-0310-9956-ffa450edef68
2013-03-05 23:12:17 +00:00
Jason Darrell Lowe
5889f54ad1
YARN-345. Many InvalidStateTransitonException errors for ApplicationImpl in Node Manager. Contributed by Robert Parker
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1452548 13f79535-47bb-0310-9956-ffa450edef68
2013-03-04 21:47:39 +00:00
Siddharth Seth
83d8065867
YARN-376. Fixes a bug which would prevent the NM knowing about completed containers and applications. Contributed by Jason Lowe.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1451473 13f79535-47bb-0310-9956-ffa450edef68
2013-03-01 05:59:54 +00:00
Alejandro Abdelnur
5d6eca08bd
MAPREDUCE-4896. mapred queue -info spits out ugly exception when queue does not exist. (sandyr via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1451452 13f79535-47bb-0310-9956-ffa450edef68
2013-03-01 01:28:18 +00:00
Kihwal Lee
27c8a87a54
YARN-269. Resource Manager not logging the health_check_script result when taking it out. Contributed by Jason Lowe
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1451341 13f79535-47bb-0310-9956-ffa450edef68
2013-02-28 21:02:06 +00:00
Robert Joseph Evans
0057600a82
YARN-426. Failure to download a public resource prevents further downloads (Jason Lowe via bobby)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1450807 13f79535-47bb-0310-9956-ffa450edef68
2013-02-27 15:30:10 +00:00
Siddharth Seth
7ea82ca558
YARN-365. Change NM heartbeat handling to not generate a scheduler event on each heartbeat. (Contributed by Xuan Gong)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1450007 13f79535-47bb-0310-9956-ffa450edef68
2013-02-26 03:32:10 +00:00
Jonathan Turner Eagles
320f7bdcb8
YARN-236. RM should point tracking URL to RM web page when app fails to start (Jason Lowe via jeagles)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1448406 13f79535-47bb-0310-9956-ffa450edef68
2013-02-20 20:57:22 +00:00
Thomas Graves
efaaf58605
YARN-400. RM can return null application resource usage report leading to NPE in client (Jason Lowe via tgraves)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1448241 13f79535-47bb-0310-9956-ffa450edef68
2013-02-20 15:35:06 +00:00
Alejandro Abdelnur
6bcbf492a2
HADOOP-9117. replace protoc ant plugin exec with a maven plugin. (tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1445956 13f79535-47bb-0310-9956-ffa450edef68
2013-02-13 22:13:44 +00:00
Siddharth Seth
7bc6040ebb
YARN-391. Formatting fixes for LCEResourceHandler classes. Contributed by Steve Loughran)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1445882 13f79535-47bb-0310-9956-ffa450edef68
2013-02-13 19:40:36 +00:00
Thomas Graves
666667464a
YARN-249. Capacity Scheduler web page should show list of active users per queue like it used to (in 1.x) (Ravi Prakash via tgraves)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1445673 13f79535-47bb-0310-9956-ffa450edef68
2013-02-13 15:40:01 +00:00
Tsz-wo Sze
a63d50f79a
YARN-377. Use the new StringUtils methods added by HADOOP-9252 and fix TestContainersMonitor. Contributed by Chris Nauroth
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1443796 13f79535-47bb-0310-9956-ffa450edef68
2013-02-07 23:53:49 +00:00
Siddharth Seth
ab16a37572
YARN-355. Fixes a bug where RM app submission could jam under load. Contributed by Daryn Sharp.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1443131 13f79535-47bb-0310-9956-ffa450edef68
2013-02-06 19:03:52 +00:00
Daryn Sharp
d272918426
YARN-357. App submission should not be synchronized (daryn)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1443016 13f79535-47bb-0310-9956-ffa450edef68
2013-02-06 15:26:30 +00:00
Arun Murthy
38bb4ad738
YARN-370. Fix SchedulerUtils to correctly round up the resource for containers. Contributed by Zhijie Shen.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1442840 13f79535-47bb-0310-9956-ffa450edef68
2013-02-06 04:24:56 +00:00
Siddharth Seth
5d679c4f43
YARN-360. Allow apps to concurrently register tokens for renewal. Contributed by Daryn Sharp.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1442441 13f79535-47bb-0310-9956-ffa450edef68
2013-02-05 00:26:16 +00:00
Hitesh Shah
b1baf11fb2
YARN-372. Move InlineDispatcher from hadoop-yarn-server-resourcemanager to hadoop-yarn-common. Contributed by Siddharth Seth.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1441668 13f79535-47bb-0310-9956-ffa450edef68
2013-02-02 00:43:54 +00:00
Thomas Graves
6384684299
YARN-343. Capacity Scheduler maximum-capacity value -1 is invalid (Xuan Gong via tgraves)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1440457 13f79535-47bb-0310-9956-ffa450edef68
2013-01-30 15:15:23 +00:00
Jason Darrell Lowe
2e9e7601e1
YARN-354. WebAppProxyServer exits immediately after startup. Contributed by Liang Xie
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1437775 13f79535-47bb-0310-9956-ffa450edef68
2013-01-23 22:05:55 +00:00
Thomas White
345bcee066
YARN-319. Submitting a job to a fair scheduler queue for which the user does not have permission causes the client to wait forever. Contributed by shenhong.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1437336 13f79535-47bb-0310-9956-ffa450edef68
2013-01-23 10:53:05 +00:00
Hitesh Shah
29ddce96a2
YARN-231. RM Restart - Add FS-based persistent store implementation for RMStateStore. Contributed by Bikas Saha
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1437245 13f79535-47bb-0310-9956-ffa450edef68
2013-01-23 00:57:37 +00:00
Alejandro Abdelnur
f85b583731
YARN-302. Fair scheduler assignmultiple should default to false. (sandyr via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1434996 13f79535-47bb-0310-9956-ffa450edef68
2013-01-18 00:48:22 +00:00