big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
5,557 Commits 2 Branches 2 Tags 264 MiB
9eb06b31d6
Commit Graph

104 Commits

Author SHA1 Message Date
Sanford Ryza
9eb06b31d6 YARN-932. TestResourceLocalizationService.testLocalizationInit can fail on JDK7. (Karthik Kambatla via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1508209 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-29 22:06:28 +00:00
Vinod Kumar Vavilapalli
412e19f8e3 YARN-245. Fixed NodeManager to handle duplicate responses from ResourceManager. Contributed by Mayank Bansal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1508157 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-29 18:15:05 +00:00
Vinod Kumar Vavilapalli
db1e0c1eef YARN-688. Fixed NodeManager to properly cleanup containers when it is shut down. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1506814 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-25 04:14:25 +00:00
Vinod Kumar Vavilapalli
3ca3137179 YARN-926. Modified ContainerManagerProtcol APIs to take in requests for multiple containers. Contributed by Jian He. 
MAPREDUCE-5412. Update MR app to use multiple containers API of ContainerManager after YARN-926. Contributed by Jian He.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1506391 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-24 03:41:03 +00:00
Vinod Kumar Vavilapalli
7a29bccd7a YARN-814. Improving diagnostics when containers fail during launch due to various reasons like invalid env etc. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1504732 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-19 00:28:18 +00:00
Vinod Kumar Vavilapalli
c6facfb10d YARN-912. Move client facing exceptions to yarn-api module. Contributed by Mayank Bansal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1504032 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-17 07:31:33 +00:00
Vinod Kumar Vavilapalli
8b9c1e68ab YARN-62. Modified NodeManagers to avoid AMs from abusing container tokens for repetitive container launches. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1503986 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-17 04:24:44 +00:00
Vinod Kumar Vavilapalli
6d12709b19 YARN-820. Fixed an invalid state transition in NodeManager caused by failing resource localization. Contributed by Mayank Bansal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1503947 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-16 23:44:56 +00:00
Vinod Kumar Vavilapalli
af0d2fc3e3 YARN-661. Fixed NM to cleanup users' local directories correctly when starting up. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1503942 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-16 23:31:02 +00:00
Bikas Saha
33b5a81509 YARN-513. Create common proxy client for communicating with RM (Xuan Gong & Jian He via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1503933 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-16 22:47:40 +00:00
Vinod Kumar Vavilapalli
4466d8653f YARN-523. Modified a test-case to validate container diagnostics on localization failures. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1503532 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-16 00:57:41 +00:00
Chris Nauroth
1db4319072 YARN-909. Disable TestLinuxContainerExecutorWithMocks on Windows. Contributed by Chuan Liu. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1503357 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-15 17:27:31 +00:00
Chris Nauroth
31fff48ec4 YARN-894. NodeHealthScriptRunner timeout checking is inaccurate on Windows. Contributed by Chuan Liu. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1501016 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-08 23:40:43 +00:00
Hitesh Shah
575495b5b1 YARN-861. TestContainerManager is failing. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1495160 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-20 20:18:04 +00:00
Siddharth Seth
ab647cfcdb YARN-848. Fix NodeManager to register with RM using the fully qualified hostname. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494385 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-18 23:49:23 +00:00
Vinod Kumar Vavilapalli
243bcd367f YARN-694. Starting to use NMTokens to authenticate all communication with NodeManagers. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494369 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-18 23:19:49 +00:00
Vinod Kumar Vavilapalli
f4d80e91ae YARN-841. Move Auxiliary service to yarn-api, annotate and document it. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494031 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-18 06:20:37 +00:00
Chris Nauroth
b02dc5b464 YARN-839. TestContainerLaunch.testContainerEnvVariables fails on Windows. Contributed by Chuan Liu. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493937 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-17 21:05:00 +00:00
Vinod Kumar Vavilapalli
b9efe6bd4a HADOOP-9649. Promoted YARN service life-cycle libraries into Hadoop Common for usage across all Hadoop projects. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493650 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-17 06:39:33 +00:00
Arun Murthy
77e134d13f YARN-387. Renamed YARN protocols for consistency. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493623 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-17 02:27:18 +00:00
Vinod Kumar Vavilapalli
29a3708724 YARN-781. Exposing LOGDIR in all containers' environment which should be used by containers for logging purposes. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493428 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-15 23:09:24 +00:00
Vinod Kumar Vavilapalli
e9f753a11e YARN-792. Moved NodeHealthStatus from yarn.api.record to yarn.server.api.record. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492943 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-14 03:17:03 +00:00
Vinod Kumar Vavilapalli
f0eb4bc342 YARN-692. Creating NMToken master key on RM and sharing it with NM as a part of RM-NM heartbeat. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492907 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-14 00:06:42 +00:00
Vinod Kumar Vavilapalli
c0cd68d8fb YARN-773. Moved YarnRuntimeException from package api.yarn to api.yarn.exceptions. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492903 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-13 23:25:14 +00:00
Vinod Kumar Vavilapalli
0928502029 YARN-530. Defined Service model strictly, implemented AbstractService for robust subclassing and migrated yarn-common services. Contributed by Steve Loughran. 
YARN-117. Migrated rest of YARN to the new service model. Contributed by Steve Louhran.
MAPREDUCE-5298. Moved MapReduce services to YARN-530 stricter lifecycle. Contributed by Steve Loughran.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492718 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-13 15:54:38 +00:00
Siddharth Seth
c6c41abf68 YARN-737. Throw some specific exceptions directly instead of wrapping them in YarnException. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1491896 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-11 18:47:16 +00:00
Kihwal Lee
78063b3a76 YARN-742. Log aggregation causes a lot of redundant setPermission calls. Contributed by Jason Lowe. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489596 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-04 20:49:34 +00:00
Vinod Kumar Vavilapalli
a83fb61ac0 YARN-635. Renamed YarnRemoteException to YarnException. Contributed by Siddharth Seth. 
MAPREDUCE-5301. Updated MR code to work with YARN-635 changes of renaming YarnRemoteException to YarnException. Contributed by Siddharth Seth


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489283 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-04 04:05:50 +00:00
Vinod Kumar Vavilapalli
d33534c4fb YARN-748. Moved BuilderUtils from yarn-common to yarn-server-common for eventual retirement. Contributed by Jian He. 
MAPREDUCE-5297. Updated MR App since BuilderUtils is no longer public after YARN-748. Contributed by Jian He.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489257 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-04 00:34:18 +00:00
Vinod Kumar Vavilapalli
a2c4233004 YARN-717. Put object creation factories for Token in the class itself and remove useless derivations for specific tokens. Contributed by Jian He. 
MAPREDUCE-5289. Updated MR App to use Token directly after YARN-717. Contributed by Jian He.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488616 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-01 21:43:17 +00:00
Siddharth Seth
39f019f413 YARN-735. Make ApplicationAttemptId, ContainerId and NodeId immutable. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488439 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-01 00:14:31 +00:00
Siddharth Seth
2692675fc3 YARN-684. ContainerManager.startContainer should use ContainerTokenIdentifier instead of the entire Container. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488085 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-31 04:13:16 +00:00
Vinod Kumar Vavilapalli
982753dc8e YARN-716. Making ApplicationID immutable. Contributed by Siddharth Seth. 
MAPREDUCE-5282. Updating MR App to use immutable ApplicationID after YARN-716. Contributed by Siddharth Seth.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1487994 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-30 20:18:39 +00:00
Siddharth Seth
b16c5638b5 YARN-719. Move RMIdentifier from Container to ContainerTokenIdentifier. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1487741 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-30 04:59:05 +00:00
Vinod Kumar Vavilapalli
5420f287cc YARN-578. Fixed NM to use SecureIOUtils for reading and aggregating logs. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1487672 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-29 23:14:59 +00:00
Jason Darrell Lowe
31c96829c7 YARN-512. Log aggregation root directory check is more expensive than it needs to be. Contributed by Maysam Yabandeh 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1487498 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-29 14:25:04 +00:00
Vinod Kumar Vavilapalli
259edf8dca YARN-571. Remove user from ContainerLaunchContext. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1485928 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-24 03:22:54 +00:00
Vinod Kumar Vavilapalli
1fac1ac98c YARN-708. Moved RecordFactory classes to hadoop-yarn-api, and put some miscellaneous fixes to the interfaces. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1485478 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-22 22:51:04 +00:00
Vinod Kumar Vavilapalli
ca80246731 YARN-617. Made ContainerTokens to be used for validation at NodeManager also in unsecure mode to prevent AMs from faking resource requirements in unsecure mode. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1483667 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-17 06:36:33 +00:00
Vinod Kumar Vavilapalli
68148989bf YARN-590. Added an optional mesage to be returned by ResourceMaanger when RM asks an RM to shutdown/resync etc so that NMs can log this message locally for better debuggability. Contributed by Mayank Bansal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1481234 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-10 23:37:44 +00:00
Vinod Kumar Vavilapalli
a35c7fd80b YARN-663. Changed ResourceTracker API and LocalizationProtocol API to throw YarnRemoteException and IOException. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1481215 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-10 21:58:59 +00:00
Vinod Kumar Vavilapalli
2638bc67a4 YARN-634. Modified YarnRemoteException to be not backed by PB and introduced a separate SerializedException record. Contributed by Siddharth Seth. 
MAPREDUCE-5239. Updated MR App to reflect YarnRemoteException changes after YARN-634. Contributed by Siddharth Seth.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1481205 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-10 21:49:28 +00:00
Vinod Kumar Vavilapalli
9eb53b9c84 YARN-632. Changed ContainerManager api to throw IOException and YarnRemoteException. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1479740 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-07 00:42:48 +00:00
Vinod Kumar Vavilapalli
92b7165a71 YARN-629. Make YarnRemoteException not be rooted at IOException. Contributed by Xuan Gong. 
MAPREDUCE-5204. Handling YarnRemoteException separately from IOException in MR app after YARN-629. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1479680 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-06 19:04:31 +00:00
Vinod Kumar Vavilapalli
46b2898a1a YARN-618. Modified RM_INVALID_IDENTIFIER to be -1 instead of zero. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1478230 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-01 23:23:20 +00:00
Suresh Srinivas
27d4592771 YARN-506. Move to common utils FileUtil#setReadable/Writable/Executable and FileUtil#canRead/Write/Execute. Contributed by Ivan Mitic. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1477408 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-29 23:09:28 +00:00
Vinod Kumar Vavilapalli
635f96e74e YARN-562. Missed files from previous commit. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476038 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-26 04:42:34 +00:00
Vinod Kumar Vavilapalli
fbb55784d9 YARN-562. Modified NM to reject any containers allocated by a previous ResourceManager. Contributed by Jian He. 
MAPREDUCE-5167. Update MR App after YARN-562 to use the new builder API for the container. Contributed by Jian He.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476034 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-26 03:50:20 +00:00
Vinod Kumar Vavilapalli
a2f97fd9cb YARN-605. Fix failing unit test in TestNMWebServices when versionInfo has parantheses like when running on a git checkout. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471608 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-24 19:42:47 +00:00
Vinod Kumar Vavilapalli
27e8c86999 YARN-561. Modified NodeManager to set key information into the environment of every container that it launches. Contributed by Xuan Gong. 
MAPREDUCE-5175. Updated MR App to not set envs that will be set by NMs anyways after YARN-561. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471156 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-23 21:39:51 +00:00