big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
5,281 Commits 2 Branches 2 Tags 264 MiB
f0eb4bc342
Commit Graph

115 Commits

Author SHA1 Message Date
Vinod Kumar Vavilapalli
f0eb4bc342 YARN-692. Creating NMToken master key on RM and sharing it with NM as a part of RM-NM heartbeat. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492907 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-14 00:06:42 +00:00
Vinod Kumar Vavilapalli
c0cd68d8fb YARN-773. Moved YarnRuntimeException from package api.yarn to api.yarn.exceptions. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492903 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-13 23:25:14 +00:00
Vinod Kumar Vavilapalli
686ac311c9 YARN-746. Renamed Service.register() and Service.unregister() to registerServiceListener() & unregisterServiceListener() respectively. Contributed by Steve Loughran. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492780 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-13 17:49:14 +00:00
Vinod Kumar Vavilapalli
0928502029 YARN-530. Defined Service model strictly, implemented AbstractService for robust subclassing and migrated yarn-common services. Contributed by Steve Loughran. 
YARN-117. Migrated rest of YARN to the new service model. Contributed by Steve Louhran.
MAPREDUCE-5298. Moved MapReduce services to YARN-530 stricter lifecycle. Contributed by Steve Loughran.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492718 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-13 15:54:38 +00:00
Alejandro Abdelnur
80eb92aff0 YARN-600. Hook up cgroups CPU settings to the number of virtual cores allocated. (sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492365 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-12 18:57:31 +00:00
Siddharth Seth
c6c41abf68 YARN-737. Throw some specific exceptions directly instead of wrapping them in YarnException. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1491896 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-11 18:47:16 +00:00
Jason Darrell Lowe
a8c330fd33 YARN-760. NodeManager throws AvroRuntimeException on failed start. Contributed by Niranjan Singh 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1490704 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-07 16:20:20 +00:00
Vinod Kumar Vavilapalli
66de4ccf60 YARN-724. Moved ProtoBase from api.records to api.records.impl.pb. Contributed by Jian He. 
MAPREDUCE-5303. Changed MR app after moving ProtoBase to package impl.pb via YARN-724. Contributed by Jian He.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489658 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-04 22:43:19 +00:00
Kihwal Lee
78063b3a76 YARN-742. Log aggregation causes a lot of redundant setPermission calls. Contributed by Jason Lowe. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489596 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-04 20:49:34 +00:00
Vinod Kumar Vavilapalli
a83fb61ac0 YARN-635. Renamed YarnRemoteException to YarnException. Contributed by Siddharth Seth. 
MAPREDUCE-5301. Updated MR code to work with YARN-635 changes of renaming YarnRemoteException to YarnException. Contributed by Siddharth Seth


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489283 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-04 04:05:50 +00:00
Vinod Kumar Vavilapalli
d33534c4fb YARN-748. Moved BuilderUtils from yarn-common to yarn-server-common for eventual retirement. Contributed by Jian He. 
MAPREDUCE-5297. Updated MR App since BuilderUtils is no longer public after YARN-748. Contributed by Jian He.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489257 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-04 00:34:18 +00:00
Siddharth Seth
a2205a3b72 YARN-720 and MAPREDUCE-5291. container-log4j.properties should not refer to mapreduce properties. Update MRApp to use YARN properties for log setup. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488829 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-03 02:35:34 +00:00
Vinod Kumar Vavilapalli
a2c4233004 YARN-717. Put object creation factories for Token in the class itself and remove useless derivations for specific tokens. Contributed by Jian He. 
MAPREDUCE-5289. Updated MR App to use Token directly after YARN-717. Contributed by Jian He.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488616 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-01 21:43:17 +00:00
Siddharth Seth
39f019f413 YARN-735. Make ApplicationAttemptId, ContainerId and NodeId immutable. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488439 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-01 00:14:31 +00:00
Siddharth Seth
2692675fc3 YARN-684. ContainerManager.startContainer should use ContainerTokenIdentifier instead of the entire Container. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488085 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-31 04:13:16 +00:00
Vinod Kumar Vavilapalli
982753dc8e YARN-716. Making ApplicationID immutable. Contributed by Siddharth Seth. 
MAPREDUCE-5282. Updating MR App to use immutable ApplicationID after YARN-716. Contributed by Siddharth Seth.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1487994 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-30 20:18:39 +00:00
Siddharth Seth
b16c5638b5 YARN-719. Move RMIdentifier from Container to ContainerTokenIdentifier. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1487741 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-30 04:59:05 +00:00
Vinod Kumar Vavilapalli
5420f287cc YARN-578. Fixed NM to use SecureIOUtils for reading and aggregating logs. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1487672 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-29 23:14:59 +00:00
Jason Darrell Lowe
31c96829c7 YARN-512. Log aggregation root directory check is more expensive than it needs to be. Contributed by Maysam Yabandeh 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1487498 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-29 14:25:04 +00:00
Vinod Kumar Vavilapalli
259edf8dca YARN-571. Remove user from ContainerLaunchContext. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1485928 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-24 03:22:54 +00:00
Vinod Kumar Vavilapalli
61885df2fa YARN-711. Copied BuilderUtil methods in individual API records as BuilderUtils is going to be dismantled. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1485888 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-23 22:47:33 +00:00
Vinod Kumar Vavilapalli
1fac1ac98c YARN-708. Moved RecordFactory classes to hadoop-yarn-api, and put some miscellaneous fixes to the interfaces. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1485478 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-22 22:51:04 +00:00
Vinod Kumar Vavilapalli
ca80246731 YARN-617. Made ContainerTokens to be used for validation at NodeManager also in unsecure mode to prevent AMs from faking resource requirements in unsecure mode. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1483667 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-17 06:36:33 +00:00
Vinod Kumar Vavilapalli
065747efab YARN-628. Fix the way YarnRemoteException is being unrolled to extract out the underlying exception. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1483207 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-16 06:58:34 +00:00
Siddharth Seth
1a119f87b4 YARN-615. Rename ContainerLaunchContext.containerTokens to tokens. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1482199 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-14 04:11:55 +00:00
Vinod Kumar Vavilapalli
68148989bf YARN-590. Added an optional mesage to be returned by ResourceMaanger when RM asks an RM to shutdown/resync etc so that NMs can log this message locally for better debuggability. Contributed by Mayank Bansal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1481234 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-10 23:37:44 +00:00
Vinod Kumar Vavilapalli
a35c7fd80b YARN-663. Changed ResourceTracker API and LocalizationProtocol API to throw YarnRemoteException and IOException. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1481215 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-10 21:58:59 +00:00
Vinod Kumar Vavilapalli
2638bc67a4 YARN-634. Modified YarnRemoteException to be not backed by PB and introduced a separate SerializedException record. Contributed by Siddharth Seth. 
MAPREDUCE-5239. Updated MR App to reflect YarnRemoteException changes after YARN-634. Contributed by Siddharth Seth.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1481205 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-10 21:49:28 +00:00
Vinod Kumar Vavilapalli
9eb53b9c84 YARN-632. Changed ContainerManager api to throw IOException and YarnRemoteException. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1479740 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-07 00:42:48 +00:00
Vinod Kumar Vavilapalli
92b7165a71 YARN-629. Make YarnRemoteException not be rooted at IOException. Contributed by Xuan Gong. 
MAPREDUCE-5204. Handling YarnRemoteException separately from IOException in MR app after YARN-629. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1479680 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-06 19:04:31 +00:00
Vinod Kumar Vavilapalli
fc275fed66 YARN-625. Move the utility method unwrapAndThrowException from YarnRemoteExceptionPBImpl to RPCUtil. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1478594 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-02 22:44:12 +00:00
Vinod Kumar Vavilapalli
46b2898a1a YARN-618. Modified RM_INVALID_IDENTIFIER to be -1 instead of zero. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1478230 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-01 23:23:20 +00:00
Suresh Srinivas
27d4592771 YARN-506. Move to common utils FileUtil#setReadable/Writable/Executable and FileUtil#canRead/Write/Execute. Contributed by Ivan Mitic. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1477408 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-29 23:09:28 +00:00
Vinod Kumar Vavilapalli
635f96e74e YARN-562. Missed files from previous commit. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476038 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-26 04:42:34 +00:00
Vinod Kumar Vavilapalli
fbb55784d9 YARN-562. Modified NM to reject any containers allocated by a previous ResourceManager. Contributed by Jian He. 
MAPREDUCE-5167. Update MR App after YARN-562 to use the new builder API for the container. Contributed by Jian He.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476034 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-26 03:50:20 +00:00
Vinod Kumar Vavilapalli
a2f97fd9cb YARN-605. Fix failing unit test in TestNMWebServices when versionInfo has parantheses like when running on a git checkout. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471608 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-24 19:42:47 +00:00
Vinod Kumar Vavilapalli
27e8c86999 YARN-561. Modified NodeManager to set key information into the environment of every container that it launches. Contributed by Xuan Gong. 
MAPREDUCE-5175. Updated MR App to not set envs that will be set by NMs anyways after YARN-561. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471156 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-23 21:39:51 +00:00
Vinod Kumar Vavilapalli
ffd7dbb34a YARN-583. Moved application level local resources to be localized under the filecache sub-directory under application directory. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1470812 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-23 04:37:45 +00:00
Vinod Kumar Vavilapalli
c570309b07 YARN-547. Fixed race conditions in public and private resource localization which used to cause duplicate downloads. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1470076 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-19 22:35:43 +00:00
Bikas Saha
edcfd4527c YARN-593. container launch on Windows does not correctly populate classpath with new process's environment variables and localized resources (Chris Nauroth via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1469998 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-19 19:29:22 +00:00
Vinod Kumar Vavilapalli
44bf8525a5 YARN-493. Fixed some shell related flaws in YARN on Windows. Contributed by Chris Nauroth. 
HADOOP-9486. Promoted Windows and Shell related utils from YARN to Hadoop Common. Contributed by Chris Nauroth.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1469667 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-19 02:14:58 +00:00
Vinod Kumar Vavilapalli
8e1c2823fc YARN-441. Removed unused utility methods for collections from two API records. Contributed by Xuan Gong. 
MAPREDUCE-5163. Update MR App to not use API utility methods for collections after YARN-441. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1469657 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-19 01:33:13 +00:00
Vinod Kumar Vavilapalli
0e01f26821 YARN-444. Moved special container exit codes from YarnConfiguration to API where they belong. Contributed by Sandy Ryza. 
MAPREDUCE-5151. Updated MR AM to use standard exit codes from the API after YARN-444. Contributed by Sandy Ryza.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1468276 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-15 23:54:09 +00:00
Vinod Kumar Vavilapalli
e4c55e17fe YARN-486. Changed NM's startContainer API to accept Container record given by RM as a direct parameter instead of as part of the ContainerLaunchContext record. Contributed by Xuan Gong. 
MAPREDUCE-5139. Update MR AM to use the modified startContainer API after YARN-486. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1467063 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-11 19:28:51 +00:00
Vinod Kumar Vavilapalli
4234bc87b3 YARN-539. Addressed memory leak of LocalResource objects NM when a resource localization fails. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1466756 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-11 02:08:11 +00:00
Vinod Kumar Vavilapalli
2e3b56f6e9 YARN-495. Changed NM reboot behaviour to be a simple resync - kill all containers and re-register with RM. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1466752 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-11 02:00:47 +00:00
Vinod Kumar Vavilapalli
add00d6d74 YARN-487. Modify path manipulation in LocalDirsHandlerService to let TestDiskFailures pass on Windows. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1466746 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-11 01:00:38 +00:00
Vinod Kumar Vavilapalli
3a54a5653b YARN-112. Fixed a race condition during localization that fails containers. Contributed by Omkar Vinit Joshi. 
MAPREDUCE-5138. Fix LocalDistributedCacheManager after YARN-112. Contributed by Omkar Vinit Joshi.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1466196 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-09 19:56:10 +00:00
Vinod Kumar Vavilapalli
b96d18bd23 YARN-99. Modify private distributed cache to localize files such that no local directory hits unix file count limits and thus prevent job failures. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1465853 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-09 01:33:38 +00:00
Bikas Saha
16fedf5473 YARN-479. NM retry behavior for connection to RM should be similar for lost heartbeats (Jian He via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1465731 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-08 19:17:16 +00:00