big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
5,975 Commits 2 Branches 2 Tags 264 MiB
21181b6553
Commit Graph

441 Commits

Author SHA1 Message Date
Vinod Kumar Vavilapalli
21181b6553 YARN-1277. Added a policy based configuration for http/https in common HttpServer and using the same in YARN - related 
to per project https config support via HADOOP-10022. Contributed by Suresh Srinivas and Omkar Vinit Joshi.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529662 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-06 18:43:36 +00:00
Vinod Kumar Vavilapalli
44f7ee7192 YARN-1278. Fixed NodeManager to not delete local resources for apps on resync command from RM - a bug caused by YARN-1149. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529657 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-06 18:32:16 +00:00
Vinod Kumar Vavilapalli
52b0ce3565 YARN-1274. Fixed NodeManager's LinuxContainerExecutor to create user, app-dir and log-dirs correctly even when there are no resources to localize for the container. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529555 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-06 00:14:55 +00:00
Arun Murthy
5e2d93b4d3 YARN-1090. Fixed CS UI to better reflect applications as non-schedulable and not as pending. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529538 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-05 22:26:42 +00:00
Sanford Ryza
54e54d45b3 YARN-1268. TestFairScheduer.testContinuousScheduling is flaky (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529529 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-05 22:04:53 +00:00
Vinod Kumar Vavilapalli
0a887a0910 YARN-1254. Fixed NodeManager to not pollute container's credentials. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529382 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-05 04:25:44 +00:00
Alejandro Abdelnur
80c5bffc4b YARN-1253. Changes to LinuxContainerExecutor to run containers as a single dedicated user in non-secure mode. (rvs via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529325 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-04 21:59:43 +00:00
Bikas Saha
cbab04727b YARN-1232. Configuration to support multiple RMs (Karthik Kambatla via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529251 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-04 18:40:18 +00:00
Sanford Ryza
d8ea364d07 YARN-1271. Text file busy errors launching containers again (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529058 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-04 01:30:19 +00:00
Hitesh Shah
d3841bd499 YARN-1149. NM throws InvalidStateTransitonException: Invalid event: APPLICATION_LOG_HANDLING_FINISHED at RUNNING. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529043 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-04 00:43:26 +00:00
Bikas Saha
8ebf37f369 YARN-1256. NM silently ignores non-existent service in StartContainerRequest (Xuan Gong via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529039 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-04 00:23:35 +00:00
Sanford Ryza
ac2cdb5f65 YARN-1236. FairScheduler setting queue name in RMApp is not working. (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529034 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-03 23:29:39 +00:00
Arun Murthy
79a11ce09d YARN-890. Ensure CapacityScheduler doesn't round-up metric for available resources. Contributed by Xuan Gong & Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529015 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-03 21:54:35 +00:00
Jonathan Turner Eagles
7b687dda09 YARN-1199. Make NM/RM Versions Available (Mit Desai via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529003 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-03 21:04:12 +00:00
Jonathan Turner Eagles
dbdb8c6f1f Revert YARN-677. Increase coverage to FairScheduler (Vadim Bondarev and Dennis Y via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1528914 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-03 15:52:30 +00:00
Sanford Ryza
2db1d4d4ee YARN-1213. Restore config to ban submitting to undeclared pools in the Fair Scheduler. (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1528696 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-03 01:17:48 +00:00
Sanford Ryza
86cc60c43b YARN-876. Node resource is added twice when node comes back from unhealthy. (Peng Zhang via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1528660 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-02 22:33:47 +00:00
Bikas Saha
0a6e275ee3 YARN-1141. Updating resource requests should be decoupled with updating blacklist (Zhijie Shen via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1528632 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-02 21:31:47 +00:00
Jonathan Turner Eagles
d14584dec5 YARN-677. Increase coverage to FairScheduler (Vadim Bondarev and Dennis Y via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1528524 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-02 15:45:11 +00:00
Sanford Ryza
4b9fa6af9d YARN-1262. TestApplicationCleanup relies on all containers assigned in a single heartbeat (Karthik Kambatla via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1528243 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-01 22:42:50 +00:00
Sanford Ryza
aa2745abe5 YARN-1228. Clean up Fair Scheduler configuration loading. (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1528201 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-01 20:14:18 +00:00
Sanford Ryza
ae05623a75 YARN-1010. FairScheduler: decouple container scheduling from nodemanager heartbeats. (Wei Yan via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1528192 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-01 19:54:50 +00:00
Vinod Kumar Vavilapalli
74d20250ff YARN-1070. Fixed race conditions in NodeManager during container-kill. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1527827 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-01 00:18:09 +00:00
Alejandro Abdelnur
84e4e4b143 YARN-1247. test-container-executor has gotten out of sync with the changes to container-executor. (rvs via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1527813 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-30 22:55:20 +00:00
Sanford Ryza
09bb12c394 YARN-1221. With Fair Scheduler, reserved MB reported in RM web UI increases indefinitely (Siqi Li via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1527794 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-30 21:59:21 +00:00
Vinod Kumar Vavilapalli
1c5b49eeaf YARN-899. Added back queue level administrator-acls so that there is no regression w.r.t 1.x. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1527282 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-29 08:21:36 +00:00
Vinod Kumar Vavilapalli
4ce930ea44 YARN-1246. Added application finish-status to ApplicationSummary for the sake of testing given ApplicationHistoryServer is not yet ready. Contributed by Arpit Gupta. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1527205 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-28 16:29:58 +00:00
Arun Murthy
96c11fe602 MAPREDUCE-5514. Fix TestRMContainerAllocator. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1526735 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-26 23:02:51 +00:00
Jonathan Turner Eagles
5a15c392a1 YARN-819. ResourceManager and NodeManager should check for a minimum allowed version (Robert Parker via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1526660 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-26 19:55:26 +00:00
Vinod Kumar Vavilapalli
db5322695a YARN-1157. Fixed ResourceManager UI to behave correctly when apps like distributed-shell do not set tracking urls. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1526371 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-26 04:18:00 +00:00
Bikas Saha
929f96ee14 YARN-1214. Register ClientToken MasterKey in SecretManager after it is saved (Jian He via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1526078 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-25 01:07:59 +00:00
Siddharth Seth
10a4289ebf YARN-1229. Define constraints on Auxiliary Service names. Change ShuffleHandler service name from mapreduce.shuffle to mapreduce_shuffle. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1526065 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-25 00:35:03 +00:00
Vinod Kumar Vavilapalli
78e995a37d YARN-1204. Added separate configuration properties for https for RM and NM without which servers enabled with https will also start on http ports. Contributed by Omkar Vinit Joshi. 
MAPREDUCE-5523. Added separate configuration properties for https for JHS without which even when https is enabled, it starts on http port itself. Contributed by Omkar Vinit Joshi.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1525947 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-24 17:20:18 +00:00
Sanford Ryza
6d030abe8c YARN-1188. The context of QueueMetrics becomes default when using FairScheduler (Akira Ajisaka via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1525516 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-23 07:06:05 +00:00
Sanford Ryza
9f6cf60ae0 YARN-1128. FifoPolicy.computeShares throws NPE on empty list of Schedulables (Karthik Kambatla via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1525313 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-22 01:46:58 +00:00
Vinod Kumar Vavilapalli
13420d01f2 YARN-1203. Changed YARN web-app proxy to handle http and https URLs from AM registration and finish correctly. Contributed by Omkar Vinit Joshi. 
MAPREDUCE-5515. Fixed MR AM's webapp to depend on a new config mapreduce.ssl.enabled to enable https and disabling it by default as MR AM needs
to set up its own certificates etc and not depend on clusters'. Contributed by Omkar Vinit Joshi.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1524864 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-19 23:35:58 +00:00
Hitesh Shah
8628c1704b YARN-353. Add Zookeeper-based store implementation for RMStateStore. Contributed by Bikas Saha, Jian He and Karthik Kambatla. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1524829 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-19 20:39:10 +00:00
Christopher Douglas
90f43b1c18 YARN-1184. ClassCastException during preemption enforcement. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1523863 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-17 01:31:09 +00:00
Vinod Kumar Vavilapalli
4ea295016e YARN-1001. Added a web-service to get statistics about per application-type per state for consumption by downstream projects. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1523855 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-17 00:24:17 +00:00
Bikas Saha
ead3dcc3de YARN-1027. Implement RMHAProtocolService (Karthik Kambatla via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1523750 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-16 18:33:59 +00:00
Arun Murthy
60a1a70197 YARN-1170. YARN & MapReduce proto definitions fixed to specify protobuf package as hadoop.yarn and hadoop.mapreduce respectively. Contibuted by Binglin Chang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1523652 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-16 14:04:04 +00:00
Alejandro Abdelnur
89c7ee9245 YARN-1137. Add support whitelist for system users to Yarn container-executor.c. (rvs via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1523587 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-16 11:01:28 +00:00
Bikas Saha
ec010a2936 YARN-540. Race condition causing RM to potentially relaunch already unregistered AMs on RM restart (Jian He via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1523376 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-15 01:01:06 +00:00
Jason Darrell Lowe
baea09be61 YARN-1189. NMTokenSecretManagerInNM is not being told when applications have finished. Contributed by Omkar Vinit Joshi 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1523158 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-14 00:18:29 +00:00
Bikas Saha
f4951e0708 YARN-1116. Populate AMRMTokens back to AMRMTokenSecretManager after RM restarts (Jian He via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1523146 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-13 23:51:50 +00:00
Jason Darrell Lowe
2950c9ff6e YARN-1194. TestContainerLogsPage fails with native builds. Contributed by Roman Shaposhnik 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1522968 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-13 15:14:07 +00:00
Chris Nauroth
f152a7e788 YARN-1078. TestNodeManagerResync, TestNodeManagerShutdown, and TestNodeStatusUpdater fail on Windows. Contributed by Chuan Liu. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1522644 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-12 15:58:34 +00:00
Thomas Graves
c22091aecb YARN-1176. RM web services ClusterMetricsInfo total nodes doesn't include unhealthy nodes (Jonathan Eagles via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1522062 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-11 22:00:08 +00:00
Jonathan Turner Eagles
220b21732b YARN-1119. Add ClusterMetrics checks to tho TestRMNodeTransitions tests (Mit Desai via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1521611 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-10 19:53:49 +00:00
Bikas Saha
bcb865314f YARN-1098. Separate out RM services into Always On and Active (Karthik Kambatla via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1521560 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-10 17:43:51 +00:00