big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
10,223 Commits 2 Branches 2 Tags 264 MiB
a380643d20
Commit Graph

1637 Commits

Author SHA1 Message Date
Vinod Kumar Vavilapalli
39063cd36f YARN-2676. Enhanced Timeline auth-filter to support proxy users. Contributed by Zhijie Shen. 2014-10-17 22:02:50 -07:00
Vinod Kumar Vavilapalli
e9c66e8fd2 YARN-2705. Fixed bugs in ResourceManager node-label manager that were causing test-failures: added a dummy in-memory labels-manager. Contributed by Wangda Tan. 2014-10-17 18:26:12 -07:00
Jian He
c3de2412eb YARN-1879. Marked Idempotent/AtMostOnce annotations to ApplicationMasterProtocol for RM fail over. Contributed by Tsuyoshi OZAWA 2014-10-17 16:35:27 -07:00
Jian He
a6aa6e42ca YARN-2588. Standby RM fails to transitionToActive if previous transitionToActive failed with ZK exception. Contributed by Rohith Sharmaks 2014-10-17 10:54:24 -07:00
Vinod Kumar Vavilapalli
abae63caf9 YARN-2699. Fixed a bug in CommonNodeLabelsManager that caused tests to fail when using ephemeral ports on NodeIDs. Contributed by Wangda Tan. 2014-10-17 08:58:08 -07:00
Jian He
0fd0ebae64 YARN-2682. Updated WindowsSecureContainerExecutor to not use DefaultContainerExecutor#getFirstApplicationDir and use getWorkingDir() instead. Contributed by Zhihai Xu 2014-10-16 18:14:34 -07:00
Jian He
233d446be1 YARN-2621. Simplify the output when the user doesn't have the access for getDomain(s). Contributed by Zhijie Shen 2014-10-16 16:37:21 -07:00
Steve Loughran
6f43491c03 YARN-2689 TestSecureRMRegistryOperations failing on windows: secure ZK won't start 2014-10-16 14:22:02 -07:00
Vinod Kumar Vavilapalli
b3056c266a YARN-2685. Fixed a bug in CommonNodeLabelsManager that caused wrong resource tracking per label when a host runs multiple node-managers. Contributed by Wangda Tan. 2014-10-15 18:47:26 -07:00
Vinod Kumar Vavilapalli
f2ea555ac6 YARN-2496. Enhanced Capacity Scheduler to have basic support for allocating resources based on node-labels. Contributed by Wangda Tan. 
YARN-2500. Ehnaced ResourceManager to support schedulers allocating resources based on node-labels. Contributed by Wangda Tan.
 2014-10-15 18:33:06 -07:00
Jian He
0af1a2b5bc YARN-2312. Deprecated old ContainerId#getId API and updated MapReduce to use ContainerId#getContainerId instead. Contributed by Tsuyoshi OZAWA 2014-10-15 15:22:07 -07:00
Zhijie Shen
1220bb72d4 YARN-2656. Made RM web services authentication filter support proxy user. Contributed by Varun Vasudev and Zhijie Shen. 2014-10-14 21:50:46 -07:00
Zhijie Shen
cdce88376a HADOOP-11181. Generalized o.a.h.s.t.d.DelegationTokenManager to handle all sub-classes of AbstractDelegationTokenIdentifier. Contributed by Zhijie Shen. 2014-10-14 11:35:38 -07:00
Karthik Kambatla
cc93e7e683 YARN-2566. DefaultContainerExecutor should pick a working directory randomly. (Zhihai Xu via kasha) 2014-10-13 16:32:01 -07:00
Karthik Kambatla
da709a2eac YARN-2641. Decommission nodes on -refreshNodes instead of next NM-RM heartbeat. (Zhihai Xu via kasha) 2014-10-13 16:23:04 -07:00
Jian He
f9680d9a16 YARN-2308. Changed CapacityScheduler to explicitly throw exception if the queue 
to which the apps were submitted is changed across RM restart. Contributed by Craig Welch & Chang Li
 2014-10-13 14:09:04 -07:00
Jason Lowe
a56ea01002 YARN-2377. Localization exception stack traces are not passed as diagnostic info. Contributed by Gera Shegalov 2014-10-13 18:31:16 +00:00
Zhijie Shen
4aed2d8e91 YARN-2651. Spun off LogRollingInterval from LogAggregationContext. Contributed by Xuan Gong. 2014-10-13 10:54:09 -07:00
Jason Lowe
344a10ad5e YARN-2667. Fix the release audit warning caused by hadoop-yarn-registry. Contributed by Yi Liu 2014-10-13 15:37:50 +00:00
Steve Loughran
ac64ff77cf YARN-2668 yarn-registry JAR won't link against ZK 3.4.5. (stevel) 2014-10-11 18:34:26 -07:00
Vinod Kumar Vavilapalli
a5ec3d0809 YARN-2501. Enhanced AMRMClient library to support requests against node labels. Contributed by Wangda Tan. 2014-10-10 19:57:39 -07:00
Vinod Kumar Vavilapalli
db7f165319 YARN-2494. Added NodeLabels Manager internal API and implementation. Contributed by Wangda Tan. 2014-10-10 11:44:21 -07:00
Zhijie Shen
cb81bac002 YARN-2583. Modified AggregatedLogDeletionService to be able to delete rolling aggregated logs. Contributed by Xuan Gong. 2014-10-10 00:11:30 -07:00
cnauroth
d3afd730ac YARN-2662. TestCgroupsLCEResourcesHandler leaks file descriptors. Contributed by Chris Nauroth. 2014-10-09 22:44:44 -07:00
Zhijie Shen
e532ed8faa YARN-2617. Fixed ApplicationSubmissionContext to still set resource for backward compatibility. Contributed by Wangda Tan. 2014-10-09 17:57:50 -07:00
Karthik Kambatla
4f426fe223 YARN-2180. [YARN-1492] In-memory backing store for cache manager. (Chris Trezzo via kasha) 2014-10-09 17:16:06 -07:00
Vinod Kumar Vavilapalli
596702a025 YARN-2544. Added admin-API objects for using node-labels. Contributed by Wangda Tan. 2014-10-09 15:46:24 -07:00
Zhijie Shen
1d4612f5ad YARN-2629. Made the distributed shell use the domain-based timeline ACLs. Contributed by Zhijie Shen. 2014-10-09 12:59:47 -07:00
Vinod Kumar Vavilapalli
180afa2f86 YARN-2493. Added user-APIs for using node-labels. Contributed by Wangda Tan. 2014-10-09 11:26:26 -07:00
Mayank Bansal
df3becf080 YARN-2598 GHS should show N/A instead of null for the inaccessible information (Zhijie Shen via mayank) 2014-10-08 17:27:53 -07:00
Steve Loughran
6a326711aa YARN-913 service registry: YARN-2652 add hadoop-yarn-registry package under hadoop-yarn 2014-10-08 13:02:25 -07:00
Jian He
e16e25ab1b YARN-2649. Fixed TestAMRMRPCNodeUpdates test failure. Contributed by Ming Ma 2014-10-08 10:58:51 -07:00
Jian He
30d56fdbb4 YARN-1857. CapacityScheduler headroom doesn't account for other AM's running. Contributed by Chen He and Craig Welch 2014-10-07 13:45:04 -07:00
Jian He
519e5a7dd2 YARN-2644. Fixed CapacityScheduler to return up-to-date headroom when AM allocates. Contributed by Craig Welch 2014-10-06 15:48:46 -07:00
Jian He
ea26cc0b4a YARN-2615. Changed ClientToAMTokenIdentifier/RM(Timeline)DelegationTokenIdentifier to use protobuf as payload. Contributed by Junping Du 2014-10-06 10:47:43 -07:00
Chris Douglas
c8212bacb1 YARN-1051. Add a system for creating reservations of cluster capacity. 
Contributed by Subru Krishnan and Carlo Curino.
 2014-10-03 17:09:02 -07:00
subru
a2986234be YARN-2611. Fixing jenkins findbugs warning and TestRMWebServicesCapacitySched for branch YARN-1051. Contributed by Subru Krishnan and Carlo Curino. 
(cherry picked from commit c47464aba407d1dafe10be23fe454f0489cc4367)
 2014-10-03 15:43:23 -07:00
subru
5e10a13bb4 YARN-2576. Making test patch pass in branch. Contributed by Subru Krishnan and Carlo Curino. 
(cherry picked from commit 90ac0be86b898aefec5471db4027554c8e1b310c)
 2014-10-03 15:43:13 -07:00
subru
6261f7cc69 YARN-2080. Integrating reservation system with ResourceManager and client-RM protocol. Contributed by Subru Krishnan and Carlo Curino. 
(cherry picked from commit 8baeaead8532898163f1006276b731a237b1a559)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java
 2014-10-03 15:42:43 -07:00
subru
3418c56bcf YARN-1712. Plan follower that synchronizes the current state of reservation subsystem with the scheduler. Contributed by Subru Krishnan and Carlo Curino. 
(cherry picked from commit 169085319b8b76641f8b9f6840a3fef06d221e2b)
 2014-10-03 15:42:10 -07:00
carlo curino
b6df0dddcd YARN-1711. Policy to enforce instantaneous and over-time quotas on user reservation. Contributed by Carlo Curino and Subru Krishnan. 
(cherry picked from commit c4918cb4cb5a267a8cfd6eace28fcfe7ad6174e8)
 2014-10-03 15:42:03 -07:00
carlo curino
f66ffcf832 YARN-1710. Logic to find allocations within a Plan that satisfy user ReservationRequest(s). Contributed by Carlo Curino and Subru Krishnan. 
(cherry picked from commit aef7928899b37262773f3dc117157bb746bf8918)
 2014-10-03 15:41:57 -07:00
subru
cf4b34282a YARN-1709. In-memory data structures used to track resources over time to enable reservations. 
(cherry picked from commit 0d8b2cd88b958b1e602fd4ea4078ef8d4742a7c3)
 2014-10-03 15:41:51 -07:00
subru
c9266df404 YARN-1708. Public YARN APIs for creating/updating/deleting reservations. 
(cherry picked from commit 3f2e3b275bcf29264a112c5d2f4eae289fe7f52c)
 2014-10-03 15:41:39 -07:00
carlo curino
1c6950354f YARN-2475. Logic for responding to capacity drops for the ReservationSystem. Contributed by Carlo Curino and Subru Krishnan. 
(cherry picked from commit f83a07f266f2c5e6eead554d8a331ed7e75e10d5)
 2014-10-03 15:41:21 -07:00
carlo curino
eb3e40b833 YARN-1707. Introduce APIs to add/remove/resize queues in the CapacityScheduler. Contributed by Carlo Curino and Subru Krishnan 
(cherry picked from commit aac47fda7fecda9fc18ade34d633eca895865a70)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
 2014-10-03 15:41:02 -07:00
Vinod Kumar Vavilapalli
34cdcaad71 YARN-2468. Enhanced NodeManager to support log handling APIs (YARN-2569) for use by long running services. Contributed by Xuan Gong. 2014-10-03 12:15:40 -07:00
Karthik Kambatla
80d11eb68e YARN-2635. TestRM, TestRMRestart, TestClientToAMTokens should run with both CS and FS. (Wei Yan and kasha via kasha) 2014-10-03 11:49:49 -07:00
Jian He
eb6ce5e97c YARN-2562. Changed ContainerId#toString() to be more readable. Contributed by Tsuyoshi OZAWA 2014-10-03 11:04:02 -07:00
Jian He
054f285526 YARN-2628. Capacity scheduler with DominantResourceCalculator carries out reservation even though slots are free. Contributed by Varun Vasudev 2014-10-02 15:13:33 -07:00
Zhijie Shen
1c93025a1b YARN-2527. Fixed the potential NPE in ApplicationACLsManager and added test cases for it. Contributed by Benoy Antony. 2014-10-02 14:56:13 -07:00
Jason Lowe
29f520052e YARN-2624. Resource Localization fails on a cluster due to existing cache directories. Contributed by Anubhav Dhoot 2014-10-02 17:39:34 +00:00
Karthik Kambatla
5e0b49da9c YARN-2254. TestRMWebServicesAppsModification should run against both CS and FS. (Zhihai Xu via kasha) 2014-10-02 10:15:04 -07:00
Jian He
3ef1cf187f YARN-2617. Fixed NM to not send duplicate container status whose app is not running. Contributed by Jun Gong 2014-10-02 10:04:09 -07:00
junping_du
c7cee9b455 YARN-1979. TestDirectoryCollection fails when the umask is unusual. (Contributed by Vinod Kumar Vavilapalli and Tsuyoshi OZAWA) 2014-10-02 08:02:59 -07:00
Vinod Kumar Vavilapalli
9e40de6af7 YARN-2446. Augmented Timeline service APIs to start taking in domains as a parameter while posting entities and events. Contributed by Zhijie Shen. 2014-10-01 17:52:03 -07:00
junping_du
0708827a93 YARN-2613. Support retry in NMClient for rolling-upgrades. (Contributed by Jian He) 2014-10-01 16:50:30 -07:00
Zhijie Shen
52bbe0f11b YARN-2630. Prevented previous AM container status from being acquired by the current restarted AM. Contributed by Jian He. 2014-10-01 15:38:11 -07:00
Vinod Kumar Vavilapalli
ba7f31c2ee YARN-1972. Added a secure container-executor for Windows. Contributed by Remus Rusanu. 2014-10-01 10:14:41 -07:00
Karthik Kambatla
17d1202c35 YARN-2179. [YARN-1492] Initial cache manager structure and context. (Chris Trezzo via kasha) 2014-09-30 19:01:47 -07:00
Jian He
bbff96be48 YARN-2602. Fixed possible NPE in ApplicationHistoryManagerOnTimelineStore. Contributed by Zhijie Shen 2014-09-30 16:44:17 -07:00
Karthik Kambatla
14d60dadc2 YARN-2594. Potential deadlock in RM when querying ApplicationResourceUsageReport. (Wangda Tan via kasha) 2014-09-30 16:12:27 -07:00
Jason Lowe
feaf139b4f YARN-2387. Resource Manager crashes with NPE due to lack of synchronization. Contributed by Mit Desai 2014-09-30 22:37:28 +00:00
Zhijie Shen
9582a50176 YARN-2627. Added the info logs of attemptFailuresValidityInterval and number of previous failed attempts. Contributed by Xuan Gong. 2014-09-30 12:57:45 -07:00
Karthik Kambatla
f7743dd07d YARN-2610. Hamlet should close table tags. (Ray Chiang via kasha) 2014-09-30 11:46:07 -07:00
Jonathan Eagles
e10eeaabce YARN-2606. Application History Server tries to access hdfs before doing secure login (Mit Desai via jeagles) 2014-09-29 16:37:23 -05:00
Jason Lowe
9c22065109 YARN-1769. CapacityScheduler: Improve reservations. Contributed by Thomas Graves 2014-09-29 14:12:18 +00:00
Jian He
5391919b09 YARN-668. Changed NMTokenIdentifier/AMRMTokenIdentifier/ContainerTokenIdentifier to use protobuf object as the payload. Contributed by Junping Du. 2014-09-26 17:48:41 -07:00
Allen Wittenauer
32870db0fb YARN-2372. There are Chinese Characters in the FairScheduler's document (Fengdong Yu via aw) 2014-09-26 12:46:45 -07:00
Allen Wittenauer
ac70c27473 YARN-2577. Clarify ACL delimiter and how to configure ACL groups only (Mikos Christine via aw) 2014-09-26 11:05:13 -07:00
Karthik Kambatla
f4357240a6 YARN-2608. FairScheduler: Potential deadlocks in loading alloc files and clock access. (Wei Yan via kasha) 2014-09-25 17:42:51 -07:00
Jason Lowe
8269bfa613 YARN-2523. ResourceManager UI showing negative value for "Decommissioned Nodes" field. Contributed by Rohith 2014-09-25 22:37:05 +00:00
Zhijie Shen
72b0881ca6 YARN-2546. Made REST API for application creation/submission use numeric and boolean types instead of the string of them. Contributed by Varun Vasudev. 2014-09-24 17:57:32 -07:00
Zhijie Shen
c86674a3a4 YARN-2581. Passed LogAggregationContext to NM via ContainerTokenIdentifier. Contributed by Xuan Gong. 2014-09-24 17:50:26 -07:00
Karthik Kambatla
39c87344e1 YARN-2596. TestWorkPreservingRMRestart fails with FairScheduler. (kasha) 2014-09-24 15:23:31 -07:00
Vinod Kumar Vavilapalli
d78b452a4f YARN-2102. Added the concept of a Timeline Domain to handle read/write ACLs on Timeline service event data. Contributed by Zhijie Shen. 2014-09-24 11:27:53 -07:00
Allen Wittenauer
034df0e2eb YARN-2161. Fix build on macosx: YARN parts (Binglin Chang via aw) 2014-09-24 08:47:55 -07:00
Allen Wittenauer
3dc28e2052 HADOOP-11092. hadoop shell commands should print usage if not given a class (aw) 2014-09-23 12:24:23 -07:00
Zhijie Shen
5338ac416a YARN-2569. Added the log handling APIs for the long running services. Contributed by Xuan Gong. 2014-09-23 10:36:57 -07:00
Karthik Kambatla
f5578207d2 YARN-2252. Intermittent failure of TestFairScheduler.testContinuousScheduling. (Ratandeep Ratti and kasha via kasha) 2014-09-23 00:03:16 -07:00
Karthik Kambatla
568d3dc2bb YARN-1959. Fix headroom calculation in FairScheduler. (Anubhav Dhoot via kasha) 2014-09-22 23:49:39 -07:00
junping_du
a9a55db065 YARN-2584. TestContainerManagerSecurity fails on trunk. (Contributed by Jian He) 2014-09-22 22:45:06 -07:00
Karthik Kambatla
43efdd30b5 YARN-2539. FairScheduler: Set the default value for maxAMShare to 0.5. (Wei Yan via kasha) 2014-09-22 16:09:52 -07:00
Karthik Kambatla
23e17ce0ff YARN-2540. FairScheduler: Queue filters not working on scheduler page in RM UI. (Ashwin Shankar via kasha) 2014-09-22 15:40:47 -07:00
Jian He
0a641496c7 YARN-1372. Ensure all completed containers are reported to the AMs across RM restart. Contributed by Anubhav Dhoot 2014-09-22 10:30:53 -07:00
Zhijie Shen
376233cdd4 MAPREDUCE-5279. Made MR headroom calculation honor cpu dimension when YARN scheduler resource type is memory plus cpu. Contributed by Peng Zhang and Varun Vasudev. 2014-09-22 09:28:47 -07:00
Karthik Kambatla
9721e2c1fe YARN-2453. TestProportionalCapacityPreemptionPolicy fails with FairScheduler. (Zhihai Xu via kasha) 2014-09-21 23:13:45 -07:00
Karthik Kambatla
c50fc92502 YARN-2452. TestRMApplicationHistoryWriter fails with FairScheduler. (Zhihai Xu via kasha) 2014-09-21 13:15:04 -07:00
Allen Wittenauer
aa1052c34b YARN-2460. Remove obsolete entries from yarn-default.xml (Ray Chiang via aw) 2014-09-19 14:07:38 -07:00
Jian He
444acf8ea7 YARN-2565. Fixed RM to not use FileSystemApplicationHistoryStore unless explicitly set. Contributed by Zhijie Shen 2014-09-19 11:26:29 -07:00
Zhijie Shen
6fe5c6b746 YARN-2568. Fixed the potential test failures due to race conditions when RM work-preserving recovery is enabled. Contributed by Jian He. 2014-09-18 21:56:56 -07:00
Jian He
eb92cc67df YARN-2563. Fixed YarnClient to call getTimeLineDelegationToken only if the Token is not present. Contributed by Zhijie Shen 2014-09-18 14:40:24 -07:00
Jason Lowe
a337f0e354 YARN-2561. MR job client cannot reconnect to AM after NM restart. Contributed by Junping Du 2014-09-18 21:34:40 +00:00
Jason Lowe
9ea7b6c063 YARN-2363. Submitted applications occasionally lack a tracking URL. Contributed by Jason Lowe 2014-09-18 20:13:16 +00:00
Vinod Kumar Vavilapalli
485c96e3cb YARN-2001. Added a time threshold for RM to wait before starting container allocations after restart/failover. Contributed by Jian He. 2014-09-18 11:03:12 -07:00
Vinod Kumar Vavilapalli
a3d9934f91 YARN-1779. Fixed AMRMClient to handle AMRMTokens correctly across ResourceManager work-preserving-restart or failover. Contributed by Jian He. 2014-09-18 10:16:18 -07:00
Jian He
ee21b13cbd YARN-2559. Fixed NPE in SystemMetricsPublisher when retrieving FinalApplicationStatus. Contributed by Zhijie Shen 2014-09-17 21:44:15 -07:00
Jian He
f4886111aa YARN-2558. Updated ContainerTokenIdentifier#read/write to use ContainerId#getContainerId. Contributed by Tsuyoshi OZAWA. 2014-09-17 15:13:59 -07:00
junping_du
90a0c03f0a YARN-1250. Generic history service should support application-acls. (Contributed by Zhijie Shen) 2014-09-16 18:20:49 -07:00
Allen Wittenauer
7971c97ec1 HADOOP-11022. User replaced functions get lost 2-3 levels deep (e.g., sbin) (aw) 2014-09-16 16:06:23 -07:00
XuanGong
8e5d6713cf YARN-2557. Add a parameter "attempt_Failures_Validity_Interval" into 
DistributedShell. Contributed by Xuan Gong
 2014-09-16 10:58:18 -07:00
Vinod Kumar Vavilapalli
9f6891d9ef YARN-2531. Added a configuration for admins to be able to override app-configs and enforce/not-enforce strict control of per-container cpu usage. Contributed by Varun Vasudev. 2014-09-16 10:14:46 -07:00
Allen Wittenauer
0c8aec43f0 YARN-2438. yarn-env.sh cleanup (aw) 2014-09-15 14:09:50 -07:00
Jian He
88e329fdfb YARN-2529. Generic history service RPC interface doesn't work when service authorization is enabled. Contributed by Zhijie Shen 2014-09-15 13:13:54 -07:00
cnauroth
9d4ec97c95 YARN-2549. TestContainerLaunch fails due to classpath problem with hamcrest classes. Contributed by Chris Nauroth. 2014-09-15 11:53:57 -07:00
Vinod Kumar Vavilapalli
14e2639fd0 YARN-611. Added an API to let apps specify an interval beyond which AM failures should be ignored towards counting max-attempts. Contributed by Xuan Gong. 2014-09-13 18:04:05 -07:00
Zhijie Shen
98588cf044 YARN-2528. Relaxed http response split vulnerability protection for the origins header and made it accept multiple origins in CrossOriginFilter. Contributed by Jonathan Eagles. 2014-09-12 21:33:01 -07:00
Jian He
a0ad975ea1 YARN-2542. Fixed NPE when retrieving ApplicationReport from TimeLineServer. Contributed by Zhijie Shen 2014-09-12 15:27:13 -07:00
XuanGong
e65ae575a0 YARN-2456. Possible livelock in CapacityScheduler when RM is recovering 
apps. Contributed by Jian He
 2014-09-12 15:21:46 -07:00
Allen Wittenauer
40364dc47c YARN-2525. yarn logs command gives error on trunk (Akira AJISAKA via aw) 2014-09-12 14:56:41 -07:00
Jonathan Eagles
54e5794806 YARN-2547. Cross Origin Filter throws UnsupportedOperationException upon destroy (Mit Desai via jeagles) 2014-09-12 15:35:17 -05:00
Jian He
3122daa802 YARN-2229. Changed the integer field of ContainerId to be long type. Contributed by Tsuyoshi OZAWA 2014-09-12 10:33:33 -07:00
Jason Lowe
78b048393a YARN-2484. FileSystemRMStateStore#readFile/writeFile should close FSData(In|Out)putStream in final block. Contributed by Tsuyoshi OZAWA 2014-09-12 14:09:14 +00:00
Zhijie Shen
469ea3dcef YARN-2538. Added logs when RM sends roll-overed AMRMToken to AM. Contributed by Xuan Gong. 2014-09-11 22:19:00 -07:00
Jian He
5633da2a01 YARN-2541. Fixed ResourceManagerRest.apt.vm table syntax error. Contributed by Jian He 2014-09-11 21:31:09 -07:00
junping_du
6b8b1608e6 YARN-2033. Merging generic-history into the Timeline Store (Contributed by Zhijie Shen) 2014-09-12 10:04:51 +08:00
Karthik Kambatla
c11ada5ea6 YARN-2534. FairScheduler: Potential integer overflow calculating totalMaxShare. (Zhihai Xu via kasha) 2014-09-11 12:06:06 -07:00
Vinod Kumar Vavilapalli
4be95175cd YARN-2440. Enabled Nodemanagers to limit the aggregate cpu usage across all containers to a preconfigured limit. Contributed by Varun Vasudev. 2014-09-10 19:22:52 -07:00
Jian He
83be3ad444 YARN-415. Capture aggregate memory allocation at the app-level for chargeback. Contributed by Eric Payne & Andrey Klochkov 2014-09-10 18:20:54 -07:00
Jian He
cbfe26370b YARN-2158. Fixed TestRMWebServicesAppsModification#testSingleAppKill test failure. Contributed by Varun Vasudev 2014-09-10 12:47:34 -07:00
XUAN
47bdfa044a YARN-2459. RM crashes if App gets rejected for any reason and HA is enabled. Contributed by Jian He 2014-09-10 11:44:41 -07:00
Vinod Kumar Vavilapalli
b67d5ba784 YARN-2448. Changed ApplicationMasterProtocol to expose RM-recognized resource types to the AMs. Contributed by Varun Vasudev. 2014-09-10 10:15:47 -07:00
Karthik Kambatla
3072c83b38 YARN-1458. FairScheduler: Zero weight can lead to livelock. (Zhihai Xu via kasha) 2014-09-10 08:26:14 -07:00
Jian He
0974f434c4 YARN-2515. Updated ConverterUtils#toContainerId to parse epoch. Contributed by Tsuyoshi OZAWA 2014-09-07 18:29:22 -07:00
Zhijie Shen
56dc496a10 YARN-2507. Documented CrossOriginFilter configurations for the timeline server. Contributed by Jonathan Eagles. 2014-09-07 18:22:40 -07:00
Zhijie Shen
a092cdf32d YARN-2512. Allowed pattern matching for origins in CrossOriginFilter. Contributed by Jonathan Eagles. 2014-09-07 17:49:06 -07:00
cnauroth
cbea1b10ef YARN-2519. Credential Provider related unit tests failed on Windows. Contributed by Xiaoyu Yao. 2014-09-06 20:05:07 -07:00
Jonathan Eagles
e6420fec0a YARN-2508. Cross Origin configuration parameters prefix are not honored (Mit Desai via jeagles) 2014-09-05 19:42:40 -05:00
Zhijie Shen
51a4faf521 YARN-2511. Allowed all origins by default when CrossOriginFilter is enabled. Contributed by Jonathan Eagles. 2014-09-04 15:01:01 -07:00
Jason Lowe
3fa5f728c4 YARN-2431. NM restart: cgroup is not removed for reacquired containers. Contributed by Jason Lowe 2014-09-04 21:11:27 +00:00
Jonathan Eagles
91d45f0f01 YARN-2509. Enable Cross Origin Filter for timeline server only and not all Yarn servers (Mit Desai via jeagles) 2014-09-04 11:53:52 -05:00
Karthik Kambatla
1dcaba9a7a YARN-2394. FairScheduler: Configure fairSharePreemptionThreshold per queue. (Wei Yan via kasha) 2014-09-03 10:27:36 -07:00
Karthik Kambatla
0f34e6f387 YARN-2395. FairScheduler: Preemption timeout should be configurable per queue. (Wei Yan via kasha) 2014-08-30 01:17:13 -07:00
Jian He
5c14bc426b YARN-1506. Changed RMNode/SchedulerNode to update resource with event notification. Contributed by Junping Du 2014-08-29 23:05:51 -07:00
Karthik Kambatla
270a271f53 YARN-2360. Fair Scheduler: Display dynamic fair share for queues on the scheduler page. (Ashwin Shankar and Wei Yan via kasha) 2014-08-29 17:15:38 -07:00
Jason Lowe
b03653f9a5 YARN-2462. TestNodeManagerResync#testBlockNewContainerRequestsOnStartAndResync should have a test timeout. Contributed by Eric Payne 2014-08-29 20:15:40 +00:00
Jian He
c686aa3533 YARN-2447. RM web service app submission doesn't pass secrets correctly. Contributed by Varun Vasudev 2014-08-29 11:40:47 -07:00
Hitesh Shah
3de66011c2 YARN-2450. Fix typos in log messages. Contributed by Ray Chiang. 2014-08-29 11:16:36 -07:00
Zhijie Shen
4bd0194e6b YARN-2449. Fixed the bug that TimelineAuthenticationFilterInitializer is not automatically added when hadoop.http.filter.initializers is not configured. Contributed by Varun Vasudev. 2014-08-29 09:40:39 -07:00
Karthik Kambatla
fa80ca49bd YARN-2405. NPE in FairSchedulerAppsBlock. (Tsuyoshi Ozawa via kasha) 2014-08-28 23:21:37 -07:00
Jian
7b3e27ab73 YARN-2406. Move RM recovery related proto to yarn_server_resourcemanager_recovery.proto. Contributed by Tsuyoshi OZAWA 2014-08-28 21:47:26 -07:00
Allen Wittenauer
d8774cc577 HADOOP-11013. CLASSPATH handling should be consolidated, debuggable (aw) 2014-08-28 18:09:25 -07:00
Jian He
e2d0ff364a YARN-2182. Updated ContainerId#toString() to append RM Epoch number. Contributed by Tsuyoshi OZAWA 2014-08-27 10:02:45 -07:00
Allen Wittenauer
9ec4a930f5 HADOOP-10996. Stop violence in the *_HOME (aw) 2014-08-27 07:00:31 -07:00
Zhijie Shen
d778abf022 YARN-2035. FileSystemApplicationHistoryStore should not make working dir when it already exists. Contributed by Jonathan Eagles. 2014-08-27 02:01:00 -07:00
Karthik Kambatla
d16bfd1d0f YARN-1326. RM should log using RMStore at startup time. (Tsuyoshi Ozawa via kasha) 2014-08-27 01:43:58 -07:00
Karthik Kambatla
0097b15e21 YARN-2393. FairScheduler: Add the notion of steady fair share. (Wei Yan via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1619845 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-22 15:44:47 +00:00
Jason Darrell Lowe
4236c6600e YARN-2434. RM should not recover containers from previously failed attempt when AM restart is not enabled. Contributed by Jian He 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1619614 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-21 22:41:34 +00:00
Jason Darrell Lowe
da4ba50269 HADOOP-10893. isolated classloader on the client side. Contributed by Sangjin Lee 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1619604 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-21 21:38:16 +00:00
Allen Wittenauer
cbbb899aed YARN-2436. [post-HADOOP-9902] yarn application help doesn't work 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1619603 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-21 21:33:35 +00:00
Allen Wittenauer
7e75226e68 YARN-2424. LCE should support non-cgroups, non-secure mode (Chris Douglas via aw) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1619421 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-21 14:57:11 +00:00
Karthik Kambatla
e1dd210fa5 YARN-1919. Potential NPE in EmbeddedElectorService#stop. (Tsuyoshi Ozawa via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1619251 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-20 22:10:52 +00:00
Jason Darrell Lowe
55a2b550b5 YARN-2034. Description for yarn.nodemanager.localizer.cache.target-size-mb is incorrect. Contributed by Chen He 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1619176 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-20 17:38:51 +00:00
Jian He
e4539e88e3 YARN-2174. Enable HTTPs for the writer REST API of TimelineServer. Contributed by Zhijie Shen 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1619160 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-20 17:05:07 +00:00
Zhijie Shen
f6a778c372 YARN-2249. Avoided AM release requests being lost on work preserving RM restart. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618972 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-19 20:33:49 +00:00
Jian He
375c221960 YARN-2409. RM ActiveToStandBy transition missing stoping previous rmDispatcher. Contributed by Rohith 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618915 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-19 17:49:39 +00:00
Allen Wittenauer
31467453ec HADOOP-9902. Shell script rewrite (aw) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618847 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-19 12:11:17 +00:00
Jian He
519c4be95a YARN-2411. Support simple user and group mappings to queues. Contributed by Ram Venkatesh 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618542 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-18 06:08:45 +00:00
Jian He
c3084d6c16 YARN-2389. Added functionality for schedulers to kill all applications in a queue. Contributed by Subramaniam Venkatraman Krishnan 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618294 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-15 23:53:57 +00:00
Jian He
7360cec692 YARN-2378. Added support for moving applications across queues in CapacityScheduler. Contributed by Subramaniam Venkatraman Krishnan 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618106 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-15 06:00:31 +00:00
Allen Wittenauer
0350ea3c72 YARN-1918. Typo in description and error message for yarn.resourcemanager.cluster-id (Anandha L Ranganathan via aw) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618070 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-14 21:53:12 +00:00
Zhijie Shen
a9023c2736 YARN-2397. Avoided loading two authentication filters for RM and TS web interfaces. Contributed by Varun Vasudev. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618054 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-14 21:17:20 +00:00
Zhijie Shen
1a535f8dcd YARN-2070. Made DistributedShell publish the short user name to the timeline server. Contributed by Robert Kanter. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617837 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-13 21:07:18 +00:00
Zhijie Shen
41aa4badf8 YARN-2277. Added cross-origin support for the timeline server web services. Contributed by Jonathan Eagles. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617832 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-13 20:29:23 +00:00
Karthik Kambatla
5197f8c3c5 YARN-1370. Fair scheduler to re-populate container allocation state. (Anubhav Dhoot via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617645 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-13 01:38:59 +00:00
Karthik Kambatla
4239695588 YARN-2399. Delete old versions of files. FairScheduler: Merge AppSchedulable and FSSchedulerApp into FSAppAttempt. (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617619 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-12 22:51:57 +00:00
Karthik Kambatla
486e718fc1 YARN-2399. FairScheduler: Merge AppSchedulable and FSSchedulerApp into FSAppAttempt. (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617600 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-12 21:43:27 +00:00
Zhijie Shen
7ad7ab1723 YARN-2317. Updated the document about how to write YARN applications. Contributed by Li Lu. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617594 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-12 21:37:34 +00:00
Jian He
0c9b8f2d7f YARN-2373. Changed WebAppUtils to use Configuration#getPassword for accessing SSL passwords. Contributed by Larry McCay 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617555 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-12 18:29:42 +00:00
Junping Du
c2febdcbaa YARN-1337. Recover containers upon nodemanager restart. (Contributed by Jason Lowe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617448 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-12 10:56:13 +00:00
Jian He
c4dc685343 YARN-2138. Cleaned up notifyDone* APIs in RMStateStore. Contributed by Varun Saxena 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617341 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-11 18:24:24 +00:00
Xuan Gong
946be75704 YARN-2400: Addendum fix for TestAMRestart failure. Contributed by Jian He 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617333 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-11 17:42:53 +00:00
Karthik Kambatla
da7b508ffc YARN-2361. RMAppAttempt state machine entries for KILLED state has duplicate event entries. (Zhihai Xu via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617190 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-11 01:42:26 +00:00
Karthik Kambatla
bdd3e2ce49 YARN-2337. ResourceManager sets ClientRMService in RMContext multiple times. (Zhihai Xu via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617183 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-11 00:13:27 +00:00
Junping Du
e91d099c4a YARN-2302. Refactor TimelineWebServices. (Contributed by Zhijie Shen) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617055 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-10 07:21:15 +00:00
Xuan Gong
743f7f30da YARN-2400. Fixed TestAMRestart fails intermittently. Contributed by Jian He: 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617028 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-09 23:31:11 +00:00
Zhijie Shen
ee3825e278 YARN-1954. Added waitFor to AMRMClient(Async). Contributed by Tsuyoshi Ozawa. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617002 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-09 18:44:51 +00:00
Karthik Kambatla
a7643f4de7 YARN-2026. Fair scheduler: Consider only active queues for computing fairshare. (Ashwin Shankar via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616915 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-09 02:10:00 +00:00
Xuan Gong
eeb4acd955 YARN-2212: ApplicationMaster needs to find a way to update the AMRMToken periodically. Contributed by Xuan Gong 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616892 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-08 21:38:24 +00:00
Karthik Kambatla
6d39525f6b YARN-2352. Add missing file. FairScheduler: Collect metrics on duration of critical methods that affect performance. (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616784 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-08 14:38:18 +00:00
Karthik Kambatla
14864e9c7c YARN-2352. FairScheduler: Collect metrics on duration of critical methods that affect performance. (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616769 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-08 14:17:54 +00:00
Jian He
8437df8ba9 YARN-2008. Fixed CapacityScheduler to calculate headroom based on max available capacity instead of configured max capacity. Contributed by Craig Welch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616580 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-07 20:00:04 +00:00
Zhijie Shen
3bfdb0091d YARN-2288. Made persisted data in LevelDB timeline store be versioned. Contributed by Junping Du. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616540 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-07 17:02:26 +00:00
Zhijie Shen
be6360593b YARN-2388. Fixed TestTimelineWebServices failure due to HADOOP-10791. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616405 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-07 05:36:50 +00:00
Karthik Kambatla
8feddc4c84 YARN-2359. Application hangs when it fails to launch AM container. (Zhihai Xu via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616375 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-07 00:06:17 +00:00
Jian He
9ebeac16eb YARN-2374. Fixed TestDistributedShell#testDSShell failure due to hostname dismatch. Contributed by Varun Vasudev 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616302 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-06 18:58:50 +00:00
Junping Du
b47f65214c YARN-2298. Move TimelineClient to yarn-common project (Contributed by Zhijie Shen) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616100 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-06 05:48:32 +00:00
Junping Du
b8f151231b YARN-1354. Recover applications upon nodemanager restart. (Contributed by Jason Lowe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1615550 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-04 13:25:37 +00:00
Junping Du
04ae560370 YARN-2370. Fix comment in o.a.h.y.server.resourcemanager.schedulerAppSchedulingInfo (Contributed by Wenwu Peng) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1615469 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-04 01:15:46 +00:00
Jian He
a41c314373 YARN-2343. Improve NMToken expire exception message. Contributed by Li Lu 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1615270 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-01 23:44:48 +00:00
Junping Du
1141edc424 YARN-2051. Fix bug in PBimpls and add more unit tests with reflection. (Contributed by Binglin Chang) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1615025 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-01 04:16:08 +00:00
Xuan Gong
e52f67e389 YARN-1994. Expose YARN/MR endpoints on multiple interfaces. Contributed by Craig Welch, Milan Potocnik,and Arpit Agarwal 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1614981 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-31 20:06:02 +00:00
Zhijie Shen
1d6e178144 YARN-2347. Consolidated RMStateVersion and NMDBSchemaVersion into Version in yarn-server-common. Contributed by Junping Du. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1614838 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-31 09:27:43 +00:00
Jian He
7e54b1c6d9 YARN-2354. DistributedShell may allocate more containers than client specified after AM restarts. Contributed by Li Lu 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1614538 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-30 03:58:59 +00:00
Karthik Kambatla
c0b49ff107 YARN-2328. FairScheduler: Verify update and continuous scheduling threads are stopped when the scheduler is stopped. (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1614432 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-29 17:41:52 +00:00
Zhijie Shen
d6532d3a77 YARN-2247. Made RM web services authenticate users via kerberos and delegation token. Contributed by Varun Vasudev. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1613821 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-27 17:55:06 +00:00
Aaron Myers
5d4677b57b YARN-1796. container-executor shouldn't require o-r permissions. Contributed by Aaron T. Myers. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1613548 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-26 01:51:35 +00:00
Jian He
d4fec34933 YARN-2211. Persist AMRMToken master key in RMStateStore for RM recovery. Contributed by Xuan Gong 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1613515 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-25 20:42:37 +00:00
Karthik Kambatla
1e553858f9 YARN-2214. FairScheduler: preemptContainerPreCheck() in FSParentQueue delays convergence towards fairness. (Ashwin Shankar via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1613459 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-25 16:13:07 +00:00
Devarajulu K
2050e0dad6 YARN-1342. Recover container tokens upon nodemanager restart. Contributed by Jason Lowe. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612995 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-24 05:02:00 +00:00
Zhijie Shen
06e5c5cb2d YARN-2300. Improved the documentation of the sample requests for RM REST API - submitting an app. Contributed by Varun Vasudev. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612981 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-24 03:12:45 +00:00
Jason Darrell Lowe
28fca92521 YARN-2147. client lacks delegation token exception details when application submit fails. Contributed by Chen He 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612950 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-23 21:40:57 +00:00
Sanford Ryza
c88402f36d YARN-2313. Livelock can occur in FairScheduler when there are lots of running apps (Tsuyoshi Ozawa via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612769 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-23 05:00:52 +00:00
Karthik Kambatla
ff77582991 YARN-2273. NPE in ContinuousScheduling thread when we lose a node. (Wei Yan via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612720 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-22 22:44:38 +00:00
Karthik Kambatla
df2abcb372 YARN-2131. Addendum2: Document -format-state-store. Add a way to format the RMStateStore. (Robert Kanter via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612634 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-22 18:11:11 +00:00
Jian He
f6c723ff0c YARN-2295. Refactored DistributedShell to use public APIs of protocol records. Contributed by Li Lu 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612626 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-22 17:49:14 +00:00
Zhijie Shen
eac0701c96 YARN-2319. Made the MiniKdc instance start/close before/after the class of TestRMWebServicesDelegationTokens. Contributed by Wenwu Peng. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612588 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-22 15:15:29 +00:00
Junping Du
afb9394c91 YARN-2242. Addendum patch. Improve exception information on AM launch crashes. (Contributed by Li Lu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612565 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-22 13:07:23 +00:00
Zhijie Shen
70dededdc9 YARN-2270. Made TestFSDownload#testDownloadPublicWithStatCache be skipped when there’s no ancestor permissions. Contributed by Akira Ajisaka. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612460 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-22 04:58:00 +00:00
Junping Du
537c361f5b YARN-2013. The diagnostics is always the ExitCodeException stack when the container crashes. (Contributed by Tsuyoshi OZAWA) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612449 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-22 03:01:58 +00:00
Karthik Kambatla
266a4ff6a5 YARN-2131. Addendum. Add a way to format the RMStateStore. (Robert Kanter via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612443 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-22 01:30:36 +00:00
Jason Darrell Lowe
1ad2d7b405 YARN-2321. NodeManager web UI can incorrectly report Pmem enforcement. Contributed by Leitao Guo 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612411 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-21 21:55:06 +00:00
Jason Darrell Lowe
8a87085820 YARN-2045. Data persisted in NM should be versioned. Contributed by Junping Du 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612285 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-21 14:43:59 +00:00
Sanford Ryza
7c71a3b876 YARN-2323. FairShareComparator creates too many Resource objects (Hong Zhiguo via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612187 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-21 05:00:27 +00:00
Karthik Kambatla
8871d8ed9f YARN-2244. FairScheduler missing handling of containers for unknown application attempts. (Anubhav Dhoot via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1611840 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-19 00:12:05 +00:00
Xuan Gong
f1b831ccfb YARN-2208. AMRMTokenManager need to have a way to roll over AMRMToken. Contributed by Xuan Gong 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1611820 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-18 21:46:29 +00:00
Junping Du
403ec8ea80 YARN-1341. Recover NMTokens upon nodemanager restart. (Contributed by Jason Lowe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1611512 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-17 23:33:22 +00:00
Jian He
3c193811ca YARN-2219. Addendum patch for YARN-2219 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1611240 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-17 03:28:39 +00:00
Vinod Kumar Vavilapalli
bda23181bf YARN-2219. Changed ResourceManager to avoid AMs and NMs getting exceptions after RM recovery but before scheduler learns about apps and app-attempts. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1611222 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-17 00:14:56 +00:00
Jian He
40e1bb9d31 YARN-2264. Fixed a race condition in DrainDispatcher which may cause random test failures. Contributed by Li Lu 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1611126 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-16 18:25:05 +00:00
Vinayakumar B
dc31d66f8a HADOOP-9921. daemon scripts should remove pid file on stop call after stop or process is found not running ( Contributed by Vinayakumar B) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1610964 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-16 10:48:15 +00:00
Vinod Kumar Vavilapalli
030580387a YARN-2233. Implemented ResourceManager web-services to create, renew and cancel delegation tokens. Contributed by Varun Vasudev. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1610876 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-15 23:00:17 +00:00
Mayank Bansal
43589a8df7 YARN-1408 Preemption caused Invalid State Event: ACQUIRED at KILLED and caused a task timeout for 30mins. (Sunil G via mayank) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1610860 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-15 21:48:58 +00:00
Vinod Kumar Vavilapalli
a6cd712ca7 YARN-2228. Augmented TimelineServer to load pseudo authentication filter when authentication = simple. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1610575 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-15 04:33:48 +00:00
Vinod Kumar Vavilapalli
c6cc6a6a8e YARN-2260. Fixed ResourceManager's RMNode to correctly remember containers when nodes resync during work-preserving RM restart. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1610557 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-14 23:32:03 +00:00
Karthik Kambatla
4d211d5fde YARN-2274. FairScheduler: Add debug information about cluster capacity, availability and reservations. (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1609942 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-12 15:10:19 +00:00
Xuan Gong
c0439b732c YARN-2269. Remove external links from YARN UI. Contributed by Craig Welch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1609590 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-10 23:31:55 +00:00
Jian He
50be8561a7 YARN-2088. Fixed a bug in GetApplicationsRequestPBImpl#mergeLocalToBuilder. Contributed by Binglin Chang 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1609584 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-10 22:36:13 +00:00
Jian He
c9fb040c87 YARN-2181. Added preemption info to logs and RM web UI. Contributed by Wangda Tan 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1609561 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-10 20:03:35 +00:00
Karthik Kambatla
8fbca62a90 YARN-2131. Add a way to format the RMStateStore. (Robert Kanter via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1609278 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-09 19:58:43 +00:00
Jian He
d751a61e5a YARN-1366. Changed AMRMClient to re-register with RM and send outstanding requests back to RM on work-preserving RM restart. Contributed by Rohith 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1609254 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-09 18:25:45 +00:00
Zhijie Shen
12c4197b35 YARN-2158. Improved assertion messages of TestRMWebServicesAppsModification. Contributed by Varun Vasudev. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1608667 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-08 05:50:04 +00:00
Jian He
6d7dbd4fed YARN-1367. Changed NM to not kill containers on NM resync if RM work-preserving restart is enabled. Contributed by Anubhav Dhoot 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1608334 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-07 04:37:59 +00:00
Sanford Ryza
5644f529f3 YARN-2250. FairScheduler.findLowestCommonAncestorQueue returns null when queues not identical (Krisztian Horvath via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1607872 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-04 15:16:43 +00:00
Junping Du
4074843fea YARN-2251. Avoid negative elapsed time in JHS/MRAM web UI and services (Contributed by Zhijie Shen) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1607833 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-04 11:18:05 +00:00
Junping Du
5cb489f9d3 YARN-2242. Improve exception information on AM launch crashes. (Contributed by Li Lu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1607655 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-03 14:15:19 +00:00
Vinod Kumar Vavilapalli
45b191e38c YARN-2232. Fixed ResourceManager to allow DelegationToken owners to be able to cancel their own tokens in secure mode. Contributed by Varun Vasudev. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1607484 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-02 21:36:42 +00:00
Karthik Kambatla
447c1c233b YARN-2241. ZKRMStateStore: On startup, show nicer messages if znodes already exist. (Robert Kanter via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1607473 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-02 20:33:26 +00:00
Steve Loughran
d1f54f4f4b YARN-2065 AM cannot create new containers after restart 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1607441 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-02 18:35:10 +00:00
Karthik Kambatla
58e398f31d YARN-2204. Explicitly enable vmem check in TestContainersMonitor#testContainerKillOnMemoryOverflow. (Anubhav Dhoot via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1607231 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-02 02:05:37 +00:00
Mayank Bansal
03a25d2cc1 YARN-2022 Preempting an Application Master container can be kept as least priority when multiple applications are marked for preemption by ProportionalCapacityPreemptionPolicy (Sunil G via mayank) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1607227 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-02 01:54:47 +00:00
Vinod Kumar Vavilapalli
075ff276ca YARN-1713. Added get-new-app and submit-app functionality to RM web services. Contributed by Varun Vasudev. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1607216 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-02 00:23:07 +00:00
Xuan Gong
e5ae7c55d1 TestRMApplicationHistoryWriter sometimes fails in trunk. Contributed by Zhijie Shen 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1606835 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-30 16:51:22 +00:00
Jian He
b0c51504c4 YARN-2052. Embedded an epoch number in container id to ensure the uniqueness of container id after RM restarts. Contributed by Tsuyoshi OZAWA 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1606557 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-29 18:24:03 +00:00
Jian He
b717d44b52 YARN-614. Changed ResourceManager to not count disk failure, node loss and RM restart towards app failures. Contributed by Xuan Gong 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1606407 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-28 23:37:46 +00:00
Zhijie Shen
55a0aa0bad YARN-2201. Made TestRMWebServicesAppsModification be independent of the changes on yarn-default.xml. Contributed by Varun Vasudev. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1606285 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-28 03:30:44 +00:00
Jason Darrell Lowe
73927ba695 YARN-2104. Scheduler queue filter failed to work because index of queue column changed. Contributed by Wangda Tan 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1606265 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-27 23:11:12 +00:00
Karthik Kambatla
f911f5495b YARN-2204. Addendum patch. TestAMRestart#testAMRestartWithExistingContainers assumes CapacityScheduler. (Robert Kanter via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1606168 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-27 18:09:41 +00:00
Ravi Prakash
f084170a0a YARN 2163. WebUI: Order of AppId in apps table should be consistent with ApplicationId.compareTo() 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1605964 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-27 06:43:46 +00:00
Vinod Kumar Vavilapalli
9571db19eb YARN-2171. Improved CapacityScheduling to not lock on nodemanager-count when AMs heartbeat in. Contributed by Jason Lowe. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1605616 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-25 21:56:42 +00:00
Karthik Kambatla
1a3a7e0c1a YARN-2204. TestAMRestart#testAMRestartWithExistingContainers assumes CapacityScheduler. (Robert Kanter via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1605548 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-25 18:50:53 +00:00
Jian He
c3f1c30e65 YARN-1365. Changed ApplicationMasterService to allow an app to re-register after RM restart. Contributed by Anubhav Dhoot 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1605263 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-25 04:42:39 +00:00
Vinod Kumar Vavilapalli
e285b98f0f YARN-2152. Added missing information into ContainerTokenIdentifier so that NodeManagers can report the same to RM when RM restarts. Contributed Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1605205 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-24 21:43:22 +00:00
Thomas Graves
1f9a0fd927 YARN-2072. RM/NM UIs and webservices are missing vcore information. (Nathan Roberts via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1605162 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-24 19:34:34 +00:00
Karthik Kambatla
c0991d11eb YARN-2109. Fix TestRM to work with both schedulers. (Anubhav Dhoot via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1605142 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-24 17:30:53 +00:00
Karthik Kambatla
db4d277117 YARN-2192. TestRMHA fails when run with a mix of Schedulers. (Anubhav Dhoot via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1605138 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-24 17:05:35 +00:00
Sanford Ryza
29c102cad0 YARN-2111. In FairScheduler.attemptScheduling, we don't count containers as assigned if they have 0 memory but non-zero cores (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1605113 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-24 15:40:39 +00:00
Vinod Kumar Vavilapalli
d16470025a YARN-2074. Changed ResourceManager to not count AM preemptions towards app failures. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1605106 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-24 15:15:12 +00:00
Devarajulu K
cb2b34b01c YARN-2195. Clean a piece of code in ResourceRequest. Contributed by Wei Yan. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1605083 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-24 13:57:31 +00:00
Jian He
59b5e9fa15 YARN-2191. Added a new test to ensure NM will clean up completed applications in the case of RM restart. Contributed by Wangda Tan 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1604949 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-23 22:52:38 +00:00
Karthik Kambatla
6fcbf9b848 YARN-2187. FairScheduler: Disable max-AM-share check by default. (Robert Kanter via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1604321 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-21 07:30:07 +00:00
Haohui Mai
22cbcd16cb HADOOP-10717. Revert r1603571. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1603620 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-18 20:37:25 +00:00
Steve Loughran
bae5837d3b HADOOP-10717. Missing JSP support in Jetty 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1603534 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-18 17:08:45 +00:00
Jason Darrell Lowe
98238a8d4a YARN-2167. LeveldbIterator should get closed in NMLeveldbStateStoreService#loadLocalizationState() within finally block. Contributed by Junping Du 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1603039 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-17 02:12:03 +00:00
Junping Du
072360d128 YARN-1339. Recover DeletionService state upon nodemanager restart. (Contributed by Jason Lowe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1603036 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-17 01:02:16 +00:00
Jian He
95897ca14b YARN-1885. Fixed a bug that RM may not send application-clean-up signal to NMs where the completed applications previously ran in case of RM restart. Contributed by Wangda Tan 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1603028 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-16 23:56:12 +00:00
Karthik Kambatla
082c2ab8b8 YARN-2159. Better logging in SchedulerNode#allocateContainer. (Ray Chiang via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1603003 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-16 20:45:38 +00:00
Vinod Kumar Vavilapalli
dc7dd1fa19 YARN-1702. Added kill app functionality to RM web services. Contributed by Varun Vasudev. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1602298 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-12 21:31:52 +00:00
Karthik Kambatla
4bc91b44c9 YARN-2155. FairScheduler: Incorrect threshold check for preemption. (Wei Yan via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1602295 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-12 21:23:32 +00:00
Zhijie Shen
deb858a835 YARN-2075. Fixed the test failure of TestRMAdminCLI. Contributed by Kenji Kikushima. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1602071 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-12 06:40:32 +00:00
Bikas Saha
2584d531e1 YARN-2148. TestNMClient failed due more exit code values added and passed to AM (Wangda Tan via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1602043 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-11 22:45:09 +00:00
Jian He
c6e29a9f06 YARN-2125. Changed ProportionalCapacityPreemptionPolicy to log CSV in debug level. Contributed by Wangda Tan 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1601980 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-11 18:31:28 +00:00
Jian He
710a8693e5 YARN-2124. Fixed NPE in ProportionalCapacityPreemptionPolicy. Contributed by Wangda Tan 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1601964 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-11 17:30:18 +00:00
Bikas Saha
ecfd43a2f1 YARN-2091. Add more values to ContainerExitStatus and pass it from NM to RM and then to app masters (Tsuyoshi OZAWA via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1601762 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-10 20:08:33 +00:00
Karthik Kambatla
5de6f72054 YARN-1424. RMAppAttemptImpl should return the DummyApplicationResourceUsageReport for all invalid accesses. (Ray Chiang via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1601742 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-10 19:03:06 +00:00
Jian He
c94f2cec3a Augmented RMStateStore with state machine. Contributed by Binglin Chang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1601491 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-09 19:44:31 +00:00
Vinod Kumar Vavilapalli
424fd9494f YARN-1368. Added core functionality of recovering container state into schedulers after ResourceManager Restart so as to preserve running work in the cluster. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1601303 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-09 03:09:21 +00:00
Karthik Kambatla
9958072d53 YARN-2132. ZKRMStateStore.ZKAction#runWithRetries doesn't log the exception it encounters. (Vamsee Yarlagadda via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1601065 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-07 06:12:59 +00:00
Karthik Kambatla
85d4c787e0 YARN-2128. FairScheduler: Incorrect calculation of amResource usage. (Wei Yan via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1601050 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-07 01:21:33 +00:00
Karthik Kambatla
969bef3c77 YARN-2122. In AllocationFileLoaderService, the reloadThread should be created in init() and started in start(). (Robert Kanter via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1601045 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-07 00:49:20 +00:00
Vinod Kumar Vavilapalli
825365d1d0 YARN-2121. Fixed NPE handling in Timeline Server's TimelineAuthenticator. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1601000 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-06 20:32:43 +00:00
Zhijie Shen
08b4aa699a YARN-2117. Fixed the issue that secret file reader is potentially not closed in TimelineAuthenticationFilterInitializer. Contributed by Chen He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1600994 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-06 20:07:58 +00:00
Zhijie Shen
d7f5d76869 YARN-2118. Fixed the type mismatch in Map#containsKey check of TimelineWebServices#injectOwnerInfo. Contributed by Ted Yu. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1600967 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-06 17:47:03 +00:00
Junping Du
0ceb742549 YARN-1977. Add tests on getApplicationRequest with filtering start time range. (Contributed by Junping Du) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1600644 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-05 13:15:44 +00:00
Karthik Kambatla
d1d6bb70f4 YARN-2061. Revisit logging levels in ZKRMStateStore. (Ray Chiang via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1600498 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-04 21:11:09 +00:00
Karthik Kambatla
731f417f67 YARN-2119. DEFAULT_PROXY_ADDRESS should use DEFAULT_PROXY_PORT. (Anubhav Dhoot via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1600484 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-04 20:12:47 +00:00
Sanford Ryza
16caa3fd18 YARN-1913. With Fair Scheduler, cluster can logjam when all resources are consumed by AMs (Wei Yan via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1599400 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-03 00:56:48 +00:00
Karthik Kambatla
0aad2d56df YARN-1550. NPE in FairSchedulerAppsBlock#render. (Anubhav Dhoot via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1599345 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-02 20:22:52 +00:00
Junping Du
790cbbf3e2 YARN-2103. Inconsistency between viaProto flag and initial value of SerializedExceptionProto.Builder (Contributed by Binglin Chang) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1599115 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-02 08:15:30 +00:00
Karthik Kambatla
a4ba451802 YARN-1474. Make schedulers services. (Tsuyoshi Ozawa via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598908 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-31 19:33:09 +00:00
Vinod Kumar Vavilapalli
23c325ad47 YARN-2115. Replaced RegisterNodeManagerRequest's ContainerStatus with a new NMContainerStatus which has more information that is needed for work-preserving RM-restart. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598790 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-31 00:20:50 +00:00
Chris Nauroth
d599adcc6a YARN-1868. YARN status web ui does not show correctly in IE 11. Contributed by Chuan Liu. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598686 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-30 18:07:23 +00:00
Junping Du
66598697a6 YARN-1338. Recover localized resource cache state upon nodemanager restart (Contributed by Jason Lowe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598640 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-30 15:37:27 +00:00
Karthik Kambatla
75112e97b4 YARN-2010. Document yarn.resourcemanager.zk-auth and its scope. (Robert Kanter via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598636 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-30 15:32:23 +00:00
Karthik Kambatla
49a3a0cd0c YARN-2054. Better defaults for YARN ZK configs for retries and retry-inteval when HA is enabled. (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598630 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-30 15:24:49 +00:00
Arpit Agarwal
4a4868e523 HADOOP-10448. Support pluggable mechanism to specify proxy user settings (Contributed by Benoy Antony) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598396 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-29 20:52:01 +00:00
Vinod Kumar Vavilapalli
cd24e955cf YARN-2112. Fixed yarn-common's pom.xml to include jackson dependencies so that both Timeline Server and client can access them. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598373 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-29 19:06:36 +00:00
Chris Nauroth
806f4d11f2 HADOOP-10602. Documentation has broken "Go Back" hyperlinks. Contributed by Akira AJISAKA. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598337 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-29 17:26:50 +00:00
Sanford Ryza
342da5b4d3 YARN-596. Use scheduling policies throughout the queue hierarchy to decide which containers to preempt (Wei Yan via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598197 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-29 04:01:24 +00:00
Zhijie Shen
001078e067 YARN-2107. Refactored timeline classes into o.a.h.y.s.timeline package. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598094 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-28 18:09:04 +00:00
Sanford Ryza
edfbc8ad4a YARN-2105. Fix TestFairScheduler after YARN-2012. (Ashwin Shankar via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1597902 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-27 23:46:22 +00:00
Vinod Kumar Vavilapalli
4dd2003373 YARN-2071. Modified levelDB store permissions to be readable only by the server user. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1597231 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-24 01:24:26 +00:00
Karthik Kambatla
7dd378c274 YARN-2096. Race in TestRMRestart#testQueueMetricsOnRMRestart. (Anubhav Dhoot via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1597223 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-23 23:51:00 +00:00
Sanford Ryza
a00b2d4f37 YARN-2073. Fair Scheduler: Add a utilization threshold to prevent preempting resources when cluster is free (Karthik Kambatla via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1597209 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-23 22:52:46 +00:00
Vinod Kumar Vavilapalli
7f0724dddb YARN-2059. Added admin ACLs support to Timeline Server. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1597207 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-23 22:40:52 +00:00
Sanford Ryza
6c56612af5 YARN-2012. Fair Scheduler: allow default queue placement rule to take an arbitrary queue (Ashwin Shankar via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1597204 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-23 22:38:52 +00:00
Vinod Kumar Vavilapalli
892ada8fb3 YARN-1937. Added owner-only ACLs support for Timeline Client and server. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1597186 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-23 21:19:15 +00:00
Vinod Kumar Vavilapalli
af8b09af75 YARN-1936. Added security support for the Timeline Client. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1597153 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-23 18:47:57 +00:00
Vinod Kumar Vavilapalli
b867b69556 YARN-2049. Added delegation-token support for the Timeline Server. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1597130 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-23 17:09:47 +00:00
Karthik Kambatla
619ec833fc YARN-2089. FairScheduler: QueuePlacementPolicy and QueuePlacementRule are missing audience annotations. (Zhihai Xu via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1596765 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-22 06:13:24 +00:00
Vinod Kumar Vavilapalli
82f3454f5a YARN-2017. Merged some of the common scheduler code. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1596753 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-22 05:32:26 +00:00
Zhijie Shen
28058894dd YARN-2081. Fixed TestDistributedShell failure after YARN-1962. Contributed by Zhiguo Hong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1596724 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-22 01:16:14 +00:00
Vinod Kumar Vavilapalli
fdb5870d49 YARN-1938. Added kerberos login for the Timeline Server. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1596710 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-21 23:20:08 +00:00
Jason Darrell Lowe
64ec743e6c YARN-2050. Fix LogCLIHelpers to create the correct FileContext. Contributed by Ming Ma 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1596310 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-20 16:19:55 +00:00
Junping Du
461ded8bd4 YARN-2066. Wrong field is referenced in GetApplicationsRequestPBImpl#mergeLocalToBuilder (Contributed by Hong Zhiguo) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1595413 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-17 01:43:15 +00:00
Ravi Prakash
c9377189a0 MAPREDUCE 5874. Creating MapReduce REST API section. Contributed by Tsuyoshi Ozawa 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1595265 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-16 16:50:58 +00:00
Jian He
0f9147c857 YARN-2053. Fixed a bug in AMS to not add null NMToken into NMTokens list from previous attempts for work-preserving AM restart. Contributed by Wangda Tan 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1595116 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-16 06:22:22 +00:00
Sanford Ryza
561c014e77 YARN-2042. String shouldn't be compared using == in QueuePlacementRule#NestedUserQueue#getQueueForApp (Chen He via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1594482 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-14 06:55:04 +00:00
Sanford Ryza
84dfae2f8a YARN-1986. In Fifo Scheduler, node heartbeat in between creating app and attempt causes NPE (Hong Zhiguo via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1594476 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-14 06:41:20 +00:00
Junping Du
b29434a5c8 YARN-1362. Distinguish between nodemanager shutdown for decommission vs shutdown for restart. (Contributed by Jason Lowe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1594421 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-14 00:20:53 +00:00
Christopher Douglas
45b42676f9 YARN-1957. Consider the max capacity of the queue when computing the ideal 
capacity for preemption. Contributed by Carlo Curino


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1594414 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-13 23:15:27 +00:00
Jonathan Turner Eagles
1c48142807 YARN-1981. Nodemanager version is not updated when a node reconnects (Jason Lowe via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1594358 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-13 20:03:58 +00:00
Vinod Kumar Vavilapalli
2f87e77cb7 YARN-1861. Fixed a bug in RM to reset leader-election on fencing that was causing both RMs to be stuck in standby mode when automatic failover is enabled. Contributed by Karthik Kambatla and Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1594356 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-13 20:00:44 +00:00
Jason Darrell Lowe
b0e80d1a47 MAPREDUCE-5814. fat jar with *-default.xml may fail when mapreduce.job.classloader=true. Contributed by Gera Shegalov 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1594326 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-13 18:55:05 +00:00
Jason Darrell Lowe
95ba9f47fc YARN-1751. Improve MiniYarnCluster for log aggregation testing. Contributed by Ming Ma 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1594275 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-13 16:43:59 +00:00
Jian He
4f6493bcf9 YARN-2016. Fix a bug in GetApplicationsRequestPBImpl to add the missed fields to proto. Contributed by Junping Du 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1594085 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-12 20:45:21 +00:00
Jian He
41344a4a69 YARN-1975. Fix yarn application CLI to print the scheme of the tracking url of failed/killed applications. Contributed by Junping Du 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1593874 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-12 00:43:35 +00:00
Junping Du
ca95af7d23 YARN-2011. Fix typo and warning in TestLeafQueue (Contributed by Chen He) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1593804 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-11 15:13:29 +00:00
Karthik Kambatla
4a19196a34 YARN-1987. Wrapper for leveldb DBIterator to aid in handling database exceptions. (Jason Lowe via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1593757 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-11 06:11:04 +00:00
Vinod Kumar Vavilapalli
58c81d56e9 YARN-1962. Changed Timeline Service client configuration to be off by default given the non-readiness of the feature yet. Contributed by Mohammad Kamrul Islam. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1593750 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-11 00:15:12 +00:00
Vinod Kumar Vavilapalli
fa6fd973cf YARN-1982. Renamed the daemon name to be TimelineServer instead of History Server and deprecated the old usage. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1593748 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-11 00:04:49 +00:00
Junping Du
72ba2b35d8 YARN-766. TestNodeManagerShutdown in branch-2 should use Shell to form the output path and a format issue in trunk. (Contributed by Siddharth Seth) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1593660 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-10 03:45:01 +00:00
Zhijie Shen
0c437e90b8 YARN-1701. Improved default paths of the timeline store and the generic history store. Contributed by Tsuyoshi Ozawa. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1593481 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-09 04:23:27 +00:00
Sanford Ryza
1c5902fba2 YARN-1864. Add missing file FSQueueType.java 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1593191 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-08 07:22:42 +00:00
Sanford Ryza
cfc97a4e88 YARN-1864. Fair Scheduler Dynamic Hierarchical User Queues (Ashwin Shankar via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1593190 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-08 07:21:11 +00:00
Arpit Agarwal
f4b687b873 YARN-2018. TestClientRMService.testTokenRenewalWrongUser fails after HADOOP-10562. (Contributed by Ming Ma) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1592783 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-06 15:45:49 +00:00
Junping Du
2ad1cee5da YARN-1201. TestAMAuthorization fails with local hostname cannot be resolved. (Wangda Tan via junping_du) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1592197 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-03 13:03:27 +00:00
Haohui Mai
d777cb8996 HADOOP-10563. Remove the dependency of jsp in trunk. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1592069 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-02 22:26:02 +00:00
Vinod Kumar Vavilapalli
ef498235ad YARN-1696. Added documentation for ResourceManager fail-over. Contributed by Karthik Kambatla, Masatake Iwasaki, Tsuyoshi OZAWA. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1591416 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-30 18:00:32 +00:00
Vinod Kumar Vavilapalli
7a241aee90 YARN-1929. Fixed a deadlock in ResourceManager that occurs when failover happens right at the time of shutdown. Contributed by Karthik Kambatla. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1591071 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-29 19:49:44 +00:00
Jason Darrell Lowe
a9775b4e49 YARN-738. TestClientRMTokens is failing irregularly while running all yarn tests. Contributed by Ming Ma 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1591030 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-29 17:47:11 +00:00
Sanford Ryza
faffedeb24 YARN-483. Improve documentation on log aggregation in yarn-default.xml (Akira Ajisaka via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1590150 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-25 21:14:03 +00:00
Jing Zhao
6d4c7df434 HADOOP-10535. Make the retry numbers in ActiveStandbyElector configurable. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1589905 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-24 23:03:30 +00:00
Jason Darrell Lowe
c02ea74551 YARN-1975. Used resources shows escaped html in CapacityScheduler and FairScheduler page. Contributed by Mit Desai 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1589859 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-24 20:02:44 +00:00
Chris Nauroth
84388525a3 YARN-1970. Prepare YARN codebase for JUnit 4.11. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1589001 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-21 23:31:18 +00:00
Ivan Mitic
4810e2b849 YARN-1865. ShellScriptBuilder does not check for some error conditions. Contributed by Remus Rusanu. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1588693 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-19 18:55:07 +00:00
Jason Darrell Lowe
d667df4ed0 YARN-1932. Javascript injection on the job status page. Contributed by Mit Desai 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1588572 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-18 21:41:37 +00:00
Jason Darrell Lowe
cda8646cfa YARN-1940. deleteAsUser() terminates early without deleting more files on error. Contributed by Rushabh S Shah 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1588546 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-18 19:24:13 +00:00
Vinod Kumar Vavilapalli
bad021534c YARN-1281. Fixed TestZKRMStateStoreZKClientConnections to not fail intermittently due to ZK-client timeouts. Contributed by Tsuyoshi Ozawa. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1588369 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-17 20:57:15 +00:00
Junping Du
bd43d2481e YARN-1947. TestRMDelegationTokens#testRMDTMasterKeyStateOnRollingMasterKey is failing intermittently. (Jian He via junping_du) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1588365 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-17 20:27:37 +00:00
Junping Du
0d087f7941 YARN-1750. TestNodeStatusUpdater#testNMRegistration is incorrect in test case. (Wangda Tan via junping_du) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1588343 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-17 19:06:48 +00:00
Junping Du
36ed8c74ea YARN-1870. FileInputStream is not closed in ProcfsBasedProcessTree#constructProcessSMAPInfo. (Fengdong Yu via junping_du) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1588324 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-17 18:05:43 +00:00
Thomas Graves
ad9edb02b9 YARN-1931. Private API change in YARN-1824 in 2.4 broke compatibility with previous releases (Sandy Ryza via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1588281 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-17 15:23:22 +00:00
Jian He
38d5ca2e47 YARN-1934. Fixed a potential NPE in ZKRMStateStore caused by handling Disconnected event from ZK. Contributed by Karthik Kambatla. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1587776 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-16 02:02:18 +00:00
Zhijie Shen
44b6261bfa YARN-1892. Improved some logs in the scheduler. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1587717 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-15 20:37:44 +00:00
Vinod Kumar Vavilapalli
eb7b33c298 YARN-1928. Fixed a race condition in TestAMRMRPCNodeUpdates which caused it to fail occassionally. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1587114 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-13 22:40:16 +00:00
Vinod Kumar Vavilapalli
c6b70f4760 YARN-1933. Fixed test issues with TestAMRestart and TestNodeHealthService. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1587104 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-13 21:51:38 +00:00
Sanford Ryza
9274626a11 YARN-1923. Make Fair Scheduler resource ratio calculations terminate faster (Anubhav Dhoot via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1586796 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-11 23:24:40 +00:00
Jonathan Turner Eagles
0a0ca4f062 HADOOP-8826. Docs still refer to 0.20.205 as stable line (Mit Desai via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1586685 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-11 15:34:12 +00:00
Vinod Kumar Vavilapalli
d6a6e982a8 YARN-1926. Changed DistributedShell to use appIDs as unique identifiers for HDFS paths and thus avoid test failures on Windows. Contributed by Varun Vasudev. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1586562 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-11 04:06:40 +00:00
Zhijie Shen
dc84800184 YARN-1924. Made ZKRMStateStore updateApplication(Attempt)StateInternal work when Application(Attempt) state hasn't been stored before. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1586547 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-11 03:36:36 +00:00
Jian He
ed78328d50 YARN-1903. Set exit code and diagnostics when container is killed at NEW/LOCALIZING state. Contributed by Zhijie Shen 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1586522 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-11 01:26:36 +00:00
Vinod Kumar Vavilapalli
cde7b17566 YARN-1914. Fixed resource-download on NodeManagers to skip permission verification of public cache files in Windows+local file-system environment. Contribued by Varun Vasudev. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1586434 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-10 19:28:31 +00:00
Zhijie Shen
3993d646d7 YARN-1920. Fixed TestFileSystemApplicationHistoryStore failure on windows. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1586414 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-10 18:57:02 +00:00
Vinod Kumar Vavilapalli
791f2f6f1b YARN-1910. Fixed a race condition in TestAMRMTokens that causes the test to fail more often on Windows. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1586192 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-10 01:36:50 +00:00
Kihwal Lee
c3f6d71726 YARN-1907. TestRMApplicationHistoryWriter#testRMWritingMassiveHistory intermittently fails. Contributed by Mit Desai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1585992 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-09 15:33:25 +00:00
Steve Loughran
b8f250a99d HADOOP-10104. Update jackson to 1.9.13 (Akira Ajisaka via stevel) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1585932 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-09 11:11:20 +00:00
Karthik Kambatla
1e1200d594 YARN-1784. TestContainerAllocation assumes CapacityScheduler. (Robert Kanter via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1585894 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-09 05:49:24 +00:00
Vinod Kumar Vavilapalli
6c2a0ce30b YARN-1908. Fixed DistributedShell to not fail in secure clusters. Contributed by Vinod Kumar Vavilapalli and Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1585849 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-08 22:11:52 +00:00
Karthik Kambatla
245012a9d9 YARN-1757. NM Recovery. Auxiliary service support. (Jason Lowe via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1585783 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-08 17:15:58 +00:00
Chris Nauroth
4eacde67f8 YARN-1905. TestProcfsBasedProcessTree must only run on Linux. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1585376 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-07 03:04:09 +00:00
Arun Murthy
05da90ee2f YARN-1898. Addendum patch to ensure /jmx and /metrics are re-directed to Active RM. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1584954 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-05 02:51:08 +00:00
Jian He
15ac36a216 YARN-1837. Fixed TestMoveApplication#testMoveRejectedByScheduler failure. Contributed by Hong Zhiguo 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1584862 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-04 18:42:35 +00:00
Zhijie Shen
b0180afc9c YARN-1898. Made Standby RM links conf, stacks, logLevel, metrics, jmx, logs and static not be redirected to Active RM. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1583833 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-02 00:01:50 +00:00
Sanford Ryza
7bd62b8da0 YARN-1889. In Fair Scheduler, avoid creating objects on each call to AppSchedulable comparator (Hong Zhiguo via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1583491 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-31 23:40:45 +00:00
Jian He
3532d96ff6 YARN-1893. Mark AtMostOnce annotation to ApplicationMasterProtocol#allocate. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1583203 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-30 21:35:54 +00:00
Jian He
72051fc52c YARN-1017. Added documentation for ResourceManager Restart. (jianhe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1582913 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-28 23:36:30 +00:00
Vinod Kumar Vavilapalli
c3dbaea6fd YARN-1891. Added documentation for NodeManager health-monitoring. Contributed by Varun Vasudev. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1582891 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-28 21:54:21 +00:00
Jonathan Turner Eagles
6d2e1fde63 YARN-1883. TestRMAdminService fails due to inconsistent entries in UserGroups (Mit Desai via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1582862 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-28 19:30:08 +00:00
Tsz-wo Sze
3f7aa79e58 HDFS-6168. Remove a deprecated constructor and the deprecated methods reportChecksumFailure, getDelegationToken(Text), renewDelegationToken and cancelDelegationToken from DistributedFileSystem. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1582856 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-28 18:29:31 +00:00
Zhijie Shen
3af3b0bda9 YARN-1873. Fixed TestDistributedShell failure when the test cases are out of order. Contributed by Mit Desai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1582060 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-26 21:18:38 +00:00
Jian He
71c2b159ab YARN-1521. Mark Idempotent/AtMostOnce annotations to the APIs in ApplicationClientProtcol, ResourceManagerAdministrationProtocol and ResourceTrackerProtocol so that they work in HA scenario. Contributed by Xuan Gong 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1581678 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-26 03:43:41 +00:00
Vinod Kumar Vavilapalli
8a9ae9e3ec YARN-1867. Fixed a bug in ResourceManager that was causing invalid ACL checks in the web-services after fail-over. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1581662 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-26 03:03:17 +00:00
Vinod Kumar Vavilapalli
dce1d20383 YARN-1452. Added documentation about the configuration and usage of generic application history and the timeline data service. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1581656 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-26 02:52:58 +00:00
Vinod Kumar Vavilapalli
d14eff7d38 YARN-1866. Fixed an issue with renewal of RM-delegation tokens on restart or fail-over. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1581448 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-25 18:21:55 +00:00
Vinod Kumar Vavilapalli
09f383254c YARN-1850. Introduced the ability to optionally disable sending out timeline-events in the TimelineClient. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1581189 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-25 02:22:39 +00:00
Jian He
0febaa1b3d YARN-1852. Fixed RMAppAttempt to not resend AttemptFailed/AttemptKilled events to already recovered Failed/Killed RMApps. Contributed by Rohith Sharmaks 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1580997 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-24 19:04:57 +00:00
Zhijie Shen
0f1eda6bbf YARN-1838. Enhanced timeline service getEntities API to get entities from a given entity ID or insertion timestamp. Contributed by Billie Rinaldi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1580960 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-24 18:14:13 +00:00
Jonathan Turner Eagles
2d89dcb121 YARN-1670. aggregated log writer can write more log data then it says is the log length (Mit Desai via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1580957 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-24 18:11:48 +00:00
Zhijie Shen
ef9634bf79 YARN-1577. Made UnmanagedAMLauncher do launchAM after the attempt reaches the LAUNCHED state. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1580164 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-22 04:35:46 +00:00
Jian He
ce56616037 YARN-1776. Fixed DelegationToken renewal to survive RM failover. Contributed by Zhijie Shen 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1580154 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-22 04:12:57 +00:00
Karthik Kambatla
cbb3914c89 YARN-1536. Cleanup: Get rid of ResourceManager#get*SecretManager() methods and use the RMContext methods instead. (Anubhav Dhoot via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1580128 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-22 02:35:52 +00:00
Vinod Kumar Vavilapalli
dc594101cc YARN-1854. Fixed test failure in TestRMHA#testStartAndTransitions. Contributed by Rohith Sharma KS. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1580097 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-22 00:29:43 +00:00
Vinod Kumar Vavilapalli
a764e9a7a4 YARN-1863. Fixed test failure in TestRMFailover after YARN-1859. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1580094 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-22 00:25:27 +00:00
Vinod Kumar Vavilapalli
06579878df YARN-1775. Enhanced ProcfsBasedProcessTree to optionally add the ability to use smaps for obtaining used memory information. Contributed by Rajesh Balamohan. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1580087 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-22 00:01:59 +00:00
Jian He
f67218809c YARN-1849. Fixed NPE in ResourceTrackerService#registerNodeManager for UAM. Contributed by Karthik Kambatla 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1580077 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-21 22:53:49 +00:00
Vinod Kumar Vavilapalli
2826a35469 YARN-1670. Fixed a bug in log-aggregation that can cause the writer to write more log-data than the log-length that it records. Contributed by Mit Desai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1580005 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-21 18:35:54 +00:00
Vinod Kumar Vavilapalli
4224e61340 YARN-1811. Fixed AMFilters in YARN to correctly accept requests from either web-app proxy or the RMs when HA is enabled. Contributed by Robert Kanter. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1579877 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-21 04:20:46 +00:00
Vinod Kumar Vavilapalli
1c49cfbeb7 YARN-1859. Fixed WebAppProxyServlet to correctly handle applications absent on the ResourceManager. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1579866 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-21 02:02:41 +00:00
Vinod Kumar Vavilapalli
bf428cc474 YARN-1855. Made Application-history server to be optional in MiniYARNCluster and thus avoid the failure of TestRMFailover#testRMWebAppRedirect. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1579838 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-21 00:18:33 +00:00