big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
9,001 Commits 2 Branches 2 Tags 264 MiB
30d56fdbb4
Commit Graph

1364 Commits

Author SHA1 Message Date
Jian He
30d56fdbb4 YARN-1857. CapacityScheduler headroom doesn't account for other AM's running. Contributed by Chen He and Craig Welch 2014-10-07 13:45:04 -07:00
Jian He
519e5a7dd2 YARN-2644. Fixed CapacityScheduler to return up-to-date headroom when AM allocates. Contributed by Craig Welch 2014-10-06 15:48:46 -07:00
Jian He
ea26cc0b4a YARN-2615. Changed ClientToAMTokenIdentifier/RM(Timeline)DelegationTokenIdentifier to use protobuf as payload. Contributed by Junping Du 2014-10-06 10:47:43 -07:00
Chris Douglas
8380ca3723 Move YARN-1051 to 2.6 2014-10-06 10:38:18 -07:00
Chris Douglas
c8212bacb1 YARN-1051. Add a system for creating reservations of cluster capacity. 
Contributed by Subru Krishnan and Carlo Curino.
 2014-10-03 17:09:02 -07:00
subru
a2986234be YARN-2611. Fixing jenkins findbugs warning and TestRMWebServicesCapacitySched for branch YARN-1051. Contributed by Subru Krishnan and Carlo Curino. 
(cherry picked from commit c47464aba407d1dafe10be23fe454f0489cc4367)
 2014-10-03 15:43:23 -07:00
subru
5e10a13bb4 YARN-2576. Making test patch pass in branch. Contributed by Subru Krishnan and Carlo Curino. 
(cherry picked from commit 90ac0be86b898aefec5471db4027554c8e1b310c)
 2014-10-03 15:43:13 -07:00
subru
6261f7cc69 YARN-2080. Integrating reservation system with ResourceManager and client-RM protocol. Contributed by Subru Krishnan and Carlo Curino. 
(cherry picked from commit 8baeaead8532898163f1006276b731a237b1a559)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java
 2014-10-03 15:42:43 -07:00
subru
3418c56bcf YARN-1712. Plan follower that synchronizes the current state of reservation subsystem with the scheduler. Contributed by Subru Krishnan and Carlo Curino. 
(cherry picked from commit 169085319b8b76641f8b9f6840a3fef06d221e2b)
 2014-10-03 15:42:10 -07:00
carlo curino
b6df0dddcd YARN-1711. Policy to enforce instantaneous and over-time quotas on user reservation. Contributed by Carlo Curino and Subru Krishnan. 
(cherry picked from commit c4918cb4cb5a267a8cfd6eace28fcfe7ad6174e8)
 2014-10-03 15:42:03 -07:00
carlo curino
f66ffcf832 YARN-1710. Logic to find allocations within a Plan that satisfy user ReservationRequest(s). Contributed by Carlo Curino and Subru Krishnan. 
(cherry picked from commit aef7928899b37262773f3dc117157bb746bf8918)
 2014-10-03 15:41:57 -07:00
subru
cf4b34282a YARN-1709. In-memory data structures used to track resources over time to enable reservations. 
(cherry picked from commit 0d8b2cd88b958b1e602fd4ea4078ef8d4742a7c3)
 2014-10-03 15:41:51 -07:00
subru
c9266df404 YARN-1708. Public YARN APIs for creating/updating/deleting reservations. 
(cherry picked from commit 3f2e3b275bcf29264a112c5d2f4eae289fe7f52c)
 2014-10-03 15:41:39 -07:00
carlo curino
1c6950354f YARN-2475. Logic for responding to capacity drops for the ReservationSystem. Contributed by Carlo Curino and Subru Krishnan. 
(cherry picked from commit f83a07f266f2c5e6eead554d8a331ed7e75e10d5)
 2014-10-03 15:41:21 -07:00
carlo curino
eb3e40b833 YARN-1707. Introduce APIs to add/remove/resize queues in the CapacityScheduler. Contributed by Carlo Curino and Subru Krishnan 
(cherry picked from commit aac47fda7fecda9fc18ade34d633eca895865a70)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
 2014-10-03 15:41:02 -07:00
Vinod Kumar Vavilapalli
34cdcaad71 YARN-2468. Enhanced NodeManager to support log handling APIs (YARN-2569) for use by long running services. Contributed by Xuan Gong. 2014-10-03 12:15:40 -07:00
Karthik Kambatla
80d11eb68e YARN-2635. TestRM, TestRMRestart, TestClientToAMTokens should run with both CS and FS. (Wei Yan and kasha via kasha) 2014-10-03 11:49:49 -07:00
Jian He
eb6ce5e97c YARN-2562. Changed ContainerId#toString() to be more readable. Contributed by Tsuyoshi OZAWA 2014-10-03 11:04:02 -07:00
Jian He
054f285526 YARN-2628. Capacity scheduler with DominantResourceCalculator carries out reservation even though slots are free. Contributed by Varun Vasudev 2014-10-02 15:13:33 -07:00
Zhijie Shen
1c93025a1b YARN-2527. Fixed the potential NPE in ApplicationACLsManager and added test cases for it. Contributed by Benoy Antony. 2014-10-02 14:56:13 -07:00
Jason Lowe
29f520052e YARN-2624. Resource Localization fails on a cluster due to existing cache directories. Contributed by Anubhav Dhoot 2014-10-02 17:39:34 +00:00
Karthik Kambatla
5e0b49da9c YARN-2254. TestRMWebServicesAppsModification should run against both CS and FS. (Zhihai Xu via kasha) 2014-10-02 10:15:04 -07:00
Jian He
3ef1cf187f YARN-2617. Fixed NM to not send duplicate container status whose app is not running. Contributed by Jun Gong 2014-10-02 10:04:09 -07:00
junping_du
c7cee9b455 YARN-1979. TestDirectoryCollection fails when the umask is unusual. (Contributed by Vinod Kumar Vavilapalli and Tsuyoshi OZAWA) 2014-10-02 08:02:59 -07:00
Vinod Kumar Vavilapalli
9e40de6af7 YARN-2446. Augmented Timeline service APIs to start taking in domains as a parameter while posting entities and events. Contributed by Zhijie Shen. 2014-10-01 17:52:03 -07:00
junping_du
0708827a93 YARN-2613. Support retry in NMClient for rolling-upgrades. (Contributed by Jian He) 2014-10-01 16:50:30 -07:00
Zhijie Shen
52bbe0f11b YARN-2630. Prevented previous AM container status from being acquired by the current restarted AM. Contributed by Jian He. 2014-10-01 15:38:11 -07:00
Vinod Kumar Vavilapalli
ba7f31c2ee YARN-1972. Added a secure container-executor for Windows. Contributed by Remus Rusanu. 2014-10-01 10:14:41 -07:00
Vinod Kumar Vavilapalli
5ca97f1e60 YARN-1063. Augmented Hadoop common winutils to have the ability to create containers as domain users. Contributed by Remus Rusanu. 
Committed as a YARN patch even though all the code changes are in common.
 2014-10-01 09:54:17 -07:00
Karthik Kambatla
17d1202c35 YARN-2179. [YARN-1492] Initial cache manager structure and context. (Chris Trezzo via kasha) 2014-09-30 19:01:47 -07:00
Jian He
bbff96be48 YARN-2602. Fixed possible NPE in ApplicationHistoryManagerOnTimelineStore. Contributed by Zhijie Shen 2014-09-30 16:44:17 -07:00
Karthik Kambatla
14d60dadc2 YARN-2594. Potential deadlock in RM when querying ApplicationResourceUsageReport. (Wangda Tan via kasha) 2014-09-30 16:12:27 -07:00
Jason Lowe
feaf139b4f YARN-2387. Resource Manager crashes with NPE due to lack of synchronization. Contributed by Mit Desai 2014-09-30 22:37:28 +00:00
Zhijie Shen
9582a50176 YARN-2627. Added the info logs of attemptFailuresValidityInterval and number of previous failed attempts. Contributed by Xuan Gong. 2014-09-30 12:57:45 -07:00
Karthik Kambatla
f7743dd07d YARN-2610. Hamlet should close table tags. (Ray Chiang via kasha) 2014-09-30 11:46:07 -07:00
Arun C. Murthy
ea32a66f7d Preparing to branch hadoop-2.6 2014-09-30 10:26:42 -07:00
Jonathan Eagles
e10eeaabce YARN-2606. Application History Server tries to access hdfs before doing secure login (Mit Desai via jeagles) 2014-09-29 16:37:23 -05:00
Jason Lowe
9c22065109 YARN-1769. CapacityScheduler: Improve reservations. Contributed by Thomas Graves 2014-09-29 14:12:18 +00:00
Jian He
5391919b09 YARN-668. Changed NMTokenIdentifier/AMRMTokenIdentifier/ContainerTokenIdentifier to use protobuf object as the payload. Contributed by Junping Du. 2014-09-26 17:48:41 -07:00
Allen Wittenauer
32870db0fb YARN-2372. There are Chinese Characters in the FairScheduler's document (Fengdong Yu via aw) 2014-09-26 12:46:45 -07:00
Allen Wittenauer
ac70c27473 YARN-2577. Clarify ACL delimiter and how to configure ACL groups only (Mikos Christine via aw) 2014-09-26 11:05:13 -07:00
Karthik Kambatla
f4357240a6 YARN-2608. FairScheduler: Potential deadlocks in loading alloc files and clock access. (Wei Yan via kasha) 2014-09-25 17:42:51 -07:00
Jason Lowe
8269bfa613 YARN-2523. ResourceManager UI showing negative value for "Decommissioned Nodes" field. Contributed by Rohith 2014-09-25 22:37:05 +00:00
Zhijie Shen
72b0881ca6 YARN-2546. Made REST API for application creation/submission use numeric and boolean types instead of the string of them. Contributed by Varun Vasudev. 2014-09-24 17:57:32 -07:00
Zhijie Shen
c86674a3a4 YARN-2581. Passed LogAggregationContext to NM via ContainerTokenIdentifier. Contributed by Xuan Gong. 2014-09-24 17:50:26 -07:00
Karthik Kambatla
39c87344e1 YARN-2596. TestWorkPreservingRMRestart fails with FairScheduler. (kasha) 2014-09-24 15:23:31 -07:00
Vinod Kumar Vavilapalli
d78b452a4f YARN-2102. Added the concept of a Timeline Domain to handle read/write ACLs on Timeline service event data. Contributed by Zhijie Shen. 2014-09-24 11:27:53 -07:00
Allen Wittenauer
034df0e2eb YARN-2161. Fix build on macosx: YARN parts (Binglin Chang via aw) 2014-09-24 08:47:55 -07:00
Allen Wittenauer
3dc28e2052 HADOOP-11092. hadoop shell commands should print usage if not given a class (aw) 2014-09-23 12:24:23 -07:00
Zhijie Shen
5338ac416a YARN-2569. Added the log handling APIs for the long running services. Contributed by Xuan Gong. 2014-09-23 10:36:57 -07:00