big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
14,349 Commits 2 Branches 2 Tags 264 MiB
d6d9cff21b
Commit Graph

513 Commits

Author SHA1 Message Date
Junping Du
2188a07e5b YARN-3333. Rename TimelineAggregator etc. to TimelineCollector. Contributed by Sangjin Lee 2016-07-10 08:45:31 -07:00
Zhijie Shen
9b56364080 YARN-3039. Implemented the app-level timeline aggregator discovery service. Contributed by Junping Du. 2016-07-10 08:45:31 -07:00
Zhijie Shen
f0e752c14b YARN-3264. Created backing storage write interface and a POC only FS based storage implementation. Contributed by Vrushali C. 
(cherry picked from commit 821b68d05d246fd57d7b7286eb2ccc075ed1eae8)
 2016-07-10 08:45:30 -07:00
Zhijie Shen
9d57c9c015 YARN-3087. Made the REST server of per-node aggregator work alone in NM daemon. Conntributed by Li Lu. 
(cherry picked from commit 41a08ad404d4278fe598d6c222b2ae0e84bae0df)
 2016-07-10 08:45:29 -07:00
Junping Du
4f0c7eaff3 YARN-3240. Implement client API to put generic entities. Contributed by Zhijie Shen 
(cherry picked from commit 4487da249f448d5c67b712cd0aa723e764eed77d)
 2016-07-10 08:45:29 -07:00
Zhijie Shen
c77f86bf63 YARN-3041. Added the overall data model of timeline service next gen. Contributed by Zhijie Shen. 
(cherry picked from commit fdff5d2625c129c20c44f90a0517b28cee1522bd)
 2016-07-10 08:45:29 -07:00
Jian He
8a9d293dd6 YARN-5233. Support for specifying a path for ATS plugin jars. Contributed by Li Lu 2016-07-06 16:44:46 -07:00
Wangda Tan
c77a1095dc YARN-1942. Deprecate toString/fromString methods from ConverterUtils and move them to records classes like ContainerId/ApplicationId, etc. (wangda) 2016-06-14 15:06:38 -07:00
Wangda Tan
eec835ec17 YARN-4887. Add allocation request ID to AM-RM protocol for identifying resource-requests explicitly. (Subru Krishnan via wangda) 2016-06-13 21:57:33 -07:00
Wangda Tan
03fc6b1bb0 YARN-3426. Add jdiff support to YARN. (vinodkv via wangda) 2016-06-10 09:51:09 -07:00
Wangda Tan
620325e816 YARN-4837. User facing aspects of 'AM blacklisting' feature need fixing. (vinodkv via wangda) 2016-06-07 15:06:42 -07:00
Ming Ma
4a1cedc010 MAPREDUCE-5044. Have AM trigger jstack on task attempts that timeout before killing them. (Eric Payne and Gera Shegalov via mingma) 2016-06-06 14:30:51 -07:00
Arun Suresh
dc26601d8f YARN-5180. Allow ResourceRequest to specify an enforceExecutionType flag. (asuresh) 2016-06-02 09:01:02 -07:00
Varun Vasudev
86fb58b7dc YARN-4007. Add support for different network setups when launching the docker container. Contributed by Sidharta Seethana. 2016-05-29 21:32:51 +05:30
Varun Vasudev
42f90ab885 YARN-4844. Add getMemorySize/getVirtualCoresSize to o.a.h.y.api.records.Resource. Contributed by Wangda Tan. 2016-05-29 21:24:16 +05:30
Arun Suresh
aa975bc781 YARN-5127. Expose ExecutionType in Container api record. (Hitesh Sharma via asuresh) 2016-05-27 14:06:32 -07:00
Carlo Curino
013532a95e YARN-4957. Add getNewReservation in ApplicationClientProtocol (Sean Po via curino) 2016-05-25 16:55:49 -07:00
Varun Vasudev
0287c49107 YARN-5016. Add support for a minimum retry interval for container retries. Contributed by Jun Gong. 2016-05-20 12:28:14 +05:30
Junping Du
22fcd819f0 YARN-5076. YARN web interfaces lack XFS protection. Contributed by Jonathan Maron. 
(cherry picked from commit 2703ec68712279494d67b0d76b7ac10e7a1628be)
 2016-05-19 14:15:21 -07:00
Arun Suresh
ccc93e7812 YARN-5075. Fix findbugs warnings in hadoop-yarn-common module. (asuresh) 2016-05-16 23:22:01 -07:00
Arun Suresh
f0ac18d001 YARN-2888. Corrective mechanisms for rebalancing NM container queues. (asuresh) 2016-05-13 13:38:36 -07:00
Andrew Wang
3c5c57af28 HADOOP-13142. Change project version from 3.0.0 to 3.0.0-alpha1. 2016-05-12 18:27:28 -07:00
Andrew Wang
ca5613af91 Revert "Update project version to 3.0.0-alpha1-SNAPSHOT." 
This reverts commit 6b53802cba.
 2016-05-12 15:32:45 -07:00
Andrew Wang
6b53802cba Update project version to 3.0.0-alpha1-SNAPSHOT. 2016-05-12 11:05:05 -07:00
Sangjin Lee
0bbe01f8d5 YARN-4577. Enable aux services to have their own custom classpath/jar file (Xuan Gong via sale) 2016-05-12 10:11:23 -07:00
Rohith Sharma K S
b7ac85259c YARN-5068. Expose scheduler queue to application master. (Harish Jaiprakash via rohithsharmaks) 2016-05-12 15:17:49 +05:30
Jason Lowe
d0da13229c YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla 2016-05-05 14:07:54 +00:00
Varun Vasudev
0f25a1bb52 YARN-3998. Add support in the NodeManager to re-launch containers. Contributed by Jun Gong. 2016-04-29 16:09:07 +05:30
Jian He
4a8508501b YARN-5009. NMLeveldbStateStoreService database can grow substantially leading to longer recovery times. Contributed by Jason Lowe 2016-04-28 21:54:11 -07:00
Jian He
dd80042c42 YARN-5008. LeveldbRMStateStore database can grow substantially leading to long recovery times. Contributed by Jason Lowe 2016-04-28 21:27:25 -07:00
Arun Suresh
341888a0aa YARN-4412. Create ClusterMonitor to compute ordered list of preferred NMs for OPPORTUNITIC containers. (asuresh) 2016-04-26 20:12:12 -07:00
Arun Suresh
c282a08f38 YARN-2885. Create AMRMProxy request interceptor and ContainerAllocator to distribute OPPORTUNISTIC containers to appropriate Nodes (asuresh) 
(cherry picked from commit 2bf025278a318b0452fdc9ece4427b4c42124e39)
 2016-04-24 22:38:33 -07:00
Arun Suresh
b2a654c5ee YARN-4335. Allow ResourceRequests to specify ExecutionType of a request ask (kkaranasos via asuresh) 
(cherry picked from commit 8ffabfdf4fec104cdaa68133385cfe9cf88b95a8)
 2016-04-22 23:20:54 -07:00
Karthik Kambatla
c8172f5f14 YARN-2883. Queuing of container requests in the NM. (Konstantinos Karanasos and Arun Suresh via kasha) 2016-04-20 09:55:50 -07:00
Jason Lowe
2a5da97f81 Revert "YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla" 
This reverts commit 1cbcd4a491.
 2016-04-11 15:51:01 +00:00
Akira Ajisaka
1ff27f9d12 YARN-4630. Remove useless boxing/unboxing code. Contributed by Kousuke Saruta. 2016-04-11 14:55:03 +09:00
Jason Lowe
1cbcd4a491 YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla 2016-04-05 13:40:19 +00:00
Arun Suresh
82621e38a0 YARN-4895. Add subtractFrom method to ResourceUtilization class. Contributed by Konstantinos Karanasos. 2016-04-01 14:57:06 -07:00
Junping Du
af1d125f9c YARN-4814. ATS 1.5 timelineclient impl call flush after every event write. Contributed by Xuan Gong. 2016-03-23 08:57:16 -07:00
Li Lu
d49cfb3504 YARN-4696. Improving EntityGroupFSTimelineStore on exception handling, test setup, and concurrency. (Steve Loughran via gtcarrera9) 2016-03-10 10:51:55 -08:00
Varun Vasudev
e51a8c1056 YARN-4737. Add CSRF filter support in YARN. Contributed by Jonathan Maron. 2016-03-07 15:26:44 +05:30
Robert Kanter
d7fdec1e6b YARN-4579. Allow DefaultContainerExecutor container log directory permissions to be configurable (rchiang via rkanter) 2016-02-25 16:36:38 -08:00
Robert Kanter
954dd57043 YARN-4697. NM aggregation thread pool is not bound by limits (haibochen via rkanter) 2016-02-24 15:00:24 -08:00
Li Lu
9e0f7b8b69 YARN-4680. TimerTasks leak in ATS V1.5 Writer. (Xuan Gong via 
gtcarrera9)
 2016-02-24 13:43:09 -08:00
Arun Suresh
23f937e3b7 YARN-2575. Create separate ACLs for Reservation create/update/delete/list ops (Sean Po via asuresh) 2016-02-11 10:47:43 -08:00
Wangda Tan
9875325d5c YARN-4340. Add list API to reservation system. (Sean Po via wangda) 2016-02-02 10:17:33 +08:00
Xuan
9fab22b366 YARN-4219. New levelDB cache storage for timeline v1.5. Contributed by 
Li Lu
 2016-01-28 14:24:22 -08:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
2085e60a96 YARN-3542. Refactored existing CPU cgroups support to use the newer and integrated ResourceHandler mechanism, and also deprecated the old LCEResourceHandler inteface hierarchy. Contributed by Varun Vasudev. 2016-01-25 16:19:36 -08:00
Akira Ajisaka
56a0c17508 YARN-4592. Remove unused GetContainerStatus proto. Contributed by Chang Li. 2016-01-25 15:20:29 +09:00
Junping Du
02f597c5db YARN-4265. Provide new timeline plugin storage to support fine-grained entity caching. Contributed by Li Lu and Jason Lowe 2016-01-17 17:37:40 -08:00
rohithsharmaks
f7736f464f YARN-4389. Allow application to enable or disable am blacklisting. (Sunil G via rohithsharmaks) 2016-01-15 21:38:26 +05:30
Wangda Tan
c0537bcd2c YARN-4571. Make app id/name available to the yarn authorizer provider for better auditing. (Jian He via wangda) 2016-01-13 13:18:31 +08:00
Wangda Tan
9e792da014 YARN-4582. Label-related invalid resource request exception should be able to properly handled by application. (Bibin A Chundatt via wangda) 2016-01-12 12:53:31 +08:00
Xuan
89022f8d4b YARN-4438. Implement RM leader election with curator. Contributed by Jian He 2016-01-07 14:33:06 -08:00
Karthik Kambatla
fb00794368 YARN-2882. Add an OPPORTUNISTIC ExecutionType. (Konstantinos Karanasos and Inigo Goiri via kasha) 2015-12-26 20:22:16 -08:00
Gera Shegalov
2c17b81569 YARN-2934. Improve handling of container's stderr. (Naganarasimha G R via gera) 2015-12-24 23:48:05 -08:00
Junping Du
882f2f0464 YARN-4234. New put APIs in TimelineClient for ats v1.5. Contributed by Xuan Gong. 2015-12-23 05:26:51 -08:00
Jian He
85c2466048 YARN-4164. Changed updateApplicationPriority API to return the updated application priority. Contributed by Rohith Sharma K S 2015-12-18 14:13:48 -08:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
4e7d32c0db YARN-1856. Added cgroups based memory monitoring for containers as another alternative to custom memory-monitoring. Contributed by Varun Vasudev. 2015-12-17 12:13:03 -08:00
Wangda Tan
7faa406f27 YARN-4225. Add preemption status to yarn queue -status for capacity scheduler. (Eric Payne via wangda) 2015-12-16 13:19:40 -08:00
Wangda Tan
79c41b1d83 YARN-4293. ResourceUtilization should be a part of yarn node CLI. (Sunil G via wangda) 2015-12-16 13:18:19 -08:00
Siddharth Seth
0f708d465f YARN-4207. Add a non-judgemental YARN app completion status. Contributed by Rich Haase. 2015-12-16 11:33:55 -08:00
Wangda Tan
dfcbbddb09 YARN-4309. Add container launch related debug information to container logs when a container fails. (Varun Vasudev via wangda) 2015-12-14 11:13:22 -08:00
Junping Du
f910e4f639 YARN-3623. Add a config to indicate the Timeline Service version. Contributed by Xuan Gong. 2015-12-10 02:53:47 -08:00
Jian He
755dda8dd8 YARN-4405. Support node label store in non-appendable file system. Contributed by Wangda Tan 2015-12-03 17:45:31 -08:00
Jason Lowe
4ac6799d4a YARN-4132. Separate configs for nodemanager to resourcemanager connection timeout and retries. Contributed by Chang Li 2015-11-24 22:35:37 +00:00
Steve Loughran
253e0404f3 YARN-4279. Mark ApplicationId and ApplicationAttemptId static methods as @Public, @Unstable. (stevel) 2015-11-18 21:41:39 +00:00
Wangda Tan
3cc73773eb YARN-4285. Display resource usage as percentage of queue and cluster in the RM UI (Varun Vasudev via wangda) 2015-10-26 13:07:39 -07:00
Rohith Sharma K S
5acdde4744 YARN-2729. Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup. (Naganarasimha G R via rohithsharmaks) 2015-10-26 15:42:42 +05:30
Akira Ajisaka
7781fe1b9e YARN-4294. [JDK8] Fix javadoc errors caused by wrong reference and illegal tag. (aajisaka) 2015-10-24 11:54:42 +09:00
Jonathan Eagles
f8adeb712d YARN-4009. CORS support for ResourceManager REST API. ( Varun Vasudev via jeagles) 2015-10-23 10:34:08 -05:00
Junping Du
0fce5f9a49 YARN-4243. Add retry on establishing Zookeeper conenction in EmbeddedElectorService#serviceInit. Contributed by Xuan Gong. 2015-10-22 13:41:09 -07:00
Jonathan Eagles
25f8f801d1 YARN-2513. Host framework UIs in YARN for use with the ATS (jeagles) 2015-10-21 15:38:08 -05:00
Varun Vasudev
e39ae0e676 YARN-4262. Allow whitelisted users to run privileged docker containers. Contributed by Sidharta Seethana. 2015-10-19 20:19:22 +05:30
Jian He
f9da5cdb2b YARN-4170. AM need to be notified with priority in AllocateResponse. Contributed by Sunil G 2015-10-16 15:26:27 -07:00
Wangda Tan
4337b263aa YARN-4162. CapacityScheduler: Add resource usage by partition and queue capacity by partition to REST API. (Naganarasimha G R via wangda) 2015-10-16 15:06:28 -07:00
Varun Vasudev
63020c54c1 YARN-4258. Add support for controlling capabilities for docker containers. Contributed by Sidharta Seethana. 2015-10-15 13:41:20 +05:30
Devaraj K
db93047881 YARN-3964. Support NodeLabelsProvider at Resource Manager side. 
Contributed by Dian Fu.
 2015-10-11 11:21:29 +05:30
Jason Lowe
a0bca2b5ad YARN-261. Ability to fail AM attempts. Contributed by Andrey Klochkov and Rohith Sharma K S 2015-10-09 14:17:38 +00:00
Jason Lowe
8d226225d0 YARN-3943. Use separate threshold configurations for disk-full detection and disk-not-full detection. Contributed by Zhihai Xu 2015-10-08 22:25:34 +00:00
Wangda Tan
30ac69c6bd YARN-4176. Resync NM nodelabels with RM periodically for distributed nodelabels. (Bibin A Chundatt via wangda) 2015-10-05 15:47:21 -07:00
Harsh J
c918f7be5e HADOOP-12458. Retries is typoed to spell Retires in parts of hadoop-yarn and hadoop-common. Contributed by Neelesh Srinivas Salian. 2015-10-03 18:37:58 +05:30
Xuan
8f08532bde YARN-1897. CLI and core support for signal container functionality. Contributed by Ming Ma 2015-10-02 18:50:47 -07:00
Jason Lowe
fdf02d1f26 YARN-3619. ContainerMetrics unregisters during getMetrics and leads to ConcurrentModificationException. Contributed by Zhihai Xu 2015-10-02 20:20:31 +00:00
Jian He
83a18add10 YARN-1449. AM-NM protocol changes to support container resizing. Contributed by Meng Ding & Wangda Tan) 2015-09-23 13:29:36 -07:00
Jian He
dfe2cb849f YARN-3866. AM-RM protocol changes to support container resizing. Contributed by Meng Ding 2015-09-23 13:29:36 -07:00
Chris Douglas
8e01b0d97a YARN-4188. Make MoveApplicationAcrossQueues abstract, newInstance static 
Contributed by Giovanni Matteo Fumarola
 2015-09-21 17:51:48 -07:00
Wangda Tan
ae5308fe1d YARN-3717. Expose app/am/queue's node-label-expression to RM web UI / CLI / REST-API. (Naganarasimha G R via wangda) 2015-09-15 11:40:50 -07:00
Junping Du
73e3a49eb0 YARN-313. Add Admin API for supporting node resource configuration in command line. (Contributed by Inigo Goiri, Kenji Kikushima and Junping Du) 2015-09-15 07:56:47 -07:00
Karthik Kambatla
81df7b586a YARN-2005. Blacklisting support for scheduling AMs. (Anubhav Dhoot via kasha) 2015-09-13 17:03:15 -07:00
Steve Loughran
7269906254 HADOOP-12087. [JDK8] Fix javadoc errors caused by incorrect or illegal tags. (Akira AJISAKA via stevel). 2015-09-13 14:25:26 +01:00
Jian He
6f72f1e600 YARN-2884. Added a proxy service in NM to proxy the the communication between AM and RM. Contributed by Kishore Chaliparambil 2015-09-08 09:35:46 +08:00
Xuan
9b78e6e33d YARN-4087. Followup fixes after YARN-2019 regarding RM behavior when 
state-store error occurs. Contributed by Jian He
 2015-09-07 17:45:47 -07:00
Wangda Tan
bcc85e3bab YARN-4024. YARN RM should avoid unnecessary resolving IP when NMs doing heartbeat. (Hong Zhiguo via wangda) 2015-09-04 15:13:53 -07:00
Jian He
a9c8ea71aa YARN-3250. Support admin cli interface in for Application Priority. Contributed by Rohith Sharma K S 2015-08-27 13:25:53 -07:00
Jian He
57c7ae1aff YARN-4014. Support user cli interface in for Application Priority. Contributed by Rohith Sharma K S 2015-08-24 20:36:44 -07:00
Xuan
37e1c3d82a YARN-221. NM should provide a way for AM to tell it not to aggregate 
logs. Contributed by Ming Ma
 2015-08-22 16:25:24 -07:00
Wangda Tan
fc07464d1a YARN-2923. Support configuration based NodeLabelsProvider Service in Distributed Node Label Configuration Setup. (Naganarasimha G R) 2015-08-20 11:51:03 -07:00
Karthik Kambatla
def12933b3 YARN-3534. Collect memory/cpu usage on the node. (Inigo Goiri via kasha) 2015-08-16 06:24:16 -07:00
rohithsharmaks
1c12adb71f YARN-4023. Publish Application Priority to TimelineServer. (Sunil G via rohithsharmaks) 2015-08-12 14:45:41 +05:30