big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
19,291 Commits 2 Branches 2 Tags 264 MiB
24a89825f0
Commit Graph

314 Commits

Author SHA1 Message Date
Jason Lowe
b0b535d9d5 YARN-6930. Admins should be able to explicitly enable specific LinuxContainerRuntime in the NodeManager. Contributed by Shane Kumpf 2017-09-07 16:17:03 -05:00
Junping Du
91cc070d67 YARN-6877. Create an abstract log reader for extendability. Contributed by Xuan Gong. 2017-08-31 16:41:43 -07:00
Varun Saxena
3d00c8f394 Made fixes for whitespace errors and checstyle warnings before merge. 2017-08-30 11:29:54 +05:30
Varun Saxena
ff43b8db7a YARN-6069. CORS support in timeline v2 (Rohith Sharma K S via Varun Saxena) 2017-08-30 11:29:52 +05:30
Sangjin Lee
580d884913 YARN-6094. Update the coprocessor to be a dynamically loaded one. Contributed by Vrushali C. 2017-08-30 11:29:52 +05:30
Sangjin Lee
643a20a358 YARN-5265. Make HBase configuration for the timeline service configurable. Contributed by Joep Rottinghuis. 2017-08-30 11:29:51 +05:30
Varun Saxena
9bb5cc3d49 YARN-3649. Allow configurable prefix for hbase table names like prod, exp, test etc (Vrushali C via Varun Saxena) 2017-08-30 11:29:50 +05:30
Junping Du
c2cb7ea1ef YARN-6876. Create an abstract log writer for extendability. Contributed by Xuan Gong. 2017-08-24 13:36:49 -07:00
Arun Suresh
8410d862d3 YARN-6979. Add flag to notify all types of container updates to NM via NodeHeartbeatResponse. (Kartheek Muthyala via asuresh) 2017-08-20 07:54:09 -07:00
Subru Krishnan
bbbf0e2a41 HADOOP-14741. Refactor curator based ZooKeeper communication into common library. (Íñigo Goiri via Subru). 2017-08-11 13:58:45 -07:00
Daniel Templeton
47b145b9b4 YARN-6757. Refactor the usage of yarn.nodemanager.linux-container-executor.cgroups.mount-path 
(Contributed by Miklos Szegedi via Daniel Templeton)
 2017-08-08 10:33:26 -07:00
Junping Du
f44b349b81 YARN-6811. [ATS1.5] All history logs should be kept under its own User Directory. Contributed by Rohith Sharma K S. 2017-08-04 16:03:56 -07:00
Carlo Curino
acda6b96a4 YARN-5412. Create a proxy chain for ResourceManager REST API in the Router. (Contributed by Giovanni Matteo Fumarola via curino) 
(cherry picked from commit b6240b92abf453affc5fd64e1eedf2d29842aa75)
 2017-08-01 17:28:28 -07:00
Subru Krishnan
7444406d6d YARN-5413. Create a proxy chain for ResourceManager Admin API in the Router. (Giovanni Matteo Fumarola via Subru). 
(cherry picked from commit 67846a5519b5905c2d925cf4c602f715b653e72c)
 2017-08-01 17:28:24 -07:00
Subru Krishnan
80e1904000 YARN-5411. Create a proxy chain for ApplicationClientProtocol in the Router. (Giovanni Matteo Fumarola via Subru). 
(cherry picked from commit 4846069061b6baa06da3b524b9e36567dd368388)
 2017-08-01 17:28:24 -07:00
Subru Krishnan
86b2bec56e YARN-6247. Share a single instance of SubClusterResolver instead of instantiating one per AM. (Botong Huang via Subru) 
(cherry picked from commit 51aeb2ce0c599176aca9466a939c3ad55df30036)
 2017-08-01 17:28:23 -07:00
Jian He
bdfad4523f YARN-3673. Create a FailoverProxy for Federation services. Contributed by Subru Krishnan 
(cherry picked from commit 3307564a5f8c8abc5fe84efcd05ee0f7dfdd921c)
 2017-08-01 17:28:20 -07:00
Jian He
5c84382397 YARN-3672. Create Facade for Federation State and Policy Store. Contributed by Subru Krishnan 
(cherry picked from commit 5e976cd2b90ccf1bccb6039edf14140677804c4e)
 2017-08-01 17:28:19 -07:00
Subru Krishnan
d19b677301 YARN-5390. Federation Subcluster Resolver. Contributed by Ellen Hui. 
(cherry picked from commit d3dc461a935c2af4ec3f0312ff0c26918c408467)
 2017-08-01 17:28:18 -07:00
Arun Suresh
077fcf6a96 YARN-6777. Support for ApplicationMasterService processing chain of interceptors. (asuresh) 2017-07-19 12:26:40 -07:00
Xuan
9845bea59c YARN-6689. PlacementRule should be configurable. (Jonathan Hung via xgong) 2017-07-12 15:40:45 -07:00
Xuan
3be2659f83 YARN-6694. Add certain envs to the default yarn.nodemanager.env-whitelist. Contributed by Jian He 2017-06-29 20:10:35 -07:00
Daniel Templeton
25d891a784 YARN-6743. yarn.resourcemanager.zk-max-znode-size.bytes description needs spaces in yarn-default.xml 
(Contributed by Lori Loberg via Daniel Templeton)
 2017-06-28 13:17:58 -07:00
Naganarasimha
740204b292 YARN-5006. ResourceManager quit due to ApplicationStateData exceed the limit size of znode in zk. Contributed by Bibin A Chundatt. 2017-06-23 07:52:41 +05:30
Daniel Templeton
2e52789edf YARN-2962. ZKRMStateStore: Limit the number of znodes under a znode (Contributed by Varun Sexena via Daniel Templeton) 2017-04-28 13:30:28 -07:00
Varun Saxena
7c2bc444b3 YARN-6342. Make TimelineV2Client's drain timeout after stop configurable (Haibo Chen via Varun Saxena) 2017-03-31 02:02:57 +05:30
Varun Saxena
229c7c9f89 YARN-6069. CORS support in timeline v2 (Rohith Sharma K S via Varun Saxena) 2017-03-11 04:09:01 +05:30
Robert Kanter
6f6dfe0202 YARN-5280. Allow YARN containers to run with Java Security Manager (gphillips via rkanter) 2017-03-01 14:53:47 -08:00
Xuan
53d372a255 YARN-6228: EntityGroupFSTimelineStore should allow configurable cache 
stores. Contributed by Li Lu
 2017-02-24 10:59:35 -08:00
Daniel Templeton
c7a36e6130 YARN-6125. The application attempt's diagnostic message should have a maximum size 
(Contributed by Andras Piros via Daniel Templeton)
 2017-02-17 13:40:58 -08:00
Junping Du
464ff479ce YARN-6113. Re-direct NM Web Service to get container logs for finished applications. Contributed by Xuan Gong. 2017-02-13 06:12:54 -08:00
Sangjin Lee
dc6ec97048 YARN-6117. SharedCacheManager does not start up. Contributed by Chris Trezzo. 2017-01-23 21:07:25 -08:00
Jason Lowe
69fa81679f YARN-5910. Support for multi-cluster delegation tokens. Contributed by Jian He 2017-01-23 17:12:51 +00:00
bibinchundatt
a9d946d4df YARN-6057. yarn.scheduler.minimum-allocation-* descriptions are incorrect about behavior when a request is out of bounds. Contributed by Julia Sommer. 2017-01-17 14:16:00 +05:30
Daniel Templeton
e6f13fe5d1 YARN-5849. Automatically create YARN control group for pre-mounted cgroups (Contributed by Miklos Szegedi via Daniel Templeton) 2017-01-11 16:00:01 -08:00
Daniel Templeton
165d01a73e YARN-5931. Document timeout interfaces CLI and REST APIs (Contributed by Rohith Sharma K S via Daniel Templeton) 2016-12-30 08:41:52 -08:00
Daniel Templeton
e216e8e233 HADOOP-13932. Fix indefinite article in comments (Contributed by LiXin Ge via Daniel Templeton) 2016-12-27 13:03:16 -08:00
Arun Suresh
2273a74c1f YARN-5646. Add documentation and update config parameter names for scheduling of OPPORTUNISTIC containers. (Konstantinos Karanasos via asuresh) 2016-12-16 08:14:34 -08:00
Steve Loughran
f66f61892a HADOOP-13852 hadoop build to allow hadoop version property to be explicitly set. Contriibuted by Steve Loughran 2016-12-12 13:58:27 +00:00
Arun Suresh
b0aace21b1 YARN-5982. Simplify opportunistic container parameters and metrics. (Konstantinos Karanasos via asuresh) 2016-12-09 16:41:25 -08:00
Akira Ajisaka
7d8e440eee Revert "HADOOP-13852 hadoop build to allow hadoop version property to be explicitly set. Contriibuted by Steve Loughran" 
This reverts commit c265515725.
 2016-12-09 12:49:27 +09:00
Steve Loughran
c265515725 HADOOP-13852 hadoop build to allow hadoop version property to be explicitly set. Contriibuted by Steve Loughran 2016-12-08 17:57:15 +00:00
Naganarasimha
3541ed8068 YARN-4330. MiniYARNCluster is showing multiple Failed to instantiate default resource calculator warning messages. Contributed by Varun Saxena 2016-11-23 14:12:23 +05:30
Arun Suresh
3219b7b4ac YARN-4597. Introduce ContainerScheduler and a SCHEDULED state to NodeManager container lifecycle. (asuresh) 2016-11-15 07:56:25 -08:00
Jian He
bcc15c6290 YARN-5611. Provide an API to update lifetime of an application. Contributed by Rohith Sharma K S 2016-11-09 16:08:05 -08:00
Arun Suresh
280357c29f YARN-5833. Addendum patch to include missing changes to yarn-default.xml 2016-11-09 07:15:11 -08:00
Wangda Tan
3de0da2a76 YARN-5698. [YARN-3368] Launch new YARN UI under hadoop web app port. (Sunil G via wangda) 2016-11-06 13:13:31 -08:00
Wangda Tan
c85cc3b56e YARN-4515. [YARN-3368] Support hosting web UI framework inside YARN RM. (Sunil G via wangda) 
YARN-5000. [YARN-3368] App attempt page is not loading when timeline server is not started (Sunil G via wangda)
YARN-5038. [YARN-3368] Application and Container pages shows wrong values when RM is stopped. (Sunil G via wangda)
 2016-11-06 13:13:31 -08:00
Daniel Templeton
773c60bd7b YARN-5800. Delete LinuxContainerExecutor comment from yarn-default.xml (Contributed by Jan Hentschel via Daniel Templeton) 2016-10-31 14:47:48 -07:00
Robert Kanter
5877f20f9c HADOOP-10075. Update jetty dependency to version 9 (rkanter) 2016-10-27 16:09:00 -07:00
Varun Saxena
dd4ed6a587 YARN-4388. Cleanup mapreduce.job.hdfs-servers from yarn-default.xml (Junping Du via Varun Saxena) 2016-10-28 02:22:25 +05:30
Karthik Kambatla
de6faae97c YARN-5388. Deprecate and remove DockerContainerExecutor. (Daniel Templeton via kasha) 2016-10-25 13:35:47 -07:00
Xuan
b733a6f862 YARN-5718. TimelineClient (and other places in YARN) shouldn't over-write HDFS client retry settings which could cause unexpected behavior. Contributed by Junping Du. 2016-10-18 11:06:47 -07:00
Karthik Kambatla
6378845f9e YARN-4464. Lower the default max applications stored in the RM and store. (Daniel Templeton via kasha) 2016-10-11 21:42:08 -07:00
Jian He
2ae5a3a5bf YARN-4205. Add a service for monitoring application life time out. Contributed by Rohith Sharma K S 2016-09-29 22:00:31 +08:00
Varun Saxena
9b0fd01d2e YARN-5599. Publish AM launch command to ATS (Rohith Sharma K S via Varun Saxena) 2016-09-28 16:10:10 +05:30
Varun Vasudev
bc2656f09f YARN-5662. Provide an option to enable ContainerMonitor. Contributed by Jian He. 2016-09-28 15:18:18 +05:30
Ray Chiang
378f624a39 YARN-5549. AMLauncher#createAMContainerLaunchContext() should not log the command to be launched indiscriminately. (Daniel Templeton via rchiang) 2016-09-02 11:14:35 -07:00
Junping Du
0da69c324d YARN-4676. Automatic and Asynchronous Decommissioning Nodes Status Tracking. Contributed by Diniel Zhi. 
(cherry picked from commit d464483bf7f0b3e3be3ba32cd6c3eee546747ab5)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceTrackerService.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNode.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
 2016-08-18 07:27:23 -07:00
Ray Chiang
aea3e65749 YARN-5137. Make DiskChecker pluggable in NodeManager. (Yufei Gu via rchiang) 2016-08-10 19:23:29 -07:00
Arun Suresh
82c9e06101 YARN-5457. Refactor DistributedScheduling framework to pull out common functionality. (asuresh) 2016-08-09 00:42:29 -07:00
Arun Suresh
e5766b1dbe YARN-5113. Refactoring and other clean-up for distributed scheduling. (Konstantinos Karanasos via asuresh) 2016-07-31 11:48:25 -07:00
Junping Du
7f3c306e2e YARN-5432. Lock already held by another process while LevelDB cache store creation for dag. Contributed by Li Lu. 2016-07-28 06:35:24 -07:00
Varun Saxena
1ff6833bba YARN-5243. fix several rebase and other miscellaneous issues before merge. (Sangjin Lee via Varun Saxena) 2016-07-10 08:46:03 -07:00
Li Lu
0a9b085f05 YARN-5189. Make HBaseTimeline[Reader|Writer]Impl default and move FileSystemTimeline*Impl. (Joep Rottinghuis and Sangjin Lee via gtcarrera9) 2016-07-10 08:46:01 -07:00
Li Lu
c2055a97d5 YARN-3150. Documenting the timeline service v2. (Sangjin Lee and Vrushali C via gtcarrera9) 2016-07-10 08:45:57 -07:00
Sangjin Lee
9bdd455dce YARN-4062. Add the flush and compaction functionality via coprocessors and scanners for flow run table (Vrushali C via sjlee) 2016-07-10 08:45:54 -07:00
Sangjin Lee
0d02ab8729 YARN-3367. Replace starting a separate thread for post entity with event loop in TimelineClient (Naganarasimha G R via sjlee) 2016-07-10 08:45:53 -07:00
Sangjin Lee
cc16683cef YARN-3995. Some of the NM events are not getting published due race condition when AM container finishes in NM (Naganarasimha G R via sjlee) 2016-07-10 08:45:50 -07:00
Li Lu
89e5c44f9e YARN-4356. Ensure the timeline service v.2 is disabled cleanly and has no 
impact when it's turned off. Contributed by Sangjin Lee.
 2016-07-10 08:45:48 -07:00
Junping Du
cf4666bb8c YARN-3623. Add a config to indicate the Timeline Service version. Contributed by Xuan Gong. 
(cherry picked from commit f910e4f639)
 2016-07-10 08:45:48 -07:00
Junping Du
8603736ef2 YARN-3949. Ensure timely flush of timeline writes. Contributed by Sangjin Lee. 
(cherry picked from commit 967bef7e0396d857913caa2574afb103a5f0b81b)
 2016-07-10 08:45:39 -07:00
Junping Du
5e3d9a477b YARN-3034. Implement RM starting its timeline collector. Contributed by Naganarasimha G R 2016-07-10 08:45:32 -07:00
Junping Du
2188a07e5b YARN-3333. Rename TimelineAggregator etc. to TimelineCollector. Contributed by Sangjin Lee 2016-07-10 08:45:31 -07:00
Zhijie Shen
9b56364080 YARN-3039. Implemented the app-level timeline aggregator discovery service. Contributed by Junping Du. 2016-07-10 08:45:31 -07:00
Zhijie Shen
309592b9e5 YARN-3240. Added ".keep" file which was missed before. 
(cherry picked from commit 6b4dc96414b3f15537670c67a0dccd048862f4f4)
 2016-07-10 08:45:29 -07:00
Jian He
8a9d293dd6 YARN-5233. Support for specifying a path for ATS plugin jars. Contributed by Li Lu 2016-07-06 16:44:46 -07:00
Wangda Tan
620325e816 YARN-4837. User facing aspects of 'AM blacklisting' feature need fixing. (vinodkv via wangda) 2016-06-07 15:06:42 -07:00
Varun Vasudev
86fb58b7dc YARN-4007. Add support for different network setups when launching the docker container. Contributed by Sidharta Seethana. 2016-05-29 21:32:51 +05:30
Varun Vasudev
0287c49107 YARN-5016. Add support for a minimum retry interval for container retries. Contributed by Jun Gong. 2016-05-20 12:28:14 +05:30
Junping Du
22fcd819f0 YARN-5076. YARN web interfaces lack XFS protection. Contributed by Jonathan Maron. 
(cherry picked from commit 2703ec68712279494d67b0d76b7ac10e7a1628be)
 2016-05-19 14:15:21 -07:00
Jason Lowe
d0da13229c YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla 2016-05-05 14:07:54 +00:00
Varun Vasudev
0f25a1bb52 YARN-3998. Add support in the NodeManager to re-launch containers. Contributed by Jun Gong. 2016-04-29 16:09:07 +05:30
Jian He
4a8508501b YARN-5009. NMLeveldbStateStoreService database can grow substantially leading to longer recovery times. Contributed by Jason Lowe 2016-04-28 21:54:11 -07:00
Jian He
dd80042c42 YARN-5008. LeveldbRMStateStore database can grow substantially leading to long recovery times. Contributed by Jason Lowe 2016-04-28 21:27:25 -07:00
Karthik Kambatla
c8172f5f14 YARN-2883. Queuing of container requests in the NM. (Konstantinos Karanasos and Arun Suresh via kasha) 2016-04-20 09:55:50 -07:00
Rohith Sharma K S
4ee7f0a3e7 YARN-4897. dataTables_wrapper change min height. Contributed by Bibin A Chundatt. 2016-04-12 10:25:56 +05:30
Jason Lowe
2a5da97f81 Revert "YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla" 
This reverts commit 1cbcd4a491.
 2016-04-11 15:51:01 +00:00
Jason Lowe
1cbcd4a491 YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla 2016-04-05 13:40:19 +00:00
Varun Vasudev
3488c4f2c9 Revert "YARN-4857. Add missing default configuration regarding preemption of CapacityScheduler. Contributed by Kai Sasaki." 
This reverts commit 0064cba169.
 2016-04-01 12:20:40 +05:30
Sangjin Lee
6d67420dbc YARN-4183. Clarify the behavior of timeline service config properties (Naganarasimha G R via sjlee) 2016-03-31 10:49:03 -07:00
Varun Vasudev
0064cba169 YARN-4857. Add missing default configuration regarding preemption of CapacityScheduler. Contributed by Kai Sasaki. 2016-03-31 14:05:49 +05:30
Varun Vasudev
e51a8c1056 YARN-4737. Add CSRF filter support in YARN. Contributed by Jonathan Maron. 2016-03-07 15:26:44 +05:30
Robert Kanter
d7fdec1e6b YARN-4579. Allow DefaultContainerExecutor container log directory permissions to be configurable (rchiang via rkanter) 2016-02-25 16:36:38 -08:00
Robert Kanter
954dd57043 YARN-4697. NM aggregation thread pool is not bound by limits (haibochen via rkanter) 2016-02-24 15:00:24 -08:00
Li Lu
9e0f7b8b69 YARN-4680. TimerTasks leak in ATS V1.5 Writer. (Xuan Gong via 
gtcarrera9)
 2016-02-24 13:43:09 -08:00
Arun Suresh
23f937e3b7 YARN-2575. Create separate ACLs for Reservation create/update/delete/list ops (Sean Po via asuresh) 2016-02-11 10:47:43 -08:00
Devaraj K
db144eb1c5 YARN-4100. Add Documentation for Distributed and Delegated-Centralized 
Node Labels feature. Contributed by Naganarasimha G R.
 2016-02-02 12:06:51 +05:30
Xuan
9fab22b366 YARN-4219. New levelDB cache storage for timeline v1.5. Contributed by 
Li Lu
 2016-01-28 14:24:22 -08:00
Junping Du
02f597c5db YARN-4265. Provide new timeline plugin storage to support fine-grained entity caching. Contributed by Li Lu and Jason Lowe 2016-01-17 17:37:40 -08:00