big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
12,700 Commits 2 Branches 2 Tags 264 MiB
b1ed28fa77
Commit Graph

2240 Commits

Author SHA1 Message Date
Jian He
1cb3299b48 YARN-4403. (AM/NM/Container)LivelinessMonitor should use monotonic time when calculating period. Contributed by Junping Du 2015-12-14 13:51:23 -08:00
Wangda Tan
07b0fb996a YARN-4418. AM Resource Limit per partition can be updated to ResourceUsage as well. (Sunil G via wangda) 2015-12-14 11:24:30 -08:00
Wangda Tan
dfcbbddb09 YARN-4309. Add container launch related debug information to container logs when a container fails. (Varun Vasudev via wangda) 2015-12-14 11:13:22 -08:00
Wangda Tan
6cb0af3c39 YARN-3946. Update exact reason as to why a submitted app is in ACCEPTED state to app's diagnostic message. (Naganarasimha G R via wangda) 2015-12-14 10:52:46 -08:00
Arun Suresh
7fb212e5e6 YARN-4358 addendum patch to fix javadoc error 2015-12-12 22:22:55 -08:00
Sangjin Lee
e8964589de YARN-4341. add doc about timeline performance tool usage (Chang Li via sjlee) 2015-12-11 09:46:32 -08:00
Xuan
21daa6c68a YARN-3623-Addendum: Improve the description for Timeline Service Version configuration in yarn-default.xml 2015-12-10 10:56:12 -08:00
Junping Du
f910e4f639 YARN-3623. Add a config to indicate the Timeline Service version. Contributed by Xuan Gong. 2015-12-10 02:53:47 -08:00
Akira Ajisaka
50edcb947c YARN-4434. NodeManager Disk Checker parameter documentation is not correct. Contributed by Weiwei Yang. 2015-12-09 16:32:33 +09:00
rohithsharmaks
a5e2e1ecb0 YARN-4421. Remove dead code in RmAppImpl.RMAppRecoveredTransition. (Daniel Templeton via rohithsharmaks) 2015-12-09 11:31:51 +05:30
rohithsharmaks
15c3e7ffe3 YARN-4431. Not necessary to do unRegisterNM() if NM get stop due to failed to connect to RM. (Junpin Du via rohithsharmaks) 2015-12-09 10:50:43 +05:30
Wangda Tan
7e4715186d YARN-4424. Fix deadlock in RMAppImpl. (Jian he via wangda) 2015-12-08 14:25:16 -08:00
Chris Douglas
9f50e13d5d YARN-4248. Followup patch adding asf-licence exclusions for json test files 2015-12-08 12:08:04 -08:00
=
c25a635459 YARN-4248. REST API for submit/update/delete Reservations. (curino) 2015-12-07 13:33:28 -08:00
Jonathan Eagles
4ff973f96a YARN-4422. Generic AHS sometimes doesn't show started, node, or logs on App page (Eric Payne via jeagles) 2015-12-07 15:04:48 -06:00
Xuan
4546c7582b YARN-4392. ApplicationCreatedEvent event time resets after RM 
restart/failover. Contributed by Naganarasimha G R and Xuan Gong
 2015-12-07 12:24:55 -08:00
rohithsharmaks
01a641bc44 YARN-3456. Improve handling of incomplete TimelineEntities. (Varun Saxena via rohithsharmaks) 2015-12-07 15:16:56 +05:30
Steve Loughran
65f395226b HADOOP-12321. Make JvmPauseMonitor an AbstractService. (Sunil G via Stevel) [includes HDFS-8947 MAPREDUCE-6462 and YARN-4072] 2015-12-06 17:43:35 +00:00
Arun Suresh
742632e346 YARN-4358. Reservation System: Improve relationship between SharingPolicy and ReservationAgent. (Carlo Curino via asuresh) 2015-12-05 21:26:16 -08:00
Jian He
e84d6ca2df Add missing file for YARN-4419 2015-12-03 19:07:28 -08:00
Jian He
755dda8dd8 YARN-4405. Support node label store in non-appendable file system. Contributed by Wangda Tan 2015-12-03 17:45:31 -08:00
Wangda Tan
a2c3bfc8c1 YARN-4292. ResourceUtilization should be a part of NodeInfo REST API. (Sunil G via wangda) 2015-12-03 14:28:32 -08:00
Jian He
9f77ccad73 YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999). Contributed by Mohammad Shahid Khan and Varun Saxena 2015-12-03 12:48:50 -08:00
Junping Du
62e9348bc1 YARN-4408. Fix issue that NodeManager still reports negative running containers. Contributed by Robert Kanter. 2015-12-03 06:36:37 -08:00
Jian He
6b9a5beb2b YARN-4398. Remove unnecessary synchronization in RMStateStore. Contributed by Ning Ding 2015-12-02 11:07:18 -08:00
Tsuyoshi Ozawa
0656d2dc83 YARN-4380. TestResourceLocalizationService.testDownloadingResourcesOnContainerKill fails intermittently. Contributed by Varun Saxena. 2015-11-26 01:10:02 +09:00
Wangda Tan
23c625ec57 YARN-4384. updateNodeResource CLI should not accept negative values for resource. (Junping Du via wangda) 2015-11-24 16:35:56 -08:00
Jason Lowe
f5acf94eca YARN-4365. FileSystemNodeLabelStore should check for root dir existence on startup. Contributed by Kuhu Shukla 2015-11-24 23:47:10 +00:00
Jason Lowe
4ac6799d4a YARN-4132. Separate configs for nodemanager to resourcemanager connection timeout and retries. Contributed by Chang Li 2015-11-24 22:35:37 +00:00
Tsuyoshi Ozawa
28dfe721b8 YARN-4387. Fix typo in FairScheduler log message. Contributed by Xin Wang. 2015-11-24 19:24:01 +09:00
Akira Ajisaka
8c0133f3d4 YARN-4298. Fix findbugs warnings in hadoop-yarn-common. Contributed by Sunil G. 2015-11-24 18:40:33 +09:00
Karthik Kambatla
52948bb20b YARN-3980. Plumb resource-utilization info in node heartbeat through to the scheduler. (Inigo Goiri via kasha) 2015-11-24 13:47:17 +05:30
Jian He
8676a118a1 YARN-4349. Support CallerContext in YARN. Contributed by Wangda Tan 2015-11-23 17:19:48 -08:00
Jason Lowe
d36b6e045f YARN-4344. NMs reconnecting with changed capabilities can lead to wrong cluster resource calculations. Contributed by Varun Vasudev 2015-11-23 20:30:26 +00:00
Arun Suresh
da1016365a YARN-3454. Add efficient merge operation to RLESparseResourceAllocation (Carlo Curino via asuresh) 2015-11-21 09:59:41 -08:00
Wangda Tan
2346fa3141 YARN-3769. Consider user limit when calculating total pending resource for preemption policy in Capacity Scheduler. (Eric Payne via wangda) 2015-11-20 15:55:50 -08:00
Jason Lowe
060cdcbe5d YARN-4374. RM capacity scheduler UI rounds user limit factor. Contributed by Chang Li 2015-11-20 23:12:29 +00:00
Xuan
f114e728da YARN-2859.addendum: fix the remaining issue from the previous patch 2015-11-19 10:08:38 -08:00
Steve Loughran
253e0404f3 YARN-4279. Mark ApplicationId and ApplicationAttemptId static methods as @Public, @Unstable. (stevel) 2015-11-18 21:41:39 +00:00
Arun Suresh
6a61928fb7 YARN-4184. Remove update reservation state api from state store as its not used by ReservationSystem (Sean Po via asuresh) 2015-11-17 15:50:34 -08:00
Jian He
fcd7888029 Revert "YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999) Contributed by Mohammad Shahid Khan" 
This reverts commit 8fbea531d7.

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-11-16 20:18:44 -08:00
Junping Du
855d52927b YARN-4354. Public resource localization fails with NPE. Contributed by Jason Lowe. 2015-11-15 04:43:57 -08:00
Wangda Tan
7ff280fca9 YARN-1509. Make AMRMClient support send increase container request and get increased/decreased containers. (Meng Ding via wangda) 2015-11-12 11:50:00 -08:00
Wangda Tan
7f55a18071 YARN-4347. Resource manager fails with Null pointer exception. (Jian He via wangda) 2015-11-12 11:23:40 -08:00
Wangda Tan
796638d9bc YARN-4287. Capacity Scheduler: Rack Locality improvement (Nathan Roberts via wangda) 2015-11-12 11:09:37 -08:00
Jason Lowe
06a70311b9 YARN-4345. yarn rmadmin -updateNodeResource doesn't work. Contributed by Junping Du 2015-11-12 14:49:06 +00:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
6351d3fa63 YARN-4183. Reverting the patch to fix behaviour change. 
Revert "YARN-4183. Enabling generic application history forces every job to get a timeline service delegation token (jeagles)"

This reverts commit c293c58954.
 2015-11-11 10:40:43 -08:00
Akira Ajisaka
23d0db551c YARN-4241. Fix typo of property name in yarn-default.xml. Contributed by Anthony Rojas. 2015-11-11 16:13:08 +09:00
Wangda Tan
c99925d6dd YARN-1510. Make NMClient support change container resources. (Meng Ding via wangda) 2015-11-10 11:45:46 -08:00
Jian He
8fbea531d7 YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999) Contributed by Mohammad Shahid Khan 2015-11-09 10:43:45 -08:00
Allen Wittenauer
b76b0ce51e HADOOP-12366. expose calculated paths (aw) 2015-11-07 08:32:56 -08:00
Varun Vasudev
73b9c7b82b HADOOP-10787. Rename/remove non-HADOOP_*, etc from the shell scripts. Contributed by Allen Wittenauer. 2015-11-04 15:56:17 +05:30
Wangda Tan
0783184f4b YARN-4326. Fix TestDistributedShell timeout as AHS in MiniYarnCluster no longer binds to default port 8188. (Meng Ding via wangda) 2015-11-03 11:18:34 -08:00
Tsuyoshi Ozawa
ce31b22739 YARN-4320. TestJobHistoryEventHandler fails as AHS in MiniYarnCluster no longer binds to default port 8188. Contributed by Varun Saxena. 2015-10-30 18:00:20 +09:00
Jian He
e5b1733e04 YARN-4127. RM fail with noAuth error if switched from failover to non-failover. Contributed by Varun Saxena 2015-10-29 15:42:57 -07:00
Jonathan Eagles
c293c58954 YARN-4183. Enabling generic application history forces every job to get a timeline service delegation token (jeagles) 2015-10-29 16:41:10 -05:00
Arun Suresh
58d1df585c YARN-4310. FairScheduler: Log skipping reservation messages at DEBUG level (asuresh) 2015-10-29 13:42:09 -07:00
Jason Lowe
e2267de207 YARN-2902. Killing a container that is localizing can orphan resources in the DOWNLOADING state. Contributed by Varun Saxena 2015-10-29 16:34:25 +00:00
Jian He
c41699965e YARN-4288. Fixed RMProxy to retry on IOException from local host. Contributed by Junping Du 2015-10-29 00:00:48 -07:00
Rohith Sharma K S
656c8f9527 YARN-4130. Duplicate declaration of ApplicationId in RMAppManager#submitApplication method. (Kai Sasaki via rohithsharmaks) 2015-10-29 12:22:44 +05:30
Xuan
27414dac66 YARN-2859. ApplicationHistoryServer binds to default port 8188 in 
MiniYARNCluster. Contributed by Vinod Kumar Vavilapalli
 2015-10-28 10:32:16 -07:00
Tsuyoshi Ozawa
9f4dfdf4eb YARN-4251. TestAMRMClientOnRMRestart#testAMRMClientOnAMRMTokenRollOverOnRMRestart is failing. Contributed by Brahma Reddy Battula. 2015-10-28 17:39:12 +09:00
Tsuyoshi Ozawa
96677bef00 YARN-3573. MiniMRYarnCluster constructor that starts the timeline server using a boolean should be marked deprecated. Contributed by Brahma Reddy Battula. 2015-10-27 12:35:01 +09:00
Akira Ajisaka
8a68630dd1 YARN-4300. [JDK8] Fix javadoc errors caused by wrong tags. (aajisaka) 2015-10-27 10:28:01 +09:00
Wangda Tan
56e4f6237a YARN-3216. Max-AM-Resource-Percentage should respect node labels. (Sunil G via wangda) 2015-10-26 16:44:39 -07:00
Wangda Tan
6f606214e7 YARN-4169. Fix racing condition of TestNodeStatusUpdaterForLabels. (Naganarasimha G R via wangda) 2015-10-26 16:36:34 -07:00
Wangda Tan
3cc73773eb YARN-4285. Display resource usage as percentage of queue and cluster in the RM UI (Varun Vasudev via wangda) 2015-10-26 13:07:39 -07:00
Jason Lowe
33a03af3c3 YARN-4284. condition for AM blacklisting is too narrow. Contributed by Sangjin Lee 2015-10-26 19:53:03 +00:00
Rohith Sharma K S
5acdde4744 YARN-2729. Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup. (Naganarasimha G R via rohithsharmaks) 2015-10-26 15:42:42 +05:30
Rohith Sharma K S
092883b34a YARN-4223. Fixed findbugs warnings in hadoop-yarn-server-nodemanager project. (varun saxena via rohithsharmaks) 2015-10-26 15:23:51 +05:30
Tsuyoshi Ozawa
ce60b4fc8b YARN-3528. Tests with 12345 as hard-coded port break jenkins. Contributed by Brahma Reddy Battula. 2015-10-26 16:45:11 +09:00
Rohith Sharma K S
b57f08c0d2 YARN-4246. NPE while listing app attempt. (Nijel S F via rohithsharmaks) 2015-10-26 11:56:36 +05:30
Allen Wittenauer
1aa735c188 YARN-3724. Use POSIX nftw(3) instead of fts(3) (Alan Burlison via aw) 2015-10-25 21:43:23 -07:00
Arun Suresh
ab8eb8770c YARN-3738. Add support for recovery of reserved apps running under dynamic queues (subru via asuresh) 2015-10-24 22:53:10 -07:00
Steve Loughran
446212a39e YARN-4296. DistributedShell Log.info is not friendly. (Xiaowei Wang via stevel) 2015-10-24 12:55:36 +01:00
Steve Loughran
14f8dd0423 YARN-4289. TestDistributedShell failing with bind exception. (Brahma Reddy Battula via stevel) 2015-10-24 12:48:15 +01:00
Akira Ajisaka
7781fe1b9e YARN-4294. [JDK8] Fix javadoc errors caused by wrong reference and illegal tag. (aajisaka) 2015-10-24 11:54:42 +09:00
Jason Lowe
d3a34a4f38 YARN-4041. Slow delegation token renewal can severely prolong RM recovery. Contributed by Sunil G 2015-10-23 20:57:01 +00:00
Ming Ma
934d96a334 YARN-2913. Fair scheduler should have ability to set MaxResourceDefault for each queue. (Siqi Li via mingma) 2015-10-23 08:36:33 -07:00
Jonathan Eagles
f8adeb712d YARN-4009. CORS support for ResourceManager REST API. ( Varun Vasudev via jeagles) 2015-10-23 10:34:08 -05:00
Junping Du
0fce5f9a49 YARN-4243. Add retry on establishing Zookeeper conenction in EmbeddedElectorService#serviceInit. Contributed by Xuan Gong. 2015-10-22 13:41:09 -07:00
Zhihai Xu
960201b79b YARN-4256. YARN fair scheduler vcores with decimal values. Contributed by Jun Gong 2015-10-22 12:28:03 -07:00
Anubhav Dhoot
2798723a54 YARN-3739. Add reservation system recovery to RM recovery process. Contributed by Subru Krishnan. 2015-10-22 06:51:00 -07:00
Jonathan Eagles
25f8f801d1 YARN-2513. Host framework UIs in YARN for use with the ATS (jeagles) 2015-10-21 15:38:08 -05:00
Arun Suresh
506d1b1dbc YARN-3985. Make ReservationSystem persist state using RMStateStore reservation APIs. (adhoot via asuresh) 2015-10-20 16:46:14 -07:00
Arun Suresh
7e2837f830 YARN-4270. Limit application resource reservation on nodes for non-node/rack specific requests (asuresh) 2015-10-19 20:00:38 -07:00
Varun Vasudev
151eab2a12 YARN-4267. Add additional logging to container launch implementations in container-executor. Contributed by Sidharta Seethana. 2015-10-19 20:56:01 +05:30
Varun Vasudev
e39ae0e676 YARN-4262. Allow whitelisted users to run privileged docker containers. Contributed by Sidharta Seethana. 2015-10-19 20:19:22 +05:30
Steve Loughran
e2d59e2c7b YARN-4155. TestLogAggregationService.testLogAggregationServiceWithInterval failing. (Bibin A Chundatt via stevel) 2015-10-18 12:03:57 +01:00
Jian He
f9da5cdb2b YARN-4170. AM need to be notified with priority in AllocateResponse. Contributed by Sunil G 2015-10-16 15:26:27 -07:00
Wangda Tan
4337b263aa YARN-4162. CapacityScheduler: Add resource usage by partition and queue capacity by partition to REST API. (Naganarasimha G R via wangda) 2015-10-16 15:06:28 -07:00
Jian He
cf23f2c2b5 YARN-4000. RM crashes with NPE if leaf queue becomes parent queue during restart. Contributed by Varun Saxena 2015-10-15 17:12:46 -07:00
Varun Vasudev
63020c54c1 YARN-4258. Add support for controlling capabilities for docker containers. Contributed by Sidharta Seethana. 2015-10-15 13:41:20 +05:30
Steve Loughran
dfa7848463 HADOOP-12478. Shell.getWinUtilsPath() has been renamed Shell.getWinutilsPath(). (stevel) 2015-10-14 20:25:47 +01:00
rohithsharmaks
d6c8bad869 YARN-4250. NPE in AppSchedulingInfo#isRequestLabelChanged. (Brahma Reddy Battula via rohithsharmaks) 2015-10-14 16:11:34 +05:30
Varun Vasudev
da1ee078f9 YARN-4255. container-executor does not clean up docker operation command files. Contributed by Sidharta Seethana. 2015-10-14 15:25:21 +05:30
Varun Vasudev
fb9c1519b3 YARN-4252. Log container-executor invocation details when exit code is non-zero. Contributed by Sidharta Seethana. 2015-10-14 15:18:08 +05:30
Varun Vasudev
8d59293089 YARN-4253. Standardize on using PrivilegedOperationExecutor for all invocations of container-executor in LinuxContainerExecutor. Contributed by Sidharta Seethana. 2015-10-14 14:58:10 +05:30
Steve Loughran
c59af2fdf8 HADOOP-10775. Shell operations to fail with meaningful errors on windows if winutils.exe not found. (stevel) 2015-10-13 21:49:28 +01:00
Varun Vasudev
5b6bae0094 YARN-4017. container-executor overuses PATH_MAX. Contributed by Sidharta Seethana 2015-10-13 12:48:30 +05:30