big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
10,634 Commits 2 Branches 2 Tags 264 MiB
d497f6ea2b
Commit Graph

1296 Commits

Author SHA1 Message Date
Sandy Ryza
6a6a59db7f YARN-3415. Non-AM containers can be counted towards amResourceUsage of a fairscheduler queue (Zhihai Xu via Sandy Ryza) 2015-04-02 13:56:08 -07:00
Zhijie Shen
8366a36ad3 YARN-3430. Made headroom data available on app attempt page of RM WebUI. Contributed by Xuan Gong. 2015-04-01 13:47:54 -07:00
Xuan
4728bdfa15 YARN-3248. Display count of nodes blacklisted by apps in the web UI. 
Contributed by Varun Vasudev
 2015-04-01 04:19:18 -07:00
Tsuyoshi Ozawa
c69ba81497 YARN-3424. Change logs for ContainerMonitorImpl's resourse monitoring from info to debug. Contributed by Anubhav Dhoot. 2015-04-01 17:44:25 +09:00
Karthik Kambatla
2daa478a64 YARN-3428. Debug log resources to be localized for a container. (kasha) 2015-03-31 17:34:47 -07:00
Karthik Kambatla
79f7f2aabf YARN-3412. RM tests should use MockRM where possible. (kasha) 2015-03-31 09:14:15 -07:00
Tsuyoshi Ozawa
b5a22e9838 YARN-3258. FairScheduler: Need to add more logging to investigate allocations. Contributed by Anubhav Dhoot. 2015-03-31 17:42:44 +09:00
Wangda Tan
2a945d24f7 YARN-2495. Allow admin specify labels from each NM (Distributed configuration for node label). (Naganarasimha G R via wangda) 2015-03-30 12:05:21 -07:00
Vinod Kumar Vavilapalli
c358368f51 YARN-3304. Cleaning up ResourceCalculatorProcessTree APIs for public use and removing inconsistencies in the default values. Contributed by Junping Du and Karthik Kambatla. 2015-03-30 10:09:40 -07:00
Ravi Prakash
e0ccea33c9 YARN-3288. Document and fix indentation in the DockerContainerExecutor code 2015-03-28 08:00:41 -07:00
Haohui Mai
e4b8d9e72d HADOOP-10670. Allow AuthenticationFilters to load secret from signature secret files. Contributed by Kai Zheng. 2015-03-25 11:12:42 -07:00
Xuan
e556198e71 YARN-2213. Change proxy-user cookie log in AmIpFilter to DEBUG. 
Contributed by Varun Saxena
 2015-03-25 04:49:43 -07:00
Wangda Tan
97a7277a2d YARN-3383. AdminService should use warn instead of info to log exception when operation fails. (Li Lu via wangda) 2015-03-24 10:38:54 -07:00
Xuan
9fae455e26 YARN-3393. Getting application(s) goes wrong when app finishes before 
starting the attempt. Contributed by Zhijie Shen
 2015-03-23 20:33:16 -07:00
Karthik Kambatla
972f1f1ab9 YARN-2868. FairScheduler: Metric for latency to allocate first container for an application. (Ray Chiang via kasha) 2015-03-23 14:07:05 -07:00
Karthik Kambatla
2bc097cd14 YARN-3241. FairScheduler handles invalid queue names inconsistently. (Zhihai Xu via kasha) 2015-03-23 13:22:03 -07:00
cnauroth
6ca1f12024 YARN-3336. FileSystem memory leak in DelegationTokenRenewer. 2015-03-23 10:45:50 -07:00
Tsuyoshi Ozawa
82eda771e0 YARN-3384. TestLogAggregationService.verifyContainerLogs fails after YARN-2777. Contributed by Naganarasimha G R. 2015-03-24 00:25:30 +09:00
Jian He
e1feb4ea1a YARN-3345. Add non-exclusive node label API. Contributed by Wangda Tan 2015-03-20 19:04:38 -07:00
Jian He
586348e4cb YARN-3356. Capacity Scheduler FiCaSchedulerApp should use ResourceUsage to track used-resources-by-label. Contributed by Wangda Tan 2015-03-20 13:54:01 -07:00
Junping Du
d81109e588 YARN-3269. Yarn.nodemanager.remote-app-log-dir could not be configured to fully qualified path. Contributed by Xuan Gong 2015-03-20 13:41:22 -07:00
Wangda Tan
6bc7710ec7 YARN-3369. Missing NullPointer check in AppSchedulingInfo causes RM to die. (Brahma Reddy Battula via wangda) 2015-03-20 10:42:05 -07:00
Jian He
4e886eb9cb YARN-3379. Fixed missing data in localityTable and ResourceRequests table in RM WebUI. Contributed by Xuan Gong 2015-03-19 22:27:21 -07:00
Devaraj K
93d0f4acc8 YARN-3357. Move TestFifoScheduler to FIFO package. Contributed by Rohith 
Sharmaks.
 2015-03-19 12:16:52 +05:30
Karthik Kambatla
20b49224eb YARN-3351. AppMaster tracking URL is broken in HA. (Anubhav Dhoot via kasha) 2015-03-18 16:30:33 -07:00
Jian He
658097d6da YARN-3273. Improve scheduler UI to facilitate scheduling analysis and debugging. Contributed Rohith Sharmaks 2015-03-17 21:30:23 -07:00
Tsuyoshi Ozawa
3bc72cc16d YARN-3205. FileSystemRMStateStore should disable FileSystem Cache to avoid get a Filesystem with an old configuration. Contributed by Zhihai Xu. 2015-03-18 11:53:19 +09:00
Jian He
968425e9f7 YARN-3305. Normalize AM resource request on app submission. Contributed by Rohith Sharmaks 2015-03-17 13:49:59 -07:00
Karthik Kambatla
32b4330456 Revert "YARN-3181. FairScheduler: Fix up outdated findbugs issues. (kasha)" 
This reverts commit c2b185def8.
 2015-03-17 12:31:15 -07:00
Jian He
487374b7fe YARN-3243. CapacityScheduler should pass headroom from parent to children to make sure ParentQueue obey its capacity limits. Contributed by Wangda Tan. 2015-03-17 10:24:23 -07:00
Devaraj K
7179f94f9d YARN-3197. Confusing log generated by CapacityScheduler. Contributed by 
Varun Saxena.
 2015-03-17 15:57:57 +05:30
Ravi Prakash
56085203c4 YARN-3339. TestDockerContainerExecutor should pull a single image and not the entire centos repository. (Ravindra Kumar Naik via raviprak) 2015-03-16 16:17:58 -07:00
Tsuyoshi Ozawa
7522a643fa YARN-3349. Treat all exceptions as failure in TestFSRMStateStore#testFSRMStateStoreClientRetry. Contributed by Zhihai Xu. 2015-03-17 08:09:55 +09:00
Tsuyoshi Ozawa
3da9a97cfb YARN-1453. [JDK8] Fix Javadoc errors caused by incorrect or illegal tags in doc comments. Contributed by Akira AJISAKA, Andrew Purtell, and Allen Wittenauer. 2015-03-16 23:19:05 +09:00
Xuan
3ff1ba2a7b YARN-3171. Sort by Application id, AppAttempt and ContainerID doesn't 
work in ATS / RM web ui. Contributed by Naganarasimha G R
 2015-03-15 20:26:10 -07:00
Jonathan Eagles
8180e676ab YARN-3267. Timelineserver applies the ACL rules after applying the limit on the number of records (Chang Li via jeagles) 2015-03-13 12:04:30 -05:00
Vinod Kumar Vavilapalli
863079bb87 YARN-3154. Added additional APIs in LogAggregationContext to avoid aggregating running logs of application when rolling is enabled. Contributed by Xuan Gong. 2015-03-12 13:32:29 -07:00
Zhijie Shen
85f6d67fa7 YARN-1884. Added nodeHttpAddress into ContainerReport and fixed the link to NM web page. Contributed by Xuan Gong. 2015-03-11 19:35:19 -07:00
Allen Wittenauer
a5cf985bf5 YARN-2280. Resource manager web service fields are not accessible (Krisztian Horvath via aw) 2015-03-10 16:32:20 -07:00
Jian He
c3003eba6f YARN-3300. Outstanding_resource_requests table should not be shown in AHS. Contributed by Xuan Gong 2015-03-09 20:46:48 -07:00
Zhijie Shen
d6e05c5ee2 YARN-3287. Made TimelineClient put methods do as the correct login context. Contributed by Daryn Sharp and Jonathan Eagles. 2015-03-09 13:54:36 -07:00
Jason Lowe
27e8ea820f YARN-3275. CapacityScheduler: Preemption happening on non-preemptable queues. Contributed by Eric Payne 2015-03-06 22:37:26 +00:00
Jian He
21101c01f2 YARN-2190. Added CPU and memory limit options to the default container executor for Windows containers. Contributed by Chuan Liu 2015-03-06 14:18:11 -08:00
Jian He
95bfd087dc YARN-1809. Synchronize RM and TimeLineServer Web-UIs. Contributed by Zhijie Shen and Xuan Gong 2015-03-05 21:20:09 -08:00
Tsuyoshi Ozawa
1b67209612 YARN-3249. Add a 'kill application' button to Resource Manager's Web UI. Contributed by Ryu Kobayashi. 2015-03-05 19:55:56 +09:00
Karthik Kambatla
8d88691d16 YARN-3242. Asynchrony in ZK-close can lead to ZKRMStateStore watcher receiving events for old client. (Zhihai Xu via kasha) 2015-03-04 19:49:05 -08:00
Karthik Kambatla
22426a1c9f YARN-3231. FairScheduler: Changing queueMaxRunningApps interferes with pending jobs. (Siqi Li via kasha) 2015-03-04 18:06:58 -08:00
Karthik Kambatla
53947f37c7 YARN-3122. Metrics for container's actual CPU usage. (Anubhav Dhoot via kasha) 2015-03-04 17:33:30 -08:00
Jian He
b2f1ec312e YARN-3222. Fixed NPE on RMNodeImpl#ReconnectNodeTransition when a node is reconnected with a different port. Contributed by Rohith Sharmaks 2015-03-03 16:28:28 -08:00
Wangda Tan
e17e5ba9d7 YARN-3272. Surface container locality info in RM web UI (Jian He via wangda) 2015-03-03 11:49:01 -08:00
Tsuyoshi Ozawa
d1c6accb6f HADOOP-11602. Fix toUpperCase/toLowerCase to use Locale.ENGLISH. (ozawa) 2015-03-03 14:17:52 +09:00
Vinod Kumar Vavilapalli
14dd647c55 YARN-3265. Fixed a deadlock in CapacityScheduler by always passing a queue's available resource-limit from the parent queue. Contributed by Wangda Tan. 2015-03-02 17:52:47 -08:00
Jian He
5d0bae550f YARN-3281. Added RMStateStore to StateMachine visualization list. Contributed by Chengbing Liu 2015-03-02 14:39:49 -08:00
Wangda Tan
edcecedc1c YARN-3262. Surface application outstanding resource requests table in RM web UI. (Jian He via wangda) 2015-02-27 16:13:32 -08:00
Tsuyoshi Ozawa
01a1621930 YARN-2820. Retry in FileSystemRMStateStore when FS's operations fail due to IOException. Contributed by Zhihai Xu. 2015-02-28 00:56:44 +09:00
Konstantin V Shvachko
8ca0d957c4 YARN-3255. RM, NM, JobHistoryServer, and WebAppProxyServer's main() should support generic options. Contributed by Konstantin Shvachko. 2015-02-26 17:12:19 -08:00
Tsuyoshi Ozawa
773b6515ac YARN-3217. Remove httpclient dependency from hadoop-yarn-server-web-proxy. Contributed by Brahma Reddy Battula. 2015-02-27 00:24:29 +09:00
Devaraj K
0d4296f0e0 YARN-3256. TestClientToAMTokens#testClientTokenRace is not running against 
all Schedulers even when using ParameterizedSchedulerTestBase. Contributed
by Anubhav Dhoot.
 2015-02-26 15:45:41 +05:30
Jason Lowe
1a68fc4346 YARN-3239. WebAppProxy does not support a final tracking url which has query fragments and params. Contributed by Jian He 2015-02-25 16:14:34 +00:00
Tsuyoshi Ozawa
6cbd9f1113 YARN-3247. TestQueueMappings should use CapacityScheduler explicitly. Contributed by Zhihai Xu. 2015-02-25 10:38:11 +09:00
Allen Wittenauer
d4ac6822e1 YARN-2980. Move health check script related functionality to hadoop-common (Varun Saxena via aw) 2015-02-24 11:25:26 -08:00
Tsuyoshi Ozawa
9cedad11d8 Revert "HADOOP-11602. Fix toUpperCase/toLowerCase to use Locale.ENGLISH. (ozawa)" 
This reverts commit 946456c6d8.

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/QuotaByStorageTypeEntry.java
 2015-02-25 00:32:04 +09:00
Xuan
fe7a302473 YARN-2797. TestWorkPreservingRMRestart should use 
ParametrizedSchedulerTestBase. Contributed by Karthik Kambatla
 2015-02-21 19:17:29 -08:00
Xuan
e3d290244c YARN-3236. Cleanup RMAuthenticationFilter#AUTH_HANDLER_PROPERTY. 
Contributed by zhihai xu
 2015-02-21 16:18:40 -08:00
Xuan
f56c65bb3e YARN-3237. AppLogAggregatorImpl fails to log error cause. Contributed by 
Rushabh S Shah
 2015-02-20 14:02:40 -08:00
Wangda Tan
ce5bf927c3 YARN-3230. Clarify application states on the web UI. (Jian He via wangda) 2015-02-20 10:39:28 -08:00
Junping Du
c33ae271c2 YARN-2799. Cleanup TestLogAggregationService based on the change in YARN-90. Contributed by Zhihai Xu 2015-02-20 09:43:39 -08:00
Jason Lowe
a64dd3d24b YARN-3194. RM should handle NMContainerStatuses sent by NM while registering if NM is Reconnected node. Contributed by Rohith 2015-02-20 15:10:10 +00:00
Jian He
c0d9b93953 YARN-933. Fixed InvalidStateTransitonException at FINAL_SAVING state in RMApp. Contributed by Rohith Sharmaks 2015-02-19 15:42:39 -08:00
Wangda Tan
d49ae725d5 YARN-3076. Add API/Implementation to YarnClient to retrieve label-to-node mapping (Varun Saxena via wangda) 2015-02-19 11:00:57 -08:00
Tsuyoshi Ozawa
946456c6d8 HADOOP-11602. Fix toUpperCase/toLowerCase to use Locale.ENGLISH. (ozawa) 2015-02-19 13:06:53 +09:00
Tsuyoshi Ozawa
b8a14efdf5 YARN-1615. Fix typos in delay scheduler's description. Contributed by Akira Ajisaka. 2015-02-19 10:24:07 +09:00
Jian He
1c03376300 YARN-1514. Utility to benchmark ZKRMStateStore#loadState for RM HA. Contributed by Tsuyoshi OZAWA 2015-02-18 16:06:55 -08:00
Jian He
f5da5566d9 YARN-3132. RMNodeLabelsManager should remove node from node-to-label mapping when node becomes deactivated. Contributed by Wangda Tan 2015-02-18 11:51:51 -08:00
Xuan
57db50cbe3 YARN-3207. Secondary filter matches entites which do not have the key 
being filtered for. Contributed by Zhijie Shen
 2015-02-17 18:17:29 -08:00
Tsuyoshi Ozawa
556386a070 YARN-1299. Improve a log message in AppSchedulingInfo by adding application id. Contributed by Ashutosh Jindal and Devaraj K. 2015-02-17 01:53:31 +09:00
Tsuyoshi Ozawa
3f32357c36 Revert "YARN-1299. Improve a log message in AppSchedulingInfo by adding application id. Contributed by Ashutosh Jindal and devaraj." 
This reverts commit 9aae81c934.
 2015-02-17 01:52:22 +09:00
Tsuyoshi Ozawa
9aae81c934 YARN-1299. Improve a log message in AppSchedulingInfo by adding application id. Contributed by Ashutosh Jindal and devaraj. 2015-02-17 01:48:22 +09:00
Tsuyoshi Ozawa
447bd7b5a6 YARN-3203. Correct a log message in AuxServices. Contributed by Brahma Reddy Battula. 2015-02-16 23:55:58 +09:00
Junping Du
ab0b958a52 YARN-2749. Fix some testcases from TestLogAggregationService fails in trunk. (Contributed by Xuan Gong) 2015-02-15 06:46:32 -08:00
cnauroth
6804d68901 YARN-2899. Run TestDockerContainerExecutorWithMocks on Linux only. Contributed by Ming Ma. 2015-02-13 21:58:54 -08:00
Jason Lowe
1a0f508b63 YARN-2847. Linux native container executor segfaults if default banned user detected. Contributed by Olaf Flebbe 2015-02-13 20:20:55 +00:00
Tsuyoshi Ozawa
110cf6b9a3 YARN-3182. Cleanup switch statement in ApplicationMasterLauncher#handle(). Contributed by Ray Chiang. 2015-02-13 14:21:21 +09:00
Akira Ajisaka
6a49e58cb8 YARN-3191. Log object should be initialized with its own class. Contributed by Rohith. 2015-02-12 17:58:54 -08:00
Xuan
99f6bd4f7a YARN-3158. Correct log messages in ResourceTrackerService. Contributed 
by Varun Saxena
 2015-02-12 16:42:15 -08:00
Jian He
18297e0972 YARN-3104. Fixed RM to not generate new AMRM tokens on every heartbeat between rolling and activation. Contributed by Jason Lowe 2015-02-12 16:02:45 -08:00
Jian He
18a594257e YARN-3124. Fixed CS LeafQueue/ParentQueue to use QueueCapacities to track capacities-by-label. Contributed by Wangda Tan 2015-02-12 14:58:09 -08:00
Karthik Kambatla
c2b185def8 YARN-3181. FairScheduler: Fix up outdated findbugs issues. (kasha) 2015-02-12 13:44:47 -08:00
Junping Du
04f5ef18f7 YARN-2079. Recover NonAggregatingLogHandler state upon nodemanager restart. (Contributed by Jason Lowe) 2015-02-12 11:46:47 -08:00
Xuan
83be450acc YARN-3147. Clean up RM web proxy code. Contributed by Steve Loughran 2015-02-12 10:57:55 -08:00
Xuan
65c69e296e YARN-3151. On Failover tracking url wrong in application cli for KILLED 
application. Contributed by Rohith
 2015-02-11 21:19:48 -08:00
Jason Lowe
b379972ab3 YARN-3074. Nodemanager dies when localizer runner tries to write to a full disk. Contributed by Varun Saxena 2015-02-11 16:33:43 +00:00
Junping Du
c541a374d8 YARN-3160. Fix non-atomic operation on nodeUpdateQueue in RMNodeImpl. (Contributed by Chengbing Liu) 2015-02-11 03:44:31 -08:00
Zhijie Shen
d5855c0e46 YARN-2246. Made the proxy tracking URL always be http(s)://proxy addr:port/proxy/<appId> to avoid duplicate sections. Contributed by Devaraj K. 2015-02-10 15:24:01 -08:00
Jason Lowe
3f5431a22f YARN-2809. Implement workaround for linux kernel panic when removing cgroup. Contributed by Nathan Roberts 2015-02-10 17:27:21 +00:00
Jason Lowe
4eb5f7fa32 YARN-3090. DeletionService can silently ignore deletion task failures. Contributed by Varun Saxena 2015-02-10 16:54:21 +00:00
Zhijie Shen
23bf6c7207 YARN-3100. Made YARN authorization pluggable. Contributed by Jian He. 2015-02-09 20:34:56 -08:00
Jian He
0af6a99a3f YARN-3094. Reset timer for liveness monitors after RM recovery. Contributed by Jun Gong 2015-02-09 13:47:08 -08:00
Karthik Kambatla
7e42088abf YARN-2990. FairScheduler's delay-scheduling always waits for node-local and rack-local delays, even for off-rack-only requests. (kasha) 2015-02-08 22:48:10 -08:00
Jason Lowe
da2fb2bc46 YARN-3143. RM Apps REST API can return NPE or entries missing id and other fields. Contributed by Jason Lowe 2015-02-06 21:47:32 +00:00