big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
16,850 Commits 2 Branches 2 Tags 264 MiB
df800f6cf3
Commit Graph

534 Commits

Author SHA1 Message Date
Junping Du
d275677e24 YARN-3634. TestMRTimelineEventHandling and TestApplication are broken. Contributed by Sangjin Lee. 
(cherry picked from commit b059dd4882fd759e4762cc11c019be4b68fb74c1)
 2016-07-10 08:45:36 -07:00
Junping Du
47f35a30bb YARN-3391. Clearly define flow ID/ flow run / flow version in API and storage. Contributed by Zhijie Shen 
(cherry picked from commit 68c6232f8423e55b4d152ef3d1d66aeb2d6a555e)
 2016-07-10 08:45:33 -07:00
Zhijie Shen
5712b8f9fd YARN-3334. NM uses timeline client to publish container metrics to new timeline service. Contributed by Junping Du. 2016-07-10 08:45:33 -07:00
Junping Du
d67c9bdb4d YARN-3040. Make putEntities operation be aware of the app's context. Contributed by Zhijie Shen 2016-07-10 08:45:32 -07:00
Junping Du
2188a07e5b YARN-3333. Rename TimelineAggregator etc. to TimelineCollector. Contributed by Sangjin Lee 2016-07-10 08:45:31 -07:00
Zhijie Shen
9b56364080 YARN-3039. Implemented the app-level timeline aggregator discovery service. Contributed by Junping Du. 2016-07-10 08:45:31 -07:00
Vinod Kumar Vavilapalli
4c9e1aeb94 YARN-5224. Added new web-services /containers/{containerid}/logs & /containers/{containerid}/logs/{filename} and using them in "yarn logs" CLI to get logs of finished containers of a running application. Contributed by Xuan Gong. 2016-07-06 14:10:54 -07:00
Arun Suresh
99e5dd68d0 YARN-5171. Extend DistributedSchedulerProtocol to notify RM of containers allocated by the Node. (Inigo Goiri via asuresh) 2016-06-22 19:04:54 -07:00
Tsuyoshi Ozawa
5d58858bb6 HADOOP-9613. [JDK8] Update jersey version to latest 1.x release. 2016-06-21 08:05:32 +09:00
Junping Du
c35fa4a0e5 YARN-5251. Yarn CLI to obtain App logs for last 'n' bytes fails. Contributed by Xuan Gong. 2016-06-17 08:24:24 -07:00
Junping Du
5dfc38ff57 YARN-5237. Fix missing log files issue in rolling log aggregation. Contributed by Xuan Gong. 2016-06-15 16:17:54 -07:00
Wangda Tan
c77a1095dc YARN-1942. Deprecate toString/fromString methods from ConverterUtils and move them to records classes like ContainerId/ApplicationId, etc. (wangda) 2016-06-14 15:06:38 -07:00
Arun Suresh
7dae2b3bc4 YARN-5212. Run existing ContainerManager tests using QueuingContainerManagerImpl. (Konstantinos Karanasos via asuresh) 2016-06-12 10:05:37 -07:00
Vinod Kumar Vavilapalli
9378d9428f YARN-5191. Renamed the newly added “download=true” option for getting logs via NMWebServices and AHSWebServices to be a better "format" option. (Xuan Gong via vinodkv) 2016-06-09 12:30:58 -07:00
Naganarasimha
1500a0a300 YARN-4308. ContainersAggregated CPU resource utilization reports negative usage in first few heartbeats. Contributed by Sunil G 2016-06-09 05:41:09 +05:30
Arun Suresh
3344ba70e0 YARN-5204. Properly report status of killed/stopped queued containers. (Konstantinos Karanasos via asuresh) 2016-06-08 08:31:32 -07:00
Arun Suresh
76f0800c21 YARN-5176. More test cases for queuing of containers at the NM. (Konstantinos Karanasos via asuresh) 2016-06-07 17:16:18 -07:00
Rohith Sharma K S
bddea5fe5f YARN-5118. Tests fails with localizer port bind exception. Contributed by Brahma Reddy Battula. 2016-06-07 11:20:15 +05:30
Ming Ma
4a1cedc010 MAPREDUCE-5044. Have AM trigger jstack on task attempts that timeout before killing them. (Eric Payne and Gera Shegalov via mingma) 2016-06-06 14:30:51 -07:00
Jian He
99cc439e29 YARN-5190. Registering/unregistering container metrics in ContainerMonitorImpl and ContainerImpl causing uncaught exception in ContainerMonitorImpl. Contributed by Junping Du 2016-06-03 11:10:42 -07:00
Arun Suresh
dc26601d8f YARN-5180. Allow ResourceRequest to specify an enforceExecutionType flag. (asuresh) 2016-06-02 09:01:02 -07:00
Xuan
0bc05e40fa YARN-5088. Improve "yarn log" command-line to read the last K bytes for the log files. Contributed by Xuan Gong 2016-06-01 13:44:21 -07:00
Varun Vasudev
86fb58b7dc YARN-4007. Add support for different network setups when launching the docker container. Contributed by Sidharta Seethana. 2016-05-29 21:32:51 +05:30
Varun Vasudev
42f90ab885 YARN-4844. Add getMemorySize/getVirtualCoresSize to o.a.h.y.api.records.Resource. Contributed by Wangda Tan. 2016-05-29 21:24:16 +05:30
Arun Suresh
4fc09a897b YARN-5117. QueuingContainerManager does not start GUARANTEED Container even if Resources are available. (Konstantinos Karanasos via asuresh) 2016-05-27 17:49:15 -07:00
Carlo Curino
013532a95e YARN-4957. Add getNewReservation in ApplicationClientProtocol (Sean Po via curino) 2016-05-25 16:55:49 -07:00
Robert Kanter
e07519b8db YARN-4766. NM should not aggregate logs older than the retention policy (haibochen via rkanter) 2016-05-25 10:25:44 -07:00
Varun Vasudev
c380a22031 YARN-857. Localization failures should be available in container diagnostics. Contributed by Vinod Kumar Vavilapalli. 2016-05-25 19:01:12 +05:30
Varun Vasudev
0287c49107 YARN-5016. Add support for a minimum retry interval for container retries. Contributed by Jun Gong. 2016-05-20 12:28:14 +05:30
Akira Ajisaka
d4274c64bc YARN-5107. TestContainerMetrics fails. (aajisaka) 2016-05-19 19:53:59 +09:00
Arun Suresh
1597630681 YARN-5110. Fix OpportunisticContainerAllocator to insert complete HostAddress in issued ContainerTokenIds. (Konstantinos Karanasos via asuresh) 2016-05-18 18:46:00 -07:00
Jason Lowe
81effb7dcd YARN-4325. Nodemanager log handlers fail to send finished/failed events in some cases. Contributed by Junping Du 2016-05-16 15:40:23 +00:00
Arun Suresh
f45bc5a83e YARN-4738. Notify the RM about the status of OPPORTUNISTIC containers (Konstantinos Karanasos via asuresh) 2016-05-15 17:54:34 -07:00
Arun Suresh
f0ac18d001 YARN-2888. Corrective mechanisms for rebalancing NM container queues. (asuresh) 2016-05-13 13:38:36 -07:00
Sangjin Lee
0bbe01f8d5 YARN-4577. Enable aux services to have their own custom classpath/jar file (Xuan Gong via sale) 2016-05-12 10:11:23 -07:00
Arun Suresh
d464f4d1c4 YARN-5049. Extend NMStateStore to save queued container information. (Konstantinos Karanasos via asuresh) 2016-05-11 19:10:17 -07:00
Varun Vasudev
72b047715c YARN-4595. Add support for configurable read-only mounts when launching Docker containers. Contributed by Billie Rinaldi. 2016-05-05 13:01:54 +05:30
Wangda Tan
7bd418e48c YARN-4984. LogAggregationService shouldn't swallow exception in handling createAppDir() which cause thread leak. (Junping Du via wangda) 2016-05-04 11:38:55 -07:00
Junping Du
e61d431275 YARN-4920. ATS/NM should support a link to dowload/get the logs in text format. Contributed by Xuan Gong. 2016-05-04 10:35:49 -07:00
Robert Kanter
9d4d30243b Remove parent's env vars from child processes 2016-04-29 09:25:51 -07:00
Varun Vasudev
0f25a1bb52 YARN-3998. Add support in the NodeManager to re-launch containers. Contributed by Jun Gong. 2016-04-29 16:09:07 +05:30
Jian He
4a8508501b YARN-5009. NMLeveldbStateStoreService database can grow substantially leading to longer recovery times. Contributed by Jason Lowe 2016-04-28 21:54:11 -07:00
Karthik Kambatla
1a3f1482e2 YARN-4795. ContainerMetrics drops records. (Daniel Templeton via kasha) 2016-04-26 06:15:36 -07:00
Arun Suresh
c282a08f38 YARN-2885. Create AMRMProxy request interceptor and ContainerAllocator to distribute OPPORTUNISTIC containers to appropriate Nodes (asuresh) 
(cherry picked from commit 2bf025278a318b0452fdc9ece4427b4c42124e39)
 2016-04-24 22:38:33 -07:00
Jing Zhao
63e5412f1a HDFS-9427. HDFS should not default to ephemeral ports. Contributed by Xiaobing Zhou. 2016-04-22 15:14:40 -07:00
Karthik Kambatla
c8172f5f14 YARN-2883. Queuing of container requests in the NM. (Konstantinos Karanasos and Arun Suresh via kasha) 2016-04-20 09:55:50 -07:00
Jason Lowe
3150ae8108 YARN-4924. NM recovery race can lead to container not cleaned up. Contributed by sandflee 2016-04-14 19:17:14 +00:00
Naganarasimha
437e9d6475 YARN-4810. NM applicationpage cause internal error 500. Contributed by Bibin A Chundatt. 2016-04-12 17:59:46 +05:30
Vinod Kumar Vavilapalli
44bbc50d91 YARN-4168. Fixed a failing test TestLogAggregationService.testLocalFileDeletionOnDiskFull. Contributed by Takashi Ohnishi. 2016-04-11 12:11:14 -07:00
Karthik Kambatla
e82f961a39 YARN-4756. Unnecessary wait in Node Status Updater during reboot. (Eric Badger via kasha) 2016-04-07 17:05:29 -07:00
Varun Vasudev
b41e65e5bc YARN-4906. Capture container start/finish time in container metrics. Contributed by Jian He. 2016-04-06 13:41:33 +05:30
Junping Du
0005816743 YARN-4916. TestNMProxy.tesNMProxyRPCRetry fails. Contributed by Tibor Kiss. 2016-04-05 09:01:08 -07:00
naganarasimha
5092c94195 YARN-4746. yarn web services should convert parse failures of appId, appAttemptId and containerId to 400. Contributed by Bibin A Chundatt 2016-04-04 16:25:03 +05:30
Jian He
0dd9bcab97 YARN-4811. Generate histograms in ContainerMetrics for actual container resource usage 2016-03-31 14:28:13 -07:00
Allen Wittenauer
0a74610d1c HADOOP-11393. Revert HADOOP_PREFIX, go back to HADOOP_HOME (aw) 2016-03-31 07:51:05 -07:00
Jason Lowe
948b758070 YARN-4773. Log aggregation performs extraneous filesystem operations when rolling log aggregation is disabled. Contributed by Jun Gong 2016-03-28 23:00:56 +00:00
Robert Kanter
22ca176dfe TestResourceLocalizationService.testPublicResourceInitializesLocalDir fails Intermittently due to IllegalArgumentException from cleanup (templedf via rkanter) 2016-03-15 10:05:10 -07:00
Vinod Kumar Vavilapalli
b2661765a5 YARN-4762. Fixed CgroupHandler's creation and usage to avoid NodeManagers crashing when LinuxContainerExecutor is enabled. (Sidharta Seethana via vinodkv) 2016-03-07 11:08:17 -08:00
Jason Lowe
059caf9989 YARN-4744. Too many signal to container failure in case of LCE. Contributed by Sidharta Seethana 2016-03-07 15:40:01 +00:00
Haohui Mai
0fa54d45b1 HADOOP-12813. Migrate TestRPC and related codes to rebase on ProtobufRpcEngine. Contributed by Kai Zheng. 2016-02-29 11:41:00 -08:00
Ming Ma
7f3139e54d YARN-4720. Skip unnecessary NN operations in log aggregation. (Jun Gong via mingma) 2016-02-26 08:40:05 -08:00
Robert Kanter
d7fdec1e6b YARN-4579. Allow DefaultContainerExecutor container log directory permissions to be configurable (rchiang via rkanter) 2016-02-25 16:36:38 -08:00
Robert Kanter
954dd57043 YARN-4697. NM aggregation thread pool is not bound by limits (haibochen via rkanter) 2016-02-24 15:00:24 -08:00
Jason Lowe
d284e187b8 YARN-2046. Out of band heartbeats are sent only on container kill and possibly too early. Contributed by Ming Ma 2016-02-23 20:49:09 +00:00
Varun Vasudev
140cb5d745 YARN-4709. NMWebServices produces incorrect JSON for containers. Contributed by Varun Saxena. 2016-02-23 12:29:25 +05:30
Varun Vasudev
fa00d3e205 YARN-4655. Log uncaught exceptions/errors in various thread pools in YARN. Contributed by Sidharta Seethana. 2016-02-11 12:06:42 +05:30
Wangda Tan
9875325d5c YARN-4340. Add list API to reservation system. (Sean Po via wangda) 2016-02-02 10:17:33 +08:00
Rohith Sharma K S
ac68666803 YARN-4543. Fix random test failure in TestNodeStatusUpdater.testStopReentrant. (Akihiro Suda via rohithsharmaks) 2016-01-29 12:29:54 +05:30
Jason Lowe
61382ff8fa YARN-4643. Container recovery is broken with delegating container runtime. Contributed by Sidharta Seethana 2016-01-28 18:59:35 +00:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
2085e60a96 YARN-3542. Refactored existing CPU cgroups support to use the newer and integrated ResourceHandler mechanism, and also deprecated the old LCEResourceHandler inteface hierarchy. Contributed by Varun Vasudev. 2016-01-25 16:19:36 -08:00
Xuan
618bfd6ac2 YARN-4496. Improve HA ResourceManager Failover detection on the client. 
Contributed by Jian He
 2016-01-22 18:20:38 -08:00
Varun Vasudev
b41a7e89d1 YARN-4578. Directories that are mounted in docker containers need to be more restrictive/container-specific. Contributed by Sidharta Seethana. 2016-01-22 14:43:14 +05:30
Wangda Tan
89d1fd5dac HADOOP-12356. Fix computing CPU usage statistics on Windows. (Inigo Goiri via wangda) 2016-01-19 21:27:38 +08:00
Varun Vasudev
3ddb92bd30 YARN-4553. Add cgroups support for docker containers. Contributed by Sidharta Seethana. 2016-01-14 14:29:29 +05:30
Jason Lowe
13de8359a1 YARN-4414. Nodemanager connection errors are retried at multiple levels. Contributed by Chang Li 2016-01-12 15:56:15 +00:00
Steve Loughran
07d1cb612c YARN-4550. Some tests in TestContainerLanch fails on non-english locale environment. (Takashi Ohnishi via stevel) 2016-01-07 14:30:20 +00:00
rohithsharmaks
791c1639ae YARN-4393. Fix intermittent test failure for TestResourceLocalizationService#testFailedDirsResourceRelease (Varun Saxana via rohithsharmaks) 2016-01-07 09:38:47 +05:30
Gera Shegalov
2c17b81569 YARN-2934. Improve handling of container's stderr. (Naganarasimha G R via gera) 2015-12-24 23:48:05 -08:00
Uma Mahesh
0f82b5d878 YARN-4480. Clean up some inappropriate imports. (Kai Zheng via umamahesh) 2015-12-19 23:10:13 -08:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
4e7d32c0db YARN-1856. Added cgroups based memory monitoring for containers as another alternative to custom memory-monitoring. Contributed by Varun Vasudev. 2015-12-17 12:13:03 -08:00
Jian He
915cd6c3f4 YARN-4402. TestNodeManagerShutdown And TestNodeManagerResync fails with bind exception. Contributed by Brahma Reddy Battula 2015-12-14 14:59:01 -08:00
Wangda Tan
dfcbbddb09 YARN-4309. Add container launch related debug information to container logs when a container fails. (Varun Vasudev via wangda) 2015-12-14 11:13:22 -08:00
Junping Du
62e9348bc1 YARN-4408. Fix issue that NodeManager still reports negative running containers. Contributed by Robert Kanter. 2015-12-03 06:36:37 -08:00
Tsuyoshi Ozawa
0656d2dc83 YARN-4380. TestResourceLocalizationService.testDownloadingResourcesOnContainerKill fails intermittently. Contributed by Varun Saxena. 2015-11-26 01:10:02 +09:00
Jason Lowe
4ac6799d4a YARN-4132. Separate configs for nodemanager to resourcemanager connection timeout and retries. Contributed by Chang Li 2015-11-24 22:35:37 +00:00
Junping Du
855d52927b YARN-4354. Public resource localization fails with NPE. Contributed by Jason Lowe. 2015-11-15 04:43:57 -08:00
Jason Lowe
e2267de207 YARN-2902. Killing a container that is localizing can orphan resources in the DOWNLOADING state. Contributed by Varun Saxena 2015-10-29 16:34:25 +00:00
Wangda Tan
6f606214e7 YARN-4169. Fix racing condition of TestNodeStatusUpdaterForLabels. (Naganarasimha G R via wangda) 2015-10-26 16:36:34 -07:00
Rohith Sharma K S
5acdde4744 YARN-2729. Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup. (Naganarasimha G R via rohithsharmaks) 2015-10-26 15:42:42 +05:30
Tsuyoshi Ozawa
ce60b4fc8b YARN-3528. Tests with 12345 as hard-coded port break jenkins. Contributed by Brahma Reddy Battula. 2015-10-26 16:45:11 +09:00
Varun Vasudev
e39ae0e676 YARN-4262. Allow whitelisted users to run privileged docker containers. Contributed by Sidharta Seethana. 2015-10-19 20:19:22 +05:30
Steve Loughran
e2d59e2c7b YARN-4155. TestLogAggregationService.testLogAggregationServiceWithInterval failing. (Bibin A Chundatt via stevel) 2015-10-18 12:03:57 +01:00
Varun Vasudev
63020c54c1 YARN-4258. Add support for controlling capabilities for docker containers. Contributed by Sidharta Seethana. 2015-10-15 13:41:20 +05:30
Steve Loughran
dfa7848463 HADOOP-12478. Shell.getWinUtilsPath() has been renamed Shell.getWinutilsPath(). (stevel) 2015-10-14 20:25:47 +01:00
Steve Loughran
c59af2fdf8 HADOOP-10775. Shell operations to fail with meaningful errors on windows if winutils.exe not found. (stevel) 2015-10-13 21:49:28 +01:00
Jason Lowe
a0bca2b5ad YARN-261. Ability to fail AM attempts. Contributed by Andrey Klochkov and Rohith Sharma K S 2015-10-09 14:17:38 +00:00
Jason Lowe
8d226225d0 YARN-3943. Use separate threshold configurations for disk-full detection and disk-not-full detection. Contributed by Zhihai Xu 2015-10-08 22:25:34 +00:00
Wangda Tan
30ac69c6bd YARN-4176. Resync NM nodelabels with RM periodically for distributed nodelabels. (Bibin A Chundatt via wangda) 2015-10-05 15:47:21 -07:00
Xuan
8f08532bde YARN-1897. CLI and core support for signal container functionality. Contributed by Ming Ma 2015-10-02 18:50:47 -07:00
Jason Lowe
fdf02d1f26 YARN-3619. ContainerMetrics unregisters during getMetrics and leads to ConcurrentModificationException. Contributed by Zhihai Xu 2015-10-02 20:20:31 +00:00
Jason Lowe
854d25b0c3 YARN-3727. For better error recovery, check if the directory exists before using it for localization. Contributed by Zhihai Xu 2015-09-30 14:59:44 +00:00
Rohith Sharma K S
8ed0d4b744 YARN-4152. NodeManager crash with NPE when LogAggregationService#stopContainer called for absent container. (Bibin A Chundatt via rohithsharmaks) 2015-09-24 11:24:14 +05:30
Jian He
c57eac5dfe YARN-3868. Recovery support for container resizing. Contributed by Meng Ding 2015-09-23 13:29:38 -07:00
Jian He
c3dc1af072 YARN-1644. RM-NM protocol changes and NodeStatusUpdater implementation to support container resizing. Contributed by Meng Ding 2015-09-23 13:29:37 -07:00
Jian He
c59ae4eeb1 YARN-1643. Make ContainersMonitor support changing monitoring size of an allocated container. Contributed by Meng Ding and Wangda Tan 2015-09-23 13:29:37 -07:00
Jian He
5f5a968d65 YARN-3867. ContainerImpl changes to support container resizing. Contributed by Meng Ding 2015-09-23 13:29:37 -07:00
Jian He
ffd820c27a YARN-1645. ContainerManager implementation to support container resizing. Contributed by Meng Ding & Wangda Tan 2015-09-23 13:29:37 -07:00
Jason Lowe
c890c51a91 YARN-4095. Avoid sharing AllocatorPerContext object in LocalDirAllocator between ShuffleHandler and LocalDirsHandlerService. Contributed by Zhihai Xu 2015-09-23 15:42:01 +00:00
Xuan
34ef1a092b YARN-4149. yarn logs -am should provide an option to fetch all the log 
files. Contributed by Varun Vasudev
 2015-09-15 14:36:30 -07:00
Wangda Tan
77666105b4 YARN-4106. NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM. (Bibin A Chundatt via wangda) 2015-09-10 09:30:09 -07:00
Zhihai Xu
16b9037dc1 YARN-4096. App local logs are leaked if log aggregation fails to initialize for the app. Contributed by Jason Lowe. 2015-09-08 12:29:54 -07:00
Jian He
6f72f1e600 YARN-2884. Added a proxy service in NM to proxy the the communication between AM and RM. Contributed by Kishore Chaliparambil 2015-09-08 09:35:46 +08:00
Varun Vasudev
1dbd8e34a7 YARN-3591. Resource localization on a bad disk causes subsequent containers failure. Contributed by Lavkesh Lahngir. 2015-09-07 11:32:12 +05:30
Rohith Sharma K S
095ab9ab5f YARN-4073. Removed unused ApplicationACLsManager in ContainerManagerImpl constructor. (Naganarasimha G R via rohithsharmaks) 2015-09-02 14:13:33 +05:30
Xuan
37e1c3d82a YARN-221. NM should provide a way for AM to tell it not to aggregate 
logs. Contributed by Ming Ma
 2015-08-22 16:25:24 -07:00
Wangda Tan
fc07464d1a YARN-2923. Support configuration based NodeLabelsProvider Service in Distributed Node Label Configuration Setup. (Naganarasimha G R) 2015-08-20 11:51:03 -07:00
Karthik Kambatla
def12933b3 YARN-3534. Collect memory/cpu usage on the node. (Inigo Goiri via kasha) 2015-08-16 06:24:16 -07:00
Jian He
38aed1a94e YARN-4005. Completed container whose app is finished is possibly not removed from NMStateStore. Contributed by Jun Gong 2015-08-13 14:46:08 -07:00
Jason Lowe
469cfcd695 YARN-3965. Add startup timestamp to nodemanager UI. Contributed by Hong Zhiguo 2015-08-03 15:53:32 +00:00
Varun Vasudev
3e6fce91a4 YARN-3853. Add docker container runtime support to LinuxContainterExecutor. Contributed by Sidharta Seethana. 2015-07-27 11:57:40 -07:00
Jason Lowe
ff9c13e0a7 YARN-3925. ContainerLogsUtils#getContainerLogFile fails to read container log files from full disks. Contributed by zhihai xu 2015-07-24 22:14:39 +00:00
Wangda Tan
06e5dd2c84 YARN-3956. Fix TestNodeManagerHardwareUtils fails on Mac (Varun Vasudev via wangda) 2015-07-22 12:01:41 -07:00
Varun Vasudev
63d0365088 YARN-2194. Addendum patch to fix failing unit test in TestPrivilegedOperationExecutor. Contributed by Sidharta Seethana. 2015-07-09 11:51:59 +05:30
Varun Vasudev
c40bdb56a7 YARN-2194. Fix bug causing CGroups functionality to fail on RHEL7. Contributed by Wei Yan. 2015-07-07 16:59:29 +05:30
Jason Lowe
b5cdf78e8e YARN-3793. Several NPEs when deleting local files on NM recovery. Contributed by Varun Saxena 2015-07-01 21:13:32 +00:00
Jian He
62e583c7dc YARN-3695. ServerProxy (NMProxy, etc.) shouldn't retry forever for non network exception. Contributed by Raju Bairishetti 2015-06-29 13:37:32 -07:00
Jason Lowe
40b256949a YARN-3850. NM fails to read files from full disks which can lead to container logs being lost and other issues. Contributed by Varun Saxena 2015-06-26 15:47:07 +00:00
Karthik Kambatla
5ebf2817e5 YARN-3842. NMProxy should retry on NMNotYetReadyException. (Robert Kanter via kasha) 2015-06-22 17:47:47 -07:00
Xuan
6c7a9d502a YARN-3834. Scrub debug logging of tokens during resource localization. Contributed by Chris Nauroth 2015-06-21 17:13:44 -07:00
Devaraj K
126321eded YARN-3747. TestLocalDirsHandlerService should delete the created test 
directory logDir2. Contributed by David Moore.
 2015-06-08 15:32:13 +05:30
Junping Du
d7e7f6aa03 YARN-41. The RM should handle the graceful shutdown of the NM. Contributed by Devaraj K. 2015-06-04 04:59:27 -07:00
cnauroth
4102e5882e YARN-3626. On Windows localized resources are not moved to the front of the classpath when they should be. Contributed by Craig Welch. 2015-05-27 14:31:49 -07:00
Vinod Kumar Vavilapalli
500a1d9c76 YARN-160. Enhanced NodeManager to automatically obtain cpu/memory values from underlying OS when configured to do so. Contributed by Varun Vasudev. 2015-05-26 11:38:35 -07:00
Vinod Kumar Vavilapalli
53fafcf061 YARN-3684. Changed ContainerExecutor's primary lifecycle methods to use a more extensible mechanism of context objects. Contributed by Sidharta Seethana. 2015-05-21 15:50:23 -07:00
Wangda Tan
b37da52a1c YARN-3565. NodeHeartbeatRequest/RegisterNodeManagerRequest should use NodeLabel object instead of String. (Naganarasimha G R via wangda) 2015-05-19 16:34:17 -07:00
Ravi Prakash
c97f32e7b9 YARN-3302. TestDockerContainerExecutor should run automatically if it can detect docker in the usual place (Ravindra Kumar Naik via raviprak) 2015-05-19 10:28:11 -07:00
Xuan
0f95921447 YARN-3626. On Windows localized resources are not moved to the front of the classpath when they should be. Contributed by Craig Welch 2015-05-13 13:10:53 -07:00
Karthik Kambatla
70fb37cd79 YARN-1287. Consolidate MockClocks. (Sebastian Wong and Anubhav Dhoot via kasha) 2015-05-09 14:34:54 -07:00
Xuan
6471d18bc7 YARN-1912. ResourceLocalizer started without any jvm memory control. 
Contributed by Masatake Iwasaki
 2015-05-08 20:01:21 -07:00
Xuan
333f9a896d YARN-3602. TestResourceLocalizationService.testPublicResourceInitializesLocalDir fails Intermittently due to IOException from cleanup. Contributed by zhihai xu 2015-05-08 17:52:54 -07:00
Xuan
088156de43 YARN-2331. Distinguish shutdown during supervision vs. shutdown for 
rolling upgrade. Contributed by Jason Lowe
 2015-05-08 15:10:43 -07:00
Robert (Bobby) Evans
bcf2890502 YARN-644: Basic null check is not performed on passed in arguments before using them in ContainerManagerImpl.startContainer 2015-05-08 11:11:01 -05:00
Akira Ajisaka
b167fe7605 YARN-1832. Fix wrong MockLocalizerStatus#equals implementation. Contributed by Hong Zhiguo. 2015-05-08 11:14:45 +09:00
Akira Ajisaka
918af8efff YARN-3577. Misspelling of threshold in log4j.properties for tests. Contributed by Brahma Reddy Battula. 2015-05-07 13:33:03 +09:00
Robert Kanter
b72507810a YARN-3491. PublicLocalizer#addResource is too slow. (zxu via rkanter) 2015-05-06 14:19:06 -07:00
Robert Kanter
ac7d152901 YARN-3363. add localization and container launch time to ContainerMetrics at NM to show these timing information for each active container. (zxu via rkanter) 2015-05-01 16:39:21 -07:00
Vinod Kumar Vavilapalli
1b3b9e5c31 YARN-2619. Added NodeManager support for disk io isolation through cgroups. Contributed by Varun Vasudev and Wei Yan. 2015-04-30 21:41:07 -07:00
Karthik Kambatla
47279c3228 YARN-3464. Race condition in LocalizerRunner kills localizer before localizing all resources. (Zhihai Xu via kasha) 2015-04-26 09:13:46 -07:00
Vinod Kumar Vavilapalli
a100be685c YARN-3366. Enhanced NodeManager to support classifying/shaping outgoing network bandwidth traffic originating from YARN containers Contributed by Sidharta Seethana. 2015-04-22 17:26:13 -07:00
Jian He
674c7ef649 YARN-3503. Expose disk utilization percentage and bad local and log dir counts in NM metrics. Contributed by Varun Vasudev 2015-04-21 20:57:02 -07:00
Jian He
1b89a3e173 YARN-3354. Add node label expression in ContainerTokenIdentifier to support RM recovery. Contributed by Wangda Tan 2015-04-15 13:57:06 -07:00
Junping Du
838b06ac87 YARN-3443. Create a 'ResourceHandler' subsystem to ease addition of support for new resource types on the NM. Contributed by Sidharta Seethana. 2015-04-13 18:35:56 -07:00
Vinod Kumar Vavilapalli
b21c72777a YARN-3365. Enhanced NodeManager to support using the 'tc' tool via container-executor for outbound network traffic control. Contributed by Sidharta Seethana. 2015-04-02 16:53:59 -07:00
Wangda Tan
2a945d24f7 YARN-2495. Allow admin specify labels from each NM (Distributed configuration for node label). (Naganarasimha G R via wangda) 2015-03-30 12:05:21 -07:00
Ravi Prakash
e0ccea33c9 YARN-3288. Document and fix indentation in the DockerContainerExecutor code 2015-03-28 08:00:41 -07:00
Tsuyoshi Ozawa
82eda771e0 YARN-3384. TestLogAggregationService.verifyContainerLogs fails after YARN-2777. Contributed by Naganarasimha G R. 2015-03-24 00:25:30 +09:00
Junping Du
d81109e588 YARN-3269. Yarn.nodemanager.remote-app-log-dir could not be configured to fully qualified path. Contributed by Xuan Gong 2015-03-20 13:41:22 -07:00
Ravi Prakash
56085203c4 YARN-3339. TestDockerContainerExecutor should pull a single image and not the entire centos repository. (Ravindra Kumar Naik via raviprak) 2015-03-16 16:17:58 -07:00
Vinod Kumar Vavilapalli
863079bb87 YARN-3154. Added additional APIs in LogAggregationContext to avoid aggregating running logs of application when rolling is enabled. Contributed by Xuan Gong. 2015-03-12 13:32:29 -07:00
Jian He
21101c01f2 YARN-2190. Added CPU and memory limit options to the default container executor for Windows containers. Contributed by Chuan Liu 2015-03-06 14:18:11 -08:00
Allen Wittenauer
d4ac6822e1 YARN-2980. Move health check script related functionality to hadoop-common (Varun Saxena via aw) 2015-02-24 11:25:26 -08:00
Junping Du
c33ae271c2 YARN-2799. Cleanup TestLogAggregationService based on the change in YARN-90. Contributed by Zhihai Xu 2015-02-20 09:43:39 -08:00
cnauroth
6804d68901 YARN-2899. Run TestDockerContainerExecutorWithMocks on Linux only. Contributed by Ming Ma. 2015-02-13 21:58:54 -08:00
Junping Du
04f5ef18f7 YARN-2079. Recover NonAggregatingLogHandler state upon nodemanager restart. (Contributed by Jason Lowe) 2015-02-12 11:46:47 -08:00
Jason Lowe
b379972ab3 YARN-3074. Nodemanager dies when localizer runner tries to write to a full disk. Contributed by Varun Saxena 2015-02-11 16:33:43 +00:00
Jason Lowe
3f5431a22f YARN-2809. Implement workaround for linux kernel panic when removing cgroup. Contributed by Nathan Roberts 2015-02-10 17:27:21 +00:00
Arun C. Murthy
02f154a001 YARN-1537. Fix race condition in TestLocalResourcesTrackerImpl.testLocalResourceCache. Contributed by Xuan Gong. 2015-02-05 23:58:52 -08:00
Xuan
b73e776abc YARN-3056. Add verification for containerLaunchDuration in 
TestNodeManagerMetrics. Contributed by zhihai Xu
 2015-02-03 15:12:54 -08:00
Robert Kanter
f7a77819a1 YARN-3022. Expose Container resource information from NodeManager for monitoring (adhoot via ranter) 2015-02-03 10:39:41 -08:00
Jian He
4e15fc0841 YARN-3011. Possible IllegalArgumentException in ResourceLocalizationService might lead NM to crash. Contributed by Varun Saxena 2015-01-27 13:31:22 -08:00
Jason Lowe
902c6ea7e4 YARN-3088. LinuxContainerExecutor.deleteAsUser can throw NPE if native executor returns an error. Contributed by Eric Payne 2015-01-26 15:40:21 +00:00
Xuan
0d6bd62102 YARN-3024. LocalizerRunner should give DIE action when all resources are 
localized. Contributed by Chengbing Liu
 2015-01-25 19:37:57 -08:00
Tsuyoshi Ozawa
3aab354e66 YARN-3082. Non thread safe access to systemCredentials in NodeHeartbeatResponse processing. Contributed by Anubhav Dhoot. 2015-01-23 16:04:18 +09:00
Karthik Kambatla
84198564ba YARN-2984. Metrics for container's actual memory usage. (kasha) 2015-01-17 05:44:04 +05:30
Junping Du
5d1cca34fa YARN-3064. TestRMRestart/TestContainerResourceUsage/TestNodeManagerResync failure with allocation timeout. (Contributed by Jian He) 2015-01-16 00:08:36 -08:00
Jian He
cc2a745f7e YARN-2997. Fixed NodeStatusUpdater to not send alreay-sent completed container statuses on heartbeat. Contributed by Chengbing Liu 2015-01-08 11:12:54 -08:00
Karthik Kambatla
954fb8581e YARN-2675. containersKilled metrics is not updated when the container is killed during localization. (Zhihai Xu via kasha) 2014-12-19 16:02:20 -08:00
cnauroth
e996a1bfd4 HADOOP-11321. copyToLocal cannot save a file to an SMB share unless the user has Full Control permissions. Contributed by Chris Nauroth. 2014-12-16 15:29:22 -08:00
Steve Loughran
3681de2039 YARN-2912 Jersey Tests failing with port in use. (varun saxena via stevel) 2014-12-12 17:11:07 +00:00
Karthik Kambatla
db73cc9124 YARN-2931. PublicLocalizer may fail until directory is initialized by LocalizeRunner. (Anubhav Dhoot via kasha) 2014-12-08 22:26:18 -08:00
Junping Du
e65b7c5ff6 YARN-1156. Enhance NodeManager AllocatedGB and AvailableGB metrics for aggregation of decimal values. (Contributed by Tsuyoshi OZAWA) 2014-12-03 04:11:18 -08:00
Karthik Kambatla
233b61e495 YARN-2679. Add metric for container launch duration. (Zhihai Xu via kasha) 2014-11-21 14:22:21 -08:00
Jason Lowe
49c38898b0 YARN-2816. NM fail to start with NPE during container recovery. Contributed by Zhihai Xu 2014-11-14 21:25:59 +00:00
Karthik Kambatla
a04143039e YARN-2236. [YARN-1492] Shared Cache uploader service on the Node Manager. (Chris Trezzo and Sanjin Lee via kasha) 2014-11-12 09:31:05 -08:00
Ravi Prakash
53f64ee516 YARN-1964. Create Docker analog of the LinuxContainerExecutor in YARN 2014-11-11 21:28:11 -08:00
Xuan
5c9a51f140 YARN-2841. RMProxy should retry EOFException. Contributed by Jian He 2014-11-10 18:25:01 -08:00
Arun C. Murthy
43cd07b408 YARN-2830. Add backwords compatible ContainerId.newInstance constructor. Contributed by Jonathan Eagles. 2014-11-09 14:57:37 -08:00
Jason Lowe
c3d475070a YARN-2825. Container leak on NM. Contributed by Jian He 2014-11-07 23:16:37 +00:00
Vinod Kumar Vavilapalli
c5a46d4c8c YARN-1922. Fixed NodeManager to kill process-trees correctly in the presence of races between the launch and the stop-container call and when root processes crash. Contributed by Billie Rinaldi. 2014-11-03 16:38:55 -08:00
Vinod Kumar Vavilapalli
58e9f24e0f YARN-2788. Fixed backwards compatiblity issues with log-aggregation feature that were caused when adding log-upload-time via YARN-2703. Contributed by Xuan Gong. 2014-11-03 13:16:29 -08:00
Vinod Kumar Vavilapalli
5c0381c96a YARN-2790. Fixed a NodeManager bug that was causing log-aggregation to fail beyond HFDS delegation-token expiry even when RM is a proxy-user (YARN-2704). Contributed by Jian He. 2014-11-01 16:32:35 -07:00
Zhijie Shen
1cd088fd9d YARN-2711. Fixed TestDefaultContainerExecutor#testContainerLaunchError failure on Windows. Contributed by Varun Vasudev. 2014-10-31 17:43:30 -07:00
Jason Lowe
73e626ad91 YARN-2755. NM fails to clean up usercache_DEL_<timestamp> dirs after YARN-661. Contributed by Siqi Li 2014-10-30 15:10:27 +00:00
Zhijie Shen
8984e9b177 YARN-2741. Made NM web UI serve logs on the drive other than C: on Windows. Contributed by Craig Welch. 2014-10-28 14:11:19 -07:00
Vinod Kumar Vavilapalli
a16d022ca4 YARN-2704. Changed ResourceManager to optionally obtain tokens itself for the sake of localization and log-aggregation for long-running services. Contributed by Jian He. 2014-10-27 15:49:47 -07:00
Zhijie Shen
f81dc3f995 YARN-2703. Added logUploadedTime into LogValue for better display. Contributed by Xuan Gong. 2014-10-24 14:10:46 -07:00
Jian He
3b12fd6cfb YARN-2198. Remove the need to run NodeManager as privileged account for Windows Secure Container Executor. Contributed by Remus Rusanu 2014-10-22 15:57:46 -07:00
Jason Lowe
6f2028bd15 YARN-90. NodeManager should identify failed disks becoming good again. Contributed by Varun Vasudev 2014-10-21 17:31:13 +00:00
Jian He
4fa1fb3193 Missing file for YARN-2701 2014-10-20 19:57:45 -07:00
Jian He
2839365f23 YARN-2701. Potential race condition in startLocalizer when using LinuxContainerExecutor. Contributed by Xuan Gong 2014-10-20 18:45:47 -07:00