big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
13,888 Commits 2 Branches 2 Tags 264 MiB
fc8485d893
Commit Graph

256 Commits

Author SHA1 Message Date
Xuan
0306d902f5 YARN-3543. ApplicationReport should be able to tell whether the 
Application is AM managed or not. Contributed by Rohith Sharma K S
 2015-08-03 15:46:00 -07:00
Wangda Tan
8acb30b016 YARN-3963. AddNodeLabel on duplicate label addition shows success. (Bibin A Chundatt via wangda) 2015-07-30 09:42:55 -07:00
Akira Ajisaka
42d4e0ae99 YARN-3958. TestYarnConfigurationFields should be moved to hadoop-yarn-api module. Contributed by Varun Saxena. 2015-07-27 11:43:25 +09:00
Jian He
393fe71771 YARN-3878. AsyncDispatcher can hang while stopping if it is configured for draining events on stop. Contributed by Varun Saxena 2015-07-21 15:05:41 -07:00
Jian He
2466460d4c Revert "YARN-3878. AsyncDispatcher can hang while stopping if it is configured for draining events on stop. (Varun Saxena via kasha)" 
This reverts commit aa067c6aa4.
 2015-07-13 14:30:35 -07:00
Akira Ajisaka
d6675606dc YARN-3069. Document missing properties in yarn-default.xml. Contributed by Ray Chiang. 2015-07-13 12:57:01 +09:00
Zhijie Shen
1ea36299a4 YARN-3116. RM notifies NM whether a container is an AM container or normal task container. Contributed by Giovanni Matteo Fumarola. 2015-07-10 18:58:10 -07:00
Karthik Kambatla
ac6048372a HADOOP-12180. Move ResourceCalculatorPlugin from YARN to Common. (Chris Douglas via kasha) 2015-07-09 09:56:40 -07:00
Karthik Kambatla
aa067c6aa4 YARN-3878. AsyncDispatcher can hang while stopping if it is configured for draining events on stop. (Varun Saxena via kasha) 2015-07-09 09:48:29 -07:00
Gera Shegalov
6f2a41e37d YARN-3768. ArrayIndexOutOfBoundsException with empty environment variables. (Zhihai Xu via gera) 2015-06-30 14:59:33 -07:00
Jian He
4672315e2d YARN-3770. SerializedException should also handle java.lang.Error on de-serialization. Contributed by Lavkesh Lahngir 2015-06-29 14:31:49 -07:00
Devaraj K
b381f88c71 YARN-3745. SerializedException should also try to instantiate internal 
exception with the default constructor. Contributed by Lavkesh Lahngir.
 2015-06-25 17:38:11 +05:30
Wangda Tan
405bbcf68c Add missing test file of YARN-3733 2015-06-04 13:18:44 -07:00
Xuan
5766a04428 YARN-3749. We should make a copy of configuration when init 
MiniYARNCluster with multiple RMs. Contributed by Chun Chen
 2015-06-03 17:20:15 -07:00
Zhijie Shen
bc85959edd Revert "YARN-1462. Made RM write application tags to timeline server and exposed them to users via generic history web UI and REST API. Contributed by Xuan Gong." 
This reverts commit 4a9ec1a824.
 2015-06-03 14:15:56 -07:00
Zhijie Shen
4a9ec1a824 YARN-1462. Made RM write application tags to timeline server and exposed them to users via generic history web UI and REST API. Contributed by Xuan Gong. 2015-05-30 21:05:36 -07:00
Devaraj K
7e509f5843 YARN-3722. Merge multiple TestWebAppUtils into 
o.a.h.yarn.webapp.util.TestWebAppUtils. Contributed by Masatake Iwasaki.
 2015-05-28 15:35:06 +05:30
Vinod Kumar Vavilapalli
500a1d9c76 YARN-160. Enhanced NodeManager to automatically obtain cpu/memory values from underlying OS when configured to do so. Contributed by Varun Vasudev. 2015-05-26 11:38:35 -07:00
Jian He
8966d42179 YARN-3609. Load node labels from storage inside RM serviceStart. Contributed by Wangda Tan 2015-05-20 16:30:07 -07:00
Wangda Tan
b37da52a1c YARN-3565. NodeHeartbeatRequest/RegisterNodeManagerRequest should use NodeLabel object instead of String. (Naganarasimha G R via wangda) 2015-05-19 16:34:17 -07:00
Wangda Tan
d4f53fc963 YARN-3579. CommonNodeLabelsManager should support NodeLabel instead of string label name when getting node-to-label/label-to-label mappings. (Sunil G via wangda) 2015-05-13 13:29:09 -07:00
Karthik Kambatla
70fb37cd79 YARN-1287. Consolidate MockClocks. (Sebastian Wong and Anubhav Dhoot via kasha) 2015-05-09 14:34:54 -07:00
Akira Ajisaka
918af8efff YARN-3577. Misspelling of threshold in log4j.properties for tests. Contributed by Brahma Reddy Battula. 2015-05-07 13:33:03 +09:00
Junping Du
57d9a972cb YARN-2454. Fix compareTo of variable UNBOUNDED in o.a.h.y.util.resource.Resources. Contributed by Xu Yang. 2015-05-02 10:18:44 -07:00
Wangda Tan
db1b674b50 YARN-2740. Fix NodeLabelsManager to properly handle node label modifications when distributed node label configuration enabled. (Naganarasimha G R via wangda) 2015-04-27 16:24:38 -07:00
Vinod Kumar Vavilapalli
f5fe35e297 YARN-3413. Changed Nodelabel attributes (like exclusivity) to be settable only via addToClusterNodeLabels but not changeable at runtime. (Wangda Tan via vinodkv) 2015-04-23 11:19:55 -07:00
Junping Du
fad9d7e85b New parameter of CLI for decommissioning node gracefully in RMAdmin CLI. Contributed by Devaraj K 2015-04-22 10:07:20 -07:00
Jian He
577d755e4b YARN-3348. Add a 'yarn top' tool to help understand cluster usage. Contributed by Varun Vasudev 2015-04-10 09:57:17 -07:00
Wangda Tan
7af086a515 YARN-3459. Fix failiure of TestLog4jWarningErrorMetricsAppender. (Varun Vasudev via wangda) 2015-04-08 10:57:48 -07:00
Xuan
d27e9241e8 YARN-3294. Allow dumping of Capacity Scheduler debug logs via web UI for 
a fixed time period. Contributed by Varun Vasudev
 2015-04-07 09:52:36 -07:00
Wangda Tan
bad070fe15 YARN-2901. Add errors and warning metrics page to RM, NM web UI. (Varun Vasudev via wangda) 2015-04-02 17:23:20 -07:00
Vinod Kumar Vavilapalli
7610925e90 YARN-3304. Addendum patch. Cleaning up ResourceCalculatorProcessTree APIs for public use and removing inconsistencies in the default values. (Junping Du and Karthik Kambatla via vinodkv) 2015-03-31 17:27:46 -07:00
Vinod Kumar Vavilapalli
c358368f51 YARN-3304. Cleaning up ResourceCalculatorProcessTree APIs for public use and removing inconsistencies in the default values. Contributed by Junping Du and Karthik Kambatla. 2015-03-30 10:09:40 -07:00
Jian He
e1feb4ea1a YARN-3345. Add non-exclusive node label API. Contributed by Wangda Tan 2015-03-20 19:04:38 -07:00
Xuan
1a4b528691 YARN-2777. Mark the end of individual log in aggregated log. Contributed 
by Varun Saxena
 2015-03-20 10:24:13 -07:00
Karthik Kambatla
20b49224eb YARN-3351. AppMaster tracking URL is broken in HA. (Anubhav Dhoot via kasha) 2015-03-18 16:30:33 -07:00
Karthik Kambatla
53947f37c7 YARN-3122. Metrics for container's actual CPU usage. (Anubhav Dhoot via kasha) 2015-03-04 17:33:30 -08:00
Wangda Tan
d49ae725d5 YARN-3076. Add API/Implementation to YarnClient to retrieve label-to-node mapping (Varun Saxena via wangda) 2015-02-19 11:00:57 -08:00
Tsuyoshi Ozawa
7bc492adab YARN-3157. Refactor the exception handling in ConverterUtils#to*Id. Contributed by Bibin A Chundatt. 2015-02-12 23:48:43 +09:00
Tsuyoshi Ozawa
4cbaa74f62 Revert "YARN-3157. Refactor the exception handling in ConverterUtils#to*Id. Contributed by Bibin A Chundatt." 
This reverts commit 95a41bf35d.
 2015-02-12 18:48:17 +09:00
Tsuyoshi Ozawa
95a41bf35d YARN-3157. Refactor the exception handling in ConverterUtils#to*Id. Contributed by Bibin A Chundatt. 2015-02-12 18:41:17 +09:00
Jonathan Eagles
af08425893 YARN-2971. RM uses conf instead of token service address to renew timeline delegation tokens (jeagles) 2015-02-09 17:56:05 -06:00
Jian He
c1957fef29 YARN-2694. Ensure only single node label specified in ResourceRequest. Contributed by Wangda Tan 2015-02-06 11:34:20 -08:00
Wangda Tan
5bd984691b YARN-3075. NodeLabelsManager implementation to retrieve label to node mapping (Varun Saxena via wangda) 2015-02-03 12:52:42 -08:00
Tsuyoshi Ozawa
7acce7d364 YARN-3029. FSDownload.unpack() uses local locale for FS case conversion, may not work everywhere. Contributed by Varun Saxena. 2015-01-30 04:42:05 +09:00
Wangda Tan
fd93e5387b YARN-3028. Better syntax for replaceLabelsOnNode in RMAdmin CLI. Contributed by Rohith Sharmaks 2015-01-27 15:23:45 -08:00
Tsuyoshi Ozawa
24aa462673 YARN-2800. Remove MemoryNodeLabelsStore and add a way to enable/disable node labels feature. Contributed by Wangda Tan. 2015-01-23 20:37:05 +09:00
Robert Kanter
f451631915 YARN-2957. Create unit test to automatically compare YarnConfiguration and yarn-default.xml. (rchiang via rkanter) 2015-01-12 12:00:30 -08:00
Jian He
a260406268 YARN-3014. Replaces labels on a host should update all NM's labels on that host. Contributed by Wangda Tan 2015-01-09 17:49:53 -08:00
Jian He
2638f4d0f0 YARN-2936. Changed YARNDelegationTokenIdentifier to set proto fields on getProto method. Contributed by Varun Saxena 2015-01-07 16:14:22 -08:00
Zhijie Shen
947578c1c1 YARN-2991. Fixed DrainDispatcher to reuse the draining code path in AsyncDispatcher. Contributed by Rohith Sharmaks. 2015-01-02 11:08:47 -08:00
Junping Du
a696fbb001 YARN-2939. Fix new findbugs warnings in hadoop-yarn-common. (Li Lu via junping_du) 2014-12-22 03:06:37 -08:00
Steve Loughran
3681de2039 YARN-2912 Jersey Tests failing with port in use. (varun saxena via stevel) 2014-12-12 17:11:07 +00:00
Jason Lowe
03ab24aa01 MAPREDUCE-5932. Provide an option to use a dedicated reduce-side shuffle log. Contributed by Gera Shegalov 2014-12-03 17:02:14 +00:00
Zhijie Shen
8f1454cc6d YARN-2165. Added the sanity check for the numeric configuration values of the timeline service. Contributed by Vasanth kumar RJ. 2014-11-26 12:09:34 -08:00
Jonathan Eagles
c298a9a845 YARN-2375. Allow enabling/disabling timeline server per framework. (Mit Desai via jeagles) 2014-11-20 23:34:35 -06:00
Jason Lowe
f2fe8a800e YARN-2857. ConcurrentModificationException in ContainerLogAppender. Contributed by Mohammad Kamrul Islam 2014-11-14 20:47:26 +00:00
Vinod Kumar Vavilapalli
0fd97f9c19 YARN-2843. Fixed NodeLabelsManager to trim inputs for hosts and labels so as to make them work correctly. Contributed by Wangda Tan. 2014-11-11 12:33:10 -08:00
Arun C. Murthy
43cd07b408 YARN-2830. Add backwords compatible ContainerId.newInstance constructor. Contributed by Jonathan Eagles. 2014-11-09 14:57:37 -08:00
Zhijie Shen
9a4e0d343e YARN-2505. Supported get/add/remove/change labels in RM REST API. Contributed by Craig Welch. 2014-11-07 20:35:46 -08:00
Vinod Kumar Vavilapalli
b76179895d YARN-2804. Fixed Timeline service to not fill the logs with JAXB bindings exceptions. Contributed by Zhijie Shen. 2014-11-04 18:00:06 -08:00
Vinod Kumar Vavilapalli
58e9f24e0f YARN-2788. Fixed backwards compatiblity issues with log-aggregation feature that were caused when adding log-upload-time via YARN-2703. Contributed by Xuan Gong. 2014-11-03 13:16:29 -08:00
Vinod Kumar Vavilapalli
71fbb474f5 YARN-2798. Fixed YarnClient to populate the renewer correctly for Timeline delegation tokens. Contributed by Zhijie Shen. 2014-11-03 12:49:42 -08:00
Jason Lowe
7bc179f9f5 YARN-2707. Potential null dereference in FSDownload. Contributed by Gera Shegalov 2014-10-31 22:53:04 +00:00
Jian He
1b4be91866 YARN-2770. Added functionality to renew/cancel TimeLineDelegationToken. Contributed by Zhijie Shen 2014-10-31 13:16:45 -07:00
Zhijie Shen
ec63a3ffbd YARN-2747. Fixed the test failure of TestAggregatedLogFormat when native I/O is enabled. Contributed by Xuan Gong. 2014-10-28 22:43:40 -07:00
Zhijie Shen
caecd9fffe YARN-2734. Skipped sub-folders in the local log dir when aggregating logs. Contributed by Xuan Gong. 2014-10-26 12:57:31 -07:00
Vinod Kumar Vavilapalli
0186645505 YARN-2743. Fixed a bug in ResourceManager that was causing RMDelegationToken identifiers to be tampered and thus causing app submission failures in secure mode. Contributed by Jian He. 2014-10-26 11:14:34 -07:00
Zhijie Shen
f81dc3f995 YARN-2703. Added logUploadedTime into LogValue for better display. Contributed by Xuan Gong. 2014-10-24 14:10:46 -07:00
Zhijie Shen
e31f0a6558 YARN-2724. Skipped uploading a local log file to HDFS if exception is raised when opening it. Contributed by Xuan Gong. 2014-10-24 11:13:44 -07:00
Zhijie Shen
0f3b6900be YARN-2209. Replaced AM resync/shutdown command with corresponding exceptions and made related MR changes. Contributed by Jian He. 2014-10-23 21:56:03 -07:00
Zhijie Shen
b2942762d7 YARN-2709. Made timeline client getDelegationToken API retry if ConnectException happens. Contributed by Li Lu. 2014-10-21 16:06:39 -07:00
Zhijie Shen
e90718fa5a YARN-2582. Fixed Log CLI and Web UI for showing aggregated logs of LRS. Contributed Xuan Gong. 2014-10-20 13:38:59 -07:00
Zhijie Shen
89427419a3 YARN-2673. Made timeline client put APIs retry if ConnectException happens. Contributed by Li Lu. 2014-10-20 12:22:17 -07:00
Vinod Kumar Vavilapalli
8256766498 YARN-2504. Enhanced RM Admin CLI to support management of node-labels. Contribyted by Wangda Tan. 2014-10-18 12:07:40 -07:00
Vinod Kumar Vavilapalli
39063cd36f YARN-2676. Enhanced Timeline auth-filter to support proxy users. Contributed by Zhijie Shen. 2014-10-17 22:02:50 -07:00
Vinod Kumar Vavilapalli
e9c66e8fd2 YARN-2705. Fixed bugs in ResourceManager node-label manager that were causing test-failures: added a dummy in-memory labels-manager. Contributed by Wangda Tan. 2014-10-17 18:26:12 -07:00
Vinod Kumar Vavilapalli
db7f165319 YARN-2494. Added NodeLabels Manager internal API and implementation. Contributed by Wangda Tan. 2014-10-10 11:44:21 -07:00
Zhijie Shen
cb81bac002 YARN-2583. Modified AggregatedLogDeletionService to be able to delete rolling aggregated logs. Contributed by Xuan Gong. 2014-10-10 00:11:30 -07:00
Zhijie Shen
e532ed8faa YARN-2617. Fixed ApplicationSubmissionContext to still set resource for backward compatibility. Contributed by Wangda Tan. 2014-10-09 17:57:50 -07:00
Vinod Kumar Vavilapalli
596702a025 YARN-2544. Added admin-API objects for using node-labels. Contributed by Wangda Tan. 2014-10-09 15:46:24 -07:00
Jian He
ea26cc0b4a YARN-2615. Changed ClientToAMTokenIdentifier/RM(Timeline)DelegationTokenIdentifier to use protobuf as payload. Contributed by Junping Du 2014-10-06 10:47:43 -07:00
subru
5e10a13bb4 YARN-2576. Making test patch pass in branch. Contributed by Subru Krishnan and Carlo Curino. 
(cherry picked from commit 90ac0be86b898aefec5471db4027554c8e1b310c)
 2014-10-03 15:43:13 -07:00
Vinod Kumar Vavilapalli
34cdcaad71 YARN-2468. Enhanced NodeManager to support log handling APIs (YARN-2569) for use by long running services. Contributed by Xuan Gong. 2014-10-03 12:15:40 -07:00
Jian He
eb6ce5e97c YARN-2562. Changed ContainerId#toString() to be more readable. Contributed by Tsuyoshi OZAWA 2014-10-03 11:04:02 -07:00
Zhijie Shen
1c93025a1b YARN-2527. Fixed the potential NPE in ApplicationACLsManager and added test cases for it. Contributed by Benoy Antony. 2014-10-02 14:56:13 -07:00
Vinod Kumar Vavilapalli
9e40de6af7 YARN-2446. Augmented Timeline service APIs to start taking in domains as a parameter while posting entities and events. Contributed by Zhijie Shen. 2014-10-01 17:52:03 -07:00
Karthik Kambatla
f7743dd07d YARN-2610. Hamlet should close table tags. (Ray Chiang via kasha) 2014-09-30 11:46:07 -07:00
Jian He
5391919b09 YARN-668. Changed NMTokenIdentifier/AMRMTokenIdentifier/ContainerTokenIdentifier to use protobuf object as the payload. Contributed by Junping Du. 2014-09-26 17:48:41 -07:00
Vinod Kumar Vavilapalli
d78b452a4f YARN-2102. Added the concept of a Timeline Domain to handle read/write ACLs on Timeline service event data. Contributed by Zhijie Shen. 2014-09-24 11:27:53 -07:00
Zhijie Shen
5338ac416a YARN-2569. Added the log handling APIs for the long running services. Contributed by Xuan Gong. 2014-09-23 10:36:57 -07:00
Vinod Kumar Vavilapalli
a3d9934f91 YARN-1779. Fixed AMRMClient to handle AMRMTokens correctly across ResourceManager work-preserving-restart or failover. Contributed by Jian He. 2014-09-18 10:16:18 -07:00
Vinod Kumar Vavilapalli
14e2639fd0 YARN-611. Added an API to let apps specify an interval beyond which AM failures should be ignored towards counting max-attempts. Contributed by Xuan Gong. 2014-09-13 18:04:05 -07:00
Jian He
3122daa802 YARN-2229. Changed the integer field of ContainerId to be long type. Contributed by Tsuyoshi OZAWA 2014-09-12 10:33:33 -07:00
Jian He
0974f434c4 YARN-2515. Updated ConverterUtils#toContainerId to parse epoch. Contributed by Tsuyoshi OZAWA 2014-09-07 18:29:22 -07:00
cnauroth
cbea1b10ef YARN-2519. Credential Provider related unit tests failed on Windows. Contributed by Xiaoyu Yao. 2014-09-06 20:05:07 -07:00
Jian He
e2d0ff364a YARN-2182. Updated ContainerId#toString() to append RM Epoch number. Contributed by Tsuyoshi OZAWA 2014-08-27 10:02:45 -07:00
Jason Darrell Lowe
da4ba50269 HADOOP-10893. isolated classloader on the client side. Contributed by Sangjin Lee 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1619604 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-21 21:38:16 +00:00
Jian He
0c9b8f2d7f YARN-2373. Changed WebAppUtils to use Configuration#getPassword for accessing SSL passwords. Contributed by Larry McCay 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617555 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-12 18:29:42 +00:00
Junping Du
b47f65214c YARN-2298. Move TimelineClient to yarn-common project (Contributed by Zhijie Shen) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616100 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-06 05:48:32 +00:00