big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
13,943 Commits 2 Branches 2 Tags 264 MiB
34f02f07d5
Commit Graph

13943 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Li Lu
34f02f07d5 Rebase to latest trunk 2016-07-10 08:45:51 -07:00
Li Lu
88f0294114 YARN-4200. Refactor reader classes in storage to nest under hbase 
specific package name. Contributed by Li Lu.
 2016-07-10 08:45:51 -07:00
Sangjin Lee
cc16683cef YARN-3995. Some of the NM events are not getting published due race condition when AM container finishes in NM (Naganarasimha G R via sjlee) 2016-07-10 08:45:50 -07:00
Varun Saxena
829cceebc0 YARN-3586. RM to only get back addresses of Collectors that NM needs to know. 
(Junping Du via Varun Saxena).
 2016-07-10 08:45:50 -07:00
Varun Saxena
eb0ac8efb1 YARN-4350. TestDistributedShell fails for V2 scenarios. (Naganarasimha G R via Varun Saxena) 2016-07-10 08:45:50 -07:00
Li Lu
8ef546c1ee YARN-4445. Unify the term flowId and flowName in timeline v2 codebase. 
Contributed by Zhan Zhang.
 2016-07-10 08:45:49 -07:00
Varun Saxena
c4d7bbda5c YARN-4460. [Bug fix] RM fails to start when SMP is enabled. (Li Lu via Varun Saxena) 2016-07-10 08:45:49 -07:00
Xuan
2e2dbf59d1 YARN-4392. ApplicationCreatedEvent event time resets after RM 
restart/failover. Contributed by Naganarasimha G R and Xuan Gong

(cherry picked from commit 4546c7582b)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java
 2016-07-10 08:45:49 -07:00
Li Lu
b7d484aead YARN-4450. TestTimelineAuthenticationFilter and 
TestYarnConfigurationFields fail. Contributed by Sangjin Lee.
 2016-07-10 08:45:49 -07:00
Li Lu
89e5c44f9e YARN-4356. Ensure the timeline service v.2 is disabled cleanly and has no 
impact when it's turned off. Contributed by Sangjin Lee.
 2016-07-10 08:45:48 -07:00
Junping Du
cf4666bb8c YARN-3623. Add a config to indicate the Timeline Service version. Contributed by Xuan Gong. 
(cherry picked from commit f910e4f639)
 2016-07-10 08:45:48 -07:00
Sangjin Lee
1f710484e5 YARN-3862. Support for fetching specific configs and metrics based on prefixes (Varun Saxena via sjlee) 2016-07-10 08:45:48 -07:00
Sangjin Lee
ae72f1dc77 YARN-4297. TestJobHistoryEventHandler and TestRMContainerAllocator failing on YARN-2928 branch (Varun Saxena via sjlee) 2016-07-10 08:45:47 -07:00
Sangjin Lee
51254a6b51 YARN-4053. Change the way metric values are stored in HBase Storage (Varun Saxena via sjlee) 2016-07-10 08:45:47 -07:00
Sangjin Lee
09649005ca YARN-4221. Store user in app to flow table (Varun Saxena via sjlee) 2016-07-10 08:45:47 -07:00
Sangjin Lee
10ec5586fb YARN-4129. Refactor the SystemMetricPublisher in RM to better support newer events (Naganarasimha G R via sjlee) 2016-07-10 08:45:46 -07:00
Sangjin Lee
e3e857866d YARN-4179. [reader implementation] support flow activity queries based on time (Varun Saxena via sjlee) 2016-07-10 08:45:46 -07:00
Li Lu
d014f2ffd2 YARN-4237 Support additional queries for ATSv2 Web UI. Contributed by 
Varun Saxena.
 2016-07-10 08:45:46 -07:00
Sangjin Lee
b51d0fef56 YARN-4178. [storage implementation] app id as string in row keys can cause incorrect ordering (Varun Saxena via sjlee) 2016-07-10 08:45:45 -07:00
Sangjin Lee
0f44b5508d YARN-3864. Implement support for querying single app and all apps for a flow run (Varun Saxena via sjlee) 2016-07-10 08:45:45 -07:00
Vrushali Channapattan
708fa8b1ae YARN-4210. HBase reader throws NPE if Get returns no rows (Varun Saxena via vrushali) 2016-07-10 08:45:45 -07:00
Vrushali Channapattan
da2b7bd08e YARN-4203. Add request/response logging & timing for each REST endpoint call (Varun Saxena via vrushali) 2016-07-10 08:45:45 -07:00
Vrushali Channapattan
d95dc89a02 YARN-4075 [reader REST API] implement support for querying for flows and flow runs (Varun Saxena via vrushali) 2016-07-10 08:45:44 -07:00
Vrushali
10fa6da7d8 YARN-4074. [timeline reader] implement support for querying for flows and flow runs (sjlee via vrushali) 2016-07-10 08:45:44 -07:00
Sangjin Lee
a68e383921 YARN-3901. Populate flow run data in the flow_run & flow activity tables (Vrushali C via sjlee) 2016-07-10 08:45:44 -07:00
Sangjin Lee
ee081018e4 YARN-4102. Add a "skip existing table" mode for timeline schema creator (Li Lu via sjlee) 2016-07-10 08:45:43 -07:00
Sangjin Lee
8d9476ec5f YARN-4058. Miscellaneous issues in NodeManager project (Naganarasimha G R via sjlee) 2016-07-10 08:45:43 -07:00
Sangjin Lee
451c13b1de YARN-3814. REST API implementation for getting raw entities in TimelineReader (Varun Saxena via sjlee) 2016-07-10 08:45:43 -07:00
Sangjin Lee
c7224a285c YARN-4064. build is broken at TestHBaseTimelineWriterImpl.java (sjlee) 2016-07-10 08:45:43 -07:00
Junping Du
7a41b5501e YARN-4025. Deal with byte representations of Longs in writer code. Contributed by Sangjin Lee and Vrushali C. 2016-07-10 08:45:42 -07:00
Junping Du
477a30f536 YARN-3045. Implement NM writing container lifecycle events to Timeline Service v2. Contributed by Naganarasimha G R. 2016-07-10 08:45:42 -07:00
Sangjin Lee
102b56ee96 YARN-3904. Refactor timelineservice.storage to add support to online and offline aggregation writers (Li Lu via sjlee) 2016-07-10 08:45:42 -07:00
Zhijie Shen
561c746cbe MAPREDUCE-6370. Made the timeline service v2 test driver write event ID. Contributed by Li Lu. 
(cherry picked from commit 827633ee9fee26e3e15343cbccb0b7905ae02170)
 2016-07-10 08:45:41 -07:00
Zhijie Shen
5088f6c76a MAPREDUCE-6337. Added a mode to replay MR job history files and put them into the timeline service v2. Contributed by Sangjin Lee. 
(cherry picked from commit 463e070a8e7c882706a96eaa20ea49bfe9982875)
 2016-07-10 08:45:41 -07:00
Junping Du
00e85e7a2b YARN-3906. Split the application table from the entity table. Contributed by Sangjin Lee. 
(cherry picked from commit bcd755eba9466ce277d3c14192c31da6462c4ab3)
 2016-07-10 08:45:41 -07:00
Sangjin Lee
9e5155be36 YARN-3049. [Storage Implementation] Implement storage reader interface to fetch raw data from HBase backend (Zhijie Shen via sjlee) 
(cherry picked from commit 07433c2ad52df9e844dbd90020c277d3df844dcd)
 2016-07-10 08:45:41 -07:00
Zhijie Shen
9422d9b50d YARN-3984. Adjusted the event column key schema and avoided missing empty event. Contributed by Vrushali C. 
(cherry picked from commit 895ccfa1ab9e701f2908586e323249f670fe5544)
 2016-07-10 08:45:40 -07:00
Zhijie Shen
57e2498cd4 YARN-3993. Changed to use the AM flag in ContainerContext determine AM container in TestPerNodeTimelineCollectorsAuxService. Contributed by Sunil G. 
(cherry picked from commit 9e48f9ff2ce08f3dcdd8d60bacb697664b92196f)
 2016-07-10 08:45:40 -07:00
Zhijie Shen
a9fab9b644 YARN-3908. Fixed bugs in HBaseTimelineWriterImpl. Contributed by Vrushali C and Sangjin Lee. 
(cherry picked from commit df0ec473a84871b0effd7ca6faac776210d7df09)
 2016-07-10 08:45:40 -07:00
Junping Du
8603736ef2 YARN-3949. Ensure timely flush of timeline writes. Contributed by Sangjin Lee. 
(cherry picked from commit 967bef7e0396d857913caa2574afb103a5f0b81b)
 2016-07-10 08:45:39 -07:00
Sangjin Lee
57d8dc2fb7 YARN-3836. add equals and hashCode to TimelineEntity and other classes in the data model (Li Lu via sjlee) 
(cherry picked from commit 2d4a8f4563c06339717ca9410b2794754603fba3)
 2016-07-10 08:45:39 -07:00
Sangjin Lee
e27642abf4 YARN-3047. [Data Serving] Set up ATS reader with basic request serving structure and lifecycle (Varun Saxena via sjlee) 
(cherry picked from commit 4c5f88fb0f04b7919738d07598b0f006a9ff91f2)
 2016-07-10 08:45:39 -07:00
Zhijie Shen
2d59bc4458 YARN-3051. Created storage oriented reader interface for fetching raw entity data and made the filesystem based implementation. Contributed by Varun Saxena. 
(cherry picked from commit 499ce52c7b645ec0b1cc8ac62dc9a3127b987a20)
 2016-07-10 08:45:38 -07:00
Sangjin Lee
22e7ae5771 YARN-3792. Test case failures in TestDistributedShell and some issue fixes related to ATSV2 (Naganarasimha G R via sjlee) 
(cherry picked from commit 84f37f1c7eefec6d139cbf091c50d6c06f734323)
 2016-07-10 08:45:38 -07:00
Sangjin Lee
92d90c3a24 YARN-3706. Generalize native HBase writer for additional tables (Joep Rottinghuis via sjlee) 
(cherry picked from commit 9137aeae0dec83f9eff40d12cae712dfd508c0c5)
 2016-07-10 08:45:38 -07:00
Sangjin Lee
6b445a5ccb YARN-3801. [JDK-8] Exclude jdk.tools from hbase-client and hbase-testing-util (Tsuyoshi Ozawa via sjlee) 
(cherry picked from commit a1bb9137af84a34bde799f45e7ab8a21e33d55e0)
 2016-07-10 08:45:38 -07:00
Zhijie Shen
f3c661e8dd YARN-3044. Made RM write app, attempt and optional container lifecycle events to timeline service v2. Contributed by Naganarasimha G R. 2016-07-10 08:45:37 -07:00
Zhijie Shen
42d7864b84 YARN-3276. Code cleanup for timeline service API records. Contributed by Junping Du. 
(cherry picked from commit d88f30ba5359f59fb71b93a55e1c1d9a1c0dff8e)
 2016-07-10 08:45:37 -07:00
Sangjin Lee
f668212529 YARN-3721. build is broken on YARN-2928 branch due to possible dependency cycle (Li Lu via sjlee) 
(cherry picked from commit a9738ceb17b50cce8844fd42bb800c7f83f15caf)
 2016-07-10 08:45:37 -07:00
Sangjin Lee
db891f51d0 YARN-3726. Fix TestHBaseTimelineWriterImpl unit test failure by fixing its test data (Vrushali C via sjlee) 
(cherry picked from commit 4c0b6d73914f2e249795deb292f508177ea54884)
 2016-07-10 08:45:36 -07:00