big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
14,163 Commits 2 Branches 2 Tags 264 MiB
9422515239
Commit Graph

14163 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Junping Du
477a30f536 YARN-3045. Implement NM writing container lifecycle events to Timeline Service v2. Contributed by Naganarasimha G R. 2016-07-10 08:45:42 -07:00
Sangjin Lee
102b56ee96 YARN-3904. Refactor timelineservice.storage to add support to online and offline aggregation writers (Li Lu via sjlee) 2016-07-10 08:45:42 -07:00
Zhijie Shen
561c746cbe MAPREDUCE-6370. Made the timeline service v2 test driver write event ID. Contributed by Li Lu. 
(cherry picked from commit 827633ee9fee26e3e15343cbccb0b7905ae02170)
 2016-07-10 08:45:41 -07:00
Zhijie Shen
5088f6c76a MAPREDUCE-6337. Added a mode to replay MR job history files and put them into the timeline service v2. Contributed by Sangjin Lee. 
(cherry picked from commit 463e070a8e7c882706a96eaa20ea49bfe9982875)
 2016-07-10 08:45:41 -07:00
Junping Du
00e85e7a2b YARN-3906. Split the application table from the entity table. Contributed by Sangjin Lee. 
(cherry picked from commit bcd755eba9466ce277d3c14192c31da6462c4ab3)
 2016-07-10 08:45:41 -07:00
Sangjin Lee
9e5155be36 YARN-3049. [Storage Implementation] Implement storage reader interface to fetch raw data from HBase backend (Zhijie Shen via sjlee) 
(cherry picked from commit 07433c2ad52df9e844dbd90020c277d3df844dcd)
 2016-07-10 08:45:41 -07:00
Zhijie Shen
9422d9b50d YARN-3984. Adjusted the event column key schema and avoided missing empty event. Contributed by Vrushali C. 
(cherry picked from commit 895ccfa1ab9e701f2908586e323249f670fe5544)
 2016-07-10 08:45:40 -07:00
Zhijie Shen
57e2498cd4 YARN-3993. Changed to use the AM flag in ContainerContext determine AM container in TestPerNodeTimelineCollectorsAuxService. Contributed by Sunil G. 
(cherry picked from commit 9e48f9ff2ce08f3dcdd8d60bacb697664b92196f)
 2016-07-10 08:45:40 -07:00
Zhijie Shen
a9fab9b644 YARN-3908. Fixed bugs in HBaseTimelineWriterImpl. Contributed by Vrushali C and Sangjin Lee. 
(cherry picked from commit df0ec473a84871b0effd7ca6faac776210d7df09)
 2016-07-10 08:45:40 -07:00
Junping Du
8603736ef2 YARN-3949. Ensure timely flush of timeline writes. Contributed by Sangjin Lee. 
(cherry picked from commit 967bef7e0396d857913caa2574afb103a5f0b81b)
 2016-07-10 08:45:39 -07:00
Sangjin Lee
57d8dc2fb7 YARN-3836. add equals and hashCode to TimelineEntity and other classes in the data model (Li Lu via sjlee) 
(cherry picked from commit 2d4a8f4563c06339717ca9410b2794754603fba3)
 2016-07-10 08:45:39 -07:00
Sangjin Lee
e27642abf4 YARN-3047. [Data Serving] Set up ATS reader with basic request serving structure and lifecycle (Varun Saxena via sjlee) 
(cherry picked from commit 4c5f88fb0f04b7919738d07598b0f006a9ff91f2)
 2016-07-10 08:45:39 -07:00
Zhijie Shen
2d59bc4458 YARN-3051. Created storage oriented reader interface for fetching raw entity data and made the filesystem based implementation. Contributed by Varun Saxena. 
(cherry picked from commit 499ce52c7b645ec0b1cc8ac62dc9a3127b987a20)
 2016-07-10 08:45:38 -07:00
Sangjin Lee
22e7ae5771 YARN-3792. Test case failures in TestDistributedShell and some issue fixes related to ATSV2 (Naganarasimha G R via sjlee) 
(cherry picked from commit 84f37f1c7eefec6d139cbf091c50d6c06f734323)
 2016-07-10 08:45:38 -07:00
Sangjin Lee
92d90c3a24 YARN-3706. Generalize native HBase writer for additional tables (Joep Rottinghuis via sjlee) 
(cherry picked from commit 9137aeae0dec83f9eff40d12cae712dfd508c0c5)
 2016-07-10 08:45:38 -07:00
Sangjin Lee
6b445a5ccb YARN-3801. [JDK-8] Exclude jdk.tools from hbase-client and hbase-testing-util (Tsuyoshi Ozawa via sjlee) 
(cherry picked from commit a1bb9137af84a34bde799f45e7ab8a21e33d55e0)
 2016-07-10 08:45:38 -07:00
Zhijie Shen
f3c661e8dd YARN-3044. Made RM write app, attempt and optional container lifecycle events to timeline service v2. Contributed by Naganarasimha G R. 2016-07-10 08:45:37 -07:00
Zhijie Shen
42d7864b84 YARN-3276. Code cleanup for timeline service API records. Contributed by Junping Du. 
(cherry picked from commit d88f30ba5359f59fb71b93a55e1c1d9a1c0dff8e)
 2016-07-10 08:45:37 -07:00
Sangjin Lee
f668212529 YARN-3721. build is broken on YARN-2928 branch due to possible dependency cycle (Li Lu via sjlee) 
(cherry picked from commit a9738ceb17b50cce8844fd42bb800c7f83f15caf)
 2016-07-10 08:45:37 -07:00
Sangjin Lee
db891f51d0 YARN-3726. Fix TestHBaseTimelineWriterImpl unit test failure by fixing its test data (Vrushali C via sjlee) 
(cherry picked from commit 4c0b6d73914f2e249795deb292f508177ea54884)
 2016-07-10 08:45:36 -07:00
Sangjin Lee
5a4278ccbd YARN-3411. [Storage implementation] explore the native HBase write schema for storage (Vrushali C via sjlee) 
(cherry picked from commit 7a3068854d27eadae1c57545988f5b2029bf119a)
 2016-07-10 08:45:36 -07:00
Junping Du
d275677e24 YARN-3634. TestMRTimelineEventHandling and TestApplication are broken. Contributed by Sangjin Lee. 
(cherry picked from commit b059dd4882fd759e4762cc11c019be4b68fb74c1)
 2016-07-10 08:45:36 -07:00
Zhijie Shen
51d092faef YARN-3529. Added mini HBase cluster and Phoenix support to timeline service v2 unit tests. Contributed by Li Lu. 2016-07-10 08:45:36 -07:00
Zhijie Shen
41fb5c7381 YARN-3134. Implemented Phoenix timeline writer to access HBase backend. Contributed by Li Lu. 
(cherry picked from commit b3b791be466be79e4e964ad068f7a6ec701e22e1)
 2016-07-10 08:45:35 -07:00
Sangjin Lee
dc1f306fdc YARN-3562. unit tests failures and issues found from findbug from earlier ATS checkins (Naganarasimha G R via sjlee) 2016-07-10 08:45:35 -07:00
Sangjin Lee
fc8485d893 YARN-3551. Consolidate data model change according to the backend implementation (Zhijie Shen via sale) 
(cherry picked from commit 557a3950bddc837469244835f5577899080115d8)
 2016-07-10 08:45:35 -07:00
Zhijie Shen
8c7b6dd2c7 MAPREDUCE-6335. Created MR job based performance test driver for the timeline service v2. Contributed by Sangjin Lee. 
(cherry picked from commit b689f5d43d3f5434a30fe52f1a7e12e1fc5c71f4)
 2016-07-10 08:45:34 -07:00
Zhijie Shen
b50a6d78f5 MAPREDUCE-6327. Made MR AM use timeline service v2 API to write history events and counters. Contributed by Junping Du. 2016-07-10 08:45:34 -07:00
Junping Du
2bdefbc4a0 YARN-3431. Sub resources of timeline entity needs to be passed to a separate endpoint. Contributed By Zhijie Shen. 
(cherry picked from commit fa5cc75245a6dba549620a8b26c7b4a8aed9838e)
 2016-07-10 08:45:34 -07:00
Sangjin Lee
11e8905d8d YARN-3390. Reuse TimelineCollectorManager for RM (Zhijie Shen via sjlee) 
(cherry picked from commit 58221188811e0f61d842dac89e1f4ad4fd8aa182)
 2016-07-10 08:45:33 -07:00
Junping Du
47f35a30bb YARN-3391. Clearly define flow ID/ flow run / flow version in API and storage. Contributed by Zhijie Shen 
(cherry picked from commit 68c6232f8423e55b4d152ef3d1d66aeb2d6a555e)
 2016-07-10 08:45:33 -07:00
Zhijie Shen
5712b8f9fd YARN-3334. NM uses timeline client to publish container metrics to new timeline service. Contributed by Junping Du. 2016-07-10 08:45:33 -07:00
Junping Du
42e49399ce YARN-3374. Collector's web server should randomly bind an available port. Contributed by Zhijie Shen 
(cherry picked from commit 3aa898e734a1e4368ddf1d0bbd31f9b4de53ceba)
 2016-07-10 08:45:32 -07:00
Junping Du
d67c9bdb4d YARN-3040. Make putEntities operation be aware of the app's context. Contributed by Zhijie Shen 2016-07-10 08:45:32 -07:00
Junping Du
5e3d9a477b YARN-3034. Implement RM starting its timeline collector. Contributed by Naganarasimha G R 2016-07-10 08:45:32 -07:00
Zhijie Shen
19c1132146 YARN-3377. Fixed test failure in TestTimelineServiceClientIntegration. Contributed by Sangjin Lee. 
(cherry picked from commit 04de2ceccd02edb00bd671a63e04855c132e9735)
 2016-07-10 08:45:31 -07:00
Junping Du
2188a07e5b YARN-3333. Rename TimelineAggregator etc. to TimelineCollector. Contributed by Sangjin Lee 2016-07-10 08:45:31 -07:00
Zhijie Shen
9b56364080 YARN-3039. Implemented the app-level timeline aggregator discovery service. Contributed by Junping Du. 2016-07-10 08:45:31 -07:00
Zhijie Shen
f0e752c14b YARN-3264. Created backing storage write interface and a POC only FS based storage implementation. Contributed by Vrushali C. 
(cherry picked from commit 821b68d05d246fd57d7b7286eb2ccc075ed1eae8)
 2016-07-10 08:45:30 -07:00
Zhijie Shen
bf54d32750 YARN-3210. Refactored timeline aggregator according to new code organization proposed in YARN-3166. Contributed by Li Lu. 
(cherry picked from commit d3ff7f06cbc66d3a23c2551e7d4c752689f46afe)
 2016-07-10 08:45:30 -07:00
Zhijie Shen
d45ff878c4 YARN-3125. Made the distributed shell use timeline service next gen and add an integration test for it. Contributed by Junping Du and Li Lu. 
(cherry picked from commit bf08f7f0ed4900ce52f98137297dd1a47ba2a536)
 2016-07-10 08:45:30 -07:00
Zhijie Shen
9d57c9c015 YARN-3087. Made the REST server of per-node aggregator work alone in NM daemon. Conntributed by Li Lu. 
(cherry picked from commit 41a08ad404d4278fe598d6c222b2ae0e84bae0df)
 2016-07-10 08:45:29 -07:00
Zhijie Shen
309592b9e5 YARN-3240. Added ".keep" file which was missed before. 
(cherry picked from commit 6b4dc96414b3f15537670c67a0dccd048862f4f4)
 2016-07-10 08:45:29 -07:00
Junping Du
4f0c7eaff3 YARN-3240. Implement client API to put generic entities. Contributed by Zhijie Shen 
(cherry picked from commit 4487da249f448d5c67b712cd0aa723e764eed77d)
 2016-07-10 08:45:29 -07:00
Zhijie Shen
c77f86bf63 YARN-3041. Added the overall data model of timeline service next gen. Contributed by Zhijie Shen. 
(cherry picked from commit fdff5d2625c129c20c44f90a0517b28cee1522bd)
 2016-07-10 08:45:29 -07:00
Zhijie Shen
7c8abec0a8 YARN-3030. Set up TS aggregator with basic request serving structure and lifecycle. Contributed by Sangjin Lee. 
(cherry picked from commit f26941b39028ac30c77547e2be2d657bb5bf044a)
 2016-07-10 08:45:28 -07:00
Zhijie Shen
0d2da08f35 YARN-3063. Bootstrapping TimelineServer next generation module. Contributed by Zhijie Shen. 
(cherry picked from commit fe7d0e7c8fc88e0721b18e1531b0899d9dbed296)
 2016-07-10 08:45:28 -07:00
Junping Du
9bdb5bebea Fix OOM issue in DistributedShell. Contributed by Jian He. 2016-07-09 05:09:53 -07:00
Yongjun Zhang
da6f1b88dd HDFS-10512. VolumeScanner may terminate due to NPE in DataNode.reportBadBlocks. Contributed by Wei-Chiu Chuang and Yiqun Lin. 2016-07-08 19:40:44 -07:00
Akira Ajisaka
932aed64d7 HDFS-10336. TestBalancer failing intermittently because of not reseting UserGroupInformation completely. Contributed by Yiqun Lin. 2016-07-08 14:39:11 -07:00