big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
13,912 Commits 2 Branches 2 Tags 264 MiB
102b56ee96
Commit Graph

1992 Commits

Author SHA1 Message Date
Sangjin Lee
102b56ee96 YARN-3904. Refactor timelineservice.storage to add support to online and offline aggregation writers (Li Lu via sjlee) 2016-07-10 08:45:42 -07:00
Zhijie Shen
5088f6c76a MAPREDUCE-6337. Added a mode to replay MR job history files and put them into the timeline service v2. Contributed by Sangjin Lee. 
(cherry picked from commit 463e070a8e7c882706a96eaa20ea49bfe9982875)
 2016-07-10 08:45:41 -07:00
Junping Du
00e85e7a2b YARN-3906. Split the application table from the entity table. Contributed by Sangjin Lee. 
(cherry picked from commit bcd755eba9466ce277d3c14192c31da6462c4ab3)
 2016-07-10 08:45:41 -07:00
Sangjin Lee
9e5155be36 YARN-3049. [Storage Implementation] Implement storage reader interface to fetch raw data from HBase backend (Zhijie Shen via sjlee) 
(cherry picked from commit 07433c2ad52df9e844dbd90020c277d3df844dcd)
 2016-07-10 08:45:41 -07:00
Zhijie Shen
9422d9b50d YARN-3984. Adjusted the event column key schema and avoided missing empty event. Contributed by Vrushali C. 
(cherry picked from commit 895ccfa1ab9e701f2908586e323249f670fe5544)
 2016-07-10 08:45:40 -07:00
Zhijie Shen
57e2498cd4 YARN-3993. Changed to use the AM flag in ContainerContext determine AM container in TestPerNodeTimelineCollectorsAuxService. Contributed by Sunil G. 
(cherry picked from commit 9e48f9ff2ce08f3dcdd8d60bacb697664b92196f)
 2016-07-10 08:45:40 -07:00
Zhijie Shen
a9fab9b644 YARN-3908. Fixed bugs in HBaseTimelineWriterImpl. Contributed by Vrushali C and Sangjin Lee. 
(cherry picked from commit df0ec473a84871b0effd7ca6faac776210d7df09)
 2016-07-10 08:45:40 -07:00
Junping Du
8603736ef2 YARN-3949. Ensure timely flush of timeline writes. Contributed by Sangjin Lee. 
(cherry picked from commit 967bef7e0396d857913caa2574afb103a5f0b81b)
 2016-07-10 08:45:39 -07:00
Sangjin Lee
e27642abf4 YARN-3047. [Data Serving] Set up ATS reader with basic request serving structure and lifecycle (Varun Saxena via sjlee) 
(cherry picked from commit 4c5f88fb0f04b7919738d07598b0f006a9ff91f2)
 2016-07-10 08:45:39 -07:00
Zhijie Shen
2d59bc4458 YARN-3051. Created storage oriented reader interface for fetching raw entity data and made the filesystem based implementation. Contributed by Varun Saxena. 
(cherry picked from commit 499ce52c7b645ec0b1cc8ac62dc9a3127b987a20)
 2016-07-10 08:45:38 -07:00
Sangjin Lee
22e7ae5771 YARN-3792. Test case failures in TestDistributedShell and some issue fixes related to ATSV2 (Naganarasimha G R via sjlee) 
(cherry picked from commit 84f37f1c7eefec6d139cbf091c50d6c06f734323)
 2016-07-10 08:45:38 -07:00
Sangjin Lee
92d90c3a24 YARN-3706. Generalize native HBase writer for additional tables (Joep Rottinghuis via sjlee) 
(cherry picked from commit 9137aeae0dec83f9eff40d12cae712dfd508c0c5)
 2016-07-10 08:45:38 -07:00
Zhijie Shen
f3c661e8dd YARN-3044. Made RM write app, attempt and optional container lifecycle events to timeline service v2. Contributed by Naganarasimha G R. 2016-07-10 08:45:37 -07:00
Sangjin Lee
f668212529 YARN-3721. build is broken on YARN-2928 branch due to possible dependency cycle (Li Lu via sjlee) 
(cherry picked from commit a9738ceb17b50cce8844fd42bb800c7f83f15caf)
 2016-07-10 08:45:37 -07:00
Sangjin Lee
db891f51d0 YARN-3726. Fix TestHBaseTimelineWriterImpl unit test failure by fixing its test data (Vrushali C via sjlee) 
(cherry picked from commit 4c0b6d73914f2e249795deb292f508177ea54884)
 2016-07-10 08:45:36 -07:00
Sangjin Lee
5a4278ccbd YARN-3411. [Storage implementation] explore the native HBase write schema for storage (Vrushali C via sjlee) 
(cherry picked from commit 7a3068854d27eadae1c57545988f5b2029bf119a)
 2016-07-10 08:45:36 -07:00
Junping Du
d275677e24 YARN-3634. TestMRTimelineEventHandling and TestApplication are broken. Contributed by Sangjin Lee. 
(cherry picked from commit b059dd4882fd759e4762cc11c019be4b68fb74c1)
 2016-07-10 08:45:36 -07:00
Zhijie Shen
51d092faef YARN-3529. Added mini HBase cluster and Phoenix support to timeline service v2 unit tests. Contributed by Li Lu. 2016-07-10 08:45:36 -07:00
Zhijie Shen
41fb5c7381 YARN-3134. Implemented Phoenix timeline writer to access HBase backend. Contributed by Li Lu. 
(cherry picked from commit b3b791be466be79e4e964ad068f7a6ec701e22e1)
 2016-07-10 08:45:35 -07:00
Sangjin Lee
dc1f306fdc YARN-3562. unit tests failures and issues found from findbug from earlier ATS checkins (Naganarasimha G R via sjlee) 2016-07-10 08:45:35 -07:00
Sangjin Lee
fc8485d893 YARN-3551. Consolidate data model change according to the backend implementation (Zhijie Shen via sale) 
(cherry picked from commit 557a3950bddc837469244835f5577899080115d8)
 2016-07-10 08:45:35 -07:00
Junping Du
2bdefbc4a0 YARN-3431. Sub resources of timeline entity needs to be passed to a separate endpoint. Contributed By Zhijie Shen. 
(cherry picked from commit fa5cc75245a6dba549620a8b26c7b4a8aed9838e)
 2016-07-10 08:45:34 -07:00
Sangjin Lee
11e8905d8d YARN-3390. Reuse TimelineCollectorManager for RM (Zhijie Shen via sjlee) 
(cherry picked from commit 58221188811e0f61d842dac89e1f4ad4fd8aa182)
 2016-07-10 08:45:33 -07:00
Junping Du
47f35a30bb YARN-3391. Clearly define flow ID/ flow run / flow version in API and storage. Contributed by Zhijie Shen 
(cherry picked from commit 68c6232f8423e55b4d152ef3d1d66aeb2d6a555e)
 2016-07-10 08:45:33 -07:00
Zhijie Shen
5712b8f9fd YARN-3334. NM uses timeline client to publish container metrics to new timeline service. Contributed by Junping Du. 2016-07-10 08:45:33 -07:00
Junping Du
42e49399ce YARN-3374. Collector's web server should randomly bind an available port. Contributed by Zhijie Shen 
(cherry picked from commit 3aa898e734a1e4368ddf1d0bbd31f9b4de53ceba)
 2016-07-10 08:45:32 -07:00
Junping Du
d67c9bdb4d YARN-3040. Make putEntities operation be aware of the app's context. Contributed by Zhijie Shen 2016-07-10 08:45:32 -07:00
Junping Du
5e3d9a477b YARN-3034. Implement RM starting its timeline collector. Contributed by Naganarasimha G R 2016-07-10 08:45:32 -07:00
Zhijie Shen
19c1132146 YARN-3377. Fixed test failure in TestTimelineServiceClientIntegration. Contributed by Sangjin Lee. 
(cherry picked from commit 04de2ceccd02edb00bd671a63e04855c132e9735)
 2016-07-10 08:45:31 -07:00
Junping Du
2188a07e5b YARN-3333. Rename TimelineAggregator etc. to TimelineCollector. Contributed by Sangjin Lee 2016-07-10 08:45:31 -07:00
Zhijie Shen
9b56364080 YARN-3039. Implemented the app-level timeline aggregator discovery service. Contributed by Junping Du. 2016-07-10 08:45:31 -07:00
Zhijie Shen
f0e752c14b YARN-3264. Created backing storage write interface and a POC only FS based storage implementation. Contributed by Vrushali C. 
(cherry picked from commit 821b68d05d246fd57d7b7286eb2ccc075ed1eae8)
 2016-07-10 08:45:30 -07:00
Zhijie Shen
bf54d32750 YARN-3210. Refactored timeline aggregator according to new code organization proposed in YARN-3166. Contributed by Li Lu. 
(cherry picked from commit d3ff7f06cbc66d3a23c2551e7d4c752689f46afe)
 2016-07-10 08:45:30 -07:00
Zhijie Shen
d45ff878c4 YARN-3125. Made the distributed shell use timeline service next gen and add an integration test for it. Contributed by Junping Du and Li Lu. 
(cherry picked from commit bf08f7f0ed4900ce52f98137297dd1a47ba2a536)
 2016-07-10 08:45:30 -07:00
Zhijie Shen
9d57c9c015 YARN-3087. Made the REST server of per-node aggregator work alone in NM daemon. Conntributed by Li Lu. 
(cherry picked from commit 41a08ad404d4278fe598d6c222b2ae0e84bae0df)
 2016-07-10 08:45:29 -07:00
Junping Du
4f0c7eaff3 YARN-3240. Implement client API to put generic entities. Contributed by Zhijie Shen 
(cherry picked from commit 4487da249f448d5c67b712cd0aa723e764eed77d)
 2016-07-10 08:45:29 -07:00
Zhijie Shen
7c8abec0a8 YARN-3030. Set up TS aggregator with basic request serving structure and lifecycle. Contributed by Sangjin Lee. 
(cherry picked from commit f26941b39028ac30c77547e2be2d657bb5bf044a)
 2016-07-10 08:45:28 -07:00
Zhijie Shen
0d2da08f35 YARN-3063. Bootstrapping TimelineServer next generation module. Contributed by Zhijie Shen. 
(cherry picked from commit fe7d0e7c8fc88e0721b18e1531b0899d9dbed296)
 2016-07-10 08:45:28 -07:00
Varun Saxena
c04c5ec501 YARN-5318. Fix intermittent test failure of TestRMAdminService#testRefreshNodesResourceWithFileSystemBasedConfigurationProvider. (Jun Gong via Varun Saxena). 2016-07-09 01:13:18 +05:30
Varun Saxena
5252562edf YARN-5297. Avoid printing a stack trace when recovering an app after the RM restarts. (Junping Du via Varun Saxena). 2016-07-09 00:09:25 +05:30
Junping Du
30ee57ceb1 YARN-4939. The decommissioning Node should keep alive during NM restart. Contributed by sandflee. 2016-07-08 04:14:53 -07:00
Vinod Kumar Vavilapalli
673e5e02fe YARN-5314. Fixed a ConcurrentModificationException in ATS v1.5 EntityGroupFSTimelineStore. Contributed by Li Lu. 2016-07-07 22:37:29 -07:00
Jian He
8a9d293dd6 YARN-5233. Support for specifying a path for ATS plugin jars. Contributed by Li Lu 2016-07-06 16:44:46 -07:00
Vinod Kumar Vavilapalli
4c9e1aeb94 YARN-5224. Added new web-services /containers/{containerid}/logs & /containers/{containerid}/logs/{filename} and using them in "yarn logs" CLI to get logs of finished containers of a running application. Contributed by Xuan Gong. 2016-07-06 14:10:54 -07:00
Wangda Tan
04f6ebb66a YARN-5294. Pass remote ip address down to YarnAuthorizationProvider. (Jian He via wangda) 2016-07-06 10:36:48 -07:00
Jian He
d792a90206 YARN-5296. NMs going OutOfMemory because ContainerMetrics leak in ContainerMonitorImpl. Contributed by Junping Du 2016-07-05 18:05:16 -07:00
Vinod Kumar Vavilapalli
ce9c006430 YARN-5214. Fixed locking in DirectoryCollection to avoid hanging NMs when various code-paths hit slow disks. Contributed by Junping Du. 2016-07-05 16:07:28 -07:00
Varun Saxena
8e672e3c71 YARN-5286. Add RPC port info in RM web service's response when getting app status. (Jun Gong via Varun Saxena). 2016-07-05 22:56:07 +05:30
Ray Chiang
0a5def155e YARN-4568. Fix message when NodeManager runs into errors initializing the recovery directory. (rchiang) 2016-07-01 15:53:46 -07:00
Jian He
c35a5a7a8d YARN-5023. TestAMRestart#testShouldNotCountFailureToMaxAttemptRetry fails. Contributed by sandflee 2016-07-01 14:29:03 -07:00