big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
16,066 Commits 2 Branches 2 Tags 264 MiB
999c8fcbef
Commit Graph

3382 Commits

Author SHA1 Message Date
Daniel Templeton
59ee8b7a88 YARN-4329. [YARN-5437] Allow fetching exact reason as to why a submitted app 
is in ACCEPTED state in Fair Scheduler (Contributed by Yufei Gu)
 2016-11-09 13:11:37 -08:00
Arun Suresh
280357c29f YARN-5833. Addendum patch to include missing changes to yarn-default.xml 2016-11-09 07:15:11 -08:00
Robert Kanter
09f43fa9c0 YARN-5736. YARN container executor config does not handle white space (miklos.szegedi@cloudera.com via rkanter) 2016-11-09 13:34:40 +01:00
Arun Suresh
283fa33feb YARN-5823. Update NMTokens in case of requests with only opportunistic containers. (Konstantinos Karanasos via asuresh) 2016-11-09 00:11:25 -08:00
Andrew Wang
e1c6ef2efa HADOOP-13789. Hadoop Common includes generated test protos in both jar and test-jar. Contributed by Sean Busbey. 2016-11-08 16:33:55 -08:00
Subru Krishnan
29e3b3417c YARN-5833. Add validation to ensure default ports are unique in Configuration. (Konstantinos Karanasos via Subru). 2016-11-08 14:38:18 -08:00
Jason Lowe
3f93ac0733 YARN-5356. NodeManager should communicate physical resource capability to ResourceManager. Contributed by Inigo Goiri 2016-11-08 22:01:26 +00:00
Arun Suresh
f38a6d03a1 YARN-5377. Fix TestQueuingContainerManager.testKillMultipleOpportunisticContainers. (Konstantinos Karanasos via asuresh) 2016-11-07 22:10:03 -08:00
Jian He
de3b4aac56 YARN-5716. Add global scheduler interface definition and update CapacityScheduler to use it. Contributed by Wangda Tan 2016-11-07 10:14:39 -08:00
sunilg
59bc84aadd YARN-4849. Addendum patch to improve pom for yarn-ui. (Wangda Tan via Sunil G) 2016-11-06 13:13:31 -08:00
Wangda Tan
c00b5d1e51 YARN-5804. New UI2 is not able to launch with jetty 9 upgrade post HADOOP-10075. (Sunil G via wangda) 
(cherry picked from commit ddebaa6b7ae1031899139d153d57df5387d60542)
 2016-11-06 13:13:31 -08:00
sunilg
013ff07bc6 YARN-5785. [YARN-3368] Accessing applications and containers list from Node page is throwing few exceptions in console. (Akhil P B and Sreenath Somarajapuram via Sunil G) 2016-11-06 13:13:31 -08:00
Sreenath Somarajapuram
fad392a22f YARN-5779. [YARN-3368] Addendum patch to document limits/notes of the new YARN UI . (Sunil G via Sreenath Somarajapuram) 2016-11-06 13:13:31 -08:00
sunilg
64c7cda7e5 YARN-5490. [YARN-3368] Fix various alignment issues and broken breadcrumb link in Node page. (Akhil P B Tan via Sunil G) 2016-11-06 13:13:31 -08:00
sunilg
12ddbbc61d YARN-5497. [YARN-3368] Use different color for Undefined and Succeeded for Final State in applications page. (Akhil P B Tan via Sunil G) 2016-11-06 13:13:31 -08:00
sunilg
bc273c43ae YARN-5500. [YARN-3368] ‘Master node' link under application tab is broken. (Akhil P B Tan via Sunil G) 2016-11-06 13:13:31 -08:00
sunilg
e93f900b55 YARN-5772. [YARN-3368] Replace old Hadoop logo with new one (Akhil P B Tan via Sunil G) 2016-11-06 13:13:31 -08:00
sunilg
825de90b96 YARN-5779. [YARN-3368] Document limits/notes of the new YARN UI (Wangda Tan via Sunil G) 2016-11-06 13:13:31 -08:00
sunilg
cb77e3eb40 YARN-5741. [YARN-3368] Update UI2 documentation for new UI2 path (Kai Sasaki and Wangda Tan via Sunil G) 2016-11-06 13:13:31 -08:00
Wangda Tan
d8be766759 YARN-5145. [YARN-3368] Move new YARN UI configuration to HADOOP_CONF_DIR. (Sunil G and Kai Sasaki via wangda) 2016-11-06 13:13:31 -08:00
sunilg
1be2fd662d YARN-4849. Addendum patch to remove unwanted files from rat exclusions. (Wangda Tan via Sunil G) 2016-11-06 13:13:31 -08:00
Wangda Tan
3de0da2a76 YARN-5698. [YARN-3368] Launch new YARN UI under hadoop web app port. (Sunil G via wangda) 2016-11-06 13:13:31 -08:00
sunilg
98b2ad7208 YARN-5682. [YARN-3368] Fix maven build to keep all generated or downloaded files in target folder (Wangda Tan via Sunil G) 2016-11-06 13:13:31 -08:00
Wangda Tan
817d5f4c96 YARN-4849. Addendum patch to fix javadocs. (Sunil G via wangda) 2016-11-06 13:13:31 -08:00
sunilg
a5be75974f YARN-4849. Addendum patch to fix ASF warnings. (Wangda Tan via Sunil G) 2016-11-06 13:13:31 -08:00
sunilg
f6574d9ff6 YARN-5503. [YARN-3368] Add missing hidden files in webapp folder for deployment (Sreenath Somarajapuram via Sunil G) 2016-11-06 13:13:31 -08:00
sunilg
7005580752 YARN-5504. [YARN-3368] Fix YARN UI build pom.xml (Sreenath Somarajapuram via Sunil G) 2016-11-06 13:13:31 -08:00
sunilg
ce2361a941 YARN-4849. Addendum patch to fix document. (Wangda Tan via Sunil G) 2016-11-06 13:13:31 -08:00
sunilg
23b0287f62 YARN-5488. [YARN-3368] Applications table overflows beyond the page boundary(Harish Jaiprakash via Sunil G) 2016-11-06 13:13:31 -08:00
Wangda Tan
b61e60fb92 YARN-5509. Build error due to preparing 3.0.0-alpha2 deployment. (Kai Sasaki via wangda) 2016-11-06 13:13:31 -08:00
sunilg
561839cb5f YARN-3334. [YARN-3368] Introduce REFRESH button in various UI pages (Sreenath Somarajapuram via Sunil G) 2016-11-06 13:13:31 -08:00
Sunil
8f584a561e YARN-5321. [YARN-3368] Add resource usage for application by node managers (Wangda Tan via Sunil G) 
YARN-5320. [YARN-3368] Add resource usage by applications and queues to cluster overview page  (Wangda Tan via Sunil G)
YARN-5322. [YARN-3368] Add a node heat chart map (Wangda Tan via Sunil G)
YARN-5347. [YARN-3368] Applications page improvements (Sreenath Somarajapuram via Sunil G)
YARN-5348. [YARN-3368] Node details page improvements (Sreenath Somarajapuram via Sunil G)
YARN-5346. [YARN-3368] Queues page improvements (Sreenath Somarajapuram via Sunil G)
YARN-5345. [YARN-3368] Cluster overview page improvements (Sreenath Somarajapuram via Sunil G)
YARN-5344. [YARN-3368] Generic UI improvements (Sreenath Somarajapuram via Sunil G)
 2016-11-06 13:13:31 -08:00
Sunil
35f08122e2 YARN-5161. [YARN-3368] Add Apache Hadoop logo in YarnUI home page. (Kai Sasaki via Sunil G) 2016-11-06 13:13:31 -08:00
Sunil
14aa58c594 YARN-5183. [YARN-3368] Support for responsive navbar when window is resized. (Kai Sasaki via Sunil G) 2016-11-06 13:13:31 -08:00
Wangda Tan
c85cc3b56e YARN-4515. [YARN-3368] Support hosting web UI framework inside YARN RM. (Sunil G via wangda) 
YARN-5000. [YARN-3368] App attempt page is not loading when timeline server is not started (Sunil G via wangda)
YARN-5038. [YARN-3368] Application and Container pages shows wrong values when RM is stopped. (Sunil G via wangda)
 2016-11-06 13:13:31 -08:00
Wangda Tan
2c4f164e16 YARN-5019. [YARN-3368] Change urls in new YARN ui from camel casing to hyphens. (Sunil G via wangda) 2016-11-06 13:13:31 -08:00
Wangda Tan
dea4a296e5 YARN-4514. [YARN-3368] Cleanup hardcoded configurations, such as RM/ATS addresses. (Sunil G via wangda) 2016-11-06 13:13:31 -08:00
Wangda Tan
266784b849 YARN-4849. [YARN-3368] cleanup code base, integrate web UI related build to mvn, and fix licenses. (wangda) 2016-11-06 13:13:31 -08:00
Wangda Tan
0a5f652071 YARN-4517. Add nodes page and fix bunch of license issues. (Varun Saxena via wangda) 2016-11-06 13:13:31 -08:00
Wangda Tan
53e661f68e YARN-4733. [YARN-3368] Initial commit of new YARN web UI. (wangda) 2016-11-06 13:13:31 -08:00
Jason Lowe
6bb741ff0e YARN-5837. NPE when getting node status of a decommissioned node after an RM restart. Contributed by Robert Kanter 2016-11-04 22:20:21 +00:00
Arun Suresh
0aafc122d4 YARN-2995. Enhance UI to show cluster resource utilization of various container Execution types. (Konstantinos Karanasos via asuresh) 2016-11-04 07:31:54 -07:00
Sunil
19b3779ae7 YARN-5802. updateApplicationPriority api in scheduler should ensure to re-insert app to correct ordering policy. Contributred by Bibin A Chundatt 2016-11-04 16:07:28 +05:30
Varun Vasudev
9ee0e3172e YARN-5822. Log ContainerRuntime initialization error in LinuxContainerExecutor. Contributed by Sidharta Seethana. 2016-11-03 22:33:43 +05:30
Jason Lowe
352cbaa7a5 YARN-4862. Handle duplicate completed containers in RMNodeImpl. Contributed by Rohith Sharma K S 2016-11-03 13:54:31 +00:00
Naganarasimha
0e75496049 YARN-5720. Update document for "rmadmin -replaceLabelOnNode". Contributred by Tao Jie 2016-11-03 12:32:22 +05:30
Varun Saxena
377919010b YARN-5815. Random failure of TestApplicationPriority.testOrderOfActivatingThePriorityApplicationOnRMRestart (Bibin A Chundatt via Varun Saxena) 2016-11-03 00:37:09 +05:30
Naganarasimha
aacf214a27 YARN-5697. Use CliParser to parse options in RMAdminCLI. Contributed by Tao jie 2016-11-02 04:44:33 +05:30
Jason Lowe
76893a4100 YARN-5001. Aggregated Logs root directory is created with wrong group if nonexistent. Contributed by Haibo Chen 2016-11-01 20:23:24 +00:00
Rohith Sharma K S
9d0c2ad5a5 YARN-3732. Change NodeHeartbeatResponse.java and RegisterNodeManagerResponse.java as abstract classes. Contributed by Devaraj K. 2016-11-01 16:25:45 +05:30
Varun Saxena
7d2d8d25ba YARN-5788. Apps not activiated and AM limit resource in UI and REST not updated after -replaceLabelsOnNode (Bibin A Chundatt via Varun Saxena) 2016-11-01 15:32:04 +05:30
Varun Saxena
310aa4686e YARN-5805. Add isDebugEnabled check for debug logs in nodemanager (Bibin A Chundatt via Varun Saxena) 2016-11-01 14:51:31 +05:30
Karthik Kambatla
733aa99313 YARN-4998. Minor cleanup to UGI use in AdminService. (Daniel Templeton via kasha) 2016-10-31 16:26:01 -07:00
Varun Saxena
07ab89e8bb YARN-5809. AsyncDispatcher possibly invokes multiple shutdown threads when handling exception (Jian He via Varun Saxena) 2016-11-01 04:37:03 +05:30
Wangda Tan
90dd3a8148 YARN-2009. CapacityScheduler: Add intra-queue preemption for app priority support. (Sunil G via wangda) 2016-10-31 15:18:31 -07:00
Daniel Templeton
773c60bd7b YARN-5800. Delete LinuxContainerExecutor comment from yarn-default.xml (Contributed by Jan Hentschel via Daniel Templeton) 2016-10-31 14:47:48 -07:00
Daniel Templeton
f3eb4c3c73 YARN-5793. Trim configuration values in DockerLinuxContainerRuntime (Contributed by Tianyin Xu via Daniel Templeton) 2016-10-31 13:51:07 -07:00
Daniel Templeton
2528bea67f YARN-4396. Log the trace information on FSAppAttempt#assignContainer (Contributed by Yiqun Li via Daniel Templeton) 2016-10-31 13:34:25 -07:00
Daniel Templeton
cc2c993a8a YARN-4907. Make all MockRM#waitForState consistent. (Contributed by Yufei Gu via Daniel Templeton) 2016-10-31 13:20:56 -07:00
Naganarasimha
e0bebbbcdd YARN-4498. Application level node labels stats to be available in REST. Contributed by Bibin A Chundatt 2016-10-31 04:38:20 +05:30
Arun Suresh
aa3cab1eb2 YARN-5799. Fix Opportunistic Allocation to set the correct value of Node Http Address. (asuresh) 2016-10-29 02:03:57 -07:00
Varun Saxena
1c8ab41e8b YARN-5773. RM recovery too slow due to LeafQueue#activateApplications (Bibin A Chundatt via Varun Saxena) 2016-10-29 13:47:39 +05:30
Chris Douglas
76a1042980 YARN-3460. TestSecureRMRegistryOperations fails with IBM_JAVA. Contributed by Pascal Oliva 2016-10-28 16:49:36 -07:00
Jason Lowe
1eae719bce YARN-4963. capacity scheduler: Make number of OFF_SWITCH assignments per heartbeat configurable. Contributed by Nathan Roberts 2016-10-28 17:30:15 +00:00
Jason Lowe
1b79c417dc YARN-5767. Fix the order that resources are cleaned up from the local Public/Private caches. Contributed by Chris Trezzo 2016-10-28 15:58:04 +00:00
Jason Lowe
7146359bfd YARN-5027. NM should clean up app log dirs after NM restart. Contributed by sandflee 2016-10-28 15:48:58 +00:00
Jason Lowe
c017171da0 HADOOP-13770. Shell.checkIsBashSupported swallowed an interrupted exception. Contributed by Wei-Chiu Chuang 2016-10-28 14:57:50 +00:00
Naganarasimha
3de130252d YARN-5794. Fix the asflicense warnings. Contributed by Yiqun Lin 2016-10-28 12:37:37 +05:30
Karthik Kambatla
4df8ed63ed YARN-4743. FairSharePolicy breaks TimSort assumption. (Zephyr Guo and Yufei Gu via kasha) 2016-10-27 17:45:48 -07:00
Subru Krishnan
b2c4f24c31 YARN-2306. Add test for leakage of reservation metrics in fair scheduler. (Hong Zhiguo and Yufei Gu via subru). 2016-10-27 17:43:13 -07:00
Robert Kanter
5877f20f9c HADOOP-10075. Update jetty dependency to version 9 (rkanter) 2016-10-27 16:09:00 -07:00
Vrushali Channapattan
022bf783aa YARN-4765 Split TestHBaseTimelineStorage into multiple test classes (Varun Saxena via Vrushali C) 
(cherry picked from commit 513dcf6817dd76fde8096ff04cd888d7c908461d)
 2016-10-27 15:37:36 -07:00
Li Lu
221582c4ab YARN-4668. Reuse objectMapper instance in Yarn. (Yiqun Lin via gtcarrera9) 2016-10-27 15:20:17 -07:00
Carlo Curino
892a8348fc YARN-3432. Cluster metrics have wrong Total Memory when there is reserved memory on CS. (Brahma Reddy Battula via curino) 2016-10-27 15:15:49 -07:00
Subru Krishnan
79ae78dcbe YARN-3568. TestAMRMTokens should use some random port. (Takashi Ohnishi via Subru). 2016-10-27 15:11:12 -07:00
Daniel Templeton
b98fc8249f YARN-4710. Reduce logging application reserved debug info in FSAppAttempt#assignContainer (Contributed by Yiqun Lin via Daniel Templeton) 2016-10-27 14:42:19 -07:00
Robert Kanter
9449519a25 YARN-5776. Checkstyle: MonitoringThread.Run method length is too long (miklos.szegedi@cloudera.com via rkanter) 2016-10-27 14:36:38 -07:00
Varun Saxena
dd4ed6a587 YARN-4388. Cleanup mapreduce.job.hdfs-servers from yarn-default.xml (Junping Du via Varun Saxena) 2016-10-28 02:22:25 +05:30
Jason Lowe
7e3c327d31 YARN-4831. Recovered containers will be killed after NM stateful restart. Contributed by Siqi Li 2016-10-27 20:42:52 +00:00
Jason Lowe
b4a8fbcbbc YARN-5172. Update yarn daemonlog documentation due to HADOOP-12847. Contributed by Wei-Chiu Chuang 2016-10-27 19:43:02 +00:00
Robert Kanter
6fbfb501f2 YARN-4456. Clean up Lint warnings in nodemanager (templedf via rkanter) 2016-10-27 12:37:01 -07:00
Varun Saxena
79aeddc88f YARN-5308. FairScheduler: Move continuous scheduling related tests to TestContinuousScheduling (Kai Sasaki via Varun Saxena) 2016-10-28 00:34:50 +05:30
Naganarasimha
b3c15e4ef7 YARN-5420. Delete org.apache.hadoop.yarn.server.resourcemanager.resource.Priority as its not necessary. Contributed by Sunil G. 2016-10-27 18:22:07 +05:30
Varun Saxena
8bd6d77a41 YARN-5757. RM REST API documentation is not up to date (Miklos Szegedi via Varun Saxena) 2016-10-27 17:21:01 +05:30
Naganarasimha
6c8830992c YARN-3848. TestNodeLabelContainerAllocation is timing out. Contributed by Varun Saxena 2016-10-27 17:10:02 +05:30
Varun Saxena
7b4e9ec3b0 YARN-5686. DefaultContainerExecutor random working dir algorigthm skews results (Vrushali C via Varun Saxena) 2016-10-27 14:02:29 +05:30
Varun Saxena
6cc7c43866 YARN-5710. Fix inconsistent naming in class ResourceRequest (Yufei Gu via Varun Saxena) 2016-10-27 13:55:17 +05:30
Varun Saxena
5c39a419a6 YARN-5752. TestLocalResourcesTrackerImpl#testLocalResourceCache times out (Eric Badger via Varun Saxena) 2016-10-27 13:41:49 +05:30
Rohith Sharma K S
e29cba61a0 YARN-4363. In TestFairScheduler, testcase should not create FairScheduler redundantly. Conntributed by Tao Jie. 2016-10-27 11:57:17 +05:30
Rohith Sharma K S
b110c4b5e8 YARN-4555. TestDefaultContainerExecutor#testContainerLaunchError fails on non-english locale environment. Contributed by Takashi Ohnishi. 2016-10-27 11:34:07 +05:30
Sangjin Lee
f511cc89b6 YARN-5433. Audit dependencies for Category-X. Contributed by Sangjin Lee. 2016-10-26 11:31:00 -07:00
Robert Kanter
44fdf00964 YARN-5753. fix NPE in AMRMClientImpl.getMatchingRequests() (haibochen via rkanter) 2016-10-25 23:59:39 -07:00
Akira Ajisaka
d3bb69a667 YARN-5575. Many classes use bare yarn. properties instead of the defined constants. Contributed by Daniel Templeton. 2016-10-26 15:32:07 +09:00
Karthik Kambatla
de6faae97c YARN-5388. Deprecate and remove DockerContainerExecutor. (Daniel Templeton via kasha) 2016-10-25 13:35:47 -07:00
Xiao Chen
c88c1dc50c YARN-5777. TestLogsCLI#testFetchApplictionLogsAsAnotherUser fails. Contributed by Akira Ajisaka. 2016-10-25 12:15:09 -07:00
Karthik Kambatla
a71fc81655 YARN-5754. Null check missing for earliest in FifoPolicy. (Yufei Gu via kasha) 2016-10-24 22:19:23 -07:00
Subru Krishnan
0a166b1347 YARN-5711. Propogate exceptions back to client when using hedging RM failover provider. 2016-10-24 18:59:51 -07:00
Robert Kanter
23d7d53a41 YARN-5679. TestAHSWebServices is failing (ajisakaa via rkanter) 2016-10-21 13:31:03 -07:00
Varun Saxena
44eb2bd7ae YARN-5747. Application timeline metric aggregation in timeline v2 will lose last round aggregation when an application finishes (Li Lu via Varun Saxena) 2016-10-22 01:14:49 +05:30
Karthik Kambatla
754cb4e30f YARN-5047. Refactor nodeUpdate across schedulers. (Ray Chiang via kasha) 2016-10-20 21:17:48 -07:00
Karthik Kambatla
a064865abf YARN-4911. Bad placement policy in FairScheduler causes the RM to crash 2016-10-20 20:57:04 -07:00
Sangjin Lee
e9c4616b5e YARN-5561. [Atsv2] : Support for ability to retrieve apps/app-attempt/containers and entities via REST. Contributed by Rohith Sharma K S. 2016-10-19 09:45:23 -07:00
Xuan
b733a6f862 YARN-5718. TimelineClient (and other places in YARN) shouldn't over-write HDFS client retry settings which could cause unexpected behavior. Contributed by Junping Du. 2016-10-18 11:06:47 -07:00
Varun Saxena
b154d3edce YARN-5743. [Atsv2] Publish queue name and RMAppMetrics to ATS (Rohith Sharma K S via Varun Saxena) 2016-10-18 23:32:52 +05:30
Robert Kanter
f5d9235914 YARN-5466. DefaultContainerExecutor needs JavaDocs (templedf via rkanter) 2016-10-17 14:29:09 -07:00
Sangjin Lee
1f304b0c7f YARN-5699. Retrospect yarn entity fields which are publishing in events info fields. Contributed by Rohith Sharma K S. 2016-10-15 13:54:40 -07:00
Chris Douglas
cf3f43e95b YARN-5717. Add tests for container-executor is_feature_enabled. Contributed by Sidharta Seethana 2016-10-13 20:49:07 -07:00
Karthik Kambatla
6476934ae5 YARN-5677. RM should transition to standby when connection is lost for an extended period. (Daniel Templeton via kasha) 2016-10-11 22:07:10 -07:00
Karthik Kambatla
6378845f9e YARN-4464. Lower the default max applications stored in the RM and store. (Daniel Templeton via kasha) 2016-10-11 21:42:08 -07:00
Jason Lowe
ecb51b857a YARN-5551. Ignore file backed pages from memory computation when smaps is enabled. Contributed by Rajesh Balamohan 2016-10-11 15:12:43 +00:00
Naganarasimha
0773ffd0f8 YARN-5057. Resourcemanager.security.TestDelegationTokenRenewer fails in trunk. Contributed by Jason Lowe. 2016-10-10 18:04:47 -04:00
Junping Du
459a4833a9 YARN-5659. getPathFromYarnURL should use standard methods. Contributed by Sergey Shelukhin. 2016-10-07 07:46:08 -07:00
Rohith Sharma K S
4d2f380d78 YARN-5101. YARN_APPLICATION_UPDATED event is parsed in ApplicationHistoryManagerOnTimelineStore#convertToApplicationReport with reversed order. Contributed by Sunil G. 2016-10-06 20:42:36 +05:30
Akira Ajisaka
272a21747e MAPREDUCE-6789. Fix TestAMWebApp failure. Contributed by Daniel Templeton. 2016-10-06 15:57:15 +09:00
Chris Douglas
d65b957776 YARN-5702. Refactor TestPBImplRecords for reuse in other YARN modules. Contributed by Subru Krishnan 2016-10-05 12:16:49 -07:00
Varun Vasudev
0992708d79 YARN-5704. Provide config knobs to control enabling/disabling new/work in progress features in container-executor. Contributed by Sidharta Seethana. 2016-10-05 15:31:38 +05:30
Jian He
31f8da22d0 YARN-3139. Improve locks in AbstractYarnScheduler/CapacityScheduler/FairScheduler. Contributed by Wangda Tan 2016-10-04 17:23:13 -07:00
Karthik Kambatla
736d33cddd YARN-4767. Network issues can cause persistent RM UI outage. (Daniel Templeton via kasha) 2016-10-03 14:35:57 -07:00
Karthik Kambatla
0da54e8848 YARN-5672. FairScheduler: Wrong queue name in log when adding application. (Wilfred Spiegelenburg via kasha) 2016-10-03 06:03:46 -07:00
Naganarasimha
6e130c308c YARN-4855. Should check if node exists when replace nodelabels. Contributeed by Tao Jie 2016-10-03 02:02:26 -04:00
Karthik Kambatla
82857037b6 YARN-5678. Log demand as demand in FSLeafQueue and FSParentQueue. (Yufei Gu via kasha) 2016-10-02 22:09:43 -07:00
Subru Krishnan
3a3697deab YARN-5384. Expose priority in ReservationSystem submission APIs. (Sean Po via Subru). 2016-09-30 19:41:43 -07:00
Robert Kanter
2549ee9d4c YARN-5693. Reduce loglevel to Debug in ContainerManagementProtocolProxy and AMRMClientImpl (yufeigu via rkanter) 2016-09-30 13:30:55 -07:00
Arun Suresh
10be45986c YARN-5486. Update OpportunisticContainerAllocatorAMService::allocate method to handle OPPORTUNISTIC container requests. (Konstantinos Karanasos via asuresh) 2016-09-29 15:11:41 -07:00
Jian He
2ae5a3a5bf YARN-4205. Add a service for monitoring application life time out. Contributed by Rohith Sharma K S 2016-09-29 22:00:31 +08:00
Robert Kanter
bcb2528a51 YARN-5400. Light cleanup in ZKRMStateStore (templedf via rkanter) 2016-09-28 14:56:41 -07:00
Varun Saxena
9b0fd01d2e YARN-5599. Publish AM launch command to ATS (Rohith Sharma K S via Varun Saxena) 2016-09-28 16:10:10 +05:30
Varun Vasudev
bc2656f09f YARN-5662. Provide an option to enable ContainerMonitor. Contributed by Jian He. 2016-09-28 15:18:18 +05:30
Wangda Tan
1831be8e73 YARN-3142. Improve locks in AppSchedulingInfo. (Varun Saxena via wangda) 2016-09-27 11:54:55 -07:00
Wangda Tan
875062b5bc HADOOP-13544. JDiff reports unncessarily show unannotated APIs and cause confusion while our javadocs only show annotated and public APIs. (vinodkv via wangda) 2016-09-27 11:26:45 -07:00
Naganarasimha
ebf528c443 YARN-5660. Wrong audit constants are used in Get/Put of priority in RMWebService. Contributed by Rohith Sharma K S 2016-09-27 16:37:59 +05:30
Arun Suresh
4815d024c5 YARN-5609. Expose upgrade and restart API in ContainerManagementProtocol. Contributed by Arun Suresh 2016-09-26 08:46:54 -07:00
Arun Suresh
2f163cd5cf Revert "YARN-5609. Expose upgrade and restart API in ContainerManagementProtocol. Contributed by Arun Suresh" 
This reverts commit fe644bafe7.
 2016-09-26 08:36:59 -07:00
Jian He
fe644bafe7 YARN-5609. Expose upgrade and restart API in ContainerManagementProtocol. Contributed by Arun Suresh 2016-09-26 22:41:16 +08:00
Akira Ajisaka
14a696f369 YARN-5663. Small refactor in ZKRMStateStore. Contributed by Oleksii Dymytrov. 2016-09-26 15:00:01 +09:00
Naganarasimha
e4e72db5f9 YARN-3877. YarnClientImpl.submitApplication swallows exceptions. Contributed by Varun Saxena 2016-09-25 17:36:30 +05:30
Naganarasimha
4d16d2d72f YARN-5664. Fix Yarn documentation to link to correct versions. Contributed by Xiao Chen 2016-09-24 23:21:03 +05:30
Kihwal Lee
6e849cb658 YARN-5622. TestYarnCLI.testGetContainers fails due to mismatched date formats. Contributed by Eric Badger. 2016-09-23 14:09:25 -05:00
Varun Saxena
b8a2d7b8fc YARN-5539. TimelineClient failed to retry on java.net.SocketTimeoutException: Read timed out (Junping Du via Varun Saxena) 2016-09-23 13:27:31 +05:30
Varun Saxena
3293a7d92d Revert "TimelineClient failed to retry on java.net.SocketTimeoutException: Read timed out" 
This reverts commit 2e6ee95716.
 2016-09-23 13:25:46 +05:30
Varun Saxena
2e6ee95716 TimelineClient failed to retry on java.net.SocketTimeoutException: Read timed out 2016-09-23 13:23:54 +05:30
Naganarasimha
d0372dc613 YARN-3692. Allow REST API to set a user generated message when killing an application. Contributed by Rohith Sharma K S 2016-09-23 06:30:49 +05:30
Robert Kanter
5ffd4b7c1e YARN-4973. YarnWebParams next.fresh.interval should be next.refresh.interval (templedf via rkanter) 2016-09-22 16:45:34 -07:00
Wangda Tan
5a58bfee30 YARN-4591. YARN Web UIs should provide a robots.txt. (Sidharta Seethana via wangda) 2016-09-20 17:20:50 -07:00
Arun Suresh
9f03b403ec YARN-5656. Fix ReservationACLsTestBase. (Sean Po via asuresh) 2016-09-20 12:27:17 -07:00
Wangda Tan
e45307c9a0 Addendum patch for fix javadocs failure which is caused by YARN-3141. (wangda) 2016-09-20 11:21:01 -07:00
Jason Lowe
c6d1d742e7 YARN-5655. TestContainerManagerSecurity#testNMTokens is asserting. Contributed by Robert Kanter 2016-09-20 14:15:06 +00:00
Jian He
2b66d9ec5b YARN-3140. Improve locks in AbstractCSQueue/LeafQueue/ParentQueue. Contributed by Wangda Tan 2016-09-20 15:03:31 +08:00
Jason Lowe
7558dbbb48 YARN-5540. Scheduler spends too much time looking at empty priorities. Contributed by Jason Lowe 2016-09-19 20:31:35 +00:00
Jian He
b8a30f2f17 YARN-3141. Improve locks in SchedulerApplicationAttempt/FSAppAttempt/FiCaSchedulerApp. Contributed by Wangda Tan 2016-09-19 17:08:01 +08:00
Varun Saxena
ea29e3bc27 YARN-5577. [Atsv2] Document object passing in infofilters with an example (Rohith Sharma K S via Varun Saxena) 2016-09-19 14:33:06 +05:30
Arun Suresh
3552c2b99d YARN-5637. Changes in NodeManager to support Container rollback and commit. (asuresh) 2016-09-18 10:55:18 -07:00
Arun Suresh
f67237cbe7 YARN-5657. Fix TestDefaultContainerExecutor. (asuresh) 2016-09-17 09:32:05 -07:00
Naganarasimha
4174b9756c YARN-5642. Typos in 9 log messages. Contributed by Mehran Hassani 2016-09-17 10:35:39 +05:30
Naganarasimha
ade7c2bc9c YARN-4232. TopCLI console support for HA mode. Contributed by Bibin A Chundatt 2016-09-17 09:52:39 +05:30
Kai Zheng
58bae35447 YARN-5163. Migrate TestClientToAMTokens and TestClientRMTokens tests from the old RPC engine. Contributed by Wei Zhou and Kai Zheng 2016-09-18 08:43:36 +08:00
Arun Suresh
40b5a59b72 YARN-5620. Core changes in NodeManager to support re-initialization of Containers with new launchContext. (asuresh) 2016-09-15 07:15:11 -07:00
Jason Lowe
e793309735 YARN-5630. NM fails to start after downgrade from 2.8 to 2.7. Contributed by Jason Lowe 2016-09-13 14:41:27 +00:00
Rohith Sharma K S
729de3e6b6 YARN-5631. Missing refreshClusterMaxPriority usage in rmadmin help message. Contributed by Kai Sasaki. 2016-09-13 15:23:24 +05:30
Allen Wittenauer
58ed4fa544 HADOOP-13341. Deprecate HADOOP_SERVERNAME_OPTS; replace with (command)_(subcommand)_OPTS 
This commit includes the following changes:

	HADOOP-13356. Add a function to handle command_subcommand_OPTS
	HADOOP-13355. Handle HADOOP_CLIENT_OPTS in a function
	HADOOP-13554. Add an equivalent of hadoop_subcmd_opts for secure opts
	HADOOP-13562. Change hadoop_subcommand_opts to use only uppercase
	HADOOP-13358. Modify HDFS to use hadoop_subcommand_opts
	HADOOP-13357. Modify common to use hadoop_subcommand_opts
	HADOOP-13359. Modify YARN to use hadoop_subcommand_opts
	HADOOP-13361. Modify hadoop_verify_user to be consistent with hadoop_subcommand_opts (ie more granularity)
	HADOOP-13564. modify mapred to use hadoop_subcommand_opts
	HADOOP-13563. hadoop_subcommand_opts should print name not actual content during debug
	HADOOP-13360. Documentation for HADOOP_subcommand_OPTS

This closes apache/hadoop#126
 2016-09-12 11:10:00 -07:00
Karthik Kambatla
c0e492e50f YARN-5616. Clean up WeightAdjuster. (Yufei Gu via kasha) 2016-09-06 22:40:20 -07:00
Varun Vasudev
e6fcfe28e3 YARN-5576. Allow resource localization while container is running. Contributed by Jian He. 2016-09-06 20:01:45 +05:30
Rohith Sharma K S
f9557127b3 YARN-5608. TestAMRMClient.setup() fails with ArrayOutOfBoundsException. Contributed by Daniel Templeton. 2016-09-06 14:09:45 +05:30
Karthik Kambatla
f6ea9be547 YARN-5264. Store all queue-specific information in FSQueue. (Yufei Gu via kasha) 2016-09-02 14:56:29 -07:00
Ray Chiang
378f624a39 YARN-5549. AMLauncher#createAMContainerLaunchContext() should not log the command to be launched indiscriminately. (Daniel Templeton via rchiang) 2016-09-02 11:14:35 -07:00
Varun Vasudev
05f5c0f631 YARN-5555. Scheduler UI: "% of Queue" is inaccurate if leaf queue is hierarchically nested. Contributed by Eric Payne. 2016-09-02 16:02:01 +05:30
Karthik Kambatla
74f4bae455 YARN-5566. Client-side NM graceful decom is not triggered when jobs finish. (Robert Kanter via kasha) 2016-09-01 14:44:01 -07:00
Varun Vasudev
08f55ccbb0 YARN-5596. Fix failing unit test in TestDockerContainerRuntime. Contributed by Sidharta Seethana. 2016-09-01 14:08:51 +05:30
Arun Suresh
d6d9cff21b YARN-5221. Expose UpdateResourceRequest API to allow AM to request for change in container properties. (asuresh) 2016-08-30 15:52:29 -07:00
Zhe Zhang
ed6ff5cd78 YARN-5550. TestYarnCLI#testGetContainers should format according to CONTAINER_PATTERN. Contributed by Jonathan Hung. 2016-08-29 14:46:00 -07:00
Jason Lowe
4cbe61407d YARN-5560. Clean up bad exception catching practices in TestYarnClient. Contributed by Sean Po 2016-08-29 16:14:55 +00:00
Wangda Tan
19c743c1bb YARN-4889. Changes in AMRMClient for identifying resource-requests explicitly. (Arun Suresh via wangda) 2016-08-26 17:14:12 -07:00
Subru Krishnan
b930dc3ec0 YARN-5327. API changes required to support recurring reservations in the YARN ReservationSystem. (Sangeetha Abdu Jyothi via Subru). 2016-08-26 16:58:47 -07:00
Karthik Kambatla
cde3a00526 YARN-5373. NPE listing wildcard directory in containerLaunch. (Daniel Templeton via kasha) 2016-08-26 11:04:33 -07:00
Junping Du
9ef632f3b0 YARN-5557. Add localize API to the ContainerManagementProtocol. Contributed by Jian He. 2016-08-26 09:04:44 -07:00
Naganarasimha
46e02ab719 YARN-3940. Application moveToQueue should check NodeLabel permission. Contributed by Bibin A Chundatt 2016-08-26 20:19:11 +05:30
Varun Vasudev
bfb4d95059 YARN-5430. Return container's ip and host from NM ContainerStatus call. Contributed by Jian He. 2016-08-26 16:30:18 +05:30
Naganarasimha
27c3b86252 YARN-5564. Fix typo in RM_SCHEDULER_RESERVATION_THRESHOLD_INCREMENT_MULTIPLE. Contributed by Ray Chiang 2016-08-26 08:47:21 +05:30
Jason Lowe
3d86110a5c YARN-5389. TestYarnClient#testReservationDelete fails. Contributed by Sean Po 2016-08-25 14:42:06 +00:00
Varun Vasudev
525d52bf7c YARN-5042. Mount /sys/fs/cgroup into Docker containers as read only mount. Contributed by luhuichun. 2016-08-25 14:18:26 +05:30
Varun Saxena
79603f5882 YARN-5537. Fix intermittent failure of TestAMRMClient#testAMRMClientWithContainerResourceChange (Bibin A Chundatt via Varun Saxena) 2016-08-25 14:10:34 +05:30
Naganarasimha
143c59e4c5 YARN-4491. yarn list command to support filtering by tags. Contributed by Varun Saxena 2016-08-24 01:53:02 +05:30
Rohith Sharma K S
0d5997d2b9 YARN-5544. TestNodeBlacklistingOnAMFailures fails on trunk. Contributed by Sunil G. 2016-08-23 14:37:39 +05:30
Wangda Tan
2da32a6ef9 YARN-4685. Disable AM blacklisting by default to mitigate situations that application get hanged. (Rohith Sharma K S via wangda) 2016-08-19 16:31:16 -07:00
Wangda Tan
444b2ea7af YARN-3388. Allocation in LeafQueue could get stuck because DRF calculator isn't well supported when computing user-limit. (Nathan Roberts via wangda) 2016-08-19 16:28:32 -07:00
Varun Saxena
091dd19e86 YARN-5533. JMX AM Used metrics for queue wrong when app submited to nodelabel partition (Bibin A Chundatt via Varun Saxena) 2016-08-19 17:30:17 +05:30
Varun Saxena
8aed374182 Revert "YARN-5533. JMX AM Used metrics for queue wrong when app submited to nodelabel partition (Bibin A Chundatt via Varun Saxena)" 
This reverts commit 59557e85a4.
 2016-08-19 16:14:16 +05:30
Varun Saxena
59557e85a4 YARN-5533. JMX AM Used metrics for queue wrong when app submited to nodelabel partition (Bibin A Chundatt via Varun Saxena) 2016-08-19 15:01:48 +05:30
Junping Du
0da69c324d YARN-4676. Automatic and Asynchronous Decommissioning Nodes Status Tracking. Contributed by Diniel Zhi. 
(cherry picked from commit d464483bf7f0b3e3be3ba32cd6c3eee546747ab5)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceTrackerService.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNode.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
 2016-08-18 07:27:23 -07:00
Varun Saxena
913a895136 YARN=5526. DrainDispacher#serviceStop blocked if setDrainEventsOnStop invoked (sandflee via Varun Saxena) 2016-08-18 11:38:13 +05:30
Karthik Kambatla
20f0eb871c YARN-4702. FairScheduler: Allow setting maxResources for ad hoc queues. (Daniel Templeton via kasha) 2016-08-17 17:40:20 -07:00
Varun Saxena
e3037c5641 YARN-5523. Yarn running container log fetching causes OutOfMemoryError (Xuan Gong via Varun Saxena) 2016-08-18 01:45:33 +05:30
Varun Vasudev
7f05ff7a4e YARN-5455. Update Javadocs for LinuxContainerExecutor. Contributed by Daniel Templeton. 2016-08-17 15:34:58 +05:30
Varun Saxena
b427ce12bc YARN-5475. Fix test failure of TestAggregatedLogFormat#testReadAcontainerLogs1 (Jun Gong via Varun Saxena) 2016-08-16 20:24:53 +05:30
Varun Saxena
ffe1fff526 YARN-5514. Clarify DecommissionType.FORCEFUL comment (Vrushali C via Varun Saxena) 2016-08-16 14:05:41 +05:30
Varun Saxena
24249115bf YARN-5521. Fix random failure of TestCapacityScheduler#testKillAllAppsInQueue (sandflee via Varun Saxena) 2016-08-16 00:03:29 +05:30
Varun Saxena
d677b68c25 YARN-5491. Fix random failure of TestCapacityScheduler#testCSQueueBlocked (Bibin A Chundatt via Varun Saxena) 2016-08-15 03:31:21 +05:30
Varun Saxena
23c6e3c4e4 YARN-5476. Non existent application reported as ACCEPTED by YarnClientImpl (Junping Du via Varun Saxena) 2016-08-12 20:37:58 +05:30
Naganarasimha
874577a67d YARN-4833. For Queue AccessControlException client retries multiple times on both RM. Contributed by Bibin A Chundatt 2016-08-12 01:09:41 +05:30
Rohith Sharma K S
5199db387d YARN-5492. TestSubmitApplicationWithRMHA is failing sporadically during precommit builds. Contributed by Vrushali C. 2016-08-11 11:50:46 +05:30
Rohith Sharma K S
666ad0b67b YARN-2398. TestResourceTrackerOnHA crashes. Contributed by Ajith S. 2016-08-11 11:27:00 +05:30
Ray Chiang
aea3e65749 YARN-5137. Make DiskChecker pluggable in NodeManager. (Yufei Gu via rchiang) 2016-08-10 19:23:29 -07:00
Naganarasimha
89c0bff719 YARN-5495. Remove import wildcard in CapacityScheduler. Contributed by Ray Chiang. 2016-08-11 00:35:19 +05:30
Jason Lowe
5c95bb315b YARN-5382. RM does not audit log kill request for active applications. Contributed by Vrushali C 2016-08-10 18:25:54 +00:00
Jason Lowe
e0b570dffb YARN-5483. Optimize RMAppAttempt#pullJustFinishedContainers. Contributed by sandflee 2016-08-10 18:04:18 +00:00
Karthik Kambatla
7992c0b42c YARN-5343. TestContinuousScheduling#testSortedNodes fails intermittently. (Yufei Gu via kasha) 2016-08-09 16:51:03 -07:00
Rohith Sharma K S
3cd386bd97 YARN-5474. Typo mistake in AMRMClient#getRegisteredTimeineClient API. Contributed by Naganarasimha G R. 2016-08-09 22:28:54 +05:30
Varun Vasudev
522ddbde79 YARN-5394. Remove bind-mount /etc/passwd for Docker containers. Contributed by Zhankun Tang. 2016-08-09 16:04:09 +05:30
Arun Suresh
82c9e06101 YARN-5457. Refactor DistributedScheduling framework to pull out common functionality. (asuresh) 2016-08-09 00:42:29 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
83a2ffec40 YARN-5470. Addedum to differentiate exactly matching of log-files with regex in yarn log CLI. Contributed by Xuan Gong. 2016-08-08 14:28:07 -07:00
Varun Saxena
37d939a453 YARN-4910. Fix incomplete log info in ResourceLocalizationService (Jun Gong via Varun Saxena) 2016-08-08 22:12:53 +05:30
Naganarasimha
131d58a24e YARN-5287. LinuxContainerExecutor fails to set proper permission. Contributed by Ying Zhang 2016-08-07 01:08:13 +05:30
Naganarasimha
d81b8163b4 YARN-4624. NPE in PartitionQueueCapacitiesInfo while accessing Schduler UI. Contributed by Brahma Reddy Battula 2016-08-06 01:13:36 +05:30
Junping Du
e605d47df0 YARN-5470. Differentiate exactly match with regex in yarn log CLI. Contributed by Xuan Gong. 2016-08-05 10:55:29 -07:00
Wangda Tan
3f100d76ff YARN-4888. Changes in scheduler to identify resource-requests explicitly by allocation-id. (Subru Krishnan via wangda) 2016-08-05 10:43:35 -07:00
Wangda Tan
e0d131f055 YARN-4091. Add REST API to retrieve scheduler activity. (Chen Ge and Sunil G via wangda) 2016-08-05 10:27:34 -07:00
Rohith Sharma K S
d9a354c2f3 YARN-5333. Some recovered apps are put into default queue when RM HA. Contributed by Jun Gong. 2016-08-05 21:35:49 +05:30
Varun Saxena
4a26221021 YARN-5429. Fix return related javadoc warnings in yarn-api (Vrushali C via Varun Saxena) 2016-08-05 16:27:53 +05:30
Varun Vasudev
08e33381f5 YARN-5459. Add support for docker rm. Contributed by Shane Kumpf. 2016-08-04 14:08:34 +05:30
Jason Lowe
db4a61dc61 YARN-5469. Increase timeout of TestAmFilter.testFilter. Contributed by Eric Badger 2016-08-03 19:51:44 +00:00
Jason Lowe
db646540f0 YARN-5462. TestNodeStatusUpdater.testNodeStatusUpdaterRetryAndNMShutdown fails intermittently. Contributed by Eric Badger 2016-08-03 19:17:25 +00:00
Jason Lowe
4d92aefd35 YARN-4280. CapacityScheduler reservations may not prevent indefinite postponement on a busy cluster. Contributed by Kuhu Shukla 2016-08-03 18:53:14 +00:00
Varun Vasudev
d848184e90 YARN-5460. Change container runtime type logging in DelegatingLinuxContainerRuntime to debug. Contributed by Shane Kumpf. 2016-08-03 12:02:39 +05:30
Chris Nauroth
b913677365 YARN-5456. container-executor support for FreeBSD, NetBSD, and others if conf path is absolute. Contributed by Allen Wittenauer. 2016-08-02 22:24:34 -07:00
Junping Du
3818393297 YARN-5226. Remove AHS enable check from LogsCLI#fetchAMContainerLogs. Contributed by Xuan Gong. 2016-08-02 10:41:46 -07:00
Varun Vasudev
7fc70c6422 YARN-5458. Rename DockerStopCommandTest to TestDockerStopCommand. Contributed by Shane Kumpf. 2016-08-02 16:56:45 +05:30
Varun Vasudev
2e7c2a13a8 YARN-5443. Add support for docker inspect command. Contributed by Shane Kumpf. 2016-08-02 16:55:10 +05:30
Varun Vasudev
95694b70cd YARN-5444. Fix failing unit tests in TestLinuxContainerExecutorWithMocks. Contributed by Yufei Gu. 2016-08-01 16:08:38 +05:30
Arun Suresh
e5766b1dbe YARN-5113. Refactoring and other clean-up for distributed scheduling. (Konstantinos Karanasos via asuresh) 2016-07-31 11:48:25 -07:00
Chris Nauroth
ef501b1a0b YARN-5121. fix some container-executor portability issues. Contributed by Allen Wittenauer. 2016-07-30 08:26:19 -07:00
Junping Du
95f2b98597 YARN-5434. Add -client|server argument for graceful decommmission. Contributed by Robert Kanter. 2016-07-29 10:26:11 -07:00
Li Lu
7086fc72ee YARN-5436. Race in AsyncDispatcher can cause random test failures in Tez (probably YARN also). (Zhiyuan Yang via gtcarrera9) 2016-07-28 16:50:57 -07:00
Subru Krishnan
4e756d7271 YARN-5203.Return ResourceRequest JAXB object in ResourceManager Cluster Applications REST API. Contributed by Ellen Hui. 2016-07-28 16:03:24 -07:00
Jason Lowe
a1890c32c5 YARN-5438. TimelineClientImpl leaking FileSystem Instances causing Long running services like HiverServer2 daemon going OOM. Contributed by Rohith Sharma K S 2016-07-28 21:47:13 +00:00
Li Lu
26de4f0de7 YARN-5440. Use AHSClient in YarnClient when TimelineServer is running. (Xuan Gong via gtcarrera9) 2016-07-28 10:23:18 -07:00
Junping Du
7f3c306e2e YARN-5432. Lock already held by another process while LevelDB cache store creation for dag. Contributed by Li Lu. 2016-07-28 06:35:24 -07:00
Varun Saxena
8d06bda337 YARN-5431. TimelineReader daemon start should allow to pass its own reader opts (Rohith Sharma K S via Varun Saxena) 2016-07-28 08:36:09 +05:30
Subru Krishnan
d2cbfd7de3 YARN-5441. Fixing minor Scheduler test case failures 2016-07-27 18:21:58 -07:00
Akira Ajisaka
54fe17a607 HADOOP-9427. Use JUnit assumptions to skip platform-specific tests. Contributed by Gergely Novák. 2016-07-27 19:41:09 +09:00
Wangda Tan
d62e121ffc YARN-5195. RM intermittently crashed with NPE while handling APP_ATTEMPT_REMOVED event when async-scheduling enabled in CapacityScheduler. (sandflee via wangda) 2016-07-26 21:22:59 -07:00
Arun Suresh
2d8d183b19 YARN-5351. ResourceRequest should take ExecutionType into account during comparison. (Konstantinos Karanasos via asuresh) 2016-07-26 19:08:30 -07:00
Wangda Tan
49969b16cd YARN-5342. Improve non-exclusive node partition resource allocation in Capacity Scheduler. (Sunil G via wangda) 2016-07-26 18:14:09 -07:00
Arun Suresh
5aace38b74 YARN-5392. Replace use of Priority in the Scheduling infrastructure with an opaque ShedulerRequestKey. (asuresh and subru) 2016-07-26 14:54:03 -07:00
Varun Saxena
da6adf5151 Revert "YARN-5431. TimelineReader daemon start should allow to pass its own reader opts (Rohith Sharma K S via Varun Saxena)" 
This reverts commit 8e2614592d.
 2016-07-26 21:13:12 +05:30
Varun Saxena
8e2614592d YARN-5431. TimelineReader daemon start should allow to pass its own reader opts (Rohith Sharma K S via Varun Saxena) 2016-07-26 20:49:06 +05:30
Chris Douglas
d383bfdcd4 YARN-5164. Use plan RLE to improve CapacityOverTimePolicy efficiency 2016-07-25 16:37:50 -07:00
mattf-horton
12aa184479 HADOOP-13382. Remove unneeded commons-httpclient dependencies from POM files in Hadoop and sub-projects, while adding it in to the only place it is still needed, hadoop-openstack. Contributed by Matt Foley. 2016-07-21 14:43:57 -07:00
Rohith Sharma K S
557a245d83 YARN-5092. TestRMDelegationTokens fails intermittently. Contributed by Jason Lowe. 2016-07-21 12:47:27 +05:30
Rohith Sharma K S
be34b2a8fd YARN-1126. Add validation of users input nodes-states options to nodes CLI. Contributed by Wei Yan. 2016-07-21 12:20:47 +05:30
Akira Ajisaka
c63afdbe14 YARN-4883. Make consistent operation name in AdminService. Contributed by Kai Sasaki. 2016-07-20 16:51:01 -07:00
Vinod Kumar Vavilapalli
1c9d2ab503 YARN-5340. Fixed a race condition in RollingLevelDBTimelineStore that caused loss of Timeline events. Contributed by Li Lu. 2016-07-20 08:36:36 -07:00
Varun Vasudev
9ccf935b2c YARN-5309. Fix SSLFactory truststore reloader thread leak in TimelineClientImpl. Contributed by Weiwei Yang. 2016-07-20 12:48:32 +05:30
Arun Suresh
8fbe6ece24 YARN-5350. Distributed Scheduling: Ensure sort order of allocatable nodes returned by the RM is not lost. (asuresh) 2016-07-19 23:03:58 -07:00
Arun Suresh
cda0a280dd YARN-5181. ClusterNodeTracker: add method to get list of nodes matching a specific resourceName. (kasha via asuresh) 2016-07-19 10:43:37 -07:00
Arun Suresh
5f2d33a551 Revert "YARN=5181. ClusterNodeTracker: add method to get list of nodes matching a specific resourceName. (kasha via asuresh)" 
This reverts commit e905a42a2c.
 2016-07-19 10:43:19 -07:00
Junping Du
dc2f4b6ac8 YARN-5213. Fix a bug in LogCLIHelpers which cause TestLogsCLI#testFetchApplictionLogs fails intermittently. Contributed by Xuan Gong. 2016-07-19 07:17:58 -07:00
Varun Saxena
fe20494a72 YARN-4996. Make TestNMReconnect.testCompareRMNodeAfterReconnect() scheduler agnostic (Kai Sasaki via Varun Saxena) 2016-07-19 16:03:28 +05:30
Andrew Wang
da456ffd62 Preparing for 3.0.0-alpha2 development 2016-07-15 19:04:17 -07:00
Ray Chiang
f5f1c81e7d YARN-5272. Handle queue names consistently in FairScheduler. (Wilfred Spiegelenburg via rchiang) 2016-07-15 14:38:50 -07:00
Arun Suresh
e905a42a2c YARN=5181. ClusterNodeTracker: add method to get list of nodes matching a specific resourceName. (kasha via asuresh) 2016-07-15 14:35:12 -07:00
Wangda Tan
24db9167f1 YARN-4484. Available Resource calculation for a queue is not correct when used with labels. (Sunil G via wangda) 2016-07-15 11:40:12 -07:00
Varun Saxena
a72cb3825a YARN-5383. Fix findbugs and checkstyle issues in ContainerExecutor. (Vrushali C via Varun Saxena) 2016-07-15 23:17:04 +05:30
Junping Du
7e5355c14e YARN-5339. Passing file to -out for YARN log CLI doesnt give warning or error code. Contributed by Xuan Gong. 2016-07-15 09:23:11 -07:00
Naganarasimha
b5ee7dbd8d YARN-5380. NMTimelinePublisher should use getMemorySize instead of getMemory. Contributed by Vrushali C 2016-07-15 14:34:48 +05:30
Sangjin Lee
e549a9af31 YARN-5379. TestHBaseTimelineStorage. testWriteApplicationToHBase() fails intermittently (Vrushali C via sjlee) 2016-07-14 15:19:03 -07:00
Akira Ajisaka
6cf017558a YARN-5159. Wrong Javadoc tag in MiniYarnCluster. Contributed by Andras Bokor. 2016-07-14 10:45:11 -07:00
Xuan
429347289c YARN-5363. For AM containers, or for containers of running-apps, "yarn logs" incorrectly only (tries to) shows syslog file-type by default. (Contributed by Vinod Kumar Vavilapalli) 2016-07-14 10:20:08 -07:00
Varun Vasudev
54bf14f80b YARN-5303. Clean up ContainerExecutor JavaDoc. Contributed by Daniel Templeton. 2016-07-14 19:28:11 +05:30
Varun Vasudev
e5e558b0a3 YARN-4759. Fix signal handling for docker containers. Contributed by Shane Kumpf. 2016-07-14 19:27:16 +05:30
Varun Vasudev
58e1850801 YARN-5298. Mount usercache and NM filecache directories into Docker container. Contributed by Sidharta Seethana. 2016-07-14 16:11:07 +05:30
Rohith Sharma K S
dbe97aa768 YARN-5299. Log Docker run command when container fails. Contributed by Varun Vasudev. 2016-07-14 11:48:10 +05:30
Vinod Kumar Vavilapalli
d18050522c YARN-5339. Fixed "yarn logs" to fail when a file is passed to -out option instead of a directory. Contributed by Xuan Gong. 2016-07-13 14:52:30 -07:00
Naganarasimha
af8f480c24 YARN-5364. timelineservice modules have indirect dependencies on mapreduce artifacts. Contributed by Sangjin Lee 2016-07-14 01:43:37 +05:30
Vinod Kumar Vavilapalli
eb47163234 YARN-5200. Enhanced "yarn logs" to be able to get a list of containers whose logs are aggregated via a "show_container_log_info" option. Contributed by Xuan Gong. 2016-07-13 10:54:15 -07:00
Varun Saxena
56142171b9 YARN-5359. FileSystemTimelineReader/Writer uses unix-specific default storage path (Sangjin Lee via Varun Saxena) 2016-07-13 21:15:41 +05:30
Rohith Sharma K S
d6d41e820a YARN-5362. TestRMRestart#testFinishedAppRemovalAfterRMRestart can fail. Contributed by sandflee. 2016-07-13 19:12:35 +05:30
Varun Saxena
06c56ff79b YARN-5353. ResourceManager can leak delegation tokens when they are shared across apps. (Jason Lowe via Varun Saxena). 2016-07-13 07:55:34 +05:30
Robert Kanter
bf6f4a3b98 YARN-4366. Fix Lint Warnings in YARN Common (templedf via rkanter) 2016-07-12 15:12:25 -07:00
Jason Lowe
10b704c594 YARN-5317. testAMRestartNotLostContainerCompleteMsg may fail. Contributed by sandflee 2016-07-12 20:27:41 +00:00
Junping Du
7705812c7e New addendum to YARN-5080. Cannot obtain logs using YARN CLI -am for either KILLED or RUNNING AM. 2016-07-12 08:29:49 -07:00
Varun Saxena
a3ac1c1943 YARN-5354. TestDistributedShell.checkTimelineV2 fails randomly for concurrent tests. (Sangjin Lee via Varun Saxena). 2016-07-12 20:41:52 +05:30
Jian He
819224dcf9 YARN-5270. Solve miscellaneous issues caused by YARN-4844. Contributed by Wangda Tan 2016-07-11 22:36:20 -07:00
Varun Saxena
0fd3980a1f YARN-5037. Fix random failure of TestRMRestart#testQueueMetricsOnRMRestart (sandflee via Varun Saxena). 2016-07-10 21:28:52 +05:30
Sangjin Lee
6cf6ab7b78 Made a number of miscellaneous fixes for javac, javadoc, and checstyle warnings. 2016-07-10 08:46:05 -07:00
Sangjin Lee
c5dbde0cc4 Fixed the build after the rebase with trunk. 2016-07-10 08:46:05 -07:00
Sangjin Lee
b313ce06b2 YARN-5174. [documentation] several updates/corrections to timeline service documentation (Sangjin Lee, Varun Saxena, Naganarasimha G R, and Li Lu via sjlee) 2016-07-10 08:46:04 -07:00
Vrushali
6d943038f6 Cleanup changes during rebase with trunk (Vrushali C) 2016-07-10 08:46:04 -07:00
Sangjin Lee
1a227744ac YARN-5070. upgrade HBase version for first merge (Vrushali C via sjlee) 2016-07-10 08:46:04 -07:00
Varun Saxena
1ff6833bba YARN-5243. fix several rebase and other miscellaneous issues before merge. (Sangjin Lee via Varun Saxena) 2016-07-10 08:46:03 -07:00
Vrushali Channapattan
32b033d57c YARN-5052 Update timeline service v2 documentation to capture information about filters (Varun Saxena via Vrushali C) 2016-07-10 08:46:03 -07:00
Varun Saxena
892b193bd7 YARN-5170. Eliminate singleton converters and static method access. (Joep Rottinghuis via Varun Saxena) 2016-07-10 08:46:03 -07:00
Vrushali Channapattan
3832795e3c YARN-5210 NPE in Distributed Shell while publishing DS_CONTAINER_START event (Varun Saxena via Vrushali C) 2016-07-10 08:46:02 -07:00
Varun Saxena
c65020075e YARN-5167. Escape occurences of encodedValues. (Sangjin Lee via Varun Saxena) 2016-07-10 08:46:02 -07:00
Varun Saxena
9c926cf432 YARN-5142. fix findbugs warnings/errors for hadoop-yarn-server-timelineservice-hbase-tests. (Vrushali C via Varun Saxena) 2016-07-10 08:46:02 -07:00
Li Lu
0a9b085f05 YARN-5189. Make HBaseTimeline[Reader|Writer]Impl default and move FileSystemTimeline*Impl. (Joep Rottinghuis and Sangjin Lee via gtcarrera9) 2016-07-10 08:46:01 -07:00
Sangjin Lee
c81a2e1d19 YARN-5015. entire time series is returned for YARN container system metrics (CPU and memory) (Varun Saxena via sjlee) 2016-07-10 08:46:01 -07:00
Sangjin Lee
497d1c7f51 YARN-5111. YARN container system metrics are not aggregated to application (Naganarasimha G R via sjlee) 2016-07-10 08:46:01 -07:00
Sangjin Lee
7b8cfa5c2f YARN-5109. timestamps are stored unencoded causing parse errors (Varun Saxena via sjlee) 2016-07-10 08:46:01 -07:00
Li Lu
d729e8211b YARN-5138. fix "no findbugs output file" error for hadoop-yarn-server-timelineservice-hbase-tests. (Vrushali C via gtcarrera9) 2016-07-10 08:46:00 -07:00
Sangjin Lee
702236129b YARN-5095. flow activities and flow runs are populated with wrong timestamp when RM restarts w/ recovery enabled (Varun Saxena via sjlee) 2016-07-10 08:46:00 -07:00
Sangjin Lee
831a3ffd6e YARN-5097. NPE in Separator.joinEncoded() (Vrushali C via sjlee) 2016-07-10 08:46:00 -07:00
Sangjin Lee
a1b6d7456f YARN-5018. Online aggregation logic should not run immediately after collectors got started (Li Lu via sjlee) 2016-07-10 08:45:59 -07:00
Sangjin Lee
8c8183e515 YARN-5093. created time shows 0 in most REST output (Varun Saxena via sjlee) 2016-07-10 08:45:59 -07:00
Sangjin Lee
d5dfee24b1 YARN-5050. Code cleanup for TestDistributedShell (Li Lu via sjlee) 2016-07-10 08:45:59 -07:00
Sangjin Lee
4a7011a45f YARN-5096 addendum. Turned another logging statement to debug. Contributed by Sangjin Lee. 2016-07-10 08:45:59 -07:00
Varun Saxena
bcdb382600 YARN-5096. timelinereader has a lot of logging that's not useful. (Sangjin Lee via Varun Saxena) 2016-07-10 08:45:58 -07:00
Varun Saxena
bc6f978cd3 YARN-5045. hbase unit tests fail due to dependency issues. (Sangjin Lee via varunsaxena) 2016-07-10 08:45:58 -07:00
Sangjin Lee
089caf49fe YARN-4447. Provide a mechanism to represent complex filters and parse them at the REST layer (Varun Saxena via sjlee) 2016-07-10 08:45:57 -07:00
Li Lu
c2055a97d5 YARN-3150. Documenting the timeline service v2. (Sangjin Lee and Vrushali C via gtcarrera9) 2016-07-10 08:45:57 -07:00
Sangjin Lee
69dc561b61 YARN-4986. Add a check in the coprocessor for table to operated on (Vrushali C via sjlee) 2016-07-10 08:45:56 -07:00
Sangjin Lee
39cce4e629 YARN-3816. [Aggregation] App-level aggregation and accumulation for YARN system metrics (Li Lu via sjlee) 2016-07-10 08:45:56 -07:00
Sangjin Lee
c2efdc415a YARN-3863. Support complex filters in TimelineReader (Varun Saxena via sjlee) 2016-07-10 08:45:55 -07:00
Varun Saxena
a3cf40e532 YARN-3461. Consolidate flow name/version/run defaults. (Sangjin Lee via Varun Saxena) 2016-07-10 08:45:55 -07:00
Sangjin Lee
84c35ac6c4 YARN-4711. NM is going down with NPE's due to single thread processing of events by Timeline client (Naganarasimha G R via sjlee) 2016-07-10 08:45:55 -07:00
Varun Saxena
6f6cc647d6 YARN-4712. CPU Usage Metric is not captured properly in YARN-2928. (Naganarasimha G R via Varun Saxena) 2016-07-10 08:45:54 -07:00
Sangjin Lee
9bdd455dce YARN-4062. Add the flush and compaction functionality via coprocessors and scanners for flow run table (Vrushali C via sjlee) 2016-07-10 08:45:54 -07:00
Varun Saxena
d56dde490b YARN-4700. ATS storage has one extra record each time the RM got restarted. (Naganarasimha G R via Varun Saxena) 2016-07-10 08:45:53 -07:00
Sangjin Lee
0d02ab8729 YARN-3367. Replace starting a separate thread for post entity with event loop in TimelineClient (Naganarasimha G R via sjlee) 2016-07-10 08:45:53 -07:00
Sangjin Lee
960af7d471 YARN-4409. Fix javadoc and checkstyle issues in timelineservice code (Varun Saxena via sjlee) 2016-07-10 08:45:53 -07:00
Sangjin Lee
9cb1287e9b YARN-4446. Refactor reader API for better extensibility (Varun Saxena via sjlee) 2016-07-10 08:45:52 -07:00
Naganarasimha
06f0b50a28 YARN-4644. TestRMRestart fails and findbugs issue in YARN-2928 branch (Varun Saxena via Naganarasimha G R) 2016-07-10 08:45:52 -07:00
Li Lu
9d40d9d34c YARN-4224. Support fetching entities by UID and change the REST 
interface to conform to current REST APIs' in YARN. (Varun Saxena via
gtcarrera9)
 2016-07-10 08:45:52 -07:00
Naganarasimha
6934b05c71 YARN-4238. createdTime and modifiedTime is not reported while publishing entities to ATSv2. (Varun Saxena via Naganarasimha G R) 2016-07-10 08:45:52 -07:00
Varun Saxena
f5380d850e YARN-4622. TestDistributedShell fails for v2 test cases after modifications for 1.5. (Naganarasimha G R via Varun Saxena) 2016-07-10 08:45:51 -07:00
Li Lu
34f02f07d5 Rebase to latest trunk 2016-07-10 08:45:51 -07:00
Li Lu
88f0294114 YARN-4200. Refactor reader classes in storage to nest under hbase 
specific package name. Contributed by Li Lu.
 2016-07-10 08:45:51 -07:00
Sangjin Lee
cc16683cef YARN-3995. Some of the NM events are not getting published due race condition when AM container finishes in NM (Naganarasimha G R via sjlee) 2016-07-10 08:45:50 -07:00
Varun Saxena
829cceebc0 YARN-3586. RM to only get back addresses of Collectors that NM needs to know. 
(Junping Du via Varun Saxena).
 2016-07-10 08:45:50 -07:00
Varun Saxena
eb0ac8efb1 YARN-4350. TestDistributedShell fails for V2 scenarios. (Naganarasimha G R via Varun Saxena) 2016-07-10 08:45:50 -07:00
Li Lu
8ef546c1ee YARN-4445. Unify the term flowId and flowName in timeline v2 codebase. 
Contributed by Zhan Zhang.
 2016-07-10 08:45:49 -07:00
Varun Saxena
c4d7bbda5c YARN-4460. [Bug fix] RM fails to start when SMP is enabled. (Li Lu via Varun Saxena) 2016-07-10 08:45:49 -07:00
Xuan
2e2dbf59d1 YARN-4392. ApplicationCreatedEvent event time resets after RM 
restart/failover. Contributed by Naganarasimha G R and Xuan Gong

(cherry picked from commit 4546c7582b)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java
 2016-07-10 08:45:49 -07:00
Li Lu
b7d484aead YARN-4450. TestTimelineAuthenticationFilter and 
TestYarnConfigurationFields fail. Contributed by Sangjin Lee.
 2016-07-10 08:45:49 -07:00
Li Lu
89e5c44f9e YARN-4356. Ensure the timeline service v.2 is disabled cleanly and has no 
impact when it's turned off. Contributed by Sangjin Lee.
 2016-07-10 08:45:48 -07:00
Junping Du
cf4666bb8c YARN-3623. Add a config to indicate the Timeline Service version. Contributed by Xuan Gong. 
(cherry picked from commit f910e4f639)
 2016-07-10 08:45:48 -07:00
Sangjin Lee
1f710484e5 YARN-3862. Support for fetching specific configs and metrics based on prefixes (Varun Saxena via sjlee) 2016-07-10 08:45:48 -07:00
Sangjin Lee
ae72f1dc77 YARN-4297. TestJobHistoryEventHandler and TestRMContainerAllocator failing on YARN-2928 branch (Varun Saxena via sjlee) 2016-07-10 08:45:47 -07:00
Sangjin Lee
51254a6b51 YARN-4053. Change the way metric values are stored in HBase Storage (Varun Saxena via sjlee) 2016-07-10 08:45:47 -07:00
Sangjin Lee
09649005ca YARN-4221. Store user in app to flow table (Varun Saxena via sjlee) 2016-07-10 08:45:47 -07:00
Sangjin Lee
10ec5586fb YARN-4129. Refactor the SystemMetricPublisher in RM to better support newer events (Naganarasimha G R via sjlee) 2016-07-10 08:45:46 -07:00
Sangjin Lee
e3e857866d YARN-4179. [reader implementation] support flow activity queries based on time (Varun Saxena via sjlee) 2016-07-10 08:45:46 -07:00
Li Lu
d014f2ffd2 YARN-4237 Support additional queries for ATSv2 Web UI. Contributed by 
Varun Saxena.
 2016-07-10 08:45:46 -07:00
Sangjin Lee
b51d0fef56 YARN-4178. [storage implementation] app id as string in row keys can cause incorrect ordering (Varun Saxena via sjlee) 2016-07-10 08:45:45 -07:00
Sangjin Lee
0f44b5508d YARN-3864. Implement support for querying single app and all apps for a flow run (Varun Saxena via sjlee) 2016-07-10 08:45:45 -07:00
Vrushali Channapattan
708fa8b1ae YARN-4210. HBase reader throws NPE if Get returns no rows (Varun Saxena via vrushali) 2016-07-10 08:45:45 -07:00
Vrushali Channapattan
da2b7bd08e YARN-4203. Add request/response logging & timing for each REST endpoint call (Varun Saxena via vrushali) 2016-07-10 08:45:45 -07:00
Vrushali Channapattan
d95dc89a02 YARN-4075 [reader REST API] implement support for querying for flows and flow runs (Varun Saxena via vrushali) 2016-07-10 08:45:44 -07:00
Vrushali
10fa6da7d8 YARN-4074. [timeline reader] implement support for querying for flows and flow runs (sjlee via vrushali) 2016-07-10 08:45:44 -07:00
Sangjin Lee
a68e383921 YARN-3901. Populate flow run data in the flow_run & flow activity tables (Vrushali C via sjlee) 2016-07-10 08:45:44 -07:00
Sangjin Lee
ee081018e4 YARN-4102. Add a "skip existing table" mode for timeline schema creator (Li Lu via sjlee) 2016-07-10 08:45:43 -07:00
Sangjin Lee
8d9476ec5f YARN-4058. Miscellaneous issues in NodeManager project (Naganarasimha G R via sjlee) 2016-07-10 08:45:43 -07:00
Sangjin Lee
451c13b1de YARN-3814. REST API implementation for getting raw entities in TimelineReader (Varun Saxena via sjlee) 2016-07-10 08:45:43 -07:00
Sangjin Lee
c7224a285c YARN-4064. build is broken at TestHBaseTimelineWriterImpl.java (sjlee) 2016-07-10 08:45:43 -07:00
Junping Du
7a41b5501e YARN-4025. Deal with byte representations of Longs in writer code. Contributed by Sangjin Lee and Vrushali C. 2016-07-10 08:45:42 -07:00
Junping Du
477a30f536 YARN-3045. Implement NM writing container lifecycle events to Timeline Service v2. Contributed by Naganarasimha G R. 2016-07-10 08:45:42 -07:00
Sangjin Lee
102b56ee96 YARN-3904. Refactor timelineservice.storage to add support to online and offline aggregation writers (Li Lu via sjlee) 2016-07-10 08:45:42 -07:00
Zhijie Shen
5088f6c76a MAPREDUCE-6337. Added a mode to replay MR job history files and put them into the timeline service v2. Contributed by Sangjin Lee. 
(cherry picked from commit 463e070a8e7c882706a96eaa20ea49bfe9982875)
 2016-07-10 08:45:41 -07:00
Junping Du
00e85e7a2b YARN-3906. Split the application table from the entity table. Contributed by Sangjin Lee. 
(cherry picked from commit bcd755eba9466ce277d3c14192c31da6462c4ab3)
 2016-07-10 08:45:41 -07:00
Sangjin Lee
9e5155be36 YARN-3049. [Storage Implementation] Implement storage reader interface to fetch raw data from HBase backend (Zhijie Shen via sjlee) 
(cherry picked from commit 07433c2ad52df9e844dbd90020c277d3df844dcd)
 2016-07-10 08:45:41 -07:00
Zhijie Shen
9422d9b50d YARN-3984. Adjusted the event column key schema and avoided missing empty event. Contributed by Vrushali C. 
(cherry picked from commit 895ccfa1ab9e701f2908586e323249f670fe5544)
 2016-07-10 08:45:40 -07:00
Zhijie Shen
57e2498cd4 YARN-3993. Changed to use the AM flag in ContainerContext determine AM container in TestPerNodeTimelineCollectorsAuxService. Contributed by Sunil G. 
(cherry picked from commit 9e48f9ff2ce08f3dcdd8d60bacb697664b92196f)
 2016-07-10 08:45:40 -07:00
Zhijie Shen
a9fab9b644 YARN-3908. Fixed bugs in HBaseTimelineWriterImpl. Contributed by Vrushali C and Sangjin Lee. 
(cherry picked from commit df0ec473a84871b0effd7ca6faac776210d7df09)
 2016-07-10 08:45:40 -07:00
Junping Du
8603736ef2 YARN-3949. Ensure timely flush of timeline writes. Contributed by Sangjin Lee. 
(cherry picked from commit 967bef7e0396d857913caa2574afb103a5f0b81b)
 2016-07-10 08:45:39 -07:00
Sangjin Lee
57d8dc2fb7 YARN-3836. add equals and hashCode to TimelineEntity and other classes in the data model (Li Lu via sjlee) 
(cherry picked from commit 2d4a8f4563c06339717ca9410b2794754603fba3)
 2016-07-10 08:45:39 -07:00
Sangjin Lee
e27642abf4 YARN-3047. [Data Serving] Set up ATS reader with basic request serving structure and lifecycle (Varun Saxena via sjlee) 
(cherry picked from commit 4c5f88fb0f04b7919738d07598b0f006a9ff91f2)
 2016-07-10 08:45:39 -07:00
Zhijie Shen
2d59bc4458 YARN-3051. Created storage oriented reader interface for fetching raw entity data and made the filesystem based implementation. Contributed by Varun Saxena. 
(cherry picked from commit 499ce52c7b645ec0b1cc8ac62dc9a3127b987a20)
 2016-07-10 08:45:38 -07:00
Sangjin Lee
22e7ae5771 YARN-3792. Test case failures in TestDistributedShell and some issue fixes related to ATSV2 (Naganarasimha G R via sjlee) 
(cherry picked from commit 84f37f1c7eefec6d139cbf091c50d6c06f734323)
 2016-07-10 08:45:38 -07:00
Sangjin Lee
92d90c3a24 YARN-3706. Generalize native HBase writer for additional tables (Joep Rottinghuis via sjlee) 
(cherry picked from commit 9137aeae0dec83f9eff40d12cae712dfd508c0c5)
 2016-07-10 08:45:38 -07:00
Zhijie Shen
f3c661e8dd YARN-3044. Made RM write app, attempt and optional container lifecycle events to timeline service v2. Contributed by Naganarasimha G R. 2016-07-10 08:45:37 -07:00
Zhijie Shen
42d7864b84 YARN-3276. Code cleanup for timeline service API records. Contributed by Junping Du. 
(cherry picked from commit d88f30ba5359f59fb71b93a55e1c1d9a1c0dff8e)
 2016-07-10 08:45:37 -07:00
Sangjin Lee
f668212529 YARN-3721. build is broken on YARN-2928 branch due to possible dependency cycle (Li Lu via sjlee) 
(cherry picked from commit a9738ceb17b50cce8844fd42bb800c7f83f15caf)
 2016-07-10 08:45:37 -07:00
Sangjin Lee
db891f51d0 YARN-3726. Fix TestHBaseTimelineWriterImpl unit test failure by fixing its test data (Vrushali C via sjlee) 
(cherry picked from commit 4c0b6d73914f2e249795deb292f508177ea54884)
 2016-07-10 08:45:36 -07:00
Sangjin Lee
5a4278ccbd YARN-3411. [Storage implementation] explore the native HBase write schema for storage (Vrushali C via sjlee) 
(cherry picked from commit 7a3068854d27eadae1c57545988f5b2029bf119a)
 2016-07-10 08:45:36 -07:00
Junping Du
d275677e24 YARN-3634. TestMRTimelineEventHandling and TestApplication are broken. Contributed by Sangjin Lee. 
(cherry picked from commit b059dd4882fd759e4762cc11c019be4b68fb74c1)
 2016-07-10 08:45:36 -07:00
Zhijie Shen
51d092faef YARN-3529. Added mini HBase cluster and Phoenix support to timeline service v2 unit tests. Contributed by Li Lu. 2016-07-10 08:45:36 -07:00
Zhijie Shen
41fb5c7381 YARN-3134. Implemented Phoenix timeline writer to access HBase backend. Contributed by Li Lu. 
(cherry picked from commit b3b791be466be79e4e964ad068f7a6ec701e22e1)
 2016-07-10 08:45:35 -07:00
Sangjin Lee
dc1f306fdc YARN-3562. unit tests failures and issues found from findbug from earlier ATS checkins (Naganarasimha G R via sjlee) 2016-07-10 08:45:35 -07:00
Sangjin Lee
fc8485d893 YARN-3551. Consolidate data model change according to the backend implementation (Zhijie Shen via sale) 
(cherry picked from commit 557a3950bddc837469244835f5577899080115d8)
 2016-07-10 08:45:35 -07:00
Junping Du
2bdefbc4a0 YARN-3431. Sub resources of timeline entity needs to be passed to a separate endpoint. Contributed By Zhijie Shen. 
(cherry picked from commit fa5cc75245a6dba549620a8b26c7b4a8aed9838e)
 2016-07-10 08:45:34 -07:00
Sangjin Lee
11e8905d8d YARN-3390. Reuse TimelineCollectorManager for RM (Zhijie Shen via sjlee) 
(cherry picked from commit 58221188811e0f61d842dac89e1f4ad4fd8aa182)
 2016-07-10 08:45:33 -07:00
Junping Du
47f35a30bb YARN-3391. Clearly define flow ID/ flow run / flow version in API and storage. Contributed by Zhijie Shen 
(cherry picked from commit 68c6232f8423e55b4d152ef3d1d66aeb2d6a555e)
 2016-07-10 08:45:33 -07:00
Zhijie Shen
5712b8f9fd YARN-3334. NM uses timeline client to publish container metrics to new timeline service. Contributed by Junping Du. 2016-07-10 08:45:33 -07:00
Junping Du
42e49399ce YARN-3374. Collector's web server should randomly bind an available port. Contributed by Zhijie Shen 
(cherry picked from commit 3aa898e734a1e4368ddf1d0bbd31f9b4de53ceba)
 2016-07-10 08:45:32 -07:00
Junping Du
d67c9bdb4d YARN-3040. Make putEntities operation be aware of the app's context. Contributed by Zhijie Shen 2016-07-10 08:45:32 -07:00
Junping Du
5e3d9a477b YARN-3034. Implement RM starting its timeline collector. Contributed by Naganarasimha G R 2016-07-10 08:45:32 -07:00
Zhijie Shen
19c1132146 YARN-3377. Fixed test failure in TestTimelineServiceClientIntegration. Contributed by Sangjin Lee. 
(cherry picked from commit 04de2ceccd02edb00bd671a63e04855c132e9735)
 2016-07-10 08:45:31 -07:00
Junping Du
2188a07e5b YARN-3333. Rename TimelineAggregator etc. to TimelineCollector. Contributed by Sangjin Lee 2016-07-10 08:45:31 -07:00
Zhijie Shen
9b56364080 YARN-3039. Implemented the app-level timeline aggregator discovery service. Contributed by Junping Du. 2016-07-10 08:45:31 -07:00
Zhijie Shen
f0e752c14b YARN-3264. Created backing storage write interface and a POC only FS based storage implementation. Contributed by Vrushali C. 
(cherry picked from commit 821b68d05d246fd57d7b7286eb2ccc075ed1eae8)
 2016-07-10 08:45:30 -07:00
Zhijie Shen
bf54d32750 YARN-3210. Refactored timeline aggregator according to new code organization proposed in YARN-3166. Contributed by Li Lu. 
(cherry picked from commit d3ff7f06cbc66d3a23c2551e7d4c752689f46afe)
 2016-07-10 08:45:30 -07:00
Zhijie Shen
d45ff878c4 YARN-3125. Made the distributed shell use timeline service next gen and add an integration test for it. Contributed by Junping Du and Li Lu. 
(cherry picked from commit bf08f7f0ed4900ce52f98137297dd1a47ba2a536)
 2016-07-10 08:45:30 -07:00
Zhijie Shen
9d57c9c015 YARN-3087. Made the REST server of per-node aggregator work alone in NM daemon. Conntributed by Li Lu. 
(cherry picked from commit 41a08ad404d4278fe598d6c222b2ae0e84bae0df)
 2016-07-10 08:45:29 -07:00
Zhijie Shen
309592b9e5 YARN-3240. Added ".keep" file which was missed before. 
(cherry picked from commit 6b4dc96414b3f15537670c67a0dccd048862f4f4)
 2016-07-10 08:45:29 -07:00
Junping Du
4f0c7eaff3 YARN-3240. Implement client API to put generic entities. Contributed by Zhijie Shen 
(cherry picked from commit 4487da249f448d5c67b712cd0aa723e764eed77d)
 2016-07-10 08:45:29 -07:00
Zhijie Shen
c77f86bf63 YARN-3041. Added the overall data model of timeline service next gen. Contributed by Zhijie Shen. 
(cherry picked from commit fdff5d2625c129c20c44f90a0517b28cee1522bd)
 2016-07-10 08:45:29 -07:00
Zhijie Shen
7c8abec0a8 YARN-3030. Set up TS aggregator with basic request serving structure and lifecycle. Contributed by Sangjin Lee. 
(cherry picked from commit f26941b39028ac30c77547e2be2d657bb5bf044a)
 2016-07-10 08:45:28 -07:00
Zhijie Shen
0d2da08f35 YARN-3063. Bootstrapping TimelineServer next generation module. Contributed by Zhijie Shen. 
(cherry picked from commit fe7d0e7c8fc88e0721b18e1531b0899d9dbed296)
 2016-07-10 08:45:28 -07:00
Junping Du
9bdb5bebea Fix OOM issue in DistributedShell. Contributed by Jian He. 2016-07-09 05:09:53 -07:00
Varun Saxena
c04c5ec501 YARN-5318. Fix intermittent test failure of TestRMAdminService#testRefreshNodesResourceWithFileSystemBasedConfigurationProvider. (Jun Gong via Varun Saxena). 2016-07-09 01:13:18 +05:30
Varun Saxena
5252562edf YARN-5297. Avoid printing a stack trace when recovering an app after the RM restarts. (Junping Du via Varun Saxena). 2016-07-09 00:09:25 +05:30
Junping Du
30ee57ceb1 YARN-4939. The decommissioning Node should keep alive during NM restart. Contributed by sandflee. 2016-07-08 04:14:53 -07:00
Vinod Kumar Vavilapalli
673e5e02fe YARN-5314. Fixed a ConcurrentModificationException in ATS v1.5 EntityGroupFSTimelineStore. Contributed by Li Lu. 2016-07-07 22:37:29 -07:00
Allen Wittenauer
a0035661c1 HADOOP-13349. HADOOP_CLASSPATH vs HADOOP_USER_CLASSPATH (aw) 2016-07-07 14:15:34 -07:00
Jian He
8a9d293dd6 YARN-5233. Support for specifying a path for ATS plugin jars. Contributed by Li Lu 2016-07-06 16:44:46 -07:00
Vinod Kumar Vavilapalli
4c9e1aeb94 YARN-5224. Added new web-services /containers/{containerid}/logs & /containers/{containerid}/logs/{filename} and using them in "yarn logs" CLI to get logs of finished containers of a running application. Contributed by Xuan Gong. 2016-07-06 14:10:54 -07:00
Jian He
d169f5052f YARN-5227. Yarn logs command: no need to specify applicationId when 
specifying containerId. Contributed by Gergely Novák
 2016-07-06 10:43:27 -07:00
Wangda Tan
04f6ebb66a YARN-5294. Pass remote ip address down to YarnAuthorizationProvider. (Jian He via wangda) 2016-07-06 10:36:48 -07:00
Jian He
d792a90206 YARN-5296. NMs going OutOfMemory because ContainerMetrics leak in ContainerMonitorImpl. Contributed by Junping Du 2016-07-05 18:05:16 -07:00
Vinod Kumar Vavilapalli
ce9c006430 YARN-5214. Fixed locking in DirectoryCollection to avoid hanging NMs when various code-paths hit slow disks. Contributed by Junping Du. 2016-07-05 16:07:28 -07:00
Varun Saxena
8e672e3c71 YARN-5286. Add RPC port info in RM web service's response when getting app status. (Jun Gong via Varun Saxena). 2016-07-05 22:56:07 +05:30
Ray Chiang
0a5def155e YARN-4568. Fix message when NodeManager runs into errors initializing the recovery directory. (rchiang) 2016-07-01 15:53:46 -07:00
Jian He
c35a5a7a8d YARN-5023. TestAMRestart#testShouldNotCountFailureToMaxAttemptRetry fails. Contributed by sandflee 2016-07-01 14:29:03 -07:00
Varun Saxena
8ade81228e YARN-5282. Fix typos in CapacityScheduler documentation. (Ray Chiang via Varun Saxena). 2016-07-01 11:38:32 +05:30
Varun Saxena
abe7fc22c1 YARN-5182. MockNodes.newNodes creates one more node per rack than requested. (Karthik Kambatla via Varun Saxena). 2016-06-30 00:13:28 +05:30
Rohith Sharma K S
26b5e6116f YARN-5262. Optimize sending RMNodeFinishedContainersPulledByAMEvent for every AM heartbeat. 2016-06-29 10:08:30 +05:30
Allen Wittenauer
23c3ff85a9 HADOOP-13209. replace slaves with workers (John Smith via aw) 2016-06-28 05:53:03 -07:00
Akira Ajisaka
a8a48c9125 YARN-5278. Remove unused argument in TestRMWebServicesForCSWithPartitions#setupQueueConfiguration. Contributed by Tao Jie. 2016-06-23 14:28:12 +09:00
Xuan
6ab5aa1c1f YARN-5266. Wrong exit code while trying to get app logs using regex via CLI. Contributed by Xuan Gong 2016-06-22 21:48:49 -07:00
Arun Suresh
99e5dd68d0 YARN-5171. Extend DistributedSchedulerProtocol to notify RM of containers allocated by the Node. (Inigo Goiri via asuresh) 2016-06-22 19:04:54 -07:00
Tsuyoshi Ozawa
5d58858bb6 HADOOP-9613. [JDK8] Update jersey version to latest 1.x release. 2016-06-21 08:05:32 +09:00
Sangjin Lee
5107a967fa YARN-4958. The file localization process should allow for wildcards to reduce the application footprint in the state store (Daniel Templeton via sjlee) 2016-06-20 09:56:53 -07:00
Junping Du
d0162f2040 YARN-5246. NMWebAppFilter web redirects drop query parameters. Contributed by Varun Vasudev. 2016-06-19 17:44:54 -07:00
Karthik Kambatla
20f2799938 YARN-5077. Fix FSLeafQueue#getFairShare() for queues with zero fairshare. (Yufei Gu via kasha) 2016-06-17 22:24:42 -07:00
Karthik Kambatla
fbbe0bb627 YARN-5082. Limit ContainerId increase in fair scheduler if the num of node app reserved reached the limit. Addendum to fix javac warning. (Arun Suresh via kasha) 2016-06-17 22:12:50 -07:00
Junping Du
c35fa4a0e5 YARN-5251. Yarn CLI to obtain App logs for last 'n' bytes fails. Contributed by Xuan Gong. 2016-06-17 08:24:24 -07:00
Junping Du
e14ee0d3b5 YARN-5083. YARN CLI for AM logs does not give any error message if entered invalid am value. Contributed by Jian He. 2016-06-16 08:55:56 -07:00
Junping Du
5dfc38ff57 YARN-5237. Fix missing log files issue in rolling log aggregation. Contributed by Xuan Gong. 2016-06-15 16:17:54 -07:00
Varun Vasudev
25064fb2fb YARN-5223. Container line in yarn logs output for a live application should include the hostname for the container. Contributed by Xuan Gong. 2016-06-15 12:42:44 +05:30
Varun Vasudev
d6284428e8 YARN-5122. "yarn logs" for running containers should print an explicit footer saying that the log may be incomplete. Contributed by Jian He. 2016-06-15 12:35:17 +05:30
Wangda Tan
c77a1095dc YARN-1942. Deprecate toString/fromString methods from ConverterUtils and move them to records classes like ContainerId/ApplicationId, etc. (wangda) 2016-06-14 15:06:38 -07:00
Wangda Tan
eec835ec17 YARN-4887. Add allocation request ID to AM-RM protocol for identifying resource-requests explicitly. (Subru Krishnan via wangda) 2016-06-13 21:57:33 -07:00
Rohith Sharma K S
28b66ae919 YARN-4989. TestWorkPreservingRMRestart#testCapacitySchedulerRecovery fails intermittently. Contributed by Ajith S. 2016-06-13 11:09:32 +05:30
Arun Suresh
7dae2b3bc4 YARN-5212. Run existing ContainerManager tests using QueuingContainerManagerImpl. (Konstantinos Karanasos via asuresh) 2016-06-12 10:05:37 -07:00
Arun Suresh
5143277958 YARN-5124. Modify AMRMClient to set the ExecutionType in the ResourceRequest. (asuresh) 2016-06-12 09:42:38 -07:00
Arun Suresh
5279af7cd4 YARN-5082. Limit ContainerId increase in fair scheduler if the num of node app reserved reached the limit (sandflee via asuresh) 2016-06-10 22:33:42 -07:00
Rohith Sharma K S
e0f4620cc7 YARN-5197. RM leaks containers if running container disappears from node update. Contributed by Jason Lowe. 2016-06-11 10:22:27 +05:30
Wangda Tan
03fc6b1bb0 YARN-3426. Add jdiff support to YARN. (vinodkv via wangda) 2016-06-10 09:51:09 -07:00
Wangda Tan
244506f9c8 YARN-5208. Run TestAMRMClient TestNMClient TestYarnClient TestClientRMTokens TestAMAuthorization tests with hadoop.security.token.service.use_ip enabled. (Rohith Sharma K S via wangda) 2016-06-10 09:34:32 -07:00
Vinod Kumar Vavilapalli
9378d9428f YARN-5191. Renamed the newly added “download=true” option for getting logs via NMWebServices and AHSWebServices to be a better "format" option. (Xuan Gong via vinodkv) 2016-06-09 12:30:58 -07:00
Naganarasimha
1500a0a300 YARN-4308. ContainersAggregated CPU resource utilization reports negative usage in first few heartbeats. Contributed by Sunil G 2016-06-09 05:41:09 +05:30
Vinod Kumar Vavilapalli
5a43583c0b YARN-5080. Addendum fix to the original patch to fix YARN logs CLI. Contributed by Xuan Gong 2016-06-08 09:49:55 -07:00
Arun Suresh
3344ba70e0 YARN-5204. Properly report status of killed/stopped queued containers. (Konstantinos Karanasos via asuresh) 2016-06-08 08:31:32 -07:00
Jason Lowe
8c8a377cac YARN-5206. RegistrySecurity includes id:pass in exception text if considered invalid. Contributed by Steve Loughran 2016-06-08 14:11:25 +00:00
Arun Suresh
76f0800c21 YARN-5176. More test cases for queuing of containers at the NM. (Konstantinos Karanasos via asuresh) 2016-06-07 17:16:18 -07:00
Xuan
58be55b6e0 YARN-5199. Close LogReader in in AHSWebServices#getStreamingOutput and 
FileInputStream in NMWebServices#getLogs. Contributed by Xuan Gong
 2016-06-07 16:07:02 -07:00
Arun Suresh
8554aee1be Addendum patch for YARN-5180 updating findbugs-exclude.xml 2016-06-07 15:59:43 -07:00
Wangda Tan
620325e816 YARN-4837. User facing aspects of 'AM blacklisting' feature need fixing. (vinodkv via wangda) 2016-06-07 15:06:42 -07:00
Rohith Sharma K S
bddea5fe5f YARN-5118. Tests fails with localizer port bind exception. Contributed by Brahma Reddy Battula. 2016-06-07 11:20:15 +05:30
Arun Suresh
3a154f75ed YARN-4525. Fix bug in RLESparseResourceAllocation.getRangeOverlapping(). (Ishai Menache and Carlo Curino via asuresh) 2016-06-06 21:18:32 -07:00
Arun Suresh
7a9b7372a1 YARN-5185. StageAllocaterGreedyRLE: Fix NPE in corner case. (Carlo Curino via asuresh) 2016-06-06 21:06:52 -07:00