big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
26,766 Commits 2 Branches 2 Tags 264 MiB
193ff1c24e
Commit Graph

1486 Commits

Author SHA1 Message Date
Eric E Payne
55339c2bdd YARN-10479. RMProxy should retry on SocketTimeout Exceptions. Contributed by Jim Brennan (Jim_Brennan) 2020-11-05 21:50:46 +00:00
Eric E Payne
0461a07c01 YARN-10475: Scale RM-NM heartbeat interval based on node utilization. Contributed by Jim Brennan (Jim_Brennan). 2020-11-02 16:55:06 +00:00
Surendra Singh Lilhore
7169ec4509
YARN-10442. RM should make sure node label file highly available. (#2390) 
* YARN-10442. RM should make sure node label file highly available.

* YARN-10442. Corrected method name.
            Added license header for file TestNodeLabelFileReplication.java
            Added Test code.

* YARN-10442. Changed property name and log.

* YARN-10442. Changed default replication for FS store file.
 2020-10-30 23:05:16 +05:30
Jim Brennan
8ee6bc2518 YARN-10471. Prevent logs for any container from becoming larger than a configurable size. Contributed by Eric Payne 2020-10-29 19:49:18 +00:00
Ayush Saxena
1e3a6efcef
HADOOP-17288. Use shaded guava from thirdparty. (#2342). Contributed by Ayush Saxena. 2020-10-17 12:01:18 +05:30
Aryan Gupta
bbbfa7d415
HADOOP-17286. Updated jQuery in yarn-common. (#2340) 2020-09-27 03:18:01 +09:00
Aryan Gupta
486ddb73f9
HADOOP-17283. Hadoop - Upgrade to jQuery 3.5.1 (#2330) 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2020-09-24 21:15:55 +09:00
Szilard Nemeth
4ffe26f9b8 [UI1] Provide a way to hide Tools section in Web UIv1. Contributed by Andras Gyori 2020-08-31 16:00:39 +02:00
Adam Antal
82a7505646 YARN-10304. Create an endpoint for remote application log directory path query. Contributed by Andras Gyori 2020-08-25 13:29:12 +02:00
Prabhu Joseph
6c2ce3d56b YARN-10389. Option to override RMWebServices with custom WebService class 
Contributed by Tanu Ajmera. Reviewed by Bilwa ST and Sunil G.
 2020-08-11 15:46:47 +05:30
Prabhu Joseph
c7e71a6c0b YARN-10361. Make custom DAO classes configurable into RMWebApp#JAXBContextResolver. 
Contributed by Bilwa ST.
 2020-08-06 13:19:52 +05:30
Prabhu Joseph
cc71d50b21 YARN-10339. Fix TimelineClient in NodeManager failing when Simple Http Auth used in Secure Cluster 
Contributed by Tarun Parimi.
 2020-07-16 23:36:49 +05:30
Ahmed Hussein
1f71c4ae71 HADOOP-17099. Replace Guava Predicate with Java8+ Predicate 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
 2020-07-15 11:39:06 -05:00
Ahmed Hussein
98fcffe93f HADOOP-17101. Replace Guava Function with Java8+ Function 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
 2020-07-15 09:53:18 -05:00
Eric Badger
48f90115b5 YARN-10348. Allow RM to always cancel tokens after app completes. Contributed by 
Jim Brennan
 2020-07-13 23:10:39 +00:00
Prabhu Joseph
3a4d05b850 YARN-8047. RMWebApp make external class pluggable. 
Contributed by Bilwa S T.
 2020-07-08 12:43:20 +05:30
Eric Yang
e8dc862d38 YARN-9809. Added node manager health status to resource manager registration call. 
Contributed by Eric Badger via eyang
 2020-06-30 11:39:16 -07:00
Vinayakumar B
e154084770
HADOOP-17046. Support downstreams' existing Hadoop-rpc implementations using non-shaded protobuf classes (#2026) 2020-06-12 23:16:33 +05:30
Szilard Nemeth
a4835db95a YARN-10296. Make ContainerPBImpl#getId/setId synchronized. Contributed by Benjamin Teke 2020-06-05 10:11:40 +02:00
Prabhu Joseph
cef0756929 YARN-9606. Set sslfactory for AuthenticatedURL() while creating LogsCLI#webServiceClient. 
Contributed by Bilwa S T. Reviewed by Akira Ajisaka.
 2020-05-20 11:36:52 +05:30
Szilard Nemeth
0debe55d6c YARN-10160. Add auto queue creation related configs to RMWebService#CapacitySchedulerQueueInfo. Contributed by Prabhu Joseph 2020-05-05 13:11:28 +02:00
Inigo Goiri
d125d39108 YARN-6973. Adding RM Cluster Id in ApplicationReport. Contributed by Bilwa S T. 2020-04-29 11:23:58 -07:00
Prabhu Joseph
9224568b0e YARN-10156. Destroy Jersey Client in TimelineConnector. 
Contributed by Tanu Ajmera.
 2020-04-27 18:49:15 +05:30
Prabhu Joseph
60fa15366e YARN-10240. Prevent Fatal CancelledException in TimelineV2Client when stopping 
Contributed by Tarun Parimi.
 2020-04-21 12:01:33 +05:30
Jonathan Hung
3601054822 YARN-9954. Configurable max application tags and max tag length. Contributed by Bilwa S T 2020-04-17 10:34:44 -07:00
Jonathan Hung
23481ad378 YARN-10212. Create separate configuration for max global AM attempts. Contributed by Bilwa S T 2020-04-09 11:06:11 -07:00
Szilard Nemeth
8e4517f251 YARN-5277. When localizers fail due to resource timestamps being out, provide more diagnostics. Contributed by Siddharth Ahuja 2020-04-09 12:31:01 +02:00
Szilard Nemeth
bffb43b00e YARN-10207. CLOSE_WAIT socket connection leaks during rendering of (corrupted) aggregated logs on the JobHistoryServer Web UI. Contributed by Siddharth Ahuja 2020-04-07 17:03:17 +02:00
Brahma Reddy Battula
8914cf9167 Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30
RogPodge
50f7f6dfd1
YARN-10210. Add a RMFailoverProxyProvider that does DNS resolution on failover. 2020-03-26 08:14:30 -07:00
RogPodge
2d294bd575
HADOOP-16938. Make non-HA proxy providers pluggable 2020-03-25 08:06:58 -07:00
Szilard Nemeth
cf9cf83a43 YARN-9354. Resources should be created with ResourceTypesTestHelper instead of TestUtils. Contributed by Andras Gyori 2020-03-10 16:44:48 +01:00
Eric Badger
2649f8b327 YARN-10173. Make pid file generation timeout configurable in case of reacquired 
container. Contributed by Adam Antal.
 2020-03-04 23:31:57 +00:00
Szilard Nemeth
4af2556b48 YARN-10143. YARN-10101 broke Yarn logs CLI. Contributed by Adam Antal 2020-02-20 15:27:15 +01:00
Prabhu Joseph
fe7d67a8a2 YARN-9521. Handle FileSystem close in ApiServiceClient 
Contributed by kyungwan nam. Reviewed by Eric Yang.
 2020-02-13 09:39:13 +05:30
Ayush Saxena
3f0a7cd17a YARN-9624. Use switch case for ProtoUtils#convertFromProtoFormat containerState. Contributed by Bilwa S T 2020-02-09 19:14:18 +05:30
Vinayakumar B
7dac7e1d13
HADOOP-16596. [pb-upgrade] Use shaded protobuf classes from hadoop-thirdparty dependency (#1635). Contributed by Vinayakumar B. 2020-02-07 14:51:24 +05:30
Szilard Nemeth
71b2c2ffe9 YARN-10101. Support listing of aggregated logs for containers belonging to an application attempt. Contributed by Adam Antal 2020-02-06 12:25:06 +01:00
Inigo Goiri
5abd0148eb YARN-9768. RM Renew Delegation token thread should timeout and retry. Contributed by Manikandan R. 2020-01-28 10:06:37 -08:00
Szilard Nemeth
9520b2ad79 YARN-10083. Provide utility to ask whether an application is in final status. Contributed by Adam Antal 2020-01-22 16:25:07 +01:00
Inigo Goiri
b4870bce3a Revert "YARN-9768. RM Renew Delegation token thread should timeout and retry. Contributed by Manikandan R." 
This reverts commit 0696828a09.
 2020-01-21 17:45:17 -08:00
Inigo Goiri
0696828a09 YARN-9768. RM Renew Delegation token thread should timeout and retry. Contributed by Manikandan R. 2020-01-21 13:41:01 -08:00
Szilard Nemeth
6d52bbbfcf YARN-9525. IFile format is not working against s3a remote folder. Contributed by Adam Antal 2020-01-20 12:36:55 +01:00
Szilard Nemeth
57aad0f43a YARN-10081. Exception message from ClientRMProxy#getRMAddress is misleading. Contributed by Ravuri Sushma Sree 2020-01-20 09:41:06 +01:00
Szilard Nemeth
13cea0412c YARN-10028. Integrate the new abstract log servlet to the JobHistory server. Contributed by Adam Antal 2020-01-14 11:00:08 +01:00
Prabhu Joseph
571795cd18 YARN-10068. Fix TimelineV2Client leaking File Descriptors. 
Contributed by Anand Srinivasan. Reviewed by Adam Antal.
 2020-01-08 12:01:30 +05:30
Szilard Nemeth
768ee22e9e YARN-10035. Add ability to filter the Cluster Applications API request by name. Contributed by Adam Antal 2020-01-06 16:26:33 +01:00
Eric Badger
412035b47a YARN-10009. In Capacity Scheduler, DRC can treat minimum user limit percent as a max when custom resource is defined. Contributed by Eric Payne. 2019-12-20 18:07:00 +00:00
Jonathan Hung
fddc3d55c3 YARN-10039. Allow disabling app submission from REST endpoints 2019-12-18 10:48:05 -08:00
Szilard Nemeth
631dbbc6f2 YARN-9923. Introduce HealthReporter interface to support multiple health checker files. Contributed by Adam Antal 2019-12-15 17:28:04 +01:00
Szilard Nemeth
4f758dd4c6 YARN-9607. Auto-configuring rollover-size of IFile format for non-appendable filesystems. Contributed by Adam Antal 2019-12-05 21:49:00 +01:00
Szilard Nemeth
c71befaf8f YARN-9781. SchedConfCli to get current stored scheduler configuration. Contributed by Prabhu Joseph 2019-12-05 20:31:24 +01:00
prabhujoseph
aa7ab2719f YARN-9991. Fix Application Tag prefix to userid. Contributed by Szilard Nemeth. 2019-11-27 01:59:12 +05:30
Szilard Nemeth
52e9ee39a1 YARN-9444. YARN API ResourceUtils's getRequestedResourcesFromConfig doesn't recognize yarn.io/gpu as a valid resource. Contributed by Gergely Pollak 2019-11-26 16:42:59 +01:00
Szilard Nemeth
dfdc6d6dd9 YARN-9886. Queue mapping based on userid passed through application tag. Contributed by Julia Kinga Marton 2019-11-19 21:14:15 +01:00
Eric Badger
0e22e9ab83 YARN-9562. Add Java changes for the new RuncContainerRuntime. Contributed by Eric Badger 2019-11-18 21:56:31 +00:00
Eric Badger
eef34f2d87 YARN-9914. Use separate configs for free disk space checking for full and not-full disks. Contributed by Jim Brennan 2019-10-25 17:13:04 +00:00
Zhankun Tang
fd84ca5161 YARN-9921. Issue in PlacementConstraint when YARN Service AM retries allocation on component failure. Contributed by Tarun Parimi 2019-10-24 10:05:00 +08:00
Szilard Nemeth
35f093f5b3 YARN-9356. Add more tests to ratio method in TestResourceCalculator. Contributed by Zoltan Siegl 2019-10-09 13:09:09 +02:00
Jonathan Hung
5462d2176f YARN-9760. Support configuring application priorities on a workflow level. Contributed by Varun Saxena 2019-10-08 11:20:13 -07:00
Szilard Nemeth
bec0864394 YARN-9808. Zero length files in container log output haven't got a header. Contributed by Adam Antal 2019-09-25 10:28:34 +02:00
Jonathan Hung
c2731d4b63 YARN-9730. Support forcing configured partitions to be exclusive based on app node label 2019-09-24 13:51:32 -07:00
Duo Zhang
bfe1dac29d
HADOOP-16560. [YARN] use protobuf-maven-plugin to generate protobuf classes (#1496) 
Signed-off-by:  Vinayakumar B <vinayakumarb@apache.org>
 2019-09-24 08:50:36 +08:00
Vinayakumar B
1654497f98
HADOOP-16557. [pb-upgrade] Upgrade protobuf.version to 3.7.1 (#1432) 
HADOOP-16557. [pb-upgrade] Upgrade protobuf.version to 3.7.1. Contributed by Vinayakumar B.
 2019-09-20 16:08:30 +05:30
Malcolm Taylor
56248f9d87
HADOOP-16556. Fix some alerts raised by LGTM. 
Contributed by Malcolm Taylor.

Change-Id: Ic60c3f4681dd9d48b3afcba7520bd1e4d3cc4231
 2019-09-19 16:00:05 +01:00
Sunil G
01d7924473 YARN-9814. JobHistoryServer can't delete aggregated files, if remote app root directory is created by NodeManager. Contributed by Adam Antal. 2019-09-18 09:27:41 +05:30
Eric Yang
10144a580e YARN-9728. Bugfix for escaping illegal xml characters for Resource Manager REST API. 
Contributed by Prabhu Joseph
 2019-09-10 17:04:39 -04:00
Jonathan Hung
e7d44e48f7 YARN-9761. Allow overriding application submissions based on server side configs. Contributed by Pralabh Kumar 2019-09-06 10:17:07 -07:00
Tao Yang
172bcd8e01 YARN-8995. Log events info in AsyncDispatcher when event queue size cumulatively reaches a certain number every time. Contributed by zhuqi. 2019-09-05 16:20:05 +08:00
Zhankun Tang
bb26514ba9 YARN-9785. Fix DominantResourceCalculator when one resource is zero. Contributed by Bibin A Chundatt, Sunil Govindan, Bilwa S T. 2019-09-03 14:55:15 +08:00
Eric Yang
717c853873 YARN-9755. Fixed RM failing to start when FileSystemBasedConfigurationProvider is configured. 
Contributed by Prabhu Joseph
 2019-08-27 13:14:59 -04:00
Szilard Nemeth
e8fa192f07 YARN-9217. Nodemanager will fail to start if GPU is misconfigured on the node or GPU drivers missing. Contributed by Peter Bacsko 2019-08-21 16:44:22 +02:00
Sunil G
094d73608f YARN-9758. Upgrade JQuery to latest version for YARN UI. Contributed by Akhil PB. 2019-08-20 18:47:44 +05:30
Szilard Nemeth
b91099efd6 YARN-9451. AggregatedLogsBlock shows wrong NM http port. Contributed by Prabhu Joseph 2019-08-12 15:05:28 +02:00
Szilard Nemeth
e0517fea33 YARN-9134. No test coverage for redefining FPGA / GPU resource types in TestResourceUtils. Contributed by Peter Bacsko 2019-08-12 14:36:07 +02:00
Abhishek Modi
cf5d8957ee YARN-9657. AbstractLivelinessMonitor add serviceName to PingChecker thread. Contributed by Bilwa S T. 2019-08-11 15:41:56 +05:30
Haibo Chen
f51702d539 YARN-9559. Create AbstractContainersLauncher for pluggable ContainersLauncher logic. (Contributed by Jonathan Hung) 2019-08-06 13:52:30 -07:00
Szilard Nemeth
54ac80176e Logging fileSize of log files under NM Local Dir. Contributed by Prabhu Joseph 2019-08-02 13:38:06 +02:00
Szilard Nemeth
18ee1092b4 YARN-9127. Create more tests to verify GpuDeviceInformationParser. Contributed by Peter Bacsko 2019-07-15 11:59:11 +02:00
Szilard Nemeth
a2a8be18cb YARN-9629. Support configurable MIN_LOG_ROLLING_INTERVAL. Contributed by Adam Antal. 2019-07-03 13:45:00 +02:00
Sunil G
0c8813f135 YARN-9327. Improve synchronisation in ProtoUtils#convertToProtoFormat block. Contributed by Bibin A Chundatt. 2019-07-02 11:44:11 +05:30
Eric Yang
f02b0e1994 YARN-9581. Add support for get multiple RM webapp URLs. 
Contributed by Prabhu Joseph
 2019-06-28 14:54:57 -04:00
Weiwei Yang
cbae241320 YARN-9623. Auto adjust max queue length of app activities to make sure activities on all nodes can be covered. Contributed by Tao Yang. 2019-06-28 23:24:53 +08:00
Adam Antal
e94e643584 YARN-9471. Cleanup in TestLogAggregationIndexFileController. Contributed by Adam Antal. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-06-10 12:18:15 -07:00
Eric Yang
cb9bc6e64c YARN-9581. Fixed yarn logs cli to access RM2. 
Contributed by Prabhu Joseph
 2019-06-06 16:41:58 -04:00
Eric Yang
294695dd57 HADOOP-16314. Make sure all web end points are covered by the same authentication filter. 
Contributed by Prabhu Joseph
 2019-06-05 18:55:13 -04:00
Akira Ajisaka
afd844059c HADOOP-16331. Fix ASF License check in pom.xml 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2019-05-29 17:25:13 +09:00
Akira Ajisaka
9f933e6446
HADOOP-16323. https everywhere in Maven settings. 2019-05-27 15:24:59 +09:00
Eric Yang
460ba7fb14 YARN-9558. Fixed LogAggregation test cases. 
Contributed by Prabhu Joseph
 2019-05-23 18:38:47 -04:00
Giovanni Matteo Fumarola
a85451c8d2 YARN-9575. Fix TestYarnConfigurationFields testcase failing. Contributed by Prabhu Joseph. 2019-05-21 11:25:30 -07:00
Sunil G
24c53e057a YARN-9546. Add configuration option for YARN Native services AM classpath. Contributed by Gergely Pollak. 2019-05-20 10:53:01 -04:00
bibinchundatt
2de1e30658 YARN-9547. ContainerStatusPBImpl default execution type is not returned. Contributed by Bilwa S T. 2019-05-15 13:21:39 +05:30
Sunil G
7d831eca64 YARN-9519. TFile log aggregation file format is not working for yarn.log-aggregation.TFile.remote-app-log-dir config. Contributed by Adam Antal. 2019-05-14 10:48:08 -07:00
Weiwei Yang
1a47c2b7ae YARN-9539.Improve cleanup process of app activities and make some conditions configurable. Contributed by Tao Yang. 2019-05-12 22:31:39 -07:00
Eric Yang
49e1292ea3 YARN-9524. Fixed TestAHSWebService and TestLogsCLI unit tests. 
Contributed by Prabhu Joseph
 2019-05-06 19:48:45 -04:00
Weiwei Yang
12b7059ddc YARN-9440. Improve diagnostics for scheduler and app activities. Contributed by Tao Yang. 2019-05-06 20:00:15 +08:00
Eric Yang
accb811e57 YARN-6929. Improved partition algorithm for yarn remote-app-log-dir. 
Contributed by Prabhu Joseph
 2019-04-30 17:04:59 -04:00
Inigo Goiri
96e3027e46 YARN-2889. Limit the number of opportunistic container allocated per AM heartbeat. Contributed by Abhishek Modi. 2019-04-22 09:49:03 -07:00
Prabhu Joseph
aa4c744aef
YARN-9470. Fix order of actual and expected expression in assert statements 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2019-04-18 15:40:37 +09:00
Eric Badger
254efc9358 YARN-9379. Can't specify docker runtime through environment. Contributed by caozhiqiang 2019-04-15 18:24:37 +00:00
Vrushali C
22362c876d YARN-9335 [atsv2] Restrict the number of elements held in timeline collector when backend is unreachable for async calls. Contributed by Abhishesk Modi. 2019-04-05 12:06:51 -07:00
Vrushali C
27039a29ae YARN-9382 Publish container killed, paused and resumed events to ATSv2. Contributed by Abhishesk Modi. 2019-04-05 12:02:43 -07:00
Eric Yang
09eabda314 YARN-9364. Remove commons-logging dependency from YARN. 
Contributed by Prabhu Joseph
 2019-03-18 19:58:42 -04:00
Eric Yang
5f6e225166 YARN-9363. Replaced debug logging with SLF4J parameterized log message. 
Contributed by Prabhu Joseph
 2019-03-18 13:57:18 -04:00
Eric Yang
2064ca015d YARN-9349. Changed logging to use slf4j api. 
Contributed by Prabhu Joseph
 2019-03-15 19:20:59 -04:00
Giovanni Matteo Fumarola
67cc24a7a4 YARN-9381. The yarn-default.xml has two identical property named yarn.timeline-service.http-cross-origin.enabled. Contributed by Abhishek Modi. 2019-03-12 15:21:03 -07:00
Rohith Sharma K S
c24af4b0d6 YARN-9336. JobHistoryServer leaks CLOSE_WAIT tcp connections when using LogAggregationIndexedFileController. Contributed by Tarun Parimi. 2019-03-12 20:57:27 +05:30
Sunil G
de15a66d78 YARN-9265. FPGA plugin fails to recognize Intel Processing Accelerator Card. Contributed by Peter Bacsko. 2019-03-08 17:39:22 +05:30
Eric Yang
39b4a37e02 YARN-9341. Fixed enentrant lock usage in YARN project. 
Contributed by Prabhu Joseph
 2019-03-07 16:47:45 -05:00
Akira Ajisaka
bd8d299ded
YARN-7477. Moving logging APIs over to slf4j in hadoop-yarn-common. Contributed by Prabhu Joseph. 2019-03-04 16:59:20 +09:00
Weiwei Yang
e20b5ef52c
YARN-9332. RackResolver tool should accept multiple hosts. Contributed by Lantao Jin. 2019-03-04 16:59:20 +09:00
Vrushali C
0ec962ac8f YARN-5336 Limit the flow name size & consider cleanup for hex chars. Contributed by Sushil Ks 2019-02-27 14:43:39 -08:00
Szilard Nemeth
29e27faf96 YARN-9318. Resources#multiplyAndRoundUp does not consider Resource Types 
(Contributed by Szilard Nemeth via Daniel Templeton)

Change-Id: Ia45f528574c2b054f6f764d1d140e592bdb7e217
 2019-02-27 10:29:17 -08:00
Giovanni Matteo Fumarola
95372657fc YARN-9287. Consecutive StringBuilder append should be reuse. Contributed by Ayush Saxena. 2019-02-25 11:45:37 -08:00
Eric Yang
3dc2523266 YARN-9184. Add a system flag to allow update to latest docker images. 
Contributed by Zhaohui Xin
 2019-02-12 16:16:35 -05:00
Eric E Payne
d1ca9432dd YARN-7171: RM UI should sort memory / cores numerically. Contributed by Ahmed Hussein 2019-02-07 16:38:11 +00:00
Rohith Sharma K S
7fa62e150c YARN-8219. Add application launch time to ATSV2. Contributed by Abhishek Modi. 2019-02-07 09:58:53 +05:30
Rohith Sharma K S
e3ec18b0c4 YARN-6735. Have a way to turn off container metrics from NMs. Contributed by Abhishek Modi. 2019-02-05 13:48:04 +05:30
Rohith Sharma K S
571838920d YARN-9149. yarn container -status misses logUrl when integrated with ATSv2. Contributed by Abhishek Modi. 2019-02-05 12:39:10 +05:30
Akira Ajisaka
1129288cf5
HADOOP-14178. Move Mockito up to version 2.23.4. Contributed by Akira Ajisaka and Masatake Iwasaki. 2019-01-29 18:29:56 -08:00
Eric E Payne
04105bbfdb YARN-6616: YARN AHS shows submitTime for jobs same as startTime. Contributed by Prabhu Joseph 2019-01-29 17:22:24 +00:00
Weiwei Yang
085f0e8ae7 YARN-9086. [CSI] Run csi-driver-adaptor as aux service. Contributed by Weiwei Yang. 2019-01-29 14:53:08 +08:00
Eric Yang
1ab69a9543 YARN-9221. Added flag to disable dynamic auxiliary service feature. 
Contributed by Billie Rinaldi
 2019-01-25 19:05:36 -05:00
Eric Yang
a33ef4fd31 YARN-8867. Added resource localization status to YARN service status call. 
Contributed by Chandni Singh
 2019-01-24 18:43:21 -05:00
Weiwei Yang
a4bd64e724 YARN-9116. Capacity Scheduler: implements queue level maximum-allocation inheritance. Contributed by Aihua Xu. 2019-01-24 17:53:43 +08:00
Takanobu Asanuma
27aa6e8899 HADOOP-16046. [JDK 11] Correct the compiler exclusion of org/apache/hadoop/yarn/webapp/hamlet/** classes for >= Java 9. Contributed by Devaraj K. 2019-01-21 10:54:58 +09:00
rahul3
0a46baecd3
YARN-9203. Fix typos in yarn-default.xml. 
This closes #437

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2019-01-17 16:18:33 +09:00
Jason Lowe
6a923464af YARN-6523. Optimize system credentials sent in node heartbeat responses. Contributed by Manikandan R 2019-01-08 16:54:05 -06:00
Weiwei Yang
0921b706f7 YARN-9037. [CSI] Ignore volume resource in resource calculators based on tags. Contributed by Sunil Govindan. 2019-01-08 14:30:53 +08:00
Sunil G
f4906ac019 YARN-9038. [CSI] Add ability to publish/unpublish volumes on node managers. Contributed by Weiwei Yang. 2019-01-04 12:10:00 +05:30
Eric Yang
f82922dcfa YARN-5168. Added exposed port information for Docker container. 
Contributed by Xun Liu
 2018-12-21 19:44:07 -05:00
Weiwei Yang
f659485ee8 YARN-8925. Updating distributed node attributes only when necessary. Contributed by Tao Yang. 2018-12-21 10:56:42 +08:00
Inigo Goiri
5df9fb16b9 YARN-9130. Add Bind_HOST configuration for Yarn Web Proxy. Contributed by Rong Tang. 2018-12-19 18:58:00 -08:00
Eric Yang
c7a5a4435e YARN-9075. Add ability to register/remove auxiliary service without restart node manager. 
Contributed by Billie Rinaldi
 2018-12-18 17:05:51 -05:00
Haibo Chen
881230da21 YARN-9051. Integrate multiple CustomResourceTypesConfigurationProvider implementations into one. (Contributed by Szilard Nemeth) 2018-12-11 11:41:32 -08:00
Haibo Chen
912b1f9d64 YARN-9019. Ratio calculation of ResourceCalculator implementations could return NaN. (Contributed by Szilard Nemeth) 2018-12-05 15:16:41 -08:00
Wangda Tan
9de8e8d049 YARN-9030. Log aggregation changes to handle filesystems which do not support setting permissions. (Suma Shivaprasad via wangda) 
Change-Id: I80f1e8196b8624e24d74494719fdedfd7061dced
 2018-11-28 13:36:21 -08:00
Suma Shivaprasad
7dc272199f YARN-9034. ApplicationCLI should have option to take clusterId. Contributed by Rohith Sharma K S. 2018-11-28 00:42:14 -08:00
Weiwei Yang
176bb3f812 YARN-8984. AMRMClient#OutstandingSchedRequests leaks when AllocationTags is null or empty. Contributed by Yang Wang. 2018-11-22 16:52:29 +08:00
Sunil G
5fb14e0635 YARN-8953. [CSI] CSI driver adaptor module support in NodeManager. Contributed by Weiwei Yang. 2018-11-20 08:02:11 +05:30
Rohith Sharma K S
ee3355be3c YARN-8303. YarnClient should contact TimelineReader for application/attempt/container report. 2018-11-16 17:58:19 +05:30
Eric Yang
4c465f5535 YARN-8761. Service AM support for decommissioning component instances. 
Contributed by Billie Rinaldi
 2018-11-12 19:53:10 -05:00
Weiwei Yang
f8c72d7b3a YARN-8880. Add configurations for pluggable plugin framework. Contributed by Zhankun Tang. 2018-11-08 12:23:00 +08:00
Sunil G
d36012b69f YARN-8854. Upgrade jquery datatable version references to v1.10.19. Contributed by Akhil PB. 2018-10-30 22:56:46 +05:30
Haibo Chen
823bb5dda8 MAPREDUCE-4669. MRAM web UI does not work with HTTPS. (Contributed by Robert Kanter) 2018-10-23 15:28:13 -07:00
Wangda Tan
071b9e19aa YARN-8908. Fix errors in yarn-default.xml related to GPU/FPGA. (Zhankun Tang via wangda) 
Change-Id: I46f511a9b51b65b069e15d5b827530110c41edab
 2018-10-19 09:49:26 -07:00
Sunil G
2202e00ba8 YARN-8868. Set HTTPOnly attribute to Cookie. Contributed by Chandni Singh. 2018-10-18 15:23:20 +05:30
Sunil G
5085e5fa9e YARN-8759. Copy of resource-types.xml is not deleted if test fails, causes other test failures. Contributed by Antal Bálint Steinbach. 2018-10-17 16:05:08 +05:30
Haibo Chen
84e22a6af4 YARN-8842. Expose metrics for custom resource types in QueueMetrics. (Contributed by Szilard Nemeth) 2018-10-16 14:14:30 -07:00
Haibo Chen
c2288ac45b YARN-8448. AM HTTPS Support for AM communication with RMWeb proxy. (Contributed by Robert Kanter) 2018-10-16 13:36:26 -07:00
Sunil G
8e5365e277 YARN-8836. Add tags and attributes in resource definition. Contributed by Weiwei Yang. 2018-10-15 15:38:42 +05:30
Vrushali C
a3edfddcf7 YARN-8834 Provide Java client for fetching Yarn specific entities from TimelineReader. Contributed by Abhishek Modi 2018-10-11 20:21:00 -07:00
Akira Ajisaka
6fa3feb577
HADOOP-15832. Upgrade BouncyCastle to 1.60. Contributed by Robert Kanter. 2018-10-10 10:16:57 +09:00
Jason Lowe
9bb2801e8c YARN-4254. ApplicationAttempt stuck for ever due to UnknownHostException. Contributed by Bibin A Chundatt 2018-10-05 15:52:46 -05:00
Akira Ajisaka
39b35036ba
YARN-8840. Add missing cleanupSSLConfig() call for TestTimelineClient test. Contributed by Aki Tanaka. 2018-10-04 09:51:42 +09:00
Sunil G
58fa96b697 Changed version in trunk to 3.3.0-SNAPSHOT. 2018-10-02 22:41:41 +05:30
Eric Yang
b237a0dd44 YARN-6456. Added config to set default container runtimes. 
Contributed by Craig Condit
 2018-09-27 15:31:18 -04:00
Eric Yang
913f87dada YARN-8665. Added Yarn service cancel upgrade option. 
Contributed by Chandni Singh
 2018-09-26 14:51:35 -04:00
Giovanni Matteo Fumarola
3090922805 YARN-8696. [AMRMProxy] FederationInterceptor upgrade: home sub-cluster heartbeat async. Contributed by Botong Huang. 2018-09-24 11:37:05 -07:00
Shane Kumpf
8e9afbfb66 YARN-8768. Javadoc error in node attributes. Contributed by Sunil Govindan. 2018-09-12 15:12:28 -06:00
bibinchundatt
c44088ac19 YARN-8739. Fix jenkins issues for Node Attributes branch. Contributed by Sunil Govindan. 2018-09-12 16:01:01 +05:30
Weiwei Yang
52194351e7 YARN-8721. Relax NE node-attribute check when attribute doesn't exist on a node. Contributed by Sunil Govindan. 2018-09-12 16:01:01 +05:30
Naganarasimha
67ae81f0e0 YARN-7863. Modify placement constraints to support node attributes. Contributed by Sunil Govindan. 2018-09-12 16:01:01 +05:30
Naganarasimha
8c947398ab YARN-8574. Allow dot in attribute values. Contributed by Bibin A Chundatt. 2018-09-12 16:01:01 +05:30
Naganarasimha
eb08543c7a YARN-8103. Add CLI interface to query node attributes. Contributed by Bibin A Chundatt. 2018-09-12 16:01:01 +05:30
bibinchundatt
8cf6a9a2bd YARN-7892. Revisit NodeAttribute class structure. Contributed by Naganarasimha G R. 2018-09-12 16:01:00 +05:30
Naganarasimha
5dc7d6e0f3 YARN-8104. Add API to fetch node to attribute mapping. Contributed by Bibin A Chundatt. 2018-09-12 16:01:00 +05:30
Naganarasimha
0a01b1350d YARN-8100. Support API interface to query cluster attributes and attribute to nodes. Contributed by Bibin A Chundatt. 2018-09-12 16:01:00 +05:30
Sunil G
b9890d1f66 YARN-7875. Node Attribute store for storing and recovering attributes. Contributed by Bibin A Chundatt. 2018-09-12 16:01:00 +05:30
Sunil G
6f4bc49c6d YARN-7988. Refactor FSNodeLabelStore code for Node Attributes store support. Contributed by Bibin A Chundatt. 2018-09-12 16:01:00 +05:30
Naganarasimha
3b3b6efe21 YARN-7871. Node attributes reporting from NM to RM. Contributed by Weiwei Yang. 2018-09-12 16:01:00 +05:30
Naganarasimha
86d024ef2a YARN-7965. NodeAttributeManager add/get API is not working properly. Contributed by Weiwei Yang. 2018-09-12 16:01:00 +05:30
Sunil G
ffcabd24c3 YARN-7856. Validate Node Attributes from NM. Contributed by Weiwei Yang. 2018-09-12 16:01:00 +05:30
Sunil G
2f7712be09 YARN-6858. Attribute Manager to store and provide node attributes in RM. Contributed by Naganarasimha G R. 2018-09-12 16:01:00 +05:30
Naganarasimha
d312b5cf9f YARN-7757. Refactor NodeLabelsProvider to be more generic and reusable for node attributes providers. Contributed by Weiwei Yang. 2018-09-12 16:01:00 +05:30
bibinchundatt
4458b2772f YARN-7840. Update PB for prefix support of node attributes. Contributed by Naganarasimha G R. 2018-09-12 16:01:00 +05:30
Naganarasimha
1f42ce907a YARN-6855. [YARN-3409] CLI Proto Modifications to support Node Attributes. Contributed by Naganarasimha G R. 2018-09-12 16:00:59 +05:30
Giovanni Matteo Fumarola
3dc2988a37 YARN-8699. Add Yarnclient#yarnclusterMetrics API implementation in router. Contributed by Bibin A Chundatt. 2018-09-07 11:32:03 -07:00
Shane Kumpf
73625168c0 YARN-8642. Add support for tmpfs mounts with the Docker runtime. Contributed by Craig Condit 2018-08-29 07:08:37 -06:00
Eric Yang
e557c6bd8d YARN-8298. Added express upgrade for YARN service. 
Contributed by Chandni Singh
 2018-08-21 19:49:26 -04:00
Giovanni Matteo Fumarola
8736fc39ac YARN-8673. [AMRMProxy] More robust responseId resync after an YarnRM master slave switch. Contributed by Botong Huang. 2018-08-20 12:22:36 -07:00
Eric Yang
74411ce0ce YARN-7417. Remove duplicated code from IndexedFileAggregatedLogsBlock 
and TFileAggregatedLogsBlock.
           Contributed by Zian Chen
 2018-08-13 16:50:00 -04:00
Sunil G
00013d6ef7 YARN-8633. Update DataTables version in yarn-common in line with JQuery 3 upgrade. Contributed by Akhil PB. 2018-08-09 12:18:32 +05:30
Sunil G
5b898c176f Revert "YARN-8633. Update DataTables version in yarn-common in line with JQuery 3 upgrade. Contributed by Akhil PB." 
This reverts commit 64901abdfa.
 2018-08-08 19:48:49 +05:30
Sunil G
64901abdfa YARN-8633. Update DataTables version in yarn-common in line with JQuery 3 upgrade. Contributed by Akhil PB. 2018-08-08 19:43:29 +05:30
Wangda Tan
c0599151bb YARN-7089. Mark the log-aggregation-controller APIs as public. (Zian Chen via wangda) 
Change-Id: I37851bdc5935d623a27d0973a206c997258716eb
 2018-08-07 12:37:32 -07:00
Shane Kumpf
bcfc9851f4 YARN-8287. Update documentation and yarn-default related to the Docker runtime. Contributed by Craig Condit 2018-08-03 18:20:49 -06:00
Sunil G
12a095a496 YARN-7159. Normalize unit of resource objects in RM to avoid unit conversion in critical path. Contributed by Manikandan R. 2018-08-02 22:29:21 +05:30
Wangda Tan
5cc8e99147 YARN-8522. Application fails with InvalidResourceRequestException. (Zian Chen via wangda) 
Change-Id: I34dd7fa49bd4d10580c4a78051033b1068d28f1e
 2018-07-31 22:34:53 -07:00
Wangda Tan
4b540bbfcf YARN-8418. App local logs could leaked if log aggregation fails to initialize for the app. (Bibin A Chundatt via wangda) 
Change-Id: I29a23ca4b219b48c92e7975cd44cddb8b0e04104
 2018-07-31 12:08:00 -07:00
Jonathan Hung
3e06a5dcea YARN-7974. Allow updating application tracking url after registration. Contributed by Jonathan Hung 2018-07-30 17:44:18 -07:00
bibinchundatt
2b39ad2698 YARN-8584. Several typos in Log Aggregation related classes. Contributed by Szilard Nemeth. 2018-07-30 23:25:19 +05:30
bibinchundatt
ff7c2eda34 YARN-8548. AllocationRespose proto setNMToken initBuilder not done. Contributed by Bilwa S T. 2018-07-24 16:17:20 +05:30
Haibo Chen
8a6bb8409c YARN-6964. Fair scheduler misuses Resources operations. (Daniel Templeton and Szilard Nemeth via Haibo Chen) 2018-07-20 10:46:28 -07:00
Haibo Chen
f354f47f99 YARN-6995. Improve use of ResourceNotFoundException in resource types code. (Daniel Templeton and Szilard Nemeth via Haibo Chen) 2018-07-19 15:35:05 -07:00
Eric Yang
121865c3f9 YARN-8299. Added CLI and REST API for query container status. 
Contributed by Chandni Singh
 2018-07-16 17:41:23 -04:00
Wangda Tan
238ffff999 YARN-8524. Single parameter Resource / LightWeightResource constructor looks confusing. (Szilard Nemeth via wangda) 
Change-Id: I4ae97548b5b8d76a6bcebb2d3d70bf8e0be3c125
 2018-07-16 10:58:00 -07:00
Akira Ajisaka
88625f5cd9
HADOOP-15531. Use commons-text instead of commons-lang in some classes to fix deprecation warnings. Contributed by Takanobu Asanuma. 2018-07-13 11:42:12 -04:00
Giovanni Matteo Fumarola
ea9b608237 YARN-7899. [AMRMProxy] Stateful FederationInterceptor for pending requests. Contributed by Botong Huang. 2018-07-09 12:27:36 -07:00
Wangda Tan
83cd84b70b YARN-8506. Make GetApplicationsRequestPBImpl thread safe. (wangda) 
Change-Id: If304567abb77a01b686d82c769bdf50728484163
 2018-07-09 11:30:08 -07:00