big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
27,456 Commits 2 Branches 2 Tags 264 MiB
0b3755347c
Commit Graph

368 Commits

Author SHA1 Message Date
Robert Kanter
d4f7e2507f MAPREDUCE-6282. Reuse historyFileAbsolute.getFileSystem in CompletedJob#loadFullHistoryData for code optimization. (zxu via rkanter) 2015-03-20 13:11:58 -07:00
Tsuyoshi Ozawa
742f9d90c0 MAPREDUCE-6268. Fix typo in Task Attempt API's URL. Contributed by Ryu Kobayashi. 2015-03-03 16:21:16 +09:00
Konstantin V Shvachko
8ca0d957c4 YARN-3255. RM, NM, JobHistoryServer, and WebAppProxyServer's main() should support generic options. Contributed by Konstantin Shvachko. 2015-02-26 17:12:19 -08:00
Allen Wittenauer
8b787e2fdb MAPREDUCE-6260. Convert site documentation to markdown (Masatake Iwasaki via aw) 2015-02-17 06:52:14 -10:00
Devaraj K
76e309ead0 MAPREDUCE-6253. Update use of Iterator to Iterable. Contributed by Ray 
Chiang.
 2015-02-12 13:45:19 +05:30
Akira Ajisaka
cc6bbfceae MAPREDUCE-5988. Fix dead links to the javadocs in mapreduce project. (aajisaka) 2015-02-04 17:46:01 -08:00
Allen Wittenauer
fd57ab2002 MAPREDUCE-6059. Speed up history server startup time (Siqi Li via aw) 2015-02-04 17:36:28 -08:00
Akira Ajisaka
342efa110a HADOOP-9907. Webapp http://hostname:port/metrics link is not working. (aajisaka) 2015-01-30 02:49:10 +09:00
Allen Wittenauer
0a05ae1782 HDFS-7566. Remove obsolete entries from hdfs-default.xml (Ray Chiang via aw) 2015-01-27 14:31:27 -08:00
Jason Lowe
56b7ec71a6 MAPREDUCE-6141. History server leveldb recovery store. Contributed by Jason Lowe 2015-01-26 16:28:55 +00:00
Allen Wittenauer
42715547f0 HADOOP-11256. Some site docs have inconsistent appearance (Masatake Iwasaki via aw) 2015-01-21 08:44:22 -08:00
Harsh J
cb99f43305 MAPREDUCE-5420. Remove mapreduce.task.tmp.dir from mapred-default.xml. Contributed by James Carman. (harsh) 2014-12-11 09:45:49 +05:30
Haohui Mai
d777a1e4ca HADOOP-11369. Fix new findbugs warnings in hadoop-mapreduce-client, non-core directories. Contributed by Li Lu. 2014-12-09 10:46:13 -08:00
Jason Lowe
0c588904f8 MAPREDUCE-6160. Potential NullPointerException in MRClientProtocol interface implementation. Contributed by Rohith 2014-12-01 22:39:22 +00:00
Jian He
78f7cdbfd6 MAPREDUCE-5568. Fixed CompletedJob in JHS to show progress percentage correctly in case the number of mappers or reducers is zero. Contributed by MinJi Kim 2014-11-25 13:02:55 -08:00
Jason Lowe
09b3dee122 MAPREDUCE-6162. mapred hsadmin fails on a secure cluster. Contributed by Jason Lowe 2014-11-17 22:59:59 +00:00
Arun C. Murthy
43cd07b408 YARN-2830. Add backwords compatible ContainerId.newInstance constructor. Contributed by Jonathan Eagles. 2014-11-09 14:57:37 -08:00
Karthik Kambatla
e8a31f2e1c MAPREDUCE-5875. Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild. (Gera Shegalov via kasha) 2014-10-11 22:49:06 -07:00
Jason Lowe
7af4c3888b MAPREDUCE-6104. TestJobHistoryParsing.testPartialJob fails in branch-2. Contributed by Mit Desai 2014-09-24 16:09:33 +00:00
Allen Wittenauer
aa1052c34b YARN-2460. Remove obsolete entries from yarn-default.xml (Ray Chiang via aw) 2014-09-19 14:07:38 -07:00
Jason Lowe
1cf3198047 MAPREDUCE-6090. mapred hsadmin getGroups fails to connect in some cases. Contributed by Robert Kanter 2014-09-18 21:00:53 +00:00
Vinod Kumar Vavilapalli
14e2639fd0 YARN-611. Added an API to let apps specify an interval beyond which AM failures should be ignored towards counting max-attempts. Contributed by Xuan Gong. 2014-09-13 18:04:05 -07:00
Jason Lowe
7f80e14209 MAPREDUCE-6075. HistoryServerFileSystemStateStore can create zero-length files. Contributed by Jason Lowe 2014-09-10 22:22:39 +00:00
Allen Wittenauer
d989ac0444 MAPREDUCE-5972. Fix typo 'programatically' in job.xml (and a few other places) (Akira AJISAKA via aw) 2014-09-08 13:15:46 -07:00
Zhijie Shen
84bc2fe402 MAPREDUCE-6032. Made MR jobs write job history files on the default FS when the current context’s FS is different. Contributed by Benjamin Zhitomirsky. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618269 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-15 20:17:45 +00:00
Jason Darrell Lowe
9579554988 MAPREDUCE-6010. HistoryServerFileSystemStateStore fails to update tokens. Contributed by Jason Lowe 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617984 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-14 15:55:03 +00:00
Jason Darrell Lowe
857d134fda MAPREDUCE-6014. New task status field in task attempts table can lead to an empty web page. Contributed by Mit Desai 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616018 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-05 21:52:22 +00:00
Xuan Gong
e52f67e389 YARN-1994. Expose YARN/MR endpoints on multiple interfaces. Contributed by Craig Welch, Milan Potocnik,and Arpit Agarwal 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1614981 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-31 20:06:02 +00:00
Jason Darrell Lowe
1f8bb1001a MAPREDUCE-5309. 2.0.4 JobHistoryParser can't parse certain failed job history files generated by 2.0.3 history server. Contributed by Rushabh S Shah 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1596295 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-20 15:46:54 +00:00
Ravi Prakash
c9377189a0 MAPREDUCE 5874. Creating MapReduce REST API section. Contributed by Tsuyoshi Ozawa 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1595265 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-16 16:50:58 +00:00
Karthik Kambatla
5e337149e3 MAPREDUCE-5774. Job overview in History UI should list reducer phases in chronological order. (Gera Shegalov via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1593890 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-12 06:15:33 +00:00
Jason Darrell Lowe
e2c18809d3 MAPREDUCE-5884. History server uses short user name when canceling tokens. Contributed by Mohammad Kamrul Islam 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1593422 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-08 21:22:26 +00:00
Chris Nauroth
06c15b6a3e MAPREDUCE-5852. Prepare MapReduce codebase for JUnit 4.11. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1589006 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-21 23:44:41 +00:00
Chris Nauroth
dd7d032457 HADOOP-10499. Remove unused parameter from ProxyUsers.authorize(). Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1588098 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-16 23:09:08 +00:00
Vinod Kumar Vavilapalli
c1127d19c4 MAPREDUCE-5826. Fixed HistoryServerFileSystemStore to use right permissions on Windows for temporary files and thus also fix the test-issue with 
TestHistoryServerFileSystemStateStoreService. Contributed by Varun Vasudev.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1586567 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-11 04:28:57 +00:00
Haohui Mai
fb1d7fb596 HDFS-5570. Deprecate hftp / hsftp and replace them with webhdfs / swebhdfs. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1584100 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-02 17:28:08 +00:00
Jason Darrell Lowe
87866d4a56 MAPREDUCE-5789. Average Reduce time is incorrect on Job Overview page. Contributed by Rushabh S Shah 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1577202 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-13 15:35:34 +00:00
Jason Darrell Lowe
658b1bf561 MAPREDUCE-5754. Preserve Job diagnostics in history. Contributed by Gera Shegalov 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1572269 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-26 21:32:34 +00:00
Vinod Kumar Vavilapalli
6b03174aae YARN-1590. Fixed ResourceManager, web-app proxy and MR JobHistoryServer to expand _HOST properly in their kerberos principles. Contributed by Mohammad Kamrul Islam. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1569537 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-18 21:50:37 +00:00
Vinod Kumar Vavilapalli
990cffdcfa YARN-1553. Modified YARN and MR to stop using HttpConfig.isSecure() and 
instead rely on the http policy framework. And also fix some bugs related
to https handling in YARN web-apps. Contributed by Haohui Mai.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1568501 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-14 20:01:02 +00:00
Karthik Kambatla
308f5faf18 MAPREDUCE-5746. Job diagnostics can implicate wrong task for a failed job. (Jason Lowe via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1567666 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-12 16:04:18 +00:00
Sanford Ryza
db80705719 MAPREDUCE-5732. Report proper queue when job has been automatically placed (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1562641 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-29 23:51:40 +00:00
Alejandro Abdelnur
3a6f8b8785 MAPREDUCE-5724. JobHistoryServer does not start if HDFS is not running. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1558852 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-16 17:10:20 +00:00
Karthik Kambatla
1152e23ed0 YARN-1463. Tests should avoid starting http-server where possible or creates spnego keytab/principals (vinodkv via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1553790 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-28 06:55:30 +00:00
Sanford Ryza
04dac63695 MAPREDUCE-5550. Task Status message (reporter.setStatus) not shown in UI with Hadoop 2.0 (Gera Shegalov via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1552808 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-20 21:00:25 +00:00
Jason Darrell Lowe
44429c4bb3 MAPREDUCE-5679. TestJobHistoryParsing has race condition. Contributed by Liyin Liang 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1551616 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-17 16:45:23 +00:00
Jonathan Turner Eagles
ee5351bf22 MAPREDUCE-5052. Job History UI and web services confusing job start time and job submit time (Chen He via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1549896 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-10 17:17:47 +00:00
Arun Murthy
eec0ec2d3f Add ACL check to JHS web-services. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1547159 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-02 19:32:17 +00:00
Sanford Ryza
84cec3c805 MAPREDUCE-4680. Job history cleaner should only check timestamps of files in old enough directories (Robert Kanter via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1536558 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-28 23:48:38 +00:00
Vinod Kumar Vavilapalli
f0799c5536 MAPREDUCE-5562. Fixed MR App Master to perform pending tasks like staging-dir cleanup, sending job-end notification correctly when unregister with RM fails. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529682 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-06 20:53:28 +00:00
Vinod Kumar Vavilapalli
21181b6553 YARN-1277. Added a policy based configuration for http/https in common HttpServer and using the same in YARN - related 
to per project https config support via HADOOP-10022. Contributed by Suresh Srinivas and Omkar Vinit Joshi.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529662 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-06 18:43:36 +00:00
Vinod Kumar Vavilapalli
82f4348f27 MAPREDUCE-5536. Fixed MR AM and JHS to respect mapreduce.jobhistory.webapp.https.address. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1528251 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-01 22:52:25 +00:00
Jason Darrell Lowe
2627e352d6 MAPREDUCE-5332. Support token-preserving restart of history server. Contributed by Jason Lowe 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1527015 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-27 18:19:41 +00:00
Colin McCabe
8a66e493ba HADOOP-9998. Provide methods to clear only part of the DNSToSwitchMapping. (Junping Du via Colin Patrick McCabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1526567 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-26 15:24:30 +00:00
Bikas Saha
524dad1109 MAPREDUCE-5505. Clients should be notified job finished only after job successfully unregistered (Zhijie Shen via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1526071 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-25 00:46:48 +00:00
Vinod Kumar Vavilapalli
13420d01f2 YARN-1203. Changed YARN web-app proxy to handle http and https URLs from AM registration and finish correctly. Contributed by Omkar Vinit Joshi. 
MAPREDUCE-5515. Fixed MR AM's webapp to depend on a new config mapreduce.ssl.enabled to enable https and disabling it by default as MR AM needs
to set up its own certificates etc and not depend on clusters'. Contributed by Omkar Vinit Joshi.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1524864 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-19 23:35:58 +00:00
Arun Murthy
60a1a70197 YARN-1170. YARN & MapReduce proto definitions fixed to specify protobuf package as hadoop.yarn and hadoop.mapreduce respectively. Contibuted by Binglin Chang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1523652 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-16 14:04:04 +00:00
Vinod Kumar Vavilapalli
5adba5597c YARN-1085. Addendum patch to address issues with the earlier patch. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1517721 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-26 22:59:09 +00:00
Vinod Kumar Vavilapalli
e86036662c YARN-1085. Modified YARN and MR2 web-apps to do HTTP authentication in secure setup with kerberos. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1517101 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-24 02:47:47 +00:00
Vinod Kumar Vavilapalli
2d614a916c MAPREDUCE-5466. Changed MR AM to not promote history files of intermediate AMs in case they are exiting because of errors and thus help history-server pick up the right history file for the last successful AM. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1516238 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-21 18:16:47 +00:00
Alejandro Abdelnur
8451ab5c01 HADOOP-9872. Improve protoc version handling and detection. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1514068 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-14 22:15:04 +00:00
Alejandro Abdelnur
19b36f0900 HADOOP-9845. Update protobuf to 2.5 from 2.4.x. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1513281 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-12 22:15:10 +00:00
Jason Darrell Lowe
c5c90ff8f5 MAPREDUCE-5425. Junit in TestJobHistoryServer failing in jdk 7. Contributed by Robert Parker 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1511464 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-07 20:02:10 +00:00
Jason Darrell Lowe
bfe5a528d8 MAPREDUCE-5428. HistoryFileManager doesn't stop threads when service is stopped. Contributed by Karthik Kambatla 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1509401 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-01 19:57:31 +00:00
Jason Darrell Lowe
8bb035509e MAPREDUCE-5411. Refresh size of loaded job cache on history server. Contributed by Ashwin Shankar 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1508220 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-29 22:33:45 +00:00
Jason Darrell Lowe
8fa3ebd134 MAPREDUCE-5386. Ability to refresh history server job retention and job cleaner settings. Contributed by Ashwin Shankar 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1507135 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-25 21:27:56 +00:00
Jason Darrell Lowe
a42e459b9e MAPREDUCE-5356. Ability to refresh aggregated log retention period and check interval. Contributed by Ashwin Shankar 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1506226 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-23 19:20:57 +00:00
Jason Darrell Lowe
cc536fe4da MAPREDUCE-5265. History server admin service to refresh user and superuser group mappings. Contributed by Ashwin Shankar 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1504645 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-18 20:41:14 +00:00
Vinod Kumar Vavilapalli
1a06175440 YARN-851. Share NMTokens using NMTokenCache (api-based) between AMRMClient and NMClient instead of memory based approach which is used currently. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1495247 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-21 00:08:13 +00:00
Jonathan Turner Eagles
1ab4659b6b MAPREDUCE-5084. fix coverage org.apache.hadoop.mapreduce.v2.app.webapp and org.apache.hadoop.mapreduce.v2.hs.webapp (Aleksey Gorshkov via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494676 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-19 16:22:12 +00:00
Vinod Kumar Vavilapalli
243bcd367f YARN-694. Starting to use NMTokens to authenticate all communication with NodeManagers. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494369 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-18 23:19:49 +00:00
Vinod Kumar Vavilapalli
5d1b453b85 YARN-610. ClientToken is no longer set in the environment of the Containers. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493968 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-17 23:13:17 +00:00
Vinod Kumar Vavilapalli
cd30058193 YARN-837. Moved yarn.ClusterInfo into MapReduce project as it doesn't belong to YARN. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493676 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-17 08:13:48 +00:00
Vinod Kumar Vavilapalli
b9efe6bd4a HADOOP-9649. Promoted YARN service life-cycle libraries into Hadoop Common for usage across all Hadoop projects. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493650 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-17 06:39:33 +00:00
Vinod Kumar Vavilapalli
d4a811edb2 YARN-826. Moved Clock and SystemClock into yarn.util package. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493639 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-17 04:49:13 +00:00
Arun Murthy
77e134d13f YARN-387. Renamed YARN protocols for consistency. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493623 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-17 02:27:18 +00:00
Alejandro Abdelnur
789c03438c MAPREDUCE-5171. Expose blacklisted nodes from the MR AM REST API. (sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493600 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-16 22:15:37 +00:00
Vinod Kumar Vavilapalli
c0cd68d8fb YARN-773. Moved YarnRuntimeException from package api.yarn to api.yarn.exceptions. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492903 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-13 23:25:14 +00:00
Vinod Kumar Vavilapalli
0928502029 YARN-530. Defined Service model strictly, implemented AbstractService for robust subclassing and migrated yarn-common services. Contributed by Steve Loughran. 
YARN-117. Migrated rest of YARN to the new service model. Contributed by Steve Louhran.
MAPREDUCE-5298. Moved MapReduce services to YARN-530 stricter lifecycle. Contributed by Steve Loughran.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492718 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-13 15:54:38 +00:00
Vinod Kumar Vavilapalli
5c097a134e YARN-635 and MAPREDUCE-5301. Bug fix as the committed patch ended up being stale. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489287 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-04 04:41:55 +00:00
Vinod Kumar Vavilapalli
a83fb61ac0 YARN-635. Renamed YarnRemoteException to YarnException. Contributed by Siddharth Seth. 
MAPREDUCE-5301. Updated MR code to work with YARN-635 changes of renaming YarnRemoteException to YarnException. Contributed by Siddharth Seth


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489283 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-04 04:05:50 +00:00
Jason Darrell Lowe
78daceaea4 MAPREDUCE-5283. Over 10 different tests have near identical implementations of AppContext. Committed by Sandy Ryza 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489271 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-04 01:50:15 +00:00
Vinod Kumar Vavilapalli
d33534c4fb YARN-748. Moved BuilderUtils from yarn-common to yarn-server-common for eventual retirement. Contributed by Jian He. 
MAPREDUCE-5297. Updated MR App since BuilderUtils is no longer public after YARN-748. Contributed by Jian He.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489257 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-04 00:34:18 +00:00
Jason Darrell Lowe
ac44e0a0d0 MAPREDUCE-5268. Improve history server startup performance. Contributed by Karthik Kambatla 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489012 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-03 14:46:30 +00:00
Vinod Kumar Vavilapalli
a2c4233004 YARN-717. Put object creation factories for Token in the class itself and remove useless derivations for specific tokens. Contributed by Jian He. 
MAPREDUCE-5289. Updated MR App to use Token directly after YARN-717. Contributed by Jian He.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488616 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-01 21:43:17 +00:00
Siddharth Seth
39f019f413 YARN-735. Make ApplicationAttemptId, ContainerId and NodeId immutable. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488439 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-01 00:14:31 +00:00
Vinod Kumar Vavilapalli
982753dc8e YARN-716. Making ApplicationID immutable. Contributed by Siddharth Seth. 
MAPREDUCE-5282. Updating MR App to use immutable ApplicationID after YARN-716. Contributed by Siddharth Seth.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1487994 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-30 20:18:39 +00:00
Vinod Kumar Vavilapalli
643155cbee MAPREDUCE-5270. Migrated MR app from using BuilderUtil factory methods to individual record factory methods. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1486271 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-25 01:46:14 +00:00
Vinod Kumar Vavilapalli
7359dc32d3 MAPREDUCE-4067. Changed MRClientProtocol api to throw IOException only (Xuan Gong via vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1481695 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-13 03:34:25 +00:00
Vinod Kumar Vavilapalli
ae393ab26e MAPREDUCE-5179. Fix unit test in TestHSWebServices which fails when versionInfo has parantheses like when running on a git checkout. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476835 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-28 21:02:25 +00:00
Vinod Kumar Vavilapalli
5fd460e688 MAPREUDUCE-5059. Change average merge time on Job overview page to be the time delta between the end of the shuffle and the start of the reduce. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1467120 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-11 22:27:45 +00:00
Thomas Graves
a5734cd38a MAPREDUCE-5007. fix coverage org.apache.hadoop.mapreduce.v2.hs (Aleksey Gorshkov via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1465017 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-05 15:43:53 +00:00
Robert Joseph Evans
f7df0cb7df MAPREDUCE-4972. Coverage fixing for org.apache.hadoop.mapreduce.jobhistory (Aleksey Gorshkov via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1458906 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-20 16:05:33 +00:00
Aaron Myers
64741f4635 HDFS-4521. Invalid network toploogies should not be cached. Contributed by Colin Patrick McCabe. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1457878 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-18 17:20:13 +00:00
Alejandro Abdelnur
8c8ff3f5e0 MAPREDUCE-4716. TestHsWebServicesJobsQuery.testJobsQueryStateInvalid fails with jdk7. (tgraves via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1457065 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-15 18:55:22 +00:00
Alejandro Abdelnur
d412a1a1a8 MAPREDUCE-4571. TestHsWebServicesJobs fails on jdk7. (tgraves via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1457061 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-15 18:50:51 +00:00
Robert Joseph Evans
1b6f0582fb YARN-237. Refreshing the RM page forgets how many rows I had in my Datatables (jian he via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1456536 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-14 16:56:10 +00:00
Robert Joseph Evans
ec13f1eb3a MAPREDUCE-5043. Fetch failure processing can cause AM event queue to backup and eventually OOM (Jason Lowe via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1452372 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-04 16:32:18 +00:00
Siddharth Seth
979fb054f8 MAPREDUCE-4693. Historyserver should provide counters for failed tasks. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1450956 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-27 21:03:19 +00:00
Jason Darrell Lowe
c2d6407070 MAPREDUCE-4989. JSONify DataTables input data for Attempts page. Contributed by Ravi Prakash 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1445448 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-13 01:15:48 +00:00
Siddharth Seth
74ffc7a74d MAPREDUCE-4946. Fix a performance problem for large jobs by reducing the number of map completion event type conversions. Contributed by Jason Lowe. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1437103 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-22 19:07:08 +00:00
Thomas Graves
e7aeb9d129 MAPREDUCE-4934. Maven RAT plugin is not checking all source files (tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1432932 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-14 14:35:45 +00:00
Robert Joseph Evans
64e4fb983e MAPREDUCE-4819. AM can rerun job after reporting final job status to the client (bobby and Bikas Saha via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1429114 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-04 20:35:56 +00:00
Thomas Graves
e17cecf550 MAPREDUCE-4894. Renewal / cancellation of JobHistory tokens (Siddharth Seth via tgraves 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1429086 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-04 20:15:43 +00:00
Jason Darrell Lowe
b72b36009a YARN-266. RM and JHS Web UIs are blank because AppsBlock is not escaping string properly. Contributed by Ravi Prakash 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1420232 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-11 15:45:10 +00:00
Jason Darrell Lowe
6c5f37e46d MAPREDUCE-4811. JobHistoryServer should show when it was started in WebUI About page. Contributed by Ravi Prakash 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1412377 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-22 00:05:26 +00:00
Robert Joseph Evans
d2b9a604bd MAPREDUCE-4802. Takes a long time to load the task list on the AM for large jobs (Ravi Prakash via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1410498 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-16 17:30:22 +00:00
Eli Collins
905b17876c MAPREDUCE-4723. Fix warnings found by findbugs 2. Contributed by Sandy Ryza 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1409601 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-15 00:16:35 +00:00
Robert Joseph Evans
7e2ecffd88 MAPREDUCE-4720. Browser thinks History Server main page JS is taking too long (Ravi Prakash via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1409580 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-15 00:01:07 +00:00
Jason Darrell Lowe
397c261433 YARN-216. Remove jquery theming support. Contributed by Robert Joseph Evans. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1409313 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-14 19:22:11 +00:00
Jonathan Turner Eagles
3ccc76c29b MAPREDUCE-4666. JVM metrics for history server. (jlowe via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1407669 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-09 22:14:06 +00:00
Robert Joseph Evans
50993e9d48 MAPREDUCE-4724. job history web ui applications page should be sorted to display last app first (tgraves via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1404305 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-31 19:13:03 +00:00
Vinod Kumar Vavilapalli
1195f844a9 MAPREDUCE-4733. Reducer can fail to make progress during shuffle if too many reducers complete consecutively. Contributed by Jason Lowe via. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1400264 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-19 20:18:46 +00:00
Robert Joseph Evans
c63811790a MAPREDUCE-4721. Task startup time in JHS is same as job startup time. (Ravi Prakash via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1399289 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-17 14:58:08 +00:00
Siddharth Seth
8ac3910ae0 MAPREDUCE-4705. Fix a bug in job history lookup, which makes older jobs inaccessible despite the presence of a valid history file. (Contributed by Jason Lowe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1395850 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-09 03:23:07 +00:00
Robert Joseph Evans
b897d6c35a MAPREDUCE-4689. JobClient.getMapTaskReports on failed job results in NPE (jlowe via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1391679 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-28 22:17:32 +00:00
Jason Darrell Lowe
03b7ad04fa MAPREDUCE-4691. Historyserver can report "Unknown job" after RM says job has completed. Contributed by Robert Joseph Evans. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1391671 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-28 21:55:45 +00:00
Arun Murthy
1eedee177e MAPREDUCE-4649. Ensure MapReduce JobHistory Daemon doens't assume HADOOP_YARN_HOME and HADOOP_MAPRED_HOME are the same. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1390224 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-25 23:49:09 +00:00
Robert Joseph Evans
77805c7ae4 MAPREDUCE-4629. Remove JobHistory.DEBUG_MODE (Karthik Kambatla via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1382088 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-07 16:45:03 +00:00
Robert Joseph Evans
5dcb594cb8 MAPREDUCE-4633. history server doesn't set permissions on all subdirs (tgraves via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1381614 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-06 14:20:43 +00:00
Vinod Kumar Vavilapalli
65e447cd8b MAPREDUCE-4569. Fixed TestHsWebServicesJobsQuery to pass on JDK7 by not depending on test order. Contributed by Thomas Graves. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1379146 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-30 21:02:55 +00:00
Robert Joseph Evans
0c2887b617 MAPREDUCE-3506. Calling getPriority on JobInfo after parsing a history log with JobHistoryParser throws a NullPointerException (Jason Lowe via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1375602 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-21 15:20:21 +00:00
Thomas Graves
7fc6ad661d YARN-25. remove old aggregated logs (Robert Evans via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1374424 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-17 20:29:38 +00:00
Alejandro Abdelnur
4bca22005f HADOOP-8681. add support for HTTPS to the web UIs. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1371525 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-09 22:52:56 +00:00
Thomas Graves
a3e695ab2c MAPREDUCE-4375. Show Configuration Tracability in MR UI (bobby via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1367539 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-31 13:39:41 +00:00
Robert Joseph Evans
8e576570a9 MAPREDUCE-4283. Display tail of aggregated logs by default (Jason Lowe via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1362608 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-17 19:01:42 +00:00
Thomas Graves
11782dd3a5 MAPREDUCE-4300. OOM in AM can turn it into a zombie. (Robert Evans via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1359399 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-09 21:08:40 +00:00
Siddharth Seth
eff9fa1aad MAPREDUCE-3921. MR AM should act on node health status changes. Contributed by Bikas Saha. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1349065 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-11 23:14:06 +00:00
Thomas Graves
358eda0bfa MAPREDUCE-3350. Per-app RM page should have the list of application-attempts like on the app JHS page (Jonathon Eagles via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1346047 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-04 17:41:52 +00:00
Harsh J
1fab229f30 HADOOP-8268. A few pom.xml across Hadoop project may fail XML validation. Contributed by Radim Kolar. (harsh) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1343272 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-28 14:51:05 +00:00
Thomas Graves
7e3d016845 MAPREDUCE-4102. job counters not available in Jobhistory webui for killed jobs (Bhallamudi Venkata Siva Kamesh via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1339174 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-16 14:08:08 +00:00
Robert Joseph Evans
aa60da6c2e MAPREDUCE-4162. Correctly set token service (Daryn Sharp via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1335567 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-08 15:07:40 +00:00
Alejandro Abdelnur
0fca4fbaf3 MAPREDUCE-4205. retrofit all JVM shutdown hooks to use ShutdownHookManager (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1333748 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-04 03:20:16 +00:00
Robert Joseph Evans
48414b0827 MAPREDUCE-4163. consistently set the bind address (Daryn Sharp via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1333579 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-03 18:35:21 +00:00
Thomas Graves
e7583d816a MAPREDUCE-3613. web service calls header contains 2 content types (tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1330560 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-25 21:13:06 +00:00
Robert Joseph Evans
796e1a48ac MAPREDUCE-4190. Improve web UI for task attempts userlog link (Tom Graves via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1329398 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-23 19:38:28 +00:00
Robert Joseph Evans
1eb1d0ae4f MAPREDUCE-3812. Lower default allocation sizes, fix allocation configurations and document them (Harsh J via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1329282 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-23 15:26:35 +00:00
Alejandro Abdelnur
df654cca49 MAPREDUCE-3867. MiniMRYarn/MiniYarn uses fixed ports (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1328109 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-19 20:33:11 +00:00
Siddharth Seth
7d04a96027 MAPREDUCE-3972. Fix locking and exception issues in JobHistory server. (Contributed by Robert Joseph Evans) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1327354 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-18 01:59:16 +00:00
Thomas White
346d50782d MAPREDUCE-4140. mapreduce classes incorrectly importing "clover.org.apache.*" classes. Contributed by Patrick Hunt 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1325352 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-12 16:22:25 +00:00
Robert Joseph Evans
2accda38a1 MAPREDUCE-4017. Add jobname to jobsummary log (tgraves and Koji Noguchi via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1311972 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-10 20:36:09 +00:00
Thomas Graves
cbb5f61090 MAPREDUCE-4059. The history server should have a separate pluggable storage/query interface. (Robert Evans via tgraves). 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1311896 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-10 18:11:26 +00:00
Robert Joseph Evans
01ce7ba3a4 MAPREDUCE-4024. RM webservices can't query on finalStatus (Tom Graves via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1308566 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-02 21:06:16 +00:00
Siddharth Seth
db3e592df0 REDUCE-3955. Change MR to use ProtobufRpcEngine from hadoop-common instead of ProtoOverHadoopRpcEngine. (Contributed by Jitendra Nath Pandey) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1306689 13f79535-47bb-0310-9956-ffa450edef68
 2012-03-29 02:02:17 +00:00
Arun Murthy
67a74198b1 Changed version in trunk to 3.0.0-SNAPSHOT. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1306200 13f79535-47bb-0310-9956-ffa450edef68
 2012-03-28 07:37:06 +00:00
Alejandro Abdelnur
7609243e58 MAPREDUCE-4007. JobClient getJob(JobID) should return NULL if the job does not exist (for backwards compatibility) (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1300750 13f79535-47bb-0310-9956-ffa450edef68
 2012-03-14 22:02:53 +00:00
Robert Joseph Evans
c3a4de0ec0 MAPREDUCE-3961. Map/ReduceSlotMillis computation incorrect (Siddharth Seth via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1297788 13f79535-47bb-0310-9956-ffa450edef68
 2012-03-06 23:21:13 +00:00
Siddharth Seth
4ce5f6553f MAPREDUCE-3944. Change the history jobs/ webservice to return partial job info for a significant performance improvement. (Contributed by Robert Joseph Evans) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1297672 13f79535-47bb-0310-9956-ffa450edef68
 2012-03-06 19:52:00 +00:00
Vinod Kumar Vavilapalli
b73738d21e MAPREDUCE-3009. Fixed node link on JobHistory webapp. Contributed by hackaravarthy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1297149 13f79535-47bb-0310-9956-ffa450edef68
 2012-03-05 18:29:29 +00:00
Siddharth Seth
99bf2902b2 MAPREDUCE-3896. Add user information to the delegation token issued by the history server. (Contributed by Vinod Kumar Vavilapalli) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1296401 13f79535-47bb-0310-9956-ffa450edef68
 2012-03-02 19:28:04 +00:00
Siddharth Seth
3cbb9d20d0 MAPREDUCE-3903. Add support for mapreduce admin users. (Contributed by Thomas Graves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1295262 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-29 20:48:09 +00:00
Vinod Kumar Vavilapalli
7475e836dc MAPREDUCE-3901. Modified JobHistory records in YARN to lazily load job and task reports so as to improve UI response times. Contributed by Siddarth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1294417 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-28 00:32:19 +00:00
Vinod Kumar Vavilapalli
d83672d71c MAPREDUCE-2793. Reneabling commented out test from previous comment. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1293521 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-25 02:30:55 +00:00
Vinod Kumar Vavilapalli
7a082ec2bd MAPREDUCE-2793. Corrected AppIDs, JobIDs, TaskAttemptIDs to be of correct format on the web pages. Contributed by Bikas Saha. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1293517 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-25 02:03:59 +00:00
Thomas Graves
582b97c3e7 MAPREDUCE-3904. Job history produced with mapreduce.cluster.acls.enabled false can not be viewed with mapreduce.cluster.acls.enabled true (Jonathon Eagles via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1293456 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-24 22:30:47 +00:00
Vinod Kumar Vavilapalli
5ee495e6f3 MAPREDUCE-3634. Fixed all daemons to crash instead of hanging around when their EventHandlers get exceptions. (vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1291598 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-21 05:08:25 +00:00
Mahadev Konar
caed1cfaf9 MAPREDUCE-3723. TestAMWebServicesJobs & TestHSWebServicesJobs incorrectly asserting tests (Bhallamudi Venkata Siva Kamesh via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1240590 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-04 18:55:03 +00:00
Vinod Kumar Vavilapalli
fe1cf3b0ac MAPREDUCE-3417. Fixed job-access-controls to work with MR AM and JobHistoryServer web-apps. Contributed by Jonathan Eagles. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1240428 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-04 01:21:12 +00:00
Siddharth Seth
b4929bcf14 MAPREDUCE-3640. Allow AMRecovery to work with partial JobHistory files. (Contributed by Arun C Murthy) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1239402 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-01 23:33:58 +00:00
Mahadev Konar
8a1719e999 MAPREDUCE-3549. write api documentation for web service apis for RM, NM, mapreduce app master, and job history server (Thomas Graves via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1234222 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-21 00:53:02 +00:00
Mahadev Konar
c6923061d0 MAPREDUCE-3698. Client cannot talk to the history server in secure mode. (mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1234120 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-20 20:44:17 +00:00
Alejandro Abdelnur
10325d9732 MAPREDUCE-3582. Move successfully passing MR1 tests to MR2 maven tree.(ahmed via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1233090 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-18 22:10:12 +00:00
Mahadev Konar
5f79f180f6 MAPREDUCE-3553. Add support for data returned when exceptions thrown from web service apis to be in either xml or in JSON. (Thomas Graves via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1230330 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-11 23:59:23 +00:00
Mahadev Konar
bc4b1f48d3 MAPREDUCE-3380. Token infrastructure for running clients which are not kerberos authenticated. (mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1229855 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-11 01:50:10 +00:00
Vinod Kumar Vavilapalli
bc374626be MAPREDUCE-3299. Added AMInfo table to the MR AM job pages to list all the job-attempts when AM restarts and recovers. (Jonathan Eagles via vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1229766 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-10 22:26:45 +00:00
Vinod Kumar Vavilapalli
e8645636ce MAPREDUCE-3511. Removed a multitude of cloned/duplicate counters in the AM thereby reducing the AM heap size and preventing full GCs. (vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1229347 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-09 21:04:58 +00:00
Arun Murthy
7440a8aa37 MAPREDUCE-3548. Added more unit tests for MR AM & JHS web-services. Contributed by Thomas Graves. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1227801 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-05 20:01:20 +00:00
Vinod Kumar Vavilapalli
fcbad14a3d MAPREDUCE-3568. Optimized Job's progress calculations in MR AM. (vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1224995 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-27 19:54:51 +00:00
Amar Kamat
264d3b7dd0 MAPREDUCE-3349. Log rack-name in JobHistory for unsuccessful tasks. (Devaraj K and Amar Kamat via amarrk) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1221578 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-21 02:58:00 +00:00
Vinod Kumar Vavilapalli
b7ae5a6cb7 MAPREDUCE-3426. Fixed MR AM in uber mode to write map intermediate outputs in the correct directory to work properly in secure mode. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1213987 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-13 23:35:11 +00:00
Vinod Kumar Vavilapalli
0ea8570be5 MAPREDUCE-2863. Support web services for YARN and MR components. (Thomas Graves via vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1213975 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-13 23:05:56 +00:00
Mahadev Konar
94bf0dacba MAPREDUCE-3450. NM port info no longer available in JobHistory. (Siddharth Seth via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1208327 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-30 08:43:00 +00:00
Mahadev Konar
cc4a468113 MAPREDUCE-3413. RM web ui applications not sorted in any order by default. (Jonathan Eagles via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1206926 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-27 23:56:46 +00:00
Mahadev Konar
ac320f5046 MAPREDUCE-3324. Not All HttpServer tools links (stacks,logs,config,metrics) are accessible through all UI servers (Jonathan Eagles via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1201914 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-14 21:30:10 +00:00
Vinod Kumar Vavilapalli
9fe9f42c8f MAPREDUCE-3342. Fixed JobHistoryServer to also show the job's queue name. Contributed by Jonathan Eagles. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1199133 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-08 07:28:56 +00:00
Vinod Kumar Vavilapalli
c27601fefe MAPREDUCE-3297. Moved log related components into yarn-common so that HistoryServer and clients can use them without depending on the yarn-server-nodemanager module. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1196986 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-03 08:02:19 +00:00
Arun Murthy
0df4878033 MAPREDUCE-3337. Added missing license headers. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1196977 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-03 07:06:53 +00:00
Thomas White
8e04fa8b84 HADOOP-7782. Aggregate project javadocs. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1195817 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-01 04:47:10 +00:00
Mahadev Konar
68fa208b1c MAPREDUCE-3103. Implement Job ACLs for MRAppMaster. (mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1195761 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-01 01:51:56 +00:00
Arun Murthy
9db078212f MAPREDUCE-3035. Fixed MR JobHistory to ensure rack information is present. Contributed by chakravarthy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1195575 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-31 17:27:42 +00:00
Vinod Kumar Vavilapalli
a75c4cf4e4 MAPREDUCE-2696. Fixed NodeManager to cleanup logs in a thread when logs' aggregation is not enabled. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1195383 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-31 09:55:29 +00:00
Vinod Kumar Vavilapalli
47a381e306 MAPREDUCE-3146. Added a MR specific command line to dump logs for a given TaskAttemptID. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1195349 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-31 06:42:06 +00:00
Vinod Kumar Vavilapalli
670fa24b48 MAPREDUCE-2989. Modified JobHistory to link to task and AM logs from the JobHistoryServer. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190174 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-28 06:45:04 +00:00
Arun Murthy
34e6de8f85 MAPREDUCE-3258. Fixed AM & JobHistory web-ui to display counters properly. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190160 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-28 05:08:04 +00:00
Mahadev Konar
9b34900c89 MAPREDUCE-3209. Jenkins reports 160 FindBugs warnings (mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190140 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-28 02:42:11 +00:00
Arun Murthy
f799e61269 MAPREDUCE-3279. Fixed TestJobHistoryParsing which assumed user name to be mapred all the time. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1189670 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-27 08:31:44 +00:00
Arun Murthy
fffdf661e3 MAPREDUCE-2821. Added missing fields (resourcePerMap & resourcePerReduce) to JobSummary logs. Contributed by Mahadev Konar. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1188528 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-25 06:26:14 +00:00
Mahadev Konar
d19cfe0164 MAPREDUCE-2746. Yarn servers can't communicate with each other with hadoop.security.authorization set to true (acmurthy via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1188522 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-25 06:07:13 +00:00
Arun Murthy
c70ee2488c MAPREDUCE-3248. Fixed log4j properties. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1188476 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-25 01:03:08 +00:00
Mahadev Konar
4186121c08 MAPREDUCE-3239. Use new createSocketAddr API in MRv2 to give better error messages on misconfig (Todd Lipcon via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1187556 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-21 21:36:58 +00:00
Vinod Kumar Vavilapalli
13e4562924 MAPREDUCE-3144. Augmented JobHistory with the information needed for serving aggregated logs. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1185976 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-19 05:21:18 +00:00
Vinod Kumar Vavilapalli
3e3abcb769 MAPREDUCE-3199. Fixed pom files to include correct log4j configuration for tests. (vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1185967 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-19 04:30:27 +00:00
Alejandro Abdelnur
15a1d9af19 MAPREDUCE-3003. Publish MR JARs to Maven snapshot repository. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1185286 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-17 17:06:42 +00:00
Mahadev Konar
68328ae926 MAPREDUCE-3187. Add names for various unnamed threads in MR2. (Todd Lipcon and Siddharth Seth via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1184904 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-16 19:27:02 +00:00
Mahadev Konar
b5e21e7e63 MAPREDUCE-3057. Job History Server goes of OutOfMemory with 1200 Jobs and Heap Size set to 10 GB. (Eric Payne via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1183545 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-14 22:49:19 +00:00
Arun Murthy
277e520579 MAPREDUCE-2666. Retrieve shuffle port number from JobHistory on MR AM restart. Contributed by Jonathan Eagles. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1182613 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-12 23:12:01 +00:00
Alejandro Abdelnur
4972959173 Reverted MAPREDUCE-3003 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1180283 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-08 00:38:41 +00:00
Alejandro Abdelnur
68d1162f7a MAPREDUCE-3003. Publish Yarn and MapReduce artifacts to Maven snapshot repository. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1180155 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-07 19:01:01 +00:00
Vinod Kumar Vavilapalli
66137cf17c MAPREDUCE-3013. Removed YarnConfiguration.YARN_SECURITY_INFO and its usage as it doesn't affect security any more. (vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1179174 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-05 11:43:05 +00:00
Arun Murthy
964d4a1666 MAPREDUCE-2996. Add uber-ness information to JobHistory. Contributed by Jonathan Eagles. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1177531 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-30 06:47:53 +00:00