big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
8,286 Commits 2 Branches 2 Tags 264 MiB
d51f81c3b6
Commit Graph

1087 Commits

Author SHA1 Message Date
Jason Darrell Lowe
d51f81c3b6 MAPREDUCE-6036. TestJobEndNotifier fails intermittently in branch-2. Contributed by chang li 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618691 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-18 18:18:22 +00:00
Zhijie Shen
f8e871d01b MAPREDUCE-6024. Shortened the time when Fetcher is stuck in retrying before concluding the failure by configuration. Contributed by Yunjiong Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618677 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-18 17:57:48 +00:00
Eric Yang
08d6201b34 MAPREDUCE-6033. Updated access check for displaying job information 
(Yu Gao via Eric Yang)



git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618648 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-18 16:13:56 +00:00
Zhijie Shen
84bc2fe402 MAPREDUCE-6032. Made MR jobs write job history files on the default FS when the current context’s FS is different. Contributed by Benjamin Zhitomirsky. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618269 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-15 20:17:45 +00:00
Jason Darrell Lowe
7378af7bca MAPREDUCE-5878. some standard JDK APIs are not part of system classes defaults. Contributed by Sangjin Lee 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618080 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-14 23:17:33 +00:00
Allen Wittenauer
2d904b3fa3 MAPREDUCE-5906. Inconsistent configuration in property "mapreduce.reduce.shuffle.input.buffer.percent" (Akira AJISAKA via aw) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618007 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-14 18:07:06 +00:00
Allen Wittenauer
e1f15d2cdf MAPREDUCE-5999. Fix dead link in InputFormat javadoc (Akira AJISAKA via aw) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618003 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-14 17:56:53 +00:00
Allen Wittenauer
7557f7bb2d MAPREDUCE-5998. CompositeInputFormat javadoc is broken (Akira AJISAKA via aw) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617995 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-14 17:18:14 +00:00
Allen Wittenauer
6be98091f7 MAPREDUCE-5950. incorrect description in distcp2 document (Akira AJISAKA via aw) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617994 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-14 17:11:39 +00:00
Allen Wittenauer
92054f2a06 MAPREDUCE-5597. Missing alternatives in javadocs for deprecated constructors in mapreduce.Job (Akira AJISAKA via aw) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617986 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-14 16:00:18 +00:00
Jason Darrell Lowe
9579554988 MAPREDUCE-6010. HistoryServerFileSystemStateStore fails to update tokens. Contributed by Jason Lowe 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617984 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-14 15:55:03 +00:00
Allen Wittenauer
e7b540eced MAPREDUCE-5363. Fix doc and spelling for TaskCompletionEvent#getTaskStatus and getStatus (Akira AJISAKA via aw) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617982 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-14 15:49:10 +00:00
Allen Wittenauer
1e016e6451 MAPREDUCE-4791. Javadoc for KeyValueTextInputFormat should include default separator and how to change it (Akira AJISAKA via aw) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617979 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-14 15:43:28 +00:00
Allen Wittenauer
b3df973adb MAPREDUCE-883. harchive: Document how to unarchive (Akira AJISAKA and Koji Noguchi via aw) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617977 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-14 15:39:52 +00:00
Allen Wittenauer
e51a5d691b MAPREDUCE-5943. Separate mapred commands from CommandManual.apt.vm (Akira AJISAKA via aw) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617806 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-13 19:34:26 +00:00
Xuan Gong
eeb4acd955 YARN-2212: ApplicationMaster needs to find a way to update the AMRMToken periodically. Contributed by Xuan Gong 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616892 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-08 21:38:24 +00:00
Jason Darrell Lowe
de2595833c MAPREDUCE-6021. MR AM should have working directory in LD_LIBRARY_PATH. Contributed by Jason Lowe 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616585 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-07 20:15:56 +00:00
Jason Darrell Lowe
857d134fda MAPREDUCE-6014. New task status field in task attempts table can lead to an empty web page. Contributed by Mit Desai 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616018 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-05 21:52:22 +00:00
Xuan Gong
e52f67e389 YARN-1994. Expose YARN/MR endpoints on multiple interfaces. Contributed by Craig Welch, Milan Potocnik,and Arpit Agarwal 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1614981 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-31 20:06:02 +00:00
Zhijie Shen
1d6e178144 YARN-2347. Consolidated RMStateVersion and NMDBSchemaVersion into Version in yarn-server-common. Contributed by Junping Du. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1614838 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-31 09:27:43 +00:00
Zhijie Shen
549bcc2c02 MAPREDUCE-6002. Made MR task avoid reporting error to AM when the task process is shutting down. Contributed by Wangda Tan. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1613743 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-27 01:37:51 +00:00
Jason Darrell Lowe
60b1e835e0 MAPREDUCE-5963. ShuffleHandler DB schema should be versioned with compatible/incompatible changes. Contributed by Junping Du 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612652 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-22 19:22:24 +00:00
Jason Darrell Lowe
c2174a5536 MAPREDUCE-5756. CombineFileInputFormat.getSplits() including directories in its results. Contributed by Jason Dere 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612400 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-21 21:24:15 +00:00
Jason Darrell Lowe
9e62bcca4e MAPREDUCE-5957. AM throws ClassNotFoundException with job classloader enabled if custom output format/committer is used. Contributed by Sangjin Lee 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612358 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-21 17:54:46 +00:00
Jian He
875592220f MAPREDUCE-5910. Make MR AM resync with RM in case of work-preserving RM-restart. Contributed by Rohith 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1611434 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-17 18:46:20 +00:00
Karthik Kambatla
43a12f3c01 MAPREDUCE-5952. LocalContainerLauncher#renameMapOutputForReduce incorrectly assumes a single dir for mapOutIndex. (Gera Shegalov via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1611196 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-16 21:24:51 +00:00
Vinod Kumar Vavilapalli
b60ef8b91b MAPREDUCE-5790. Made it easier to enable hprof profile options by default. Contributed by Gera Shegalov. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1610578 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-15 04:49:01 +00:00
Zhijie Shen
64306aa1b5 MAPREDUCE-5956. Made MR AM not use maxAttempts to determine if the current attempt is the last retry. Contributed by Wangda Tan. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1609649 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-11 08:45:40 +00:00
Vinod Kumar Vavilapalli
4b2ded8202 MAPREDUCE-5517. Fixed MapReduce ApplicationMaster to not validate reduce side resource configuration for deciding uber-mode on map-only jobs. Contributed by Siqi Li. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1608595 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-07 20:14:01 +00:00
Chris Nauroth
286ae1edd0 MAPREDUCE-5866. TestFixedLengthInputFormat fails in windows. Contributed by Varun Vasudev. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1608585 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-07 19:56:44 +00:00
Vinod Kumar Vavilapalli
d85f90e5b3 MAPREDUCE-5868. Fixed an issue with TestPipeApplication that was causing the nightly builds to fail. Contributed by Akira Ajisaka. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1608579 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-07 19:45:57 +00:00
Zhijie Shen
c6a09d2110 MAPREDUCE-5900. Changed to the interpret container preemption exit code as a task attempt killing event. Contributed by Mayank Bansal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1607512 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-03 01:43:56 +00:00
Jason Darrell Lowe
cfd9344540 MAPREDUCE-5939. StartTime showing up as the epoch time in JHS UI after upgrade. Contributed by Chen He 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1605892 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-26 19:56:09 +00:00
Vinod Kumar Vavilapalli
e285b98f0f YARN-2152. Added missing information into ContainerTokenIdentifier so that NodeManagers can report the same to RM when RM restarts. Contributed Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1605205 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-24 21:43:22 +00:00
Vinod Kumar Vavilapalli
1355ff6132 MAPREDUCE-5830. Added back the private API HostUtil.getTaskLogUrl(..) for binary compatibility with older clients like Hive 0.13. Contributed by Akira Ajisaka. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1604230 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-20 18:28:21 +00:00
Karthik Kambatla
7b9c074b76 MAPREDUCE-5844. Add a configurable delay to reducer-preemption. (Maysam Yabandeh via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1603957 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-19 17:22:56 +00:00
Karthik Kambatla
bd23a2ff22 MAPREDUCE-5896. InputSplits should indicate which locations have the block cached in memory. (Sandy Ryza via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1603670 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-18 23:28:50 +00:00
Jian He
0abddac476 MAPREDUCE-5924. Changed TaskAttemptImpl to ignore TA_COMMIT_PENDING event at COMMIT_PENDING state. Contributed by Zhijie Shen 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1603338 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-18 00:51:39 +00:00
Chris Nauroth
180ac8dc1d MAPREDUCE-5912. Task.calculateOutputSize does not handle Windows files after MAPREDUCE-5196. Contributed by Remus Rusanu. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1602282 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-12 20:02:45 +00:00
Karthik Kambatla
fc7c8f9bf2 HADOOP-10686. Writables are not always configured. (Abraham Elmahrek via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1602079 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-12 07:23:20 +00:00
Chris Nauroth
473e83ab71 MAPREDUCE-5920. Add Xattr option in DistCp docs. Contributed by Yi Liu. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1601983 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-11 18:36:06 +00:00
Karthik Kambatla
40ba8c17c1 MAPREDUCE-5777. Support utf-8 text with Byte Order Marker. (Zhihai Xu via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1600977 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-06 18:37:03 +00:00
Chris Nauroth
806f4d11f2 HADOOP-10602. Documentation has broken "Go Back" hyperlinks. Contributed by Akira AJISAKA. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598337 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-29 17:26:50 +00:00
Devarajulu K
61c59c0704 MAPREDUCE-5895. Close streams properly to avoid leakage in TaskLog. Contributed by Kousuke Saruta. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598209 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-29 07:05:26 +00:00
Jason Darrell Lowe
4bb4de93d6 MAPREDUCE-5862. Line records longer than 2x split size aren't handled correctly. Contributed by bc Wong 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598111 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-28 19:37:52 +00:00
Jason Darrell Lowe
1f8bb1001a MAPREDUCE-5309. 2.0.4 JobHistoryParser can't parse certain failed job history files generated by 2.0.3 history server. Contributed by Rushabh S Shah 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1596295 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-20 15:46:54 +00:00
Andrew Wang
4a839f0cc1 Fix CRLFs in TestKillAMPreemptionPolicy.java 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1596058 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-19 21:18:55 +00:00
Devarajulu K
f74e446355 MAPREDUCE-5867. Fix NPE in KillAMPreemptionPolicy related to ProportionalCapacityPreemptionPolicy. Contributed by Sunil G. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1595754 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-19 06:42:29 +00:00
Ravi Prakash
c9377189a0 MAPREDUCE 5874. Creating MapReduce REST API section. Contributed by Tsuyoshi Ozawa 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1595265 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-16 16:50:58 +00:00
Junping Du
1e5d2c612b MAPREDUCE-5861. finishedSubMaps field in LocalContainerLauncher does not need to be volatile. (Contributed by Tsuyoshi OZAWA) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1594795 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-15 07:07:18 +00:00