big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
3,886 Commits 2 Branches 2 Tags 264 MiB
1195f844a9
Commit Graph

505 Commits

Author SHA1 Message Date
Vinod Kumar Vavilapalli
1195f844a9 MAPREDUCE-4733. Reducer can fail to make progress during shuffle if too many reducers complete consecutively. Contributed by Jason Lowe via. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1400264 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-19 20:18:46 +00:00
Robert Joseph Evans
a234472d1c MAPREDUCE-4479. Fix parameter order in assertEquals() in TestCombineInputFileFormat.java (Mariappan Asokan via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1400192 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-19 17:51:33 +00:00
Vinod Kumar Vavilapalli
022f7b4a25 MAPREDUCE-4596. Split StateMachine state from states seen by MRClientProtocol for Job, Task and TaskAttempt. Contributed by Siddarth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1399976 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-19 05:57:57 +00:00
Suresh Srinivas
adcf7d10b9 MAPREDUCE-4735. Make arguments in TestDFSIO case insensitive. Contributed by Brandon Li. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1399952 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-19 02:37:59 +00:00
Robert Joseph Evans
c63811790a MAPREDUCE-4721. Task startup time in JHS is same as job startup time. (Ravi Prakash via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1399289 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-17 14:58:08 +00:00
Robert Joseph Evans
0e9f40df97 MAPREDUCE-4521. mapreduce.user.classpath.first incompatibility with 0.20/1.x (Ravi Prakash via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1399011 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-16 22:00:03 +00:00
Suresh Srinivas
94ed14c48c HADOOP-8911. CRLF characters in source and text files. Contributed Raja Aluri. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1397432 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-12 04:35:42 +00:00
Arun Murthy
f174adfc15 MAPREDUCE-4616. Improve javadoc for MultipleOutputs. Contributed by Tony Burton. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1397182 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-11 17:21:45 +00:00
Harsh J
0aa8188d18 MAPREDUCE-3678. The Map tasks logs should have the value of input split it processed. Contributed by Harsh J. (harsh) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1396032 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-09 13:39:26 +00:00
Harsh J
5c3a331040 MAPREDUCE-4574. Fix TotalOrderParitioner to work with non-WritableComparable key types. Contributed by Harsh J. (harsh) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1395936 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-09 09:58:21 +00:00
Siddharth Seth
8ac3910ae0 MAPREDUCE-4705. Fix a bug in job history lookup, which makes older jobs inaccessible despite the presence of a valid history file. (Contributed by Jason Lowe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1395850 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-09 03:23:07 +00:00
Siddharth Seth
c074cfd6f0 YARN-134. Fixes ClientToAMSecretManager creates keys without checking for validity of the appID. (Contributed by Vinod Kumar Vavilapalli) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1395841 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-09 01:56:05 +00:00
Robert Joseph Evans
49b20c2ed1 MAPREDUCE-4554. Job Credentials are not transmitted if security is turned off (Benoy Antony via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1395769 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-08 20:50:40 +00:00
Harsh J
7137503eb7 MAPREDUCE-4695. Fix LocalRunner on trunk after MAPREDUCE-3223 broke it. Contributed by Harsh J. (harsh) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1392446 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-01 17:10:11 +00:00
Robert Joseph Evans
b897d6c35a MAPREDUCE-4689. JobClient.getMapTaskReports on failed job results in NPE (jlowe via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1391679 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-28 22:17:32 +00:00
Jason Darrell Lowe
03b7ad04fa MAPREDUCE-4691. Historyserver can report "Unknown job" after RM says job has completed. Contributed by Robert Joseph Evans. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1391671 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-28 21:55:45 +00:00
Harsh J
6d77365a83 MAPREDUCE-3223. Remove MR1 configs from mapred-default.xml. Contributed by Todd Lipcon (harsh) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1391498 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-28 15:08:48 +00:00
Arun Murthy
036530fc12 Reverted MAPREDUCE-4253. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1390652 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-26 18:41:08 +00:00
Thomas Graves
050fd3a117 MAPREDUCE-4647. We should only unjar jobjar if there is a lib directory in it. (Robert Evans via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1390557 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-26 15:22:21 +00:00
Harsh J
462ba6a3de MAPREDUCE-4686. hadoop-mapreduce-client-core fails compilation in Eclipse due to missing Avro-generated classes. Contributed by Chris Nauroth. (harsh) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1390446 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-26 12:52:41 +00:00
Arun Murthy
1eedee177e MAPREDUCE-4649. Ensure MapReduce JobHistory Daemon doens't assume HADOOP_YARN_HOME and HADOOP_MAPRED_HOME are the same. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1390224 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-25 23:49:09 +00:00
Konstantin Shvachko
33a3efcd18 MAPREDUCE-4651. Benchmarking random reads with DFSIO. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1390159 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-25 21:32:29 +00:00
Konstantin Shvachko
b26ed9d6f8 MAPREDUCE-4645. Provide a random seed to Slive to make the sequence of file names deterministic. Contributed by Ravi Prakash. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1389568 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-24 20:54:20 +00:00
Thomas White
d89828573d HADOOP-8780. Update DeprecatedProperties apt file. Contributed by Ahmed Radwan 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1384833 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-14 16:05:30 +00:00
Aaron Myers
d8e386f01a HADOOP-8755. Print thread dump when tests fail due to timeout. Contributed by Andrey Klochkov. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1384627 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-14 01:44:23 +00:00
Vinod Kumar Vavilapalli
229a79bbc3 MAPREDUCE-4646. Fixed MR framework to send diagnostic information correctly to clients in case of failed jobs also. Contributed by Jason Lowe. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1383709 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-12 00:59:15 +00:00
Eli Collins
dc1fd84bd1 HDFS-3910. DFSTestUtil#waitReplication should timeout. Contributed by Eli Collins 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1383618 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-11 21:24:12 +00:00
Thomas White
3b46295c28 MAPREDUCE-4607. Race condition in ReduceTask completion can result in Task being incorrectly failed. Contributed by Bikas Saha. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1383422 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-11 14:04:03 +00:00
Alejandro Abdelnur
a7998921a7 MAPREDUCE-4642. MiniMRClientClusterFactory should not use job.setJar() (rkanter via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1382093 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-07 17:05:11 +00:00
Robert Joseph Evans
77805c7ae4 MAPREDUCE-4629. Remove JobHistory.DEBUG_MODE (Karthik Kambatla via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1382088 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-07 16:45:03 +00:00
Arun Murthy
0056914773 MAPREDUCE-4638. MR AM supplies MapReduce jars in classpath rather than rely on YARN. Contributed by Arun C Murthy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1381868 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-07 04:56:32 +00:00
Robert Joseph Evans
09038b741d MAPREDUCE-4641. Exception in commitJob marks job as successful in job history (Jason Lowe via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1381784 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-06 22:20:15 +00:00
Robert Joseph Evans
5dcb594cb8 MAPREDUCE-4633. history server doesn't set permissions on all subdirs (tgraves via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1381614 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-06 14:20:43 +00:00
Eli Collins
aa049397f1 Fix MAPREDUCE-4580 build breakage. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1381315 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-05 19:34:03 +00:00
Suresh Srinivas
60af999c73 YARN-84. Use Builder to build RPC server. Contributed by Brandon Li 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1380921 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-04 22:24:04 +00:00
Konstantin Shvachko
ab986d7cf6 MAPREDUCE-2786. Add compression option for TestDFSIO. Contributed by Plamen Jeliazkov. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1380310 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-03 18:54:27 +00:00
Jonathan Turner Eagles
94129df354 MAPREDUCE-4604. In mapred-default, mapreduce.map.maxattempts & mapreduce.reduce.maxattempts defaults are set to 4 as well as mapreduce.job.maxtaskfailures.per.tracker. (Ravi Prakash via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1379617 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-31 21:25:25 +00:00
Thomas Graves
25e96e455b MAPREDUCE-4611. MR AM dies badly when Node is decommissioned (Robert Evans via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1379599 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-31 20:43:46 +00:00
Robert Joseph Evans
63f941d2ad MAPREDUCE-4614. Simplify debugging a job's tokens (daryn via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1379595 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-31 20:37:41 +00:00
Robert Joseph Evans
015256524c MAPREDUCE-4612. job summary file permissions not set when its created (tgraves via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1379584 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-31 20:25:27 +00:00
Vinod Kumar Vavilapalli
65e447cd8b MAPREDUCE-4569. Fixed TestHsWebServicesJobsQuery to pass on JDK7 by not depending on test order. Contributed by Thomas Graves. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1379146 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-30 21:02:55 +00:00
Robert Joseph Evans
6f6e170325 HADOOP-8726. The Secrets in Credentials are not available to MR tasks (daryn and Benoy Antony via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1379100 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-30 19:58:07 +00:00
Thomas White
2f1a133dcf MAPREDUCE-4610. Support deprecated mapreduce.job.counters.limit property in MR2. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1379022 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-30 17:12:14 +00:00
Thomas White
95af351615 MAPREDUCE-4608. hadoop-mapreduce-client is missing some dependencies. Contributed by Alejandro Abdelnur. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1378873 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-30 09:57:45 +00:00
Vinod Kumar Vavilapalli
cfe25b8828 MAPREDUCE-4579. Split TestTaskAttempt into two so as to pass tests on jdk7. Contributed by Thomas Graves. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1377943 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-28 02:03:26 +00:00
Siddharth Seth
24e47ebc18 MAPREDUCE-4580. Change MapReduce to use the yarn-client module. (Contributed by Vinod Kumar Vavilapalli) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1377922 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-28 00:40:02 +00:00
Siddharth Seth
ffd2e01604 YARN-39. RM-NM secret-keys should be randomly generated and rolled every so often. (Contributed by Vinod Kumar Vavilapalli and Siddharth Seth) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1377180 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-25 02:18:49 +00:00
Alejandro Abdelnur
7b541d619f MAPREDUCE-4408. allow jobs to set a JAR that is in the distributed cached (rkanter via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1377149 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-24 23:36:55 +00:00
Robert Joseph Evans
aeb364d8b5 HADOOP-8225. DistCp fails when invoked by Oozie (daryn via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1376618 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-23 18:06:56 +00:00
Alejandro Abdelnur
f909a1d4d7 MAPREDUCE-4470. Fix TestCombineFileInputFormat.testForEmptyFile (ikatsov via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1376325 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-23 00:22:14 +00:00