big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
13,648 Commits 2 Branches 2 Tags 264 MiB
bddea5fe5f
Commit Graph

2376 Commits

Author SHA1 Message Date
Ming Ma
4a1cedc010 MAPREDUCE-5044. Have AM trigger jstack on task attempts that timeout before killing them. (Eric Payne and Gera Shegalov via mingma) 2016-06-06 14:30:51 -07:00
Arun Suresh
dc26601d8f YARN-5180. Allow ResourceRequest to specify an enforceExecutionType flag. (asuresh) 2016-06-02 09:01:02 -07:00
Kai Zheng
93d8a7f2a2 Revert "HADOOP-12579. Deprecate and remove WriteableRPCEngine. Contributed by Kai Zheng" 
This reverts commit a6c79f92d5.
 2016-06-01 08:41:15 +08:00
Varun Vasudev
42f90ab885 YARN-4844. Add getMemorySize/getVirtualCoresSize to o.a.h.y.api.records.Resource. Contributed by Wangda Tan. 2016-05-29 21:24:16 +05:30
Carlo Curino
013532a95e YARN-4957. Add getNewReservation in ApplicationClientProtocol (Sean Po via curino) 2016-05-25 16:55:49 -07:00
Kai Zheng
a6c79f92d5 HADOOP-12579. Deprecate and remove WriteableRPCEngine. Contributed by Kai Zheng 2016-05-27 04:29:42 +08:00
Jian He
ae353ea969 MAPREDUCE-6703. Add flag to allow MapReduce AM to request for OPPORTUNISTIC containers. Contributed by Arun Suresh 2016-05-24 19:47:50 -07:00
Akira Ajisaka
3b3b63081b MAPREDUCE-6607. Enable regex pattern matching when mapreduce.task.files.preserve.filepattern is set. Contributed by Kai Sasaki. 2016-05-23 07:24:31 +09:00
Jian He
21d2b90213 MAPREDUCE-6696. Add a configuration to limit the number of map tasks allowed per job. Contributed by Zhihai Xu 2016-05-19 19:25:16 -07:00
Junping Du
22fcd819f0 YARN-5076. YARN web interfaces lack XFS protection. Contributed by Jonathan Maron. 
(cherry picked from commit 2703ec68712279494d67b0d76b7ac10e7a1628be)
 2016-05-19 14:15:21 -07:00
Robert Kanter
992a49353f MAPREDUCE-6686. Add a way to download the job config from the mapred CLI (rkanter) 2016-05-18 12:16:21 -07:00
Junping Du
f6ef876fe1 MAPREDUCE-6657. Job history server can fail on startup when NameNode is in start phase. Contributed by Haibo Chen. 2016-05-17 14:41:47 -07:00
Jason Lowe
34fddd1e91 MAPREDUCE-6698. Increase timeout on TestUnnecessaryBlockingOnHistoryFileInfo.testTwoThreadsQueryingDifferentJobOfSameUser. Contributed by Haibo Chen 2016-05-17 15:01:33 +00:00
Jason Lowe
12fa4ec141 MAPREDUCE-6701. application master log can not be available when clicking jobhistory's am logs link. Contributed by Haibo Chen 2016-05-17 14:54:13 +00:00
Akira Ajisaka
b357930526 MAPREDUCE-6693. ArrayIndexOutOfBoundsException occurs when the length of the job name is equal to mapreduce.jobhistory.jobname.limit. Contributed by Ajith S. 2016-05-17 20:19:01 +09:00
Allen Wittenauer
730bc746f9 HADOOP-12930. Dynamic subcommands for hadoop shell scripts (aw) 
This commit contains the following JIRA issues:

    HADOOP-12931. bin/hadoop work for dynamic subcommands
    HADOOP-12932. bin/yarn work for dynamic subcommands
    HADOOP-12933. bin/hdfs work for dynamic subcommands
    HADOOP-12934. bin/mapred work for dynamic subcommands
    HADOOP-12935. API documentation for dynamic subcommands
    HADOOP-12936. modify hadoop-tools to take advantage of dynamic subcommands
    HADOOP-13086. enable daemonization of dynamic commands
    HADOOP-13087. env var doc update for dynamic commands
    HADOOP-13088. fix shellprofiles in hadoop-tools to allow replacement
    HADOOP-13089. hadoop distcp adds client opts twice when dynamic
    HADOOP-13094. hadoop-common unit tests for dynamic commands
    HADOOP-13095. hadoop-hdfs unit tests for dynamic commands
    HADOOP-13107. clean up how rumen is executed
    HADOOP-13108. dynamic subcommands need a way to manipulate arguments
    HADOOP-13110. add a streaming subcommand to mapred
    HADOOP-13111. convert hadoop gridmix to be dynamic
    HADOOP-13115. dynamic subcommand docs should talk about exit vs. continue program flow
    HADOOP-13117. clarify daemonization and security vars for dynamic commands
    HADOOP-13120. add a --debug message when dynamic commands have been used
    HADOOP-13121. rename sub-project shellprofiles to match the rest of Hadoop
    HADOOP-13129. fix typo in dynamic subcommand docs
    HADOOP-13151. Underscores should be escaped in dynamic subcommands document
    HADOOP-13153. fix typo in debug statement for dynamic subcommands
 2016-05-16 17:54:45 -07:00
Jason Lowe
9227dfc25f MAPREDUCE-6558. multibyte delimiters with compressed input files generate duplicate records. Contributed by Wilfred Spiegelenburg 2016-05-13 14:33:48 +00:00
Andrew Wang
3c5c57af28 HADOOP-13142. Change project version from 3.0.0 to 3.0.0-alpha1. 2016-05-12 18:27:28 -07:00
Andrew Wang
ca5613af91 Revert "Update project version to 3.0.0-alpha1-SNAPSHOT." 
This reverts commit 6b53802cba.
 2016-05-12 15:32:45 -07:00
Andrew Wang
6b53802cba Update project version to 3.0.0-alpha1-SNAPSHOT. 2016-05-12 11:05:05 -07:00
Steve Loughran
7eddecd357 MAPREDUCE-6639 Process hangs in LocatedFileStatusFetcher if FileSystem.get throws. Ryan Blue via stevel 2016-05-12 19:00:32 +01:00
Jason Lowe
5ffb54694b MAPREDUCE-6684. High contention on scanning of user directory under immediate_done in Job History Server. Contributed by Haibo Chen 2016-05-10 16:03:49 +00:00
Eric Payne
cd35b692de MAPREDUCE-6678. Allow ShuffleHandler readahead without drop-behind. Contributed by Nathan Roberts. 2016-05-10 15:45:01 +00:00
Jason Lowe
c9bb96fa81 MAPREDUCE-6689. MapReduce job can infinitely increase number of reducer resource requests. Contributed by Wangda Tan 2016-05-06 22:25:47 +00:00
Vinod Kumar Vavilapalli
8d48266720 MAPREDUCE-6514. Fixed MapReduce ApplicationMaster to properly updated resources ask after ramping down of all reducers avoiding job hangs. Contributed by Varun Saxena and Wangda Tan. 2016-05-05 19:03:27 -07:00
Robert Kanter
9d3fcdfbb3 MAPREDUCE-6675. TestJobImpl.testUnusableNode failed (haibochen via rkanter) 2016-05-04 22:42:21 -07:00
Robert Kanter
ecce3b7d53 MAPREDUCE-6677. LocalContainerAllocator doesn't specify resource of the containers allocated (haibochen via rkanter) 2016-05-04 22:19:20 -07:00
Akira Ajisaka
4ee4e5ca2b MAPREDUCE-6526. Remove usage of metrics v1 from hadoop-mapreduce. (aajisaka) 2016-05-03 10:46:11 +09:00
Chris Nauroth
26ac0f8f0b MAPREDUCE-6672. TestTeraSort fails on Windows. Contributed by Tibor Kiss. 2016-04-29 11:33:32 -07:00
Jing Zhao
63e5412f1a HDFS-9427. HDFS should not default to ephemeral ports. Contributed by Xiaobing Zhou. 2016-04-22 15:14:40 -07:00
Harsh J
7da5847cf1 MAPREDUCE-2398. MRBench: setting the baseDir parameter has no effect. Contributed by Michael Noll and Wilfred Spiegelenburg. 2016-04-21 17:53:51 +05:30
Jian He
1e48eefe58 MAPREDUCE-6680. JHS UserLogDir scan algorithm sometime could skip directory with update in CloudFS (Azure FileSystem, S3, etc. Contributed by Junping Du 2016-04-20 19:02:10 -07:00
Xuan
e6c0742012 YARN-4965. Distributed shell AM failed due to ClientHandlerException thrown by jersey. Contributed by Junping Du 2016-04-16 19:39:18 -07:00
Eric Payne
cc8b83a8e8 MAPREDUCE-6649. getFailureInfo not returning any failure info. Contributed by Eric Badger 2016-04-16 22:07:27 +00:00
Wangda Tan
8b2880c0b6 MAPREDUCE-6513. MR job got hanged forever when one NM unstable for some time. (Varun Saxena via wangda) 2016-04-14 11:00:53 -07:00
Eric Payne
1fec06e037 MAPREDUCE-6633. AM should retry map attempts if the reduce task encounters commpression related errors. Contributed by Rushabh Shah 2016-04-09 16:51:57 +00:00
Robert Kanter
3be1ab485f MAPREDUCE-6647. MR usage counters use the resources requested instead of the resources allocated (haibochen via rkanter) 2016-04-06 17:15:43 -07:00
Junping Du
de96d7c88a MAPREDUCE-6670. TestJobListCache#testEviction sometimes fails on Windows with timeout. Contributed by Gergely Novák. 2016-04-06 08:32:35 -07:00
Allen Wittenauer
0a74610d1c HADOOP-11393. Revert HADOOP_PREFIX, go back to HADOOP_HOME (aw) 2016-03-31 07:51:05 -07:00
Akira Ajisaka
690d8a368d MAPREDUCE-6663. [NNBench] Refactor nnbench as a Tool implementation. Contributed by Brahma Reddy Battula. 2016-03-30 11:42:54 +09:00
Akira Ajisaka
0050fa5f1c MAPREDUCE-6543. Migrate MR client test cases part 2. Contributed by Dustin Cote. 2016-03-29 18:17:52 +09:00
Rohith Sharma K S
0ef8bbfd87 MAPREDUCE-6662. Clear ASF Warnings on test data files. Contributed by Vinayakumar B 2016-03-29 10:32:21 +08:00
Junping Du
90fcb16c0f MAPREDUCE-6655. Fix a typo (STRICT_IE6) in Encrypted Shuffle. Contributed by Wei-Chiu Chuang. 2016-03-28 05:30:42 -07:00
Akira Ajisaka
115be193df Revert "MAPREDUCE-6543. Migrate MR client test cases part 2. Contributed by Dustin Cote." 
This reverts commit 2c268cc936.
 2016-03-28 16:05:35 +09:00
Jason Lowe
3f622a143c MAPREDUCE-6535. TaskID default constructor results in NPE on toString(). Contributed by Daniel Templeton 2016-03-25 20:04:32 +00:00
Akira Ajisaka
2c268cc936 MAPREDUCE-6543. Migrate MR client test cases part 2. Contributed by Dustin Cote. 2016-03-25 15:17:27 +09:00
Allen Wittenauer
738155063e HADOOP-12857. rework hadoop-tools (aw) 2016-03-23 13:46:38 -07:00
Akira Ajisaka
368c77376a MAPREDUCE-6656. [NNBench] OP_DELETE operation isn't working after MAPREDUCE-6363. Contributed by J.Andreina. 2016-03-23 23:22:43 +09:00
Akira Ajisaka
0bfe5a0167 MAPREDUCE-6580. Test failure: TestMRJobsWithProfiler. Contributed by Eric Badger. 2016-03-23 13:18:06 +09:00
Li Lu
57ead18a85 MAPREDUCE-6110. JobHistoryServer CLI throws NullPointerException with job ids that do not exist. (Kai Sasaki via gtcarrera9) 2016-03-22 14:27:32 -07:00