big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
13,945 Commits 2 Branches 2 Tags 264 MiB
6934b05c71
Commit Graph

2401 Commits

Author SHA1 Message Date
Naganarasimha
6934b05c71 YARN-4238. createdTime and modifiedTime is not reported while publishing entities to ATSv2. (Varun Saxena via Naganarasimha G R) 2016-07-10 08:45:52 -07:00
Li Lu
8ef546c1ee YARN-4445. Unify the term flowId and flowName in timeline v2 codebase. 
Contributed by Zhan Zhang.
 2016-07-10 08:45:49 -07:00
Li Lu
89e5c44f9e YARN-4356. Ensure the timeline service v.2 is disabled cleanly and has no 
impact when it's turned off. Contributed by Sangjin Lee.
 2016-07-10 08:45:48 -07:00
Sangjin Lee
ae72f1dc77 YARN-4297. TestJobHistoryEventHandler and TestRMContainerAllocator failing on YARN-2928 branch (Varun Saxena via sjlee) 2016-07-10 08:45:47 -07:00
Zhijie Shen
561c746cbe MAPREDUCE-6370. Made the timeline service v2 test driver write event ID. Contributed by Li Lu. 
(cherry picked from commit 827633ee9fee26e3e15343cbccb0b7905ae02170)
 2016-07-10 08:45:41 -07:00
Zhijie Shen
5088f6c76a MAPREDUCE-6337. Added a mode to replay MR job history files and put them into the timeline service v2. Contributed by Sangjin Lee. 
(cherry picked from commit 463e070a8e7c882706a96eaa20ea49bfe9982875)
 2016-07-10 08:45:41 -07:00
Sangjin Lee
fc8485d893 YARN-3551. Consolidate data model change according to the backend implementation (Zhijie Shen via sale) 
(cherry picked from commit 557a3950bddc837469244835f5577899080115d8)
 2016-07-10 08:45:35 -07:00
Zhijie Shen
8c7b6dd2c7 MAPREDUCE-6335. Created MR job based performance test driver for the timeline service v2. Contributed by Sangjin Lee. 
(cherry picked from commit b689f5d43d3f5434a30fe52f1a7e12e1fc5c71f4)
 2016-07-10 08:45:34 -07:00
Zhijie Shen
b50a6d78f5 MAPREDUCE-6327. Made MR AM use timeline service v2 API to write history events and counters. Contributed by Junping Du. 2016-07-10 08:45:34 -07:00
Akira Ajisaka
9d46a49c74 HADOOP-13320. Fix arguments check in documentation for WordCount v2.0. Contributed by niccolo becchi. 
This closes #108
 2016-07-07 15:30:39 -07:00
Steve Loughran
96fa0f848b HADOOP-12709 Cut s3:// from trunk. Contributed by Mingliang Liu. 2016-06-29 16:04:50 +01:00
Allen Wittenauer
23c3ff85a9 HADOOP-13209. replace slaves with workers (John Smith via aw) 2016-06-28 05:53:03 -07:00
Rohith Sharma K S
4fd37eed90 MAPREDUCE-5803. Counters page display all task nevertheless of task type( Map or Reduce). Contributed by Kai Sasaki. 2016-06-28 09:42:42 +05:30
Akira Ajisaka
0f23cd497b MAPREDUCE-6725. Javadoc for CLI#listEvents() contains no-existent param. Contributed by shenyinjie. 2016-06-25 00:26:24 +09:00
Akira Ajisaka
69c21d2db0 MAPREDUCE-6641. TestTaskAttempt fails in trunk. Contributed by Haibo Chen. 2016-06-23 13:26:10 +09:00
Tsuyoshi Ozawa
79a7289165 MAPREDUCE-6721. mapreduce.reduce.shuffle.memory.limit.percent=0.0 should be legal to enforce shuffle to disk. (Gera Shegalov via ozawa) 
This closes #102
 2016-06-22 17:20:13 -07:00
Jian He
d8107fcd1c MAPREDUCE-6197. Cache MapOutputLocations in ShuffleHandler. Contributed by Junping Du 2016-06-21 14:25:58 -07:00
Sangjin Lee
605b4b6136 MAPREDUCE-6719. The list of -libjars archives should be replaced with a wildcard in the distributed cache to reduce the application footprint in the state store (Daniel Templeton via sjlee) 2016-06-21 11:25:11 -07:00
Tsuyoshi Ozawa
5d58858bb6 HADOOP-9613. [JDK8] Update jersey version to latest 1.x release. 2016-06-21 08:05:32 +09:00
Akira Ajisaka
51d497fa93 MAPREDUCE-6542. HistoryViewer uses SimpleDateFormat, but SimpleDateFormat is not threadsafe. Contributed by zhangyubiao. 2016-06-17 11:03:54 +09:00
Wangda Tan
c77a1095dc YARN-1942. Deprecate toString/fromString methods from ConverterUtils and move them to records classes like ContainerId/ApplicationId, etc. (wangda) 2016-06-14 15:06:38 -07:00
Akira Ajisaka
e3ba9ad3f1 HADOOP-13270. BZip2CompressionInputStream finds the same compression marker twice in corner case, causing duplicate data blocks. Contributed by Kai Sasaki. 2016-06-14 10:18:41 +09:00
Akira Ajisaka
0b7b8a3776 MAPREDUCE-6741. Refactor UncompressedSplitLineReader.fillBuffer(). Contributed by Daniel Templeton. 2016-06-10 19:15:36 +09:00
Gera Shegalov
0af96a1c08 MAPREDUCE-6240. Hadoop client displays confusing error message. (gera) 2016-06-08 12:59:37 -07:00
Robert Kanter
733f3f18d5 MAPREDUCE-6702. TestMiniMRChildTask.testTaskEnv and TestMiniMRChildTask.testTaskOldEnv are failing (ajisakaa via rkanter) 2016-06-07 15:46:06 -07:00
Ming Ma
4a1cedc010 MAPREDUCE-5044. Have AM trigger jstack on task attempts that timeout before killing them. (Eric Payne and Gera Shegalov via mingma) 2016-06-06 14:30:51 -07:00
Arun Suresh
dc26601d8f YARN-5180. Allow ResourceRequest to specify an enforceExecutionType flag. (asuresh) 2016-06-02 09:01:02 -07:00
Kai Zheng
93d8a7f2a2 Revert "HADOOP-12579. Deprecate and remove WriteableRPCEngine. Contributed by Kai Zheng" 
This reverts commit a6c79f92d5.
 2016-06-01 08:41:15 +08:00
Varun Vasudev
42f90ab885 YARN-4844. Add getMemorySize/getVirtualCoresSize to o.a.h.y.api.records.Resource. Contributed by Wangda Tan. 2016-05-29 21:24:16 +05:30
Carlo Curino
013532a95e YARN-4957. Add getNewReservation in ApplicationClientProtocol (Sean Po via curino) 2016-05-25 16:55:49 -07:00
Kai Zheng
a6c79f92d5 HADOOP-12579. Deprecate and remove WriteableRPCEngine. Contributed by Kai Zheng 2016-05-27 04:29:42 +08:00
Jian He
ae353ea969 MAPREDUCE-6703. Add flag to allow MapReduce AM to request for OPPORTUNISTIC containers. Contributed by Arun Suresh 2016-05-24 19:47:50 -07:00
Akira Ajisaka
3b3b63081b MAPREDUCE-6607. Enable regex pattern matching when mapreduce.task.files.preserve.filepattern is set. Contributed by Kai Sasaki. 2016-05-23 07:24:31 +09:00
Jian He
21d2b90213 MAPREDUCE-6696. Add a configuration to limit the number of map tasks allowed per job. Contributed by Zhihai Xu 2016-05-19 19:25:16 -07:00
Junping Du
22fcd819f0 YARN-5076. YARN web interfaces lack XFS protection. Contributed by Jonathan Maron. 
(cherry picked from commit 2703ec68712279494d67b0d76b7ac10e7a1628be)
 2016-05-19 14:15:21 -07:00
Robert Kanter
992a49353f MAPREDUCE-6686. Add a way to download the job config from the mapred CLI (rkanter) 2016-05-18 12:16:21 -07:00
Junping Du
f6ef876fe1 MAPREDUCE-6657. Job history server can fail on startup when NameNode is in start phase. Contributed by Haibo Chen. 2016-05-17 14:41:47 -07:00
Jason Lowe
34fddd1e91 MAPREDUCE-6698. Increase timeout on TestUnnecessaryBlockingOnHistoryFileInfo.testTwoThreadsQueryingDifferentJobOfSameUser. Contributed by Haibo Chen 2016-05-17 15:01:33 +00:00
Jason Lowe
12fa4ec141 MAPREDUCE-6701. application master log can not be available when clicking jobhistory's am logs link. Contributed by Haibo Chen 2016-05-17 14:54:13 +00:00
Akira Ajisaka
b357930526 MAPREDUCE-6693. ArrayIndexOutOfBoundsException occurs when the length of the job name is equal to mapreduce.jobhistory.jobname.limit. Contributed by Ajith S. 2016-05-17 20:19:01 +09:00
Allen Wittenauer
730bc746f9 HADOOP-12930. Dynamic subcommands for hadoop shell scripts (aw) 
This commit contains the following JIRA issues:

    HADOOP-12931. bin/hadoop work for dynamic subcommands
    HADOOP-12932. bin/yarn work for dynamic subcommands
    HADOOP-12933. bin/hdfs work for dynamic subcommands
    HADOOP-12934. bin/mapred work for dynamic subcommands
    HADOOP-12935. API documentation for dynamic subcommands
    HADOOP-12936. modify hadoop-tools to take advantage of dynamic subcommands
    HADOOP-13086. enable daemonization of dynamic commands
    HADOOP-13087. env var doc update for dynamic commands
    HADOOP-13088. fix shellprofiles in hadoop-tools to allow replacement
    HADOOP-13089. hadoop distcp adds client opts twice when dynamic
    HADOOP-13094. hadoop-common unit tests for dynamic commands
    HADOOP-13095. hadoop-hdfs unit tests for dynamic commands
    HADOOP-13107. clean up how rumen is executed
    HADOOP-13108. dynamic subcommands need a way to manipulate arguments
    HADOOP-13110. add a streaming subcommand to mapred
    HADOOP-13111. convert hadoop gridmix to be dynamic
    HADOOP-13115. dynamic subcommand docs should talk about exit vs. continue program flow
    HADOOP-13117. clarify daemonization and security vars for dynamic commands
    HADOOP-13120. add a --debug message when dynamic commands have been used
    HADOOP-13121. rename sub-project shellprofiles to match the rest of Hadoop
    HADOOP-13129. fix typo in dynamic subcommand docs
    HADOOP-13151. Underscores should be escaped in dynamic subcommands document
    HADOOP-13153. fix typo in debug statement for dynamic subcommands
 2016-05-16 17:54:45 -07:00
Jason Lowe
9227dfc25f MAPREDUCE-6558. multibyte delimiters with compressed input files generate duplicate records. Contributed by Wilfred Spiegelenburg 2016-05-13 14:33:48 +00:00
Andrew Wang
3c5c57af28 HADOOP-13142. Change project version from 3.0.0 to 3.0.0-alpha1. 2016-05-12 18:27:28 -07:00
Andrew Wang
ca5613af91 Revert "Update project version to 3.0.0-alpha1-SNAPSHOT." 
This reverts commit 6b53802cba.
 2016-05-12 15:32:45 -07:00
Andrew Wang
6b53802cba Update project version to 3.0.0-alpha1-SNAPSHOT. 2016-05-12 11:05:05 -07:00
Steve Loughran
7eddecd357 MAPREDUCE-6639 Process hangs in LocatedFileStatusFetcher if FileSystem.get throws. Ryan Blue via stevel 2016-05-12 19:00:32 +01:00
Jason Lowe
5ffb54694b MAPREDUCE-6684. High contention on scanning of user directory under immediate_done in Job History Server. Contributed by Haibo Chen 2016-05-10 16:03:49 +00:00
Eric Payne
cd35b692de MAPREDUCE-6678. Allow ShuffleHandler readahead without drop-behind. Contributed by Nathan Roberts. 2016-05-10 15:45:01 +00:00
Jason Lowe
c9bb96fa81 MAPREDUCE-6689. MapReduce job can infinitely increase number of reducer resource requests. Contributed by Wangda Tan 2016-05-06 22:25:47 +00:00
Vinod Kumar Vavilapalli
8d48266720 MAPREDUCE-6514. Fixed MapReduce ApplicationMaster to properly updated resources ask after ramping down of all reducers avoiding job hangs. Contributed by Varun Saxena and Wangda Tan. 2016-05-05 19:03:27 -07:00