13420d01f2
MAPREDUCE-5515. Fixed MR AM's webapp to depend on a new config mapreduce.ssl.enabled to enable https and disabling it by default as MR AM needs to set up its own certificates etc and not depend on clusters'. Contributed by Omkar Vinit Joshi. git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1524864 13f79535-47bb-0310-9956-ffa450edef68
6992 lines
260 KiB
Plaintext
6992 lines
260 KiB
Plaintext
Hadoop MapReduce Change Log
|
|
|
|
Trunk (Unreleased)
|
|
|
|
INCOMPATIBLE CHANGES
|
|
|
|
NEW FEATURES
|
|
|
|
MAPREDUCE-778. Rumen Anonymizer. (Amar Kamat and Chris Douglas via amarrk)
|
|
|
|
MAPREDUCE-2669. Add new examples for Mean, Median, and Standard Deviation.
|
|
(Plamen Jeliazkov via shv)
|
|
|
|
MAPREDUCE-4887. Add RehashPartitioner, to smooth distributions
|
|
with poor implementations of Object#hashCode(). (Radim Kolar via cutting)
|
|
|
|
MAPREDUCE-5232. Add a configuration to be able to log classpath and other
|
|
system properties on mapreduce JVMs startup. (Sangjin Lee via vinodkv)
|
|
|
|
IMPROVEMENTS
|
|
|
|
MAPREDUCE-3481. [Gridmix] Improve Gridmix STRESS mode. (amarrk)
|
|
|
|
MAPREDUCE-3597. [Rumen] Rumen should provide APIs to access all the
|
|
job-history related information.
|
|
|
|
MAPREDUCE-3375. [Gridmix] Memory Emulation system tests.
|
|
(Vinay Thota via amarrk)
|
|
|
|
MAPREDUCE-2733. [Gridmix] Gridmix3 cpu emulation system tests.
|
|
(Vinay Thota via amarrk)
|
|
|
|
MAPREDUCE-2836. Provide option to fail jobs when submitted to non-existent
|
|
fair scheduler pools. (Ahmed Radwan via todd)
|
|
|
|
MAPREDUCE-3171. normalize nodemanager native code compilation with common/hdfs
|
|
native. (tucu)
|
|
|
|
MAPREDUCE-3149. Add a test to verify that TokenCache handles file system
|
|
uri with no authority. (John George via jitendra)
|
|
|
|
MAPREDUCE-3169. Create a new MiniMRCluster equivalent which only provides
|
|
client APIs cross MR1 and MR2 (Ahmed via tucu)
|
|
|
|
MAPREDUCE-2944. Improve checking of input for JobClient.displayTasks()
|
|
(XieXianshan via harsh)
|
|
|
|
MAPREDUCE-3956. Remove the use of the deprecated Syncable.sync() method from
|
|
TeraOutputFormat in the terasort example. (szetszwo)
|
|
|
|
MAPREDUCE-3935. Annotate Counters.Counter and Counters.Group as @Public.
|
|
(tomwhite)
|
|
|
|
HADOOP-8285 MR changes for Use ProtoBuf for RpcPayLoadHeader (sanjay radia)
|
|
|
|
MAPREDUCE-3302. Remove the last dependency call from
|
|
org.apache.hadoop.record package in MR. (harsh)
|
|
|
|
MAPREDUCE-2384. The job submitter should make sure to validate
|
|
jobs before creation of necessary files. (harsh)
|
|
|
|
MAPREDUCE-4371. Check for cyclic dependencies in Jobcontrol job DAG
|
|
(madhukara phatak via bobby)
|
|
|
|
MAPREDUCE-4686. hadoop-mapreduce-client-core fails compilation in Eclipse
|
|
due to missing Avro-generated classes (Chris Nauroth via harsh)
|
|
|
|
MAPREDUCE-4735. Make arguments in TestDFSIO case insensitive.
|
|
(Brandon Li via suresh)
|
|
|
|
MAPREDUCE-5014. Extend Distcp to accept a custom CopyListing.
|
|
(Srikanth Sundarrajan via amareshwari)
|
|
|
|
BUG FIXES
|
|
|
|
MAPREDUCE-4272. SortedRanges.Range#compareTo is not spec compliant.
|
|
(Yu Gao via llu)
|
|
|
|
MAPREDUCE-3194. "mapred mradmin" command is broken in mrv2
|
|
(Jason Lowe via bobby)
|
|
|
|
MAPREDUCE-3462. Fix Gridmix JUnit testcase failures.
|
|
(Ravi Prakash and Ravi Gummadi via amarrk)
|
|
|
|
MAPREDUCE-3349. Log rack-name in JobHistory for unsuccessful tasks.
|
|
(Devaraj K and Amar Kamat via amarrk)
|
|
|
|
MAPREDUCE-3412. Fix 'ant docs'. (amarrk)
|
|
|
|
MAPREDUCE-3346. [Rumen] LoggedTaskAttempt#getHostName() returns null.
|
|
(amarrk)
|
|
|
|
MAPREDUCE-2950. [Gridmix] TestUserResolve fails in trunk.
|
|
(Ravi Gummadi via amarrk)
|
|
|
|
MAPREDUCE-2784. [Gridmix] Bug fixes in ExecutionSummarizer and
|
|
ResourceUsageMatcher. (amarrk)
|
|
|
|
MAPREDUCE-2978. Fixed test-patch to make Jenkins report correct number of
|
|
findBugs, correct links to findBugs artifacts and no links to the
|
|
artifacts when there are no warnings. (Tom White via vinodkv).
|
|
|
|
MAPREDUCE-3664. Federation Documentation has incorrect configuration example.
|
|
(Brandon Li via jitendra)
|
|
|
|
MAPREDUCE-1740. NPE in getMatchingLevelForNodes when node locations are
|
|
variable depth (ahmed via tucu) [IMPORTANT: this is dead code in trunk]
|
|
|
|
MAPREDUCE-3990. MRBench allows Long-sized input-lines value
|
|
but parses CLI argument as an Integer. (harsh)
|
|
|
|
MAPREDUCE-3868. Make Raid Compile. (Weiyan Wang via schen)
|
|
|
|
MAPREDUCE-4685. DBCount should not use ACCESS. (Viji via harsh)
|
|
|
|
MAPREDUCE-3223. Remove MR1 configs from mapred-default.xml (tlipcon via harsh)
|
|
|
|
MAPREDUCE-4695. Fix LocalRunner on trunk after MAPREDUCE-3223 broke it
|
|
(harsh)
|
|
|
|
MAPREDUCE-4574. Fix TotalOrderParitioner to work with
|
|
non-WritableComparable key types. (harsh)
|
|
|
|
MAPREDUCE-4884. Streaming tests fail to start MiniMRCluster due to missing
|
|
queue configuration. (Chris Nauroth via suresh)
|
|
|
|
MAPREDUCE-5012. Typo in javadoc for IdentityMapper class. (Adam Monsen
|
|
via suresh)
|
|
|
|
MAPREDUCE-4885. Streaming tests have multiple failures on Windows. (Chris
|
|
Nauroth via bikas)
|
|
|
|
MAPREDUCE-4987. TestMRJobs#testDistributedCache fails on Windows due to
|
|
classpath problems and unexpected behavior of symlinks (Chris Nauroth via
|
|
bikas)
|
|
|
|
MAPREDUCE-5191. TestQueue#testQueue fails with timeout on Windows. (Ivan
|
|
Mitic via hitesh)
|
|
|
|
Release 2.3.0 - UNRELEASED
|
|
|
|
INCOMPATIBLE CHANGES
|
|
|
|
NEW FEATURES
|
|
|
|
MAPREDUCE-5265. History server admin service to refresh user and superuser
|
|
group mappings (Ashwin Shankar via jlowe)
|
|
|
|
MAPREDUCE-5356. Ability to refresh aggregated log retention period and
|
|
check interval (Ashwin Shankar via jlowe)
|
|
|
|
MAPREDUCE-5386. Ability to refresh history server job retention and job
|
|
cleaner settings (Ashwin Shankar via jlowe)
|
|
|
|
MAPREDUCE-5411. Refresh size of loaded job cache on history server (Ashwin
|
|
Shankar via jlowe)
|
|
|
|
IMPROVEMENTS
|
|
|
|
MAPREDUCE-434. LocalJobRunner limited to single reducer (Sandy Ryza and
|
|
Aaron Kimball via Sandy Ryza)
|
|
|
|
OPTIMIZATIONS
|
|
|
|
MAPREDUCE-5484. YarnChild unnecessarily loads job conf twice (Sandy Ryza)
|
|
|
|
MAPREDUCE-5487. In task processes, JobConf is unnecessarily loaded again
|
|
in Limits (Sandy Ryza)
|
|
|
|
BUG FIXES
|
|
|
|
MAPREDUCE-5316. job -list-attempt-ids command does not handle illegal
|
|
task-state (Ashwin Shankar via jlowe)
|
|
|
|
MAPREDUCE-5380. Invalid mapred command should return non-zero exit code
|
|
(Stephen Chu via jlowe)
|
|
|
|
MAPREDUCE-5404. HSAdminServer does not use ephemeral ports in minicluster
|
|
mode (Ted Yu via jlowe)
|
|
|
|
Release 2.2.0 - UNRELEASED
|
|
|
|
INCOMPATIBLE CHANGES
|
|
|
|
NEW FEATURES
|
|
|
|
IMPROVEMENTS
|
|
|
|
OPTIMIZATIONS
|
|
|
|
BUG FIXES
|
|
|
|
MAPREDUCE-5504. mapred queue -info inconsistent with types (Kousuke Saruta
|
|
via tgraves)
|
|
|
|
MAPREDUCE-5488. Changed MR client to keep trying to reach the application
|
|
when it sees that on attempt's AM is down. (Jian He via vinodkv)
|
|
|
|
MAPREDUCE-5515. Fixed MR AM's webapp to depend on a new config
|
|
mapreduce.ssl.enabled to enable https and disabling it by default as MR AM
|
|
needs to set up its own certificates etc and not depend on clusters'.
|
|
(Omkar Vinit Joshi via vinodkv)
|
|
|
|
Release 2.1.1-beta - 2013-09-23
|
|
|
|
INCOMPATIBLE CHANGES
|
|
|
|
NEW FEATURES
|
|
|
|
IMPROVEMENTS
|
|
|
|
MAPREDUCE-5478. TeraInputFormat unnecessarily defines its own FileSplit
|
|
subclass (Sandy Ryza)
|
|
|
|
MAPREDUCE-5497. Changed MRAppMaster to sleep only after doing everything else
|
|
but just before ClientService to avoid race conditions during RM restart.
|
|
(Jian He via vinodkv)
|
|
|
|
MAPREDUCE-5379. Include token tracking ids in jobconf. (kkambatl via tucu)
|
|
|
|
OPTIMIZATIONS
|
|
|
|
MAPREDUCE-5446. TestJobHistoryEvents and TestJobHistoryParsing have race
|
|
conditions (jlowe via kihwal)
|
|
|
|
MAPREDUCE-5462. In map-side sort, swap entire meta entries instead of
|
|
indexes for better cache performance. (Sandy Ryza)
|
|
|
|
MAPREDUCE-1981. Improve getSplits performance by using listLocatedStatus
|
|
(Hairong Kuang and Jason Lowe via jlowe)
|
|
|
|
BUG FIXES
|
|
|
|
MAPREDUCE-5385. Fixed a bug with JobContext getCacheFiles API. (Omkar Vinit
|
|
Joshi via vinodkv)
|
|
|
|
MAPREDUCE-5428. HistoryFileManager doesn't stop threads when service is
|
|
stopped (Karthik Kambatla via jlowe)
|
|
|
|
MAPREDUCE-5251. Reducer should not implicate map attempt if it has
|
|
insufficient space to fetch map output (Ashwin Shankar via jlowe)
|
|
|
|
MAPREDUCE-5317. Stale files left behind for failed jobs (Ravi Prakash via
|
|
jlowe)
|
|
|
|
MAPREDUCE-5358. MRAppMaster throws invalid transitions for JobImpl
|
|
(Devaraj K via jlowe)
|
|
|
|
MAPREDUCE-3193. FileInputFormat doesn't read files recursively in the
|
|
input path dir (Devaraj K via jlowe)
|
|
|
|
MAPREDUCE-5440. TestCopyCommitter Fails on JDK7 (Robert Parker via jlowe)
|
|
|
|
MAPREDUCE-5367. Local jobs all use same local working directory
|
|
(Sandy Ryza)
|
|
|
|
MAPREDUCE-5425. Junit in TestJobHistoryServer failing in jdk 7 (Robert
|
|
Parker via jlowe)
|
|
|
|
MAPREDUCE-5454. TestDFSIO fails intermittently on JDK7 (Karthik Kambatla
|
|
via Sandy Ryza)
|
|
|
|
MAPREDUCE-5001. LocalJobRunner has race condition resulting in job
|
|
failures (Sandy Ryza via jlowe)
|
|
|
|
MAPREDUCE-5466. Changed MR AM to not promote history files of intermediate
|
|
AMs in case they are exiting because of errors and thus help history-server
|
|
pick up the right history file for the last successful AM. (Jian He via
|
|
vinodkv)
|
|
|
|
MAPREDUCE-5468. Fix MR AM recovery for map-only jobs. (vinodkv via
|
|
acmurthy)
|
|
|
|
MAPREDUCE-5470. LocalJobRunner does not work on Windows. (Sandy Ryza via
|
|
cnauroth)
|
|
|
|
MAPREDUCE-5476. Changed MR AM recovery code to cleanup staging-directory
|
|
only after unregistering from the RM. (Jian He via vinodkv)
|
|
|
|
MAPREDUCE-5483. revert MAPREDUCE-5357. (rkanter via tucu)
|
|
|
|
MAPREDUCE-5441. Changed MR AM to return RUNNING state if exiting when RM
|
|
commands to reboot, so that client can continue to track the overall job.
|
|
(Jian He via vinodkv)
|
|
|
|
MAPREDUCE-5475. MRClientService does not verify ACLs properly (jlowe)
|
|
|
|
MAPREDUCE-5414. TestTaskAttempt fails in JDK7 with NPE (Nemon Lou via
|
|
devaraj)
|
|
|
|
MAPREDUCE-5020. Compile failure with JDK8 (Trevor Robinson via tgraves)
|
|
|
|
MAPREDUCE-5164. mapred job and queue commands omit HADOOP_CLIENT_OPTS
|
|
(Nemon Lou via devaraj)
|
|
|
|
MAPREDUCE-5493. Cleanup in-memory & on-disk segments to prevent leak on
|
|
shuffle completion. (jlowe via acmurthy)
|
|
|
|
Release 2.1.0-beta - 2013-08-22
|
|
|
|
INCOMPATIBLE CHANGES
|
|
|
|
MAPREDUCE-4067. Changed MRClientProtocol api to throw IOException only (Xuan
|
|
Gong via vinodkv)
|
|
|
|
MAPREDUCE-5234. Change mapred.TaskReport and mapreduce.TaskReport for binary
|
|
compatibility with mapred in 1.x but incompatible with 0.23.x. (Mayank Bansal
|
|
via vinodkv)
|
|
|
|
MAPREDUCE-5156. Change hadoop examples ProgramDriver to be able to run
|
|
1.x examples jar on top of YARN. This change breaks 0.23.x direct usages of
|
|
ProgramDriver. (Zhijie Shen via vinodkv)
|
|
|
|
MAPREDUCE-5233. Add methods that are changed or removed from JobControl.Job
|
|
when compared to 1.x. This breaks 0.23.x users of one API in Job. (Mayank
|
|
Bansal via vinodkv)
|
|
|
|
MAPREDUCE-5237. Add methods that were removed from ClusterStatus back into
|
|
2.x so as to be compatible with 1.x. Incompatible as
|
|
ClusterStatus.UNINITIALIZED_MEMORY_VALUE is a long now and so breaks 0.23.x
|
|
but it shouldn't be a big deal in reality. (Zhijie Shen via vinodkv)
|
|
|
|
NEW FEATURES
|
|
|
|
HADOOP-8562. Enhancements to support Hadoop on Windows Server and Windows
|
|
Azure environments. (See breakdown of tasks below for subtasks and
|
|
contributors)
|
|
|
|
IMPROVEMENTS
|
|
|
|
MAPREDUCE-3008. Improvements to cumulative CPU emulation for short running
|
|
tasks in Gridmix. (amarrk via tgraves)
|
|
|
|
MAPREDUCE-5033. mapred shell script should respect usage flags
|
|
(--help -help -h). (Andrew Wang via atm)
|
|
|
|
MAPREDUCE-4892. Modify CombineFileInputFormat to not skew input slits'
|
|
allocation on small clusters. (Bikas Saha via vinodkv)
|
|
|
|
MAPREDUCE-4990. Construct debug strings conditionally in
|
|
ShuffleHandler.Shuffle#sendMapOutput(). (kkambatl via tucu)
|
|
|
|
MAPREDUCE-4875. coverage fixing for org.apache.hadoop.mapred
|
|
(Aleksey Gorshkov via bobby)
|
|
|
|
MAPREDUCE-5129. Allow tags to JobHistory for deeper analytics. (billie via
|
|
acmurthy)
|
|
|
|
MAPREDUCE-3787. [Gridmix] Optimize job monitoring and STRESS mode for
|
|
faster job submission. (amarrk via tgraves)
|
|
|
|
MAPREDUCE-5079. Changes job recovery to restore state directly from job
|
|
history, instaed of simulating state machine events.
|
|
(Jason Lowe and Robert Parker via sseth)
|
|
|
|
MAPREDUCE-4981. Add WordMean, WordMedian, WordStandardDeviation
|
|
to ExamplesDriver. (Plamen Jeliazkov via shv)
|
|
|
|
MAPREUDUCE-5059. Change average merge time on Job overview page to be the
|
|
time delta between the end of the shuffle and the start of the reduce.
|
|
(Omkar Vinit Joshi via vinodkv)
|
|
|
|
MAPREDUCE-4985. Add compression option to TestDFSIO usage.
|
|
(Plamen Jeliazkov via shv)
|
|
|
|
MAPREDUCE-5152. Make MR App to simply pass through the container from RM
|
|
instead of extracting and populating information itself to start any
|
|
container. (vinodkv)
|
|
|
|
MAPREDUCE-5175. Updated MR App to not set envs that will be set by NMs
|
|
anyways after YARN-561. (Xuan Gong via vinodkv)
|
|
|
|
MAPREDUCE-5069. add concrete common implementations of
|
|
CombineFileInputFormat (Sangjin Lee via bobby)
|
|
|
|
MAPREDUCE-5145. Changed default max-attempts to be more than one for MR jobs
|
|
inline with YARN. (Zhijie Shen via vinodkv)
|
|
|
|
MAPREDUCE-5036. Default shuffle handler port should not be 8080.
|
|
(Sandy Ryza via tomwhite)
|
|
|
|
MAPREDUCE-5159. Change ValueAggregatorJob to add APIs which can support
|
|
binary compatibility with hadoop-1 examples. (Zhijie Shen via vinodkv)
|
|
|
|
MAPREDUCE-5157. Bring back old sampler related code so that we can support
|
|
binary compatibility with hadoop-1 sorter example. (Zhijie Shen via vinodkv)
|
|
|
|
MAPREDUCE-5222. Bring back some methods and constants in Jobclient for
|
|
binary compatibility with mapred in 1.x. (Karthik Kambatla via vinodkv)
|
|
|
|
MAPREDUCE-5235. Bring back old fields and exceptions in Counters for
|
|
binary compatibility with mapred in 1.x. (Mayank Bansal via vinodkv)
|
|
|
|
MAPREDUCE-5246. Specify application-type at the time of job submission after
|
|
YARN-563. (Mayank Bansal via vinodkv)
|
|
|
|
MAPREDUCE-5230. Bring back NLineInputFormat.createFileSplit for binary
|
|
compatibility with mapred in 1.x (Mayank Bansal via vinodkv)
|
|
|
|
MAPREDUCE-5270. Migrated MR app from using BuilderUtil factory methods to
|
|
individual record factory methods. (Jian He via vinodkv)
|
|
|
|
MAPREDUCE-5263. Bring back old methods and fields in
|
|
filecache.DistributedCache for binary compatibility with mapred in 1.x.
|
|
(Zhijie Shen via vinodkv)
|
|
|
|
MAPREDUCE-5228. Bring back FileInputFormat.Counter and
|
|
FileOuputFormat.Counter for binary compatibility with 1.x mapred APIs.
|
|
(Mayank Bansal via vinodkv)
|
|
|
|
MAPREDUCE-5176. Add annotation for tagging tasks as responsive to
|
|
preemption. (Carlo Curino, cdouglas)
|
|
|
|
MAPREDUCE-5275. Bring back a couple of APIs in mapreduce.security.TokenCache
|
|
for binary compatibility with 1.x mapreduce APIs. (Mayank Bansal via vinodkv)
|
|
|
|
MAPREDUCE-5231. Bring back a constructor in mapred's
|
|
DBInputFormat.DBRecordReader for binary compatibility with 1.x mapred APIs.
|
|
(Zhijie Shen via vinodkv)
|
|
|
|
MAPREDUCE-5273. Bring back a couple of protected variables in mapred and
|
|
mapreduce CombineFileRecordReader for binary compatibility with 1.x APIs.
|
|
(Mayank Bansal via vinodkv)
|
|
|
|
MAPREDUCE-5280. Bring back removed constructor and a method in mapreduce
|
|
ClusterMetrics for binary compatibility with 1.x APIs. (Mayank Bansal via
|
|
vinodkv)
|
|
|
|
MAPREDUCE-5289. Updated MR App to use Token directly after YARN-717. (Jian He
|
|
via vinodkv)
|
|
|
|
MAPREDUCE-5229. Put back FileOutputCommitter.TEMP_DIR_NAME in mapreduce for
|
|
binary compatibility with 1.x APIs. (Zhijie Shen via vinodkv)
|
|
|
|
MAPREDUCE-5274. Bring back SecureShuffleUtils.toHex in mapreduce for binary
|
|
compatibility with 1.x APIs. (Mayank Bansal via vinodkv)
|
|
|
|
MAPREDUCE-5300. Fix backward incompatibility for
|
|
o.a.h.mapreduce.filecache.DistributedCache. (Zhijie Shen via acmurthy)
|
|
|
|
MAPREDUCE-5283. Over 10 different tests have near identical
|
|
implementations of AppContext (Sandy Ryza via jlowe)
|
|
|
|
MAPREDUCE-5199. Removing ApplicationTokens file as it is no longer needed.
|
|
(Daryn Sharp via vinodkv)
|
|
|
|
MAPREDUCE-5192. Allow for alternate resolutions of TaskCompletionEvents.
|
|
(cdouglas via acmurthy)
|
|
|
|
MAPREDUCE-5184. Document compatibility for MapReduce applications in
|
|
hadoop-2 vis-a-vis hadoop-1. (Zhijie Shen via acmurthy)
|
|
|
|
MAPREDUCE-5194. Heed interrupts during Fetcher shutdown. (cdouglas)
|
|
|
|
MAPREDUCE-5326. Added version to shuffle header. (Zhijie Shen via
|
|
acmurthy)
|
|
|
|
MAPREDUCE-5333. Add test that verifies MRAM works correctly when sending
|
|
requests with non-normalized capabilities. (ywskycn via tucu)
|
|
|
|
MAPREDUCE-5398. MR changes for YARN-513 (Jian He via bikas)
|
|
|
|
OPTIMIZATIONS
|
|
|
|
MAPREDUCE-4974. Optimising the LineRecordReader initialize() method
|
|
(Gelesh via bobby)
|
|
|
|
MAPREDUCE-5268. Improve history server startup performance (Karthik
|
|
Kambatla via jlowe)
|
|
|
|
MAPREDUCE-5352. Optimize node local splits generated by
|
|
CombineFileInputFormat. (sseth)
|
|
|
|
BUG FIXES
|
|
|
|
MAPREDUCE-4671. AM does not tell the RM about container requests which are
|
|
no longer needed. (Bikas Saha via sseth)
|
|
|
|
MAPREDUCE-4994. -jt generic command line option does not work. (sandyr via tucu)
|
|
|
|
MAPREDUCE-5000. Fixes getCounters when speculating by fixing the selection
|
|
of the best attempt for a task. (Jason Lowe via sseth)
|
|
|
|
MAPREDUCE-4994. Addendum fixing testcases failures. (sandyr via tucu)
|
|
|
|
MAPREDUCE-4846. Some JobQueueInfo methods are public in MR1 but protected
|
|
in MR2. (Sandy Ryza via tomwhite)
|
|
|
|
MAPREDUCE-5013. mapred.JobStatus compatibility: MR2 missing constructors
|
|
from MR1. (Sandy Ryza via tomwhite)
|
|
|
|
MAPREDUCE-4951. Container preemption interpreted as task failure.
|
|
(Sandy Ryza via tomwhite)
|
|
|
|
MAPREDUCE-5008. Merger progress miscounts with respect to EOF_MARKER.
|
|
(Sandy Ryza via tomwhite)
|
|
|
|
MAPREDUCE-4693. History server should include counters for failed tasks.
|
|
(Xuan Gong via sseth)
|
|
|
|
MAPREDUCE-4896. mapred queue -info spits out ugly exception when queue does
|
|
not exist. (sandyr via tucu)
|
|
|
|
MAPREDUCE-3685. Fix bugs in MergeManager to ensure compression codec is
|
|
appropriately used and that on-disk segments are correctly sorted on
|
|
file-size. (Anty Rao and Ravi Prakash via acmurthy)
|
|
|
|
MAPREDUCE-4571. TestHsWebServicesJobs fails on jdk7. (tgraves via tucu)
|
|
|
|
MAPREDUCE-4716. TestHsWebServicesJobsQuery.testJobsQueryStateInvalid
|
|
fails with jdk7. (tgraves via tucu)
|
|
|
|
MAPREDUCE-5075. DistCp leaks input file handles since ThrottledInputStream
|
|
does not close the wrapped InputStream. (Chris Nauroth via szetszwo)
|
|
|
|
MAPREDUCE-3872. Fix an event handling races in ContainerLauncherImpl.
|
|
(Robert Kanter via sseth)
|
|
|
|
MAPREDUCE-5062. Fix MR AM to read max-retries from the RM. (Zhijie Shen via
|
|
vinodkv)
|
|
|
|
MAPREDUCE-3829. [Gridmix] Gridmix should give better error message when
|
|
input data directory already exists and -generate opton is
|
|
given.(ravigummadi via tgraves)
|
|
|
|
MAPREDUCE-2722. [Gridmix] Gridmix simulated job's map's hdfsBytesRead
|
|
counter is wrong when compressed input is used.(ravigummadi via tgraves)
|
|
|
|
MAPREDUCE-3953. [Gridmix] Gridmix throws NPE and does not simulate a
|
|
job if the trace contains null taskStatus for a task. (ravigummadi via
|
|
tgraves)
|
|
|
|
MAPREDUCE-4087. [Gridmix] GenerateDistCacheData job of Gridmix can
|
|
become slow in some cases (ravigummadi via tgraves).
|
|
|
|
MAPREDUCE-5077. Remove mapreduce.util.ResourceCalculatorPlugin and related
|
|
code. (Karthik Kambatla via sseth)
|
|
|
|
MAPREDUCE-4083. [Gridmix] NPE in cpu emulation. (amarrk via tgraves)
|
|
|
|
MAPREDUCE-4100. [Gridmix] Bug fixed in compression emulation feature for
|
|
map only jobs. (amarrk via tgraves)
|
|
|
|
MAPREDUCE-4356. [Rumen] Provide access to the method
|
|
ParsedTask.obtainTaskAttempts(). (ravigummadi via tgraves)
|
|
|
|
MAPREDUCE-4149. [Rumen] Rumen fails to parse certain counter
|
|
strings. (ravigummadi via tgraves)
|
|
|
|
MAPREDUCE-3757. [Rumen] Fixed Rumen Folder to adjust shuffleFinished and
|
|
sortFinished times when needed. (Ravi Gummadi via tgraves)
|
|
|
|
MAPREDUCE-5138. Fix LocalDistributedCacheManager after YARN-112. (Omkar Vinit
|
|
Joshi via vinodkv)
|
|
|
|
MAPREDUCE-5086. MR app master deletes staging dir when sent a reboot
|
|
command from the RM. (Jian He via jlowe)
|
|
|
|
MAPREDUCE-5113. Streaming input/output types are ignored with java
|
|
mapper/reducer. (sandyr via tucu)
|
|
|
|
MAPREDUCE-5098. Fix findbugs warnings in gridmix. (kkambatl via tucu)
|
|
|
|
MAPREDUCE-5137. AM web UI: clicking on Map Task results in 500 error
|
|
(Thomas Graves via jlowe)
|
|
|
|
MAPREDUCE-5136. TestJobImpl->testJobNoTasks fails with IBM JAVA (Amir
|
|
Sanjar via jlowe)
|
|
|
|
MAPREDUCE-5139. Update MR AM to use the modified startContainer API after
|
|
YARN-486. (Xuan Gong via vinodkv)
|
|
|
|
MAPREDUCE-5151. Update MR AM to use standard exit codes from the API after
|
|
YARN-444. (Sandy Ryza via vinodkv)
|
|
|
|
MAPREDUCE-5140. MR part of YARN-514 (Zhijie Shen via bikas)
|
|
|
|
MAPREDUCE-5128. mapred-default.xml is missing a bunch of history server
|
|
configs. (sandyr via tucu)
|
|
|
|
MAPREDUCE-4898. FileOutputFormat.checkOutputSpecs and
|
|
FileOutputFormat.setOutputPath incompatible with MR1. (rkanter via tucu)
|
|
|
|
MAPREDUCE-5078. TestMRAppMaster fails on Windows due to mismatched path
|
|
separators. (Chris Nauroth via sseth)
|
|
|
|
MAPREDUCE-4932. mapreduce.job#getTaskCompletionEvents incompatible with
|
|
Hadoop 1. (rkanter via tucu)
|
|
|
|
MAPREDUCE-5163. Update MR App to not use API utility methods for collections
|
|
after YARN-441. (Xuan Gong via vinodkv)
|
|
|
|
MAPREDUCE-5066. Added a timeout for the job.end.notification.url. (Ivan
|
|
Mitic via acmurthy)
|
|
|
|
MAPREDUCE-5146. application classloader may be used too early to load
|
|
classes. (Sangjin Lee via tomwhite)
|
|
|
|
MAPREDUCE-4737. Ensure that mapreduce APIs are semantically consistent
|
|
with mapred API w.r.t Mapper.cleanup and Reducer.cleanup; in the sense that
|
|
cleanup is now called even if there is an error. The old mapred API
|
|
already ensures that Mapper.close and Reducer.close are invoked during
|
|
error handling. Note that it is an incompatible change, however end-users
|
|
can override Mapper.run and Reducer.run to get the old (inconsistent)
|
|
behaviour. (acmurthy)
|
|
|
|
MAPREDUCE-5166. Fix ConcurrentModificationException due to insufficient
|
|
synchronization on updates to task Counters. (Sandy Ryza via acmurthy)
|
|
|
|
MAPREDUCE-5181. RMCommunicator should not use AMToken from the env.
|
|
(Vinod Kumar Vavilapalli via sseth)
|
|
|
|
MAPREDUCE-5178. Update MR App to set progress in ApplicationReport after
|
|
YARN-577. (Hitesh Shah via vinodkv)
|
|
|
|
MAPREDUCE-5167. Update MR App after YARN-562 to use the new builder API
|
|
for the container. (Jian He via vinodkv)
|
|
|
|
MAPREDUCE-5179. Fix unit test in TestHSWebServices which fails when
|
|
versionInfo has parantheses like when running on a git checkout. (Hitesh Shah
|
|
via vinodkv)
|
|
|
|
MAPREDUCE-5193. A few MR tests use block sizes which are smaller than the
|
|
default minimum block size. (Andrew Wang via atm)
|
|
|
|
MAPREDUCE-5205. Fixed MR App to load tokens correctly. (vinodkv)
|
|
|
|
MAPREDUCE-5204. Handling YarnRemoteException separately from IOException in
|
|
MR app after YARN-629. (Xuan Gong via vinodkv)
|
|
|
|
MAPREDUCE-5209. Fix units in a ShuffleScheduler log message.
|
|
(Tsuyoshi OZAWA via cdouglas)
|
|
|
|
MAPREDUCE-5212. Handling YarnRemoteException separately from IOException in
|
|
MR App's use of ClientRMProtocol after YARN-631. (Xuan Gong via vinodkv)
|
|
|
|
MAPREDUCE-5226. Handling YarnRemoteException separately from IOException in
|
|
MR App's use of AMRMProtocol after YARN-630. (Xuan Gong via vinodkv)
|
|
|
|
MAPREDUCE-4942. mapreduce.Job has a bunch of methods that throw
|
|
InterruptedException so its incompatible with MR1. (rkanter via tucu)
|
|
|
|
MAPREDUCE-5239. Updated MR App to reflect YarnRemoteException changes after
|
|
YARN-634. (Siddharth Seth via vinodkv)
|
|
|
|
MAPREDUCE-5208. Modified ShuffleHandler to use SecureIOUtils for reading
|
|
local files. (Omkar Vinit Joshi via vinodkv)
|
|
|
|
MAPREDUCE-5220. Setter methods in TaskCompletionEvent are public in MR1 and
|
|
protected in MR2. (sandyr via tucu)
|
|
|
|
MAPREDUCE-5240. Fix a bug in MRAppMaster because of which OutputCommitter
|
|
could not access credentials set by the user. (vinodkv)
|
|
|
|
MAPREDUCE-5244. Two functions changed their visibility in JobStatus.
|
|
(zjshen via tucu)
|
|
|
|
MAPREDUCE-4927. Historyserver 500 error due to NPE when accessing specific
|
|
counters page for failed job. (Ashwin Shankar via jlowe)
|
|
|
|
MAPREDUCE-5257. Fix issues in TestContainerLauncherImpl after YARN-617.
|
|
(Omkar Vinit Joshi via vinodkv)
|
|
|
|
MAPREDUCE-5282. Updating MR App to use immutable ApplicationID after
|
|
YARN-716. (Siddharth Seth via vinodkv)
|
|
|
|
MAPREDUCE-5286. Change MapReduce to use ContainerTokenIdentifier instead
|
|
of the entire Container in the startContainer call - YARN-684.
|
|
(Vinod Kumar Vavilapalli via sseth)
|
|
|
|
MAPREDUCE-5299. Fix backward incompatibility for TaskCompletionEvent by
|
|
adding back setTaskID. (Zhijie Shen via acmurthy)
|
|
|
|
MAPREDUCE-5296. Fix backward incompatibility for JobControl by adding the
|
|
omitted addJob. (Zhijie Shen via acmurthy)
|
|
|
|
MAPREDUCE-5245. Added back constants to JobConf to fix incompatibilities.
|
|
(Zhijie Shen via acmurthy)
|
|
|
|
MAPREDUCE-5297. Updated MR App since BuilderUtils is no longer public
|
|
after YARN-748. (Jian He via vinodkv)
|
|
|
|
MAPREDUCE-5301. Updated MR code to work with YARN-635 changes of renaming
|
|
YarnRemoteException to YarnException. (Siddharth Seth via vinodkv)
|
|
|
|
MAPREDUCE-5308. Shuffling to memory can get out-of-sync when fetching
|
|
multiple compressed map outputs (Nathan Roberts via jlowe)
|
|
|
|
MAPREDUCE-5315. DistCp reports success even on failure. (mithun and jlowe
|
|
via daryn)
|
|
|
|
MAPREDUCE-5259. TestTaskLog fails on Windows because of path separators
|
|
missmatch. (Ivan Mitic via cnauroth)
|
|
|
|
MAPREDUCE-4019. -list-attempt-ids is not working (Ashwin Shankar,
|
|
Devaraj K, and B Anil Kumar via jlowe)
|
|
|
|
MAPREDUCE-5334. Fix failing unit tests - TestContainerLauncher,
|
|
TestContainerLauncherImpl. (Vinod Kumar Vavilapalli via sseth)
|
|
|
|
MAPREDUCE-5325. MR changes related to YARN-727. ClientRMProtocol.getAllApplications
|
|
should accept ApplicationType as a parameter. (Xuan Gong via hitesh)
|
|
|
|
MAPREDUCE-5291. Change MR App to use updated property names in
|
|
container-log4j.properties. (Zhijie Shen via sseth)
|
|
|
|
MAPREDUCE-5303. Changed MR app after moving ProtoBase to package impl.pb via
|
|
YARN-724. (Jian He via vinodkv)
|
|
|
|
MAPREDUCE-5312. TestRMNMInfo is failing. (sandyr via tucu)
|
|
|
|
MAPREDUCE-5304. mapreduce.Job killTask/failTask/getTaskCompletionEvents
|
|
methods have incompatible signature changes. (kkambatl via tucu)
|
|
|
|
MAPREDUCE-5298. Moved MapReduce services to YARN-530 stricter lifecycle.
|
|
(Steve Loughran via vinodkv)
|
|
|
|
MAPREDUCE-5319. Set user.name in job.xml. (Xuan Gong via acmurthy)
|
|
|
|
MAPREDUCE-5310. MRAM should not normalize allocation request capabilities.
|
|
(tucu)
|
|
|
|
MAPREDUCE-5213. Re-assess TokenCache methods marked @Private.
|
|
(kkambatl via tucu)
|
|
|
|
MAPREDUCE-5412. Update MR app to use multiple containers API of
|
|
ContainerManager after YARN-926. (Jian He via vinodkv)
|
|
|
|
MAPREDUCE-5421. Fixed TestNonExistentJob failure after YARN-873. (Junping Du
|
|
via vinodkv)
|
|
|
|
MAPREDUCE-5419. TestSlive is getting FileNotFound Exception (Robert Parker
|
|
via jlowe)
|
|
|
|
MAPREDUCE-5399. Unnecessary Configuration instantiation in IFileInputStream
|
|
slows down merge. (Stanislav Barton via Sandy Ryza)
|
|
|
|
BREAKDOWN OF HADOOP-8562 SUBTASKS
|
|
|
|
MAPREDUCE-4739. Some MapReduce tests fail to find winutils.
|
|
(Chris Nauroth via suresh)
|
|
|
|
MAPREDUCE-4780. MapReduce distribution build fails on Windows.
|
|
(Chris Nauroth via suresh)
|
|
|
|
MAPREDUCE-4790. MapReduce build script would be more readable using abspath.
|
|
(Chris Nauroth via suresh)
|
|
|
|
MAPREDUCE-4869. Fix TestMapReduceChildJVM. (Chris Nauroth via acmurthy)
|
|
|
|
MAPREDUCE-4870. Fix TestMRJobsWithHistoryService. (Chris Nauroth via acmurthy)
|
|
|
|
MAPREDUCE-4983. Fixed various platform specific assumptions in various tests,
|
|
so that they can pass on Windows too. (Chris Nauroth via vinodkv)
|
|
|
|
HADOOP-9372. Fix bad timeout annotations on tests.
|
|
(Arpit Agarwal via suresh)
|
|
|
|
MAPREDUCE-5177. Use common utils FileUtil#setReadable/Writable/Executable &
|
|
FileUtil#canRead/Write/Execute. (Ivan Mitic via suresh)
|
|
|
|
MAPREDUCE-5355. MiniMRYarnCluster with localFs does not work on Windows.
|
|
(Chuan Liu via cnauroth)
|
|
|
|
MAPREDUCE-5359. JobHistory should not use File.separator to match timestamp
|
|
in path. (Chuan Liu via cnauroth)
|
|
|
|
MAPREDUCE-5357. Job staging directory owner checking could fail on Windows.
|
|
(Chuan Liu via cnauroth)
|
|
|
|
MAPREDUCE-5360. TestMRJobClient fails on Windows due to path format.
|
|
(Chuan Liu via cnauroth)
|
|
|
|
MAPREDUCE-5366. TestMRAsyncDiskService fails on Windows. (Chuan Liu via
|
|
cnauroth)
|
|
|
|
MAPREDUCE-5187. Create mapreduce command scripts on Windows. (Chuan Liu via
|
|
cnauroth)
|
|
|
|
MAPREDUCE-4374. Fix child task environment variable config and add support
|
|
for Windows. (Chuan Liu via cnauroth)
|
|
|
|
Release 2.0.5-alpha - 06/06/2013
|
|
|
|
INCOMPATIBLE CHANGES
|
|
|
|
NEW FEATURES
|
|
|
|
IMPROVEMENTS
|
|
|
|
OPTIMIZATIONS
|
|
|
|
BUG FIXES
|
|
|
|
MAPREDUCE-5240 inside of FileOutputCommitter the initialized Credentials cache
|
|
appears to be empty. (vinodkv)
|
|
|
|
Release 2.0.4-alpha - 2013-04-25
|
|
|
|
INCOMPATIBLE CHANGES
|
|
|
|
NEW FEATURES
|
|
|
|
IMPROVEMENTS
|
|
|
|
OPTIMIZATIONS
|
|
|
|
BUG FIXES
|
|
|
|
MAPREDUCE-5006. Fix failing streaming tests due to MAPREDUCE-4994.
|
|
(Sandy Ryza via tomwhite)
|
|
|
|
MAPREDUCE-5088. MR Client gets an renewer token exception while Oozie is
|
|
submitting a job (Daryn Sharp via cos)
|
|
|
|
MAPREDUCE-5117. Changed MRClientProtocolPBClientImpl to be closeable and thus
|
|
fix failures in renewal of HistoryServer's delegations tokens. (Siddharth
|
|
Seth via vinodkv)
|
|
|
|
MAPREDUCE-5083. MiniMRCluster should use a random component when creating an
|
|
actual cluster (Siddharth Seth via hitesh)
|
|
|
|
MAPREDUCE-5094. Disabled memory monitoring by default in MiniMRYarnCluster
|
|
to avoid some downstream tests failing. (Siddharth Seth via vinodkv)
|
|
|
|
Release 2.0.3-alpha - 2013-02-06
|
|
|
|
INCOMPATIBLE CHANGES
|
|
|
|
MAPREDUCE-4123. Remove the 'mapred groups' command, which is no longer
|
|
supported. (Devaraj K via sseth)
|
|
|
|
MAPREDUCE-4938. Use token request messages defined in hadoop common.
|
|
(suresh)
|
|
|
|
NEW FEATURES
|
|
|
|
MAPREDUCE-4520. Added support for MapReduce applications to request for
|
|
CPU cores along-with memory post YARN-2. (acmurthy)
|
|
|
|
MAPREDUCE-4810. Added new admin command options for MR AM. (Jerry Chen via
|
|
vinodkv)
|
|
|
|
MAPREDUCE-4049. Experimental api to allow for alternate shuffle plugins.
|
|
(Avner BenHanoch via acmurthy)
|
|
|
|
MAPREDUCE-4807. Allow MapOutputBuffer to be pluggable. (masokan via tucu)
|
|
|
|
MAPREDUCE-4808. Refactor MapOutput and MergeManager to facilitate reuse
|
|
by Shuffle implementations. (masokan via tucu)
|
|
|
|
IMPROVEMENTS
|
|
|
|
MAPREDUCE-3678. The Map tasks logs should have the value of input
|
|
split it processed. (harsh)
|
|
|
|
MAPREDUCE-4616. Improve javadoc for MultipleOutputs. (Tony Burton via
|
|
acmurthy)
|
|
|
|
HADOOP-8911. CRLF characters in source and text files.
|
|
(Raja Aluri via suresh)
|
|
|
|
MAPREDUCE-4723. Fix warnings found by findbugs 2. (Sandy Ryza via eli)
|
|
|
|
MAPREDUCE-4703. Add the ability to start the MiniMRClientCluster using
|
|
the configurations used before it is being stopped. (ahmed.radwan via tucu)
|
|
|
|
MAPREDUCE-4845. ClusterStatus.getMaxMemory() and getUsedMemory() exist in
|
|
MR1 but not MR2. (Sandy Ryza via tomwhite)
|
|
|
|
MAPREDUCE-4899. Implemented a MR specific plugin for tracking finished
|
|
applications that YARN's ResourceManager doesn't keep track off anymore
|
|
(Derek Dagit via vinodkv)
|
|
|
|
MAPREDUCE-4920. Use security token protobuf definition from hadoop common.
|
|
(Suresh Srinivas via vinodkv)
|
|
|
|
MAPREDUCE-4907. TrackerDistributedCacheManager issues too many getFileStatus
|
|
calls. (sandyr via tucu)
|
|
|
|
MAPREDUCE-4949. Enable multiple pi jobs to run in parallel. (sandyr via tucu)
|
|
|
|
MAPREDUCE-4809. Change visibility of classes for pluggable sort changes.
|
|
(masokan via tucu)
|
|
|
|
MAPREDUCE-4838. Add additional fields like Locality, Avataar to the
|
|
JobHistory logs. (Zhijie Shen via sseth)
|
|
|
|
MAPREDUCE-4971. Minor extensibility enhancements to Counters &
|
|
FileOutputFormat. (Arun C Murthy via sseth)
|
|
|
|
MAPREDUCE-4977. Documentation for pluggable shuffle and pluggable sort.
|
|
(tucu)
|
|
|
|
OPTIMIZATIONS
|
|
|
|
MAPREDUCE-4893. Fixed MR ApplicationMaster to do optimal assignment of
|
|
containers to get maximum locality. (Bikas Saha via vinodkv)
|
|
|
|
BUG FIXES
|
|
|
|
MAPREDUCE-4607. Race condition in ReduceTask completion can result in Task
|
|
being incorrectly failed. (Bikas Saha via tomwhite)
|
|
|
|
MAPREDUCE-4646. Fixed MR framework to send diagnostic information correctly
|
|
to clients in case of failed jobs also. (Jason Lowe via vinodkv)
|
|
|
|
MAPREDUCE-4674. Hadoop examples secondarysort has a typo
|
|
"secondarysrot" in the usage. (Robert Justice via eli)
|
|
|
|
MAPREDUCE-4681. Fix unit tests broken by HDFS-3910. (acmurthy)
|
|
|
|
MAPREDUCE-4712. mr-jobhistory-daemon.sh doesn't accept --config
|
|
(Vinod Kumar Vavilapalli via tgraves)
|
|
|
|
MAPREDUCE-4654. TestDistCp is ignored. (Sandy Ryza via tomwhite)
|
|
|
|
MAPREDUCE-4736. Remove obsolete option [-rootDir] from TestDFSIO.
|
|
(Brandon Li via suresh)
|
|
|
|
MAPREDUCE-4637. Handle TaskAttempt diagnostic updates while in the NEW and
|
|
UNASSIGNED states. (Mayank Bansal via sseth)
|
|
|
|
MAPREDUCE-1806. CombineFileInputFormat does not work with paths not on default FS. (Gera Shegalov via tucu)
|
|
|
|
MAPREDUCE-4777. In TestIFile, testIFileReaderWithCodec relies on
|
|
testIFileWriterWithCodec. (Sandy Ryza via tomwhite)
|
|
|
|
MAPREDUCE-4800. Cleanup o.a.h.mapred.MapTaskStatus - remove unused
|
|
code. (kkambatl via tucu)
|
|
|
|
MAPREDUCE-4861. Cleanup: Remove unused mapreduce.security.token.DelegationTokenRenewal.
|
|
(kkambatl via tucu)
|
|
|
|
MAPREDUCE-4856. TestJobOutputCommitter uses same directory as
|
|
TestJobCleanup. (Sandy Ryza via tomwhite)
|
|
|
|
MAPREDUCE-4278. Cannot run two local jobs in parallel from the same
|
|
gateway. (Sandy Ryza via tomwhite)
|
|
|
|
MAPREDUCE-1700. User supplied dependencies may conflict with MapReduce
|
|
system JARs. (tomwhite)
|
|
|
|
MAPREDUCE-4936. JobImpl uber checks for cpu are wrong (Arun C Murthy via
|
|
jlowe)
|
|
|
|
MAPREDUCE-4924. flakey test: org.apache.hadoop.mapred.TestClusterMRNotification.testMR.
|
|
(rkanter via tucu)
|
|
|
|
MAPREDUCE-4923. Add toString method to TaggedInputSplit. (sandyr via tucu)
|
|
|
|
MAPREDUCE-4948. Fix a failing unit test TestYARNRunner.testHistoryServerToken.
|
|
(Junping Du via sseth)
|
|
|
|
MAPREDUCE-4803. Remove duplicate copy of TestIndexCache. (Mariappan Asokan
|
|
via sseth)
|
|
|
|
MAPREDUCE-2264. Job status exceeds 100% in some cases.
|
|
(devaraj.k and sandyr via tucu)
|
|
|
|
MAPREDUCE-4969. TestKeyValueTextInputFormat test fails with Open JDK 7.
|
|
(Arpit Agarwal via suresh)
|
|
|
|
MAPREDUCE-4953. HadoopPipes misuses fprintf. (Andy Isaacson via atm)
|
|
|
|
Release 2.0.2-alpha - 2012-09-07
|
|
|
|
INCOMPATIBLE CHANGES
|
|
|
|
NEW FEATURES
|
|
|
|
MAPREDUCE-987. Exposing MiniDFS and MiniMR clusters as a single process
|
|
command-line. (ahmed via tucu)
|
|
|
|
MAPREDUCE-4417. add support for encrypted shuffle (tucu)
|
|
|
|
MAPREDUCE-4355. Add RunningJob.getJobStatus() (kkambatl via tucu)
|
|
|
|
MAPREDUCE-3451. Port Fair Scheduler to MR2 (pwendell via tucu)
|
|
|
|
MAPREDUCE-4438. Add a simple, generic client to run 'easy' AMs in YARN.
|
|
(Bikas Saha via acmurthy)
|
|
|
|
IMPROVEMENTS
|
|
|
|
MAPREDUCE-4157. ResourceManager should not kill apps that are well behaved
|
|
(Jason Lowe via bobby)
|
|
|
|
MAPREDUCE-4511. Add IFile readahead (ahmed via tucu)
|
|
|
|
MAPREDUCE-4408. allow jobs to set a JAR that is in the distributed cached
|
|
(rkanter via tucu)
|
|
|
|
MAPREDUCE-4440. Changed SchedulerApp and SchedulerNode to be a minimal
|
|
interface to allow schedulers to maintain their own. (acmurthy)
|
|
|
|
MAPREDUCE-4146. Support limits on task status string length and number of
|
|
block locations in branch-2. (Ahmed Radwan via tomwhite)
|
|
|
|
MAPREDUCE-3871. Allow symlinking in LocalJobRunner DistributedCache.
|
|
(tomwhite)
|
|
|
|
MAPREDUCE-3921. MR AM should act on node health status changes.
|
|
(Bikas Saha via sseth)
|
|
|
|
MAPREDUCE-2220. Fix new API FileOutputFormat-related typos in
|
|
mapred-default.xml (Rui Kubo via harsh)
|
|
|
|
MAPREDUCE-3907. Document entries mapred-default.xml for the
|
|
jobhistory server. (Eugene Koontz via harsh)
|
|
|
|
MAPREDUCE-3906. Fix inconsistency in documentation regarding
|
|
mapreduce.jobhistory.principal. (Eugene Koontz via harsh)
|
|
|
|
MAPREDUCE-4432. Confusing warning message when GenericOptionsParser
|
|
is not used. (Gabriel Reid via harsh)
|
|
|
|
MAPREDUCE-4427. Added an 'unmanaged' mode for AMs so as to ease
|
|
development of new applications. (Bikas Saha via acmurthy)
|
|
|
|
MAPREDUCE-4447. Remove aop from cruft from the ant build. (eli)
|
|
|
|
MAPREDUCE-3289. Make use of fadvise in the NM's shuffle handler.
|
|
(Todd Lipcon and Siddharth Seth via sseth)
|
|
|
|
MAPREDUCE-4580. Change MapReduce to use the yarn-client module.
|
|
(Vinod Kumar Vavilapalli via sseth)
|
|
|
|
MAPREDUCE-4579. Split TestTaskAttempt into two so as to pass tests on
|
|
jdk7. (Thomas Graves via vinodkv)
|
|
|
|
MAPREDUCE-4638. MR AM supplies MapReduce jars in classpath rather than
|
|
rely on YARN. (acmurthy)
|
|
|
|
BUG FIXES
|
|
|
|
MAPREDUCE-4422. YARN_APPLICATION_CLASSPATH needs a documented default value in
|
|
YarnConfiguration. (ahmed via tucu)
|
|
|
|
MAPREDUCE-4406. Users should be able to specify the MiniCluster ResourceManager
|
|
and JobHistoryServer ports. (ahmed via tucu)
|
|
|
|
MAPREDUCE-4407. Add hadoop-yarn-server-tests-<version>-tests.jar to hadoop dist
|
|
package. (ahmed via tucu)
|
|
|
|
MAPREDUCE-4465. Update description of yarn.nodemanager.address property.
|
|
(bowang via tucu)
|
|
|
|
MAPREDUCE-4342. Distributed Cache gives inconsistent result if cache files
|
|
get deleted from tasktracker. (mayank_bansal via tucu)
|
|
|
|
MAPREDUCE-4498. Remove hsqldb jar from Hadoop runtime classpath. (rkanter via tucu)
|
|
|
|
MAPREDUCE-4494. TestFifoScheduler failing with Metrics source QueueMetrics,q0=default
|
|
already exists!. (ahmed.radwan via tucu)
|
|
|
|
MAPREDUCE-4484. Incorrect IS_MINI_YARN_CLUSTER property name in YarnConfiguration.
|
|
(ahmed.radwan via tucu)
|
|
|
|
MAPREDUCE-4562. Support for "FileSystemCounter" legacy counter group name
|
|
for compatibility reasons is creating incorrect counter name.
|
|
(Jarek Jarcec Cecho via tomwhite)
|
|
|
|
MAPREDUCE-4068. Jars in lib subdirectory of the submittable JAR are not added to the
|
|
classpath (rkanter via tucu)
|
|
|
|
MAPREDUCE-4577. HDFS-3672 broke
|
|
TestCombineFileInputFormat.testMissingBlocks() test. (atm)
|
|
|
|
MAPREDUCE-4470. Fix TestCombineFileInputFormat.testForEmptyFile (ikatsov via tucu)
|
|
|
|
MAPREDUCE-4608. hadoop-mapreduce-client is missing some dependencies.
|
|
(tucu via tomwhite)
|
|
|
|
MAPREDUCE-4610. Support deprecated mapreduce.job.counters.limit property in
|
|
MR2. (tomwhite)
|
|
|
|
MAPREDUCE-4629. Remove JobHistory.DEBUG_MODE (Karthik Kambatla via bobby)
|
|
|
|
MAPREDUCE-4642. MiniMRClientClusterFactory should not use job.setJar() (rkanter via tucu)
|
|
|
|
MAPREDUCE-4148. MapReduce should not have a compile-time dependency on
|
|
HDFS. (tomwhite)
|
|
|
|
MAPREDUCE-4250. hadoop-config.sh missing variable exports, causes Yarn
|
|
jobs to fail with ClassNotFoundException MRAppMaster. (phunt via tucu)
|
|
|
|
MAPREDUCE-4002. MultiFileWordCount job fails if the input path is not
|
|
from default file system. (Bhallamudi Venkata Siva Kamesh via todd)
|
|
|
|
MAPREDUCE-4274 MapOutputBuffer should use native byte order for kvmeta.
|
|
(todd via bobby)
|
|
|
|
MAPREDUCE-4262. NM gives wrong log message saying "Connected to
|
|
ResourceManager" before trying to connect. (Devaraj K via tgraves)
|
|
|
|
MAPREDUCE-4276. Allow setting yarn.nodemanager.delete.debug-delay-sec
|
|
property to "-1" for easier container debugging. (ahmed via tucu)
|
|
|
|
MAPREDUCE-4224. TestFifoScheduler throws
|
|
org.apache.hadoop.metrics2.MetricsException (Devaraj K via tgraves)
|
|
|
|
MAPREDUCE-3493. Add the default mapreduce.shuffle.port property
|
|
to mapred-default.xml (Madhukara Phatak via harsh)
|
|
|
|
MAPREDUCE-4307. TeraInputFormat calls FileSystem.getDefaultBlockSize()
|
|
without a Path - Failure when using ViewFileSystem. (Ahmed Radwan via eli)
|
|
|
|
MAPREDUCE-4313. TestTokenCache doesn't compile due
|
|
TokenCache.getDelegationToken compilation error (bobby)
|
|
|
|
MAPREDUCE-3873. Fixed NodeManagers' decommissioning at RM to accept IP
|
|
addresses also. (xieguiming via vinodkv)
|
|
|
|
MAPREDUCE-4306. Fix distributed shell to work with users other than the one
|
|
running the daemons. (Ahmed Radwan via sseth)
|
|
|
|
MAPREDUCE-4031. Prevent a Node Manager hang during shutdown.
|
|
(Devaraj K via sseth)
|
|
|
|
MAPREDUCE-4336. Distributed Shell fails when used with the CapacityScheduler
|
|
(ahmed via tucu)
|
|
|
|
MAPREDUCE-4290. Fix Yarn Applicaiton Status to MR JobState conversion.
|
|
(Devaraj K via sseth)
|
|
|
|
MAPREDUCE-2289. Permissions race can make getStagingDir fail on local filesystem
|
|
(ahmed via tucu)
|
|
|
|
MAPREDUCE-4372. Deadlock in Resource Manager (Devaraj K via bobby)
|
|
|
|
MAPREDUCE-4376. TestClusterMRNotification times out (Kihwal Lee via bobby)
|
|
|
|
MAPREDUCE-4383. HadoopPipes.cc needs to include unistd.h.
|
|
(Andy Isaacson via eli)
|
|
|
|
MAPREDUCE-2739. Update installation docs (remove YarnClientFactory) (bowang via tucu)
|
|
|
|
MAPREDUCE-3993. Graceful handling of codec errors during decompression
|
|
(kkambatl via tucu)
|
|
|
|
MAPREDUCE-4416. Some tests fail if Clover is enabled (Kihwal Lee via bobby)
|
|
|
|
MAPREDUCE-4441. Fix build issue caused by MR-3451 (kkambatl via tucu)
|
|
|
|
HADOOP-8499. Lower min.user.id to 500 for the tests.
|
|
(Colin Patrick McCabe via eli)
|
|
|
|
MAPREDUCE-4395. Possible NPE at ClientDistributedCacheManager
|
|
#determineTimestamps (Bhallamudi via bobby)
|
|
|
|
MAPREDUCE-4380. Empty Userlogs directory is getting created under logs
|
|
directory (Devaraj K via bobby)
|
|
|
|
MAPREDUCE-4649. Ensure MapReduce JobHistory Daemon doens't assume
|
|
HADOOP_YARN_HOME and HADOOP_MAPRED_HOME are the same. (vinodkv via
|
|
acmurthy)
|
|
|
|
Release 2.0.0-alpha - 05-23-2012
|
|
|
|
INCOMPATIBLE CHANGES
|
|
|
|
MAPREDUCE-3545. Remove Avro RPC. (suresh)
|
|
|
|
NEW FEATURES
|
|
|
|
IMPROVEMENTS
|
|
|
|
MAPREDUCE-2887. Due to HADOOP-7524, change RPC to allow multiple protocols
|
|
including multuple versions of the same protocol (Sanjay Radia)
|
|
|
|
MAPREDUCE-2934. MR portion of HADOOP-7607 - Simplify the RPC proxy cleanup
|
|
process (atm)
|
|
|
|
HADOOP-7862. MR changes to work with HADOOP 7862: Move the support for
|
|
multiple protocols to lower layer so that Writable, PB and Avro can all
|
|
use it (Sanjay Radia)
|
|
|
|
MAPREDUCE-3909 Javadoc the Service interfaces (stevel)
|
|
|
|
MAPREDUCE-3885. Avoid an unnecessary copy for all requests/responses in
|
|
MRs ProtoOverHadoopRpcEngine. (Devaraj Das via sseth)
|
|
|
|
MAPREDUCE-3991. Streaming FAQ has some wrong instructions about input files
|
|
splitting. (harsh)
|
|
|
|
MAPREDUCE-3773. Add queue metrics with buckets for job run times. (omalley
|
|
via acmurthy)
|
|
|
|
MAPREDUCE-3970. Add ServiceOperations class to aid working with Services
|
|
(stevel)
|
|
|
|
MAPREDUCE-3353. Add a channel between RM and AM to get information on
|
|
nodes. (Bikas Saha via acmurthy)
|
|
|
|
MAPREDUCE-3955. Change MR to use ProtobufRpcEngine from hadoop-common
|
|
instead of ProtoOverHadoopRpcEngine. (Jitendra Nath Pandey via sseth)
|
|
|
|
MAPREDUCE-4103. Fix HA docs for changes to shell command fencer args (todd)
|
|
|
|
MAPREDUCE-4093. Improve RM WebApp start up when proxy address is not set
|
|
(Devaraj K vai bobby)
|
|
|
|
MAPREDUCE-4138. Reduce memory usage of counters due to non-static nested
|
|
classes. (tomwhite)
|
|
|
|
MAPREDUCE-3883. Document yarn.nodemanager.delete.debug-delay-sec
|
|
configuration property (Eugene Koontz via tgraves)
|
|
|
|
MAPREDUCE-4219. make default container-executor.conf.dir be a path
|
|
relative to the container-executor binary. (rvs via tucu)
|
|
|
|
MAPREDUCE-4205. retrofit all JVM shutdown hooks to use ShutdownHookManager
|
|
(tucu)
|
|
|
|
OPTIMIZATIONS
|
|
|
|
BUG FIXES
|
|
|
|
MAPREDUCE-3740. Fixed broken mapreduce compilation after the patch for
|
|
HADOOP-7965. (Devaraj K via vinodkv)
|
|
|
|
MAPREDUCE-3818. Fixed broken compilation in TestSubmitJob after the patch
|
|
for HDFS-2895. (Suresh Srinivas via vinodkv)
|
|
|
|
MAPREDUCE-2942. TestNMAuditLogger.testNMAuditLoggerWithIP failing (Thomas
|
|
Graves via mahadev)
|
|
|
|
MAPREDUCE-3933. Failures because MALLOC_ARENA_MAX is not set (ahmed via tucu)
|
|
|
|
MAPREDUCE-3728. ShuffleHandler can't access results when configured in a
|
|
secure mode (ahmed via tucu)
|
|
|
|
MAPREDUCE-3952. In MR2, when Total input paths to process == 1,
|
|
CombinefileInputFormat.getSplits() returns 0 split. (zhenxiao via tucu)
|
|
|
|
MAPREDUCE-3578. Starting nodemanager as root gives "Unknown -jvm option"
|
|
(tomwhite)
|
|
|
|
MAPREDUCE-3348. Fixed a bug in MR client to redirect to JobHistoryServer
|
|
correctly when RM forgets the app. (Devaraj K via vinodkv)
|
|
|
|
MAPREDUCE-3974. TestSubmitJob in MR1 tests doesn't compile after HDFS-162
|
|
merge. (atm)
|
|
|
|
MAPREDUCE-4007. JobClient getJob(JobID) should return NULL if the job
|
|
does not exist (for backwards compatibility) (tucu)
|
|
|
|
MAPREDUCE-3431 NPE in Resource Manager shutdown. (stevel)
|
|
|
|
MAPREDUCE-4010. TestWritableJobConf fails on trunk (tucu via bobby)
|
|
|
|
MAPREDUCE-3992. Reduce fetcher doesn't verify HTTP status code of response
|
|
(todd)
|
|
|
|
MAPREDUCE-4066. Use default value when fetching MR_AM_STAGING_DIR
|
|
(xieguiming via harsh)
|
|
|
|
MAPREDUCE-3377. Added a unit test to ensure OutputCommitter.checkOutputSpecs
|
|
is called prior to copying job.xml. (Jane Chen via acmurthy)
|
|
|
|
MAPREDUCE-4081. TestMROutputFormat.java does not compile (Jason Lowe via
|
|
bobby)
|
|
|
|
MAPREDUCE-4082. hadoop-mapreduce-client-app's mrapp-generated-classpath
|
|
file should not be in the module JAR (tucu)
|
|
|
|
MAPREDUCE-3916. various issues with running yarn proxyserver (devaraj via tucu)
|
|
|
|
MAPREDUCE-4091. tools testcases failing because of MAPREDUCE-4082 (tucu)
|
|
|
|
MAPREDUCE-4095. TestJobInProgress#testLocality uses a bogus topology.
|
|
(Colin Patrick McCabe via eli)
|
|
|
|
MAPREDUCE-4098. TestMRApps testSetClasspath fails (tucu)
|
|
|
|
MAPREDUCE-4097. tools testcases fail because missing mrapp-generated-classpath
|
|
file in classpath (rvs via tucu)
|
|
|
|
MAPREDUCE-4113. Fix tests org.apache.hadoop.mapred.TestClusterMRNotification
|
|
(Devaraj K via bobby)
|
|
|
|
MAPREDUCE-4112. Fix tests org.apache.hadoop.mapred.TestClusterMapReduceTestCase
|
|
(Devaraj K via bobby)
|
|
|
|
MAPREDUCE-4111. Fix tests in org.apache.hadoop.mapred.TestJobName (Devaraj
|
|
K via bobby)
|
|
|
|
MAPREDUCE-4110. Fix tests in org.apache.hadoop.mapred.TestMiniMRClasspath &
|
|
org.apache.hadoop.mapred.TestMiniMRWithDFSWithDistinctUsers (Devaraj K via
|
|
bobby)
|
|
|
|
MAPREDUCE-4105. Yarn RackResolver ignores rack configurations.
|
|
(Ahmed Radwan via tomwhite)
|
|
|
|
MAPREDUCE-3869. Fix classpath for DistributedShell application. (Devaraj K
|
|
via sseth)
|
|
|
|
MAPREDUCE-4057. Update RAID for the HA and fsdataset changes. (Devaraj K
|
|
via szetszwo)
|
|
|
|
MAPREDUCE-4076. Stream job fails with ZipException when use yarn jar
|
|
command (Devaraj K via bobby)
|
|
|
|
MAPREDUCE-4108. Fix tests in org.apache.hadoop.util.TestRunJar
|
|
(Devaraj K via tgraves)
|
|
|
|
MAPREDUCE-4107. Fix tests in org.apache.hadoop.ipc.TestSocketFactory
|
|
(Devaraj K via tgraves)
|
|
|
|
MAPREDUCE-4147. YARN should not have a compile-time dependency on HDFS.
|
|
(tomwhite)
|
|
|
|
MAPREDUCE-4008. ResourceManager throws MetricsException on start up
|
|
saying QueueMetrics MBean already exists (Devaraj K via tgraves)
|
|
|
|
MAPREDUCE-3867. MiniMRYarn/MiniYarn uses fixed ports (tucu)
|
|
|
|
MAPREDUCE-4141. clover integration broken, also mapreduce poms are
|
|
pulling in clover as a dependency. (phunt via tucu)
|
|
|
|
MAPREDUCE-4193. broken doc link for yarn-default.xml in site.xml.
|
|
(phunt via tomwhite)
|
|
|
|
MAPREDUCE-4202. TestYarnClientProtocolProvider is broken (Daryn Sharp via
|
|
bobby)
|
|
|
|
MAPREDUCE-3173. MRV2 UI doesn't work properly without internet (Devaraj K
|
|
via bobby)
|
|
|
|
MAPREDUCE-3958. RM: Remove RMNodeState and replace it with NodeState
|
|
(Bikas Saha via bobby)
|
|
|
|
MAPREDUCE-4231. Update RAID to use the new BlockCollection interface.
|
|
(szetszwo)
|
|
|
|
MAPREDUCE-4483. 2.0 build does not work (John George via bobby)
|
|
|
|
MAPREDUCE-4444. nodemanager fails to start when one of the local-dirs is
|
|
bad (Jason Lowe via bobby)
|
|
|
|
Release 0.23.10 - UNRELEASED
|
|
|
|
INCOMPATIBLE CHANGES
|
|
|
|
NEW FEATURES
|
|
|
|
IMPROVEMENTS
|
|
|
|
OPTIMIZATIONS
|
|
|
|
MAPREDUCE-1981. Improve getSplits performance by using listLocatedStatus
|
|
(Hairong Kuang and Jason Lowe via jlowe)
|
|
|
|
BUG FIXES
|
|
|
|
MAPREDUCE-3193. FileInputFormat doesn't read files recursively in the
|
|
input path dir (Devaraj K via jlowe)
|
|
|
|
MAPREDUCE-5380. Invalid mapred command should return non-zero exit code
|
|
(Stephen Chu via jlowe)
|
|
|
|
MAPREDUCE-5317. Stale files left behind for failed jobs (Ravi Prakash via
|
|
jlowe)
|
|
|
|
MAPREDUCE-5251. Reducer should not implicate map attempt if it has
|
|
insufficient space to fetch map output (Ashwin Shankar via jlowe)
|
|
|
|
MAPREDUCE-5419. TestSlive is getting FileNotFound Exception (Robert Parker
|
|
via jlowe)
|
|
|
|
MAPREDUCE-5440. TestCopyCommitter Fails on JDK7 (Robert Parker via jlowe)
|
|
|
|
MAPREDUCE-5001. LocalJobRunner has race condition resulting in job
|
|
failures (Sandy Ryza via jlowe)
|
|
|
|
MAPREDUCE-5475. MRClientService does not verify ACLs properly (jlowe)
|
|
|
|
MAPREDUCE-5504. mapred queue -info inconsistent with types (Kousuke Saruta
|
|
via tgraves)
|
|
|
|
Release 0.23.9 - 2013-07-08
|
|
|
|
INCOMPATIBLE CHANGES
|
|
|
|
NEW FEATURES
|
|
|
|
IMPROVEMENTS
|
|
|
|
MAPREDUCE-5084. fix coverage org.apache.hadoop.mapreduce.v2.app.webapp and
|
|
org.apache.hadoop.mapreduce.v2.hs.webapp (Aleksey Gorshkov via jeagles)
|
|
|
|
OPTIMIZATIONS
|
|
|
|
MAPREDUCE-5268. Improve history server startup performance (Karthik
|
|
Kambatla via jlowe)
|
|
|
|
BUG FIXES
|
|
|
|
MAPREDUCE-5308. Shuffling to memory can get out-of-sync when fetching
|
|
multiple compressed map outputs (Nathan Roberts via jlowe)
|
|
|
|
MAPREDUCE-5315. DistCp reports success even on failure. (mithun and jlowe
|
|
via daryn)
|
|
|
|
MAPREDUCE-4019. -list-attempt-ids is not working (Ashwin Shankar,
|
|
Devaraj K, and B Anil Kumar via jlowe)
|
|
|
|
MAPREDUCE-5316. job -list-attempt-ids command does not handle illegal
|
|
task-state (Ashwin Shankar via jlowe)
|
|
|
|
Release 0.23.8 - 2013-06-05
|
|
|
|
INCOMPATIBLE CHANGES
|
|
|
|
NEW FEATURES
|
|
|
|
IMPROVEMENTS
|
|
|
|
MAPREDUCE-5065. DistCp should skip checksum comparisons if block-sizes
|
|
are different on source/target (Mithun Radhakrishnan via kihwal)
|
|
|
|
OPTIMIZATIONS
|
|
|
|
BUG FIXES
|
|
|
|
MAPREDUCE-5015. Coverage fix for org.apache.hadoop.mapreduce.tools.CLI
|
|
(Aleksey Gorshkov via tgraves)
|
|
|
|
MAPREDUCE-5147. Maven build should create
|
|
hadoop-mapreduce-client-app-VERSION.jar directly (Robert Parker via tgraves)
|
|
|
|
MAPREDUCE-4927. Historyserver 500 error due to NPE when accessing specific
|
|
counters page for failed job. (Ashwin Shankar via jlowe)
|
|
|
|
Release 0.23.7 - 2013-04-18
|
|
|
|
INCOMPATIBLE CHANGES
|
|
|
|
NEW FEATURES
|
|
|
|
IMPROVEMENTS
|
|
|
|
MAPREDUCE-4905. test org.apache.hadoop.mapred.pipes
|
|
(Aleksey Gorshkov via bobby)
|
|
|
|
MAPREDUCE-4989. JSONify DataTables input data for Attempts page (Ravi
|
|
Prakash via jlowe)
|
|
|
|
MAPREDUCE-5027. Shuffle does not limit number of outstanding connections
|
|
(Robert Parker via jeagles)
|
|
|
|
MAPREDUCE-4972. Coverage fixing for org.apache.hadoop.mapreduce.jobhistory
|
|
(Aleksey Gorshkov via bobby)
|
|
|
|
OPTIMIZATIONS
|
|
|
|
MAPREDUCE-4946. Fix a performance problem for large jobs by reducing the
|
|
number of map completion event type conversions. (Jason Lowe via sseth)
|
|
|
|
MAPREDUCE-4822. Unnecessary conversions in History Events. (Chu Tong via
|
|
jlowe)
|
|
|
|
BUG FIXES
|
|
|
|
MAPREDUCE-4458. Warn if java.library.path is used for AM or Task
|
|
(Robert Parker via jeagles)
|
|
|
|
MAPREDUCE-4992. AM hangs in RecoveryService when recovering tasks with
|
|
speculative attempts (Robert Parker via jlowe)
|
|
|
|
MAPREDUCE-5009. Killing the Task Attempt slated for commit does not clear
|
|
the value from the Task commitAttempt member (Robert Parker via jeagles)
|
|
|
|
MAPREDUCE-4871. AM uses mapreduce.jobtracker.split.metainfo.maxsize but
|
|
mapred-default has mapreduce.job.split.metainfo.maxsize (Jason Lowe via
|
|
jeagles)
|
|
|
|
MAPREDUCE-4794. DefaultSpeculator generates error messages on normal
|
|
shutdown (Jason Lowe via jeagles)
|
|
|
|
MAPREDUCE-5043. Fetch failure processing can cause AM event queue to
|
|
backup and eventually OOM (Jason Lowe via bobby)
|
|
|
|
MAPREDUCE-5023. History Server Web Services missing Job Counters (Ravi
|
|
Prakash via tgraves)
|
|
|
|
MAPREDUCE-5060. Fetch failures that time out only count against the first
|
|
map task (Robert Joseph Evans via jlowe)
|
|
|
|
MAPREDUCE-5042. Reducer unable to fetch for a map task that was recovered
|
|
(Jason Lowe via bobby)
|
|
|
|
MAPREDUCE-5053. java.lang.InternalError from decompression codec cause
|
|
reducer to fail (Robert Parker via jeagles)
|
|
|
|
MAPREDUCE-4991. coverage for gridmix (Aleksey Gorshkov via tgraves)
|
|
|
|
MAPREDUCE-5007. fix coverage org.apache.hadoop.mapreduce.v2.hs (Aleksey
|
|
Gorshkov via tgraves)
|
|
|
|
MAPREDUCE-5137. AM web UI: clicking on Map Task results in 500 error
|
|
(Thomas Graves via jlowe)
|
|
|
|
Release 0.23.6 - 2013-02-06
|
|
|
|
INCOMPATIBLE CHANGES
|
|
|
|
NEW FEATURES
|
|
|
|
IMPROVEMENTS
|
|
|
|
MAPREDUCE-4811. JobHistoryServer should show when it was started in WebUI
|
|
About page (Ravi Prakash via jlowe)
|
|
|
|
OPTIMIZATIONS
|
|
|
|
BUG FIXES
|
|
|
|
MAPREDUCE-4802. Takes a long time to load the task list on the AM for
|
|
large jobs (Ravi Prakash via bobby)
|
|
|
|
MAPREDUCE-4764. repair TestBinaryTokenFile (Ivan A. Veselovsky via bobby)
|
|
|
|
MAPREDUCE-4825. JobImpl.finished doesn't expect ERROR as a final job state
|
|
(jlowe via bobby)
|
|
|
|
MAPREDUCE-4817. Hardcoded task ping timeout kills tasks localizing large
|
|
amounts of data (tgraves)
|
|
|
|
MAPREDUCE-4836. Elapsed time for running tasks on AM web UI tasks page is 0
|
|
(Ravi Prakash via jeagles)
|
|
|
|
MAPREDUCE-4842. Shuffle race can hang reducer (Mariappan Asokan via jlowe)
|
|
|
|
MAPREDUCE-4833. Task can get stuck in FAIL_CONTAINER_CLEANUP (Robert
|
|
Parker via jlowe)
|
|
|
|
MAPREDUCE-4793. Problem with adding resources when using both -files and
|
|
-file to hadoop streaming (jlowe)
|
|
|
|
MAPREDUCE-4890. Invalid TaskImpl state transitions when task fails while
|
|
speculating (jlowe)
|
|
|
|
MAPREDUCE-4902. Fix typo "receievd" should be "received" in log output
|
|
(Albert Chu via jlowe)
|
|
|
|
MAPREDUCE-4813. AM timing out during job commit (jlowe via bobby)
|
|
|
|
MAPREDUCE-4279. getClusterStatus() fails with null pointer exception when
|
|
running jobs in local mode (Devaraj K via bobby)
|
|
|
|
MAPREDUCE-4832. MR AM can get in a split brain situation (jlowe)
|
|
|
|
MAPREDUCE-4894. Renewal / cancellation of JobHistory tokens (Siddharth
|
|
Seth via tgraves)
|
|
|
|
MAPREDUCE-4819. AM can rerun job after reporting final job status to the
|
|
client (bobby and Bikas Saha via bobby)
|
|
|
|
MAPREDUCE-4913. TestMRAppMaster#testMRAppMasterMissingStaging occasionally
|
|
exits (Jason Lowe via tgraves)
|
|
|
|
MAPREDUCE-4848. TaskAttemptContext cast error during AM recovery (Jerry
|
|
Chen via jlowe)
|
|
|
|
MAPREDUCE-4921. JobClient should acquire HS token with RM principal
|
|
(daryn via bobby)
|
|
|
|
MAPREDUCE-4934. Maven RAT plugin is not checking all source files (tgraves)
|
|
|
|
MAPREDUCE-4678. Running the Pentomino example with defaults throws
|
|
java.lang.NegativeArraySizeException (Chris McConnell via harsh)
|
|
|
|
MAPREDUCE-4925. The pentomino option parser may be buggy.
|
|
(Karthik Kambatla via harsh)
|
|
|
|
Release 0.23.5 - 2012-11-28
|
|
|
|
INCOMPATIBLE CHANGES
|
|
|
|
NEW FEATURES
|
|
|
|
IMPROVEMENTS
|
|
|
|
MAPREDUCE-4596. Split StateMachine state from states seen by MRClientProtocol
|
|
for Job, Task and TaskAttempt. (Siddarth Seth via vinodkv)
|
|
|
|
MAPREDUCE-4752. Reduce MR AM memory usage through String Interning (Robert
|
|
Evans via tgraves)
|
|
|
|
MAPREDUCE-4266. remove Ant remnants from MR (tgraves via bobby)
|
|
|
|
MAPREDUCE-4666. JVM metrics for history server (jlowe via jeagles)
|
|
|
|
OPTIMIZATIONS
|
|
|
|
MAPREDUCE-4720. Browser thinks History Server main page JS is taking too
|
|
long (Ravi Prakash via bobby)
|
|
|
|
BUG FIXES
|
|
|
|
MAPREDUCE-4554. Job Credentials are not transmitted if security is turned
|
|
off (Benoy Antony via bobby)
|
|
|
|
MAPREDUCE-4705. Fix a bug in job history lookup, which makes older jobs
|
|
inaccessible despite the presence of a valid history file. (Jason Lowe
|
|
via sseth)
|
|
|
|
MAPREDUCE-4521. mapreduce.user.classpath.first incompatibility with 0.20/1.x
|
|
(Ravi Prakash via bobby)
|
|
|
|
MAPREDUCE-4721. Task startup time in JHS is same as job startup time.
|
|
(Ravi Prakash via bobby)
|
|
|
|
MAPREDUCE-4479. Fix parameter order in assertEquals() in
|
|
TestCombineInputFileFormat.java (Mariappan Asokan via bobby)
|
|
|
|
MAPREDUCE-4733. Reducer can fail to make progress during shuffle if too many
|
|
reducers complete consecutively. (Jason Lowe via vinodkv)
|
|
|
|
MAPREDUCE-4740. only .jars can be added to the Distributed Cache
|
|
classpath. (Robert Joseph Evans via jlowe)
|
|
|
|
MAPREDUCE-4229. Intern counter names in the JT (Miomir Boljanovic and bobby via daryn)
|
|
|
|
MAPREDUCE-4741. WARN and ERROR messages logged during normal AM shutdown.
|
|
(Vinod Kumar Vavilapalli via jlowe)
|
|
|
|
MAPREDUCE-4730. Fix Reducer's EventFetcher to scale the map-completion
|
|
requests slowly to avoid HADOOP-8942. (Jason Lowe via vinodkv)
|
|
|
|
MAPREDUCE-4748. Invalid event: T_ATTEMPT_SUCCEEDED at SUCCEEDED. (jlowe)
|
|
|
|
MAPREDUCE-4724. job history web ui applications page should be sorted to
|
|
display last app first (tgraves via bobby)
|
|
|
|
MAPREDUCE-4746. The MR Application Master does not have a config to set
|
|
environment variables (Rob Parker via bobby)
|
|
|
|
MAPREDUCE-4729. job history UI not showing all job attempts. (Vinod
|
|
Kumar Vavilapalli via jlowe)
|
|
|
|
MAPREDUCE-4763 repair test TestUmbilicalProtocolWithJobToken (Ivan A.
|
|
Veselovsky via bobby)
|
|
|
|
MAPREDUCE-4771. KeyFieldBasedPartitioner not partitioning properly when
|
|
configured (jlowe via bobby)
|
|
|
|
MAPREDUCE-4772. Fetch failures can take way too long for a map to be
|
|
restarted (bobby)
|
|
|
|
MAPREDUCE-4782. NLineInputFormat skips first line of last InputSplit
|
|
(Mark Fuhs via bobby)
|
|
|
|
MAPREDUCE-4774. JobImpl does not handle asynchronous task events in FAILED
|
|
state (jlowe via bobby)
|
|
|
|
MAPREDUCE-4751. AM stuck in KILL_WAIT for days (vinodkv via bobby)
|
|
|
|
MAPREDUCE-4787. TestJobMonitorAndPrint is broken (Rob Parker via bobby)
|
|
|
|
MAPREDUCE-4425. Speculation + Fetch failures can lead to a hung job (jlowe
|
|
via bobby)
|
|
|
|
MAPREDUCE-4786. Job End Notification retry interval is 5 milliseconds by
|
|
default (Ravi Prakash via bobby)
|
|
|
|
MAPREDUCE-4517. Too many INFO messages written out during AM to RM heartbeat
|
|
(Jason Lowe via tgraves)
|
|
|
|
MAPREDUCE-4797. LocalContainerAllocator can loop forever trying to contact
|
|
the RM (jlowe via bobby)
|
|
|
|
MAPREDUCE-4801. ShuffleHandler can generate large logs due to prematurely
|
|
closed channels (jlowe via bobby)
|
|
|
|
Release 0.23.4
|
|
|
|
INCOMPATIBLE CHANGES
|
|
|
|
NEW FEATURES
|
|
|
|
IMPROVEMENTS
|
|
|
|
MAPREDUCE-2786. Add compression option for TestDFSIO.
|
|
(Plamen Jeliazkov via shv)
|
|
|
|
MAPREDUCE-4645. Provide a random seed to Slive to make the sequence
|
|
of file names deterministic. (Ravi Prakash via shv)
|
|
|
|
MAPREDUCE-4651. Benchmarking random reads with DFSIO. (shv)
|
|
|
|
OPTIMIZATIONS
|
|
|
|
BUG FIXES
|
|
|
|
MAPREDUCE-4647. We should only unjar jobjar if there is a lib directory
|
|
in it. (Robert Evans via tgraves)
|
|
|
|
MAPREDUCE-4691. Historyserver can report "Unknown job" after RM says job
|
|
has completed (Robert Joseph Evans via jlowe)
|
|
|
|
MAPREDUCE-4689. JobClient.getMapTaskReports on failed job results in NPE
|
|
(jlowe via bobby)
|
|
|
|
Release 0.23.3
|
|
|
|
INCOMPATIBLE CHANGES
|
|
|
|
MAPREDUCE-4072. User set java.library.path seems to overwrite default
|
|
creating problems native lib loading (Anupam Seth via bobby)
|
|
|
|
MAPREDUCE-3812. Lower default allocation sizes, fix allocation
|
|
configurations and document them (Harsh J via bobby)
|
|
|
|
NEW FEATURES
|
|
|
|
IMPROVEMENTS
|
|
|
|
MAPREDUCE-4059. The history server should have a separate pluggable
|
|
storage/query interface. (Robert Evans via tgraves)
|
|
|
|
MAPREDUCE-3942. Randomize master key generation for
|
|
ApplicationTokenSecretManager and roll it every so often. (Vinod Kumar
|
|
Vavilapalli via sseth)
|
|
|
|
MAPREDUCE-4151. RM scheduler web page should filter apps to those that
|
|
are relevant to scheduling (Jason Lowe via tgraves)
|
|
|
|
MAPREDUCE-4134. Remove references of mapred.child.ulimit etc. since they
|
|
are not being used any more (Ravi Prakash via bobby)
|
|
|
|
MAPREDUCE-3972. Fix locking and exception issues in JobHistory server.
|
|
(Robert Joseph Evans via sseth)
|
|
|
|
MAPREDUCE-4161. create sockets consistently (Daryn Sharp via bobby)
|
|
|
|
MAPREDUCE-4079. Allow MR AppMaster to limit ephemeral port range.
|
|
(bobby via tgraves)
|
|
|
|
MAPREDUCE-4210. Expose listener address for WebApp (Daryn Sharp via bobby)
|
|
|
|
MAPREDUCE-4162. Correctly set token service (Daryn Sharp via bobby)
|
|
|
|
MAPREDUCE-4301. Dedupe some strings in MRAM for memory savings
|
|
(bobby via tgraves)
|
|
|
|
MAPREDUCE-4267. mavenize pipes (tgraves via bobby)
|
|
|
|
MAPREDUCE-4375. Show Configuration Tracability in MR UI (bobby
|
|
via tgraves)
|
|
|
|
MAPREDUCE-4569. Fixed TestHsWebServicesJobsQuery to pass on JDK7 by not
|
|
depending on test order. (Thomas Graves via vinodkv)
|
|
|
|
OPTIMIZATIONS
|
|
|
|
MAPREDUCE-3850. Avoid redundant calls for tokens in TokenCache (Daryn
|
|
Sharp via bobby)
|
|
|
|
BUG FIXES
|
|
|
|
MAPREDUCE-4092. commitJob Exception does not fail job (Jon Eagles via
|
|
bobby)
|
|
|
|
MAPREDUCE-4089. Hung Tasks never time out. (Robert Evans via tgraves)
|
|
|
|
MAPREDUCE-4024. RM webservices can't query on finalStatus (Tom Graves
|
|
via bobby)
|
|
|
|
MAPREDUCE-4060. Multiple SLF4J binding warning (Jason Lowe via bobby)
|
|
|
|
MAPREDUCE-3983. TestTTResourceReporting can fail, and should just be
|
|
deleted (Ravi Prakash via bobby)
|
|
|
|
MAPREDUCE-4012 Hadoop Job setup error leaves no useful info to users
|
|
(when LinuxTaskController is used). (tgraves)
|
|
|
|
MAPREDUCE-4062. AM Launcher thread can hang forever (tgraves via bobby)
|
|
|
|
MAPREDUCE-3988. mapreduce.job.local.dir doesn't point to a single directory
|
|
on a node. (Eric Payne via bobby)
|
|
|
|
MAPREDUCE-3999. Tracking link gives an error if the AppMaster hasn't
|
|
started yet (Ravi Prakash via bobby)
|
|
|
|
MAPREDUCE-4020. Web services returns incorrect JSON for deep queue tree
|
|
(Anupam Seth via tgraves)
|
|
|
|
MAPREDUCE-3672. Killed maps shouldn't be counted towards
|
|
JobCounter.NUM_FAILED_MAPS. (Anupam Seth via tgraves)
|
|
|
|
MAPREDUCE-3682 Tracker URL says AM tasks run on localhost.
|
|
(Ravi Prakash via tgraves)
|
|
|
|
MAPREDUCE-3082. Archive command take wrong path for input file with current
|
|
directory (John George via bobby)
|
|
|
|
MAPREDUCE-3650. testGetTokensForHftpFS() fails (Ravi Prakash via bobby)
|
|
|
|
MAPREDUCE-3621. TestDBJob and TestDataDrivenDBInputFormat ant tests fail
|
|
(Ravi Prakash via tgraves)
|
|
|
|
MAPREDUCE-4073. CS assigns multiple off-switch containers when using
|
|
multi-level-queues (Siddharth Seth via bobby)
|
|
|
|
MAPREDUCE-4051. Remove the empty hadoop-mapreduce-project/assembly/all.xml
|
|
file (Ravi Prakash via bobby)
|
|
|
|
MAPREDUCE-4117. mapred job -status throws NullPointerException (Devaraj K
|
|
via bobby)
|
|
|
|
MAPREDUCE-4099. ApplicationMaster may fail to remove staging directory
|
|
(Jason Lowe via bobby)
|
|
|
|
MAPREDUCE-4017. Add jobname to jobsummary log (tgraves and Koji Noguchi
|
|
via bobby)
|
|
|
|
MAPREDUCE-4040. History links should use hostname rather than IP address.
|
|
(Bhallamudi Venkata Siva Kamesh via sseth)
|
|
|
|
MAPREDUCE-4099 amendment. ApplicationMaster will remove staging directory
|
|
after the history service is stopped. (Jason Lowe via sseth)
|
|
|
|
MAPREDUCE-3932. Fix the TaskAttempt state machine to handle
|
|
CONTIANER_LAUNCHED and CONTIANER_LAUNCH_FAILED events in additional
|
|
states. (Robert Joseph Evans via sseth)
|
|
|
|
MAPREDUCE-4140. mapreduce classes incorrectly importing
|
|
"clover.org.apache.*" classes. (Patrick Hunt via tomwhite)
|
|
|
|
MAPREDUCE-4050. For tasks without assigned containers, changes the node
|
|
text on the UI to N/A instead of a link to null. (Bhallamudi Venkata Siva
|
|
Kamesh via sseth)
|
|
|
|
MAPREDUCE-4128. AM Recovery expects all attempts of a completed task to
|
|
also be completed. (Bikas Saha via bobby)
|
|
|
|
MAPREDUCE-4144. Fix a NPE in the ResourceManager when handling node
|
|
updates. (Jason Lowe via sseth)
|
|
|
|
MAPREDUCE-4156. ant build fails compiling JobInProgress (tgraves)
|
|
|
|
MAPREDUCE-4160. some mrv1 ant tests fail with timeout - due to 4156
|
|
(tgraves)
|
|
|
|
MAPREDUCE-4074. Client continuously retries to RM When RM goes down
|
|
before launching Application Master (xieguiming via tgraves)
|
|
|
|
MAPREDUCE-4159. Job is running in Uber mode after setting
|
|
"mapreduce.job.ubertask.maxreduces" to zero (Devaraj K via bobby)
|
|
|
|
MAPREDUCE-4165. Committing is misspelled as commiting in task logs
|
|
(John Eagles via bobby)
|
|
|
|
MAPREDUCE-4129. Lots of unneeded counters log messages (Ahmed Radwan via
|
|
bobby)
|
|
|
|
MAPREDUCE-3947. yarn.app.mapreduce.am.resource.mb not documented
|
|
(Devaraj K via bobby)
|
|
|
|
MAPREDUCE-4190. Improve web UI for task attempts userlog link (Tom Graves
|
|
via bobby)
|
|
|
|
MAPREDUCE-4133. MR over viewfs is broken (John George via bobby)
|
|
|
|
MAPREDUCE-4194. ConcurrentModificationError in DirectoryCollection
|
|
(Jonathan Eagles via bobby)
|
|
|
|
MAPREDUCE-3613. web service calls header contains 2 content types
|
|
(tgraves)
|
|
|
|
MAPREDUCE-4169. Container Logs appear in unsorted order (Jonathan Eagles
|
|
via bobby)
|
|
|
|
MAPREDUCE-4189. TestContainerManagerSecurity is failing (Devaraj K via
|
|
bobby)
|
|
|
|
MAPREDUCE-4209. junit dependency in hadoop-mapreduce-client is missing
|
|
scope test (Radim Kolar via bobby)
|
|
|
|
MAPREDUCE-4206. Sorting by Last Health-Update on the RM nodes page sorts
|
|
does not work correctly (Jonathon Eagles via tgraves)
|
|
|
|
MAPREDUCE-4212. TestJobClientGetJob sometimes fails
|
|
(Daryn Sharp via tgraves)
|
|
|
|
MAPREDUCE-4211. Error conditions (missing appid, appid not found) are
|
|
masked in the RM app page (Jonathan Eagles via bobby)
|
|
|
|
MAPREDUCE-4163. consistently set the bind address (Daryn Sharp via bobby)
|
|
|
|
MAPREDUCE-4048. NullPointerException exception while accessing the
|
|
Application Master UI (Devaraj K via bobby)
|
|
|
|
MAPREDUCE-4220. RM apps page starttime/endtime sorts are incorrect
|
|
(Jonathan Eagles via bobby)
|
|
|
|
MAPREDUCE-4226. ConcurrentModificationException in FileSystemCounterGroup.
|
|
(tomwhite)
|
|
|
|
MAPREDUCE-4215. RM app page shows 500 error on appid parse error
|
|
(Jonathon Eagles via tgraves)
|
|
|
|
MAPREDUCE-4237. TestNodeStatusUpdater can fail if localhost has a domain
|
|
associated with it (bobby)
|
|
|
|
MAPREDUCE-4233. NPE can happen in RMNMNodeInfo. (bobby)
|
|
|
|
MAPREDUCE-4238. mavenize data_join. (tgraves)
|
|
|
|
MAPREDUCE-4102. job counters not available in Jobhistory webui for
|
|
killed jobs (Bhallamudi Venkata Siva Kamesh via tgraves)
|
|
|
|
MAPREDUCE-3543. Mavenize Gridmix. (tgraves)
|
|
|
|
MAPREDUCE-4197. Include the hsqldb jar in the hadoop-mapreduce tar
|
|
file (Ravi Prakash via tgraves)
|
|
|
|
MAPREDUCE-4269. documentation: Gridmix has javadoc warnings in
|
|
StressJobFactory (Jonathon Eagles via tgraves).
|
|
|
|
MAPREDUCE-3870. Invalid App Metrics
|
|
(Bhallamudi Venkata Siva Kamesh via tgraves).
|
|
|
|
MAPREDUCE-4152. map task left hanging after AM dies trying to connect to RM
|
|
(Tom Graves via bobby)
|
|
|
|
MAPREDUCE-4297. Usersmap file in gridmix should not fail on empty lines
|
|
(Ravi Prakash via bobby)
|
|
|
|
MAPREDUCE-4302. NM goes down if error encountered during log aggregation
|
|
(Daryn Sharp via bobby)
|
|
|
|
MAPREDUCE-3350. Per-app RM page should have the list of application-attempts
|
|
like on the app JHS page (Jonathon Eagles via tgraves)
|
|
|
|
MAPREDUCE-3842. Stop webpages from automatic refreshing (tgraves)
|
|
|
|
MAPREDUCE-3927. Shuffle hang when set map.failures.percent
|
|
(Bhallamudi Venkata Siva Kamesh via tgraves)
|
|
|
|
MAPREDUCE-4311. Capacity scheduler.xml does not accept decimal values for
|
|
capacity and maximum-capacity settings (Karthik Kambatla via tgraves)
|
|
|
|
MAPREDUCE-4341. add types to capacity scheduler properties documentation
|
|
(Karthik Kambatla via tgraves)
|
|
|
|
MAPREDUCE-4270. Move the data_join test classes to the correct path.
|
|
(Thomas Graves via sseth)
|
|
|
|
MAPREDUCE-3889. job client tries to use /tasklog interface, but that
|
|
doesn't exist anymore (Devaraj K via bobby)
|
|
|
|
MAPREDUCE-4320. gridmix mainClass wrong in pom.xml (tgraves)
|
|
|
|
MAPREDUCE-4295. RM crashes due to DNS issue (tgraves)
|
|
|
|
MAPREDUCE-4228. mapreduce.job.reduce.slowstart.completedmaps is not working
|
|
properly (Jason Lowe via bobby)
|
|
|
|
MAPREDUCE-4392. Counters.makeCompactString() changed behavior from 0.20
|
|
(Jason Lowe via bobby)
|
|
|
|
MAPREDUCE-4384. Race conditions in IndexCache (Kihwal Lee via tgraves)
|
|
|
|
MAPREDUCE-4387. RM gets fatal error and exits during TestRM
|
|
(Kihwal Lee via tgraves)
|
|
|
|
MAPREDUCE-4379. Node Manager throws java.lang.OutOfMemoryError: Java heap
|
|
space due to org.apache.hadoop.fs.LocalDirAllocator.contexts (Devaraj K
|
|
via bobby)
|
|
|
|
MAPREDUCE-4402. TestFileInputFormat fails intermittently (Jason Lowe via
|
|
bobby)
|
|
|
|
MAPREDUCE-4300. OOM in AM can turn it into a zombie. (Robert Evans via
|
|
tgraves)
|
|
|
|
MAPREDUCE-4252. MR2 job never completes with 1 pending task (Tom White via
|
|
bobby)
|
|
|
|
MAPREDUCE-3940. ContainerTokens should have an expiry interval. (Siddharth
|
|
Seth and Vinod Kumar Vavilapalli via vinodkv)
|
|
|
|
MAPREDUCE-4419. ./mapred queue -info <queuename> -showJobs displays all
|
|
the jobs irrespective of <queuename> (Devaraj K via bobby)
|
|
|
|
MAPREDUCE-4299. Terasort hangs with MR2 FifoScheduler (Tom White via
|
|
bobby)
|
|
|
|
MAPREDUCE-4437. Race in MR ApplicationMaster can cause reducers to never be
|
|
scheduled (Jason Lowe via bobby)
|
|
|
|
MAPREDUCE-4449. Incorrect MR_HISTORY_STORAGE property name in JHAdminConfig
|
|
(Ahmed Radwan via bobby)
|
|
|
|
MAPREDUCE-4283. Display tail of aggregated logs by default (Jason Lowe via
|
|
bobby)
|
|
|
|
MAPREDUCE-4448. Fix NM crash during app cleanup if aggregation didn't
|
|
init. (Jason Lowe via daryn)
|
|
|
|
MAPREDUCE-3893. allow capacity scheduler configs maximum-applications and
|
|
maximum-am-resource-percent configurable on a per queue basis (tgraves via
|
|
bobby)
|
|
|
|
MAPREDUCE-4467. IndexCache failures due to missing synchronization
|
|
(Kihwal Lee via tgraves)
|
|
|
|
MAPREDUCE-4423. Potential infinite fetching of map output (Robert Evans
|
|
via tgraves)
|
|
|
|
MAPREDUCE-4456. LocalDistributedCacheManager can get an
|
|
ArrayIndexOutOfBounds when creating symlinks (Robert Evans via tgraves)
|
|
|
|
MAPREDUCE-4496. AM logs link is missing user name (Jason Lowe via bobby)
|
|
|
|
MAPREDUCE-4493. Distibuted Cache Compatability Issues (Robert Evans
|
|
via tgraves)
|
|
|
|
MAPREDUCE-4492. Configuring total queue capacity between 100.5 and 99.5 at
|
|
perticular level is sucessfull (Mayank Bansal via bobby)
|
|
|
|
MAPREDUCE-4457. mr job invalid transition TA_TOO_MANY_FETCH_FAILURE at
|
|
FAILED (Robert Evans via tgraves)
|
|
|
|
MAPREDUCE-4234. SortValidator.java is incompatible with multi-user or
|
|
parallel use (due to a /tmp file with static name) (Robert Evans via
|
|
jeagles)
|
|
|
|
MAPREDUCE-4504. SortValidator writes to wrong directory (Robert Evans
|
|
via tgraves)
|
|
|
|
MAPREDUCE-4503. Should throw InvalidJobConfException if duplicates found in
|
|
cacheArchives or cacheFiles (Robert Evans via jeagles)
|
|
|
|
MAPREDUCE-3782. teragen terasort jobs fail when using webhdfs:// (Jason
|
|
Lowe via bobby)
|
|
|
|
MAPREDUCE-4053. Counters group names deprecation is wrong, iterating over
|
|
group names deprecated names don't show up (Robert Evans via tgraves)
|
|
|
|
MAPREDUCE-3506. Calling getPriority on JobInfo after parsing a history log
|
|
with JobHistoryParser throws a NullPointerException (Jason Lowe via bobby)
|
|
|
|
MAPREDUCE-4570. ProcfsBasedProcessTree#constructProcessInfo() prints a
|
|
warning if procfsDir/<pid>/stat is not found. (Ahmed Radwan via bobby)
|
|
|
|
MAPREDUCE-4600. TestTokenCache.java from MRV1 no longer compiles (daryn
|
|
via bobby)
|
|
|
|
MAPREDUCE-4612. job summary file permissions not set when its created
|
|
(tgraves via bobby)
|
|
|
|
MAPREDUCE-4614. Simplify debugging a job's tokens (daryn via bobby)
|
|
|
|
MAPREDUCE-4611. MR AM dies badly when Node is decommissioned (Robert
|
|
Evans via tgraves)
|
|
|
|
MAPREDUCE-4604. In mapred-default, mapreduce.map.maxattempts &
|
|
mapreduce.reduce.maxattempts defaults are set to 4 as well as
|
|
mapreduce.job.maxtaskfailures.per.tracker. (Ravi Prakash via jeagles)
|
|
|
|
MAPREDUCE-4633. history server doesn't set permissions on all subdirs
|
|
(tgraves via bobby)
|
|
|
|
MAPREDUCE-4641. Exception in commitJob marks job as successful in job
|
|
history (Jason Lowe via bobby)
|
|
|
|
Release 0.23.2 - UNRELEASED
|
|
|
|
INCOMPATIBLE CHANGES
|
|
|
|
NEW FEATURES
|
|
|
|
IMPROVEMENTS
|
|
|
|
MAPREDUCE-3849. Change TokenCache's reading of the binary token file
|
|
(Daryn Sharp via bobby)
|
|
|
|
MAPREDUCE-3854. Fixed and reenabled tests related to MR child JVM's
|
|
environmental variables in TestMiniMRChildTask. (Tom White via vinodkv)
|
|
|
|
MAPREDUCE-3877 Add a test to formalise the current state transitions
|
|
of the yarn lifecycle. (stevel)
|
|
|
|
MAPREDUCE-3866. Fixed the bin/yarn script to not print the command line
|
|
unnecessarily. (vinodkv)
|
|
|
|
MAPREDUCE-3730. Modified RM to allow restarted NMs to be able to join the
|
|
cluster without waiting for expiry. (Jason Lowe via vinodkv)
|
|
|
|
MAPREDUCE-2793. Corrected AppIDs, JobIDs, TaskAttemptIDs to be of correct
|
|
format on the web pages. (Bikas Saha via vinodkv)
|
|
|
|
MAPREDUCE-3614. Fixed MR AM to close history file quickly and send a correct
|
|
final state to the RM when it is killed. (Ravi Prakash via vinodkv)
|
|
|
|
MAPREDUCE-3497. Added docs for YARN CLI. (tgraves via acmurthy)
|
|
|
|
MAPREDUCE-3954. Added new envs to separate heap size for different daemons
|
|
started via bin scripts. (Robert Joseph Evans via vinodkv)
|
|
|
|
MAPREDUCE-4025. AM can crash if task attempt reports bogus progress value
|
|
(Jason Lowe via bobby)
|
|
|
|
MAPREDUCE-4034. Unable to view task logs on history server with
|
|
mapreduce.job.acl-view-job=* (Jason Lowe and Siddarth Seth via bobby)
|
|
|
|
MAPREDUCE-4043. Secret keys set in Credentials are not seen by tasks
|
|
(Jason Lowe via bobby)
|
|
|
|
MAPREDUCE-3989. Cap space usage of default log4j rolling policy.
|
|
(Patrick Hunt via eli)
|
|
|
|
OPTIMIZATIONS
|
|
|
|
MAPREDUCE-3901. Modified JobHistory records in YARN to lazily load job and
|
|
task reports so as to improve UI response times. (Siddarth Seth via vinodkv)
|
|
|
|
MAPREDUCE-2855. Passing a cached class-loader to ResourceBundle creator to
|
|
minimize counter names lookup time. (Siddarth Seth via vinodkv)
|
|
|
|
MAPREDUCE-3944. Change the history jobs/ webservice to return partial job
|
|
info for a significant performance improvement. (Robert Joseph Evans via
|
|
sseth)
|
|
|
|
BUG FIXES
|
|
MAPREDUCE-3918 proc_historyserver no longer in command line arguments for
|
|
HistoryServer (Jon Eagles via bobby)
|
|
|
|
MAPREDUCE-3862. Nodemanager can appear to hang on shutdown due to lingering
|
|
DeletionService threads (Jason Lowe via bobby)
|
|
|
|
MAPREDUCE-3680. FifoScheduler web service rest API can print out invalid
|
|
JSON. (B Anil Kumar via tgraves)
|
|
|
|
MAPREDUCE-3852. Test TestLinuxResourceCalculatorPlugin failing. (Thomas
|
|
Graves via mahadev)
|
|
|
|
MAPREDUCE-3736. Variable substitution depth too large for fs.default.name
|
|
causes jobs to fail (ahmed via tucu).
|
|
|
|
MAPREDUCE-3864. Fix cluster setup docs for correct SecondaryNameNode
|
|
HTTPS parameters. (todd)
|
|
|
|
MAPREDUCE-3583. Change pid to String and stime to BigInteger in order to
|
|
avoid NumberFormatException caused by overflow. (Zhihong Yu via szetszwo)
|
|
|
|
MAPREDUCE-3634. Fixed all daemons to crash instead of hanging around when
|
|
their EventHandlers get exceptions. (vinodkv)
|
|
|
|
MAPREDUCE-3798. Fixed failing TestJobCleanup.testCusomCleanup() and moved it
|
|
to the maven build. (Ravi Prakash via vinodkv)
|
|
|
|
MAPREDUCE-3884. PWD should be first in the classpath of MR tasks (tucu)
|
|
|
|
MAPREDUCE-3878. Null user on filtered jobhistory job page (Jonathon Eagles
|
|
via tgraves)
|
|
|
|
MAPREDUCE-3738. MM can hang during shutdown if AppLogAggregatorImpl thread
|
|
dies unexpectedly (Jason Lowe via sseth)
|
|
|
|
MAPREDUCE-3904 Job history produced with mapreduce.cluster.acls.enabled
|
|
false can not be viewed with mapreduce.cluster.acls.enabled true
|
|
(Jonathon Eagles via tgraves)
|
|
|
|
MAPREDUCE-3910. Fixed a bug in CapacityScheduler LeafQueue which was causing
|
|
app-submission to fail. (John George via vinodkv)
|
|
|
|
MAPREDUCE-3686. Fixed two bugs in Counters because of which web app displays
|
|
zero counter values for framework counters. (Bhallamudi Venkata Siva Kamesh
|
|
via vinodkv)
|
|
|
|
MAPREDUCE-3913. RM application webpage is unresponsive after 2000 jobs
|
|
(Jason Lowe via tgraves)
|
|
|
|
MAPREDUCE-3922. Fixed build to not compile 32bit container-executor binary
|
|
by default on all platforms. (Hitesh Shah via vinodkv)
|
|
|
|
MAPREDUCE-3790. Broken pipe on streaming job can lead to truncated output for
|
|
a successful job (Jason Lowe via bobby)
|
|
|
|
MAPREDUCE-3816. capacity scheduler web ui bar graphs for used capacity wrong
|
|
(tgraves via bobby)
|
|
|
|
MAPREDUCE-3930. Fixed an NPE while accessing the AM page/webservice for a
|
|
task attempt without an assigned container. (Robert Joseph Evans via
|
|
sseth)
|
|
|
|
MAPREDUCE-3931. Changed PB implementation of LocalResource to take locks
|
|
so that race conditions don't fail tasks by inadvertantly changing the
|
|
timestamps. (Siddarth Seth via vinodkv)
|
|
|
|
MAPREDUCE-3687. If AM dies before it returns new tracking URL, proxy
|
|
redirects to http://N/A/ and doesn't return error code (Ravi Prakash via
|
|
bobby)
|
|
|
|
MAPREDUCE-3920. Revise yarn default port number selection
|
|
(Dave Thompson via tgraves)
|
|
|
|
MAPREDUCE-3903. Add support for mapreduce admin users. (Thomas Graves via
|
|
sseth)
|
|
|
|
MAPREDUCE-3706. Fix circular redirect error in job-attempts page. (bobby
|
|
via acmurthy)
|
|
|
|
MAPREDUCE-3896. Add user information to the delegation token issued by the
|
|
history server. (Vinod Kumar Vavilapalli via sseth)
|
|
|
|
MAPREDUCE-3792. Fix "bin/mapred job -list" to display all jobs instead of
|
|
only the jobs owned by the user. (Jason Lowe via vinodkv)
|
|
|
|
MAPREDUCE-3929. Fixed output of 'bin/mapred queue -showacl' command to
|
|
clarify ACLs for users. (John George via acmurthy)
|
|
|
|
MAPREDUCE-3960. Fix web-proxy to forward request to AM with configured
|
|
hostname or IP. (tgraves via acmurthy)
|
|
|
|
MAPREDUCE-3897. Fixed computation of maxActiveAppsPerUser for queues by
|
|
using capacity and not max-capacity since we are already scaling it by
|
|
userLimitFactor. (Eric Payne via acmurthy)
|
|
|
|
MAPREDUCE-3009. Fixed node link on JobHistory webapp. (chackaravarthy via
|
|
vinodkv)
|
|
|
|
MAPREDUCE-3964. ResourceManager does not have JVM metrics (Jason Lowe via
|
|
bobby)
|
|
|
|
MAPREDUCE-3034. Ensure NodeManager reboots itself on direction from
|
|
ResourceManager. (Devaraj K & Eric Payne via acmurthy)
|
|
|
|
MAPREDUCE-3976. TestRMContainerAllocator failing (Jason Lowe via bobby)
|
|
|
|
MAPREDUCE-3961. Map/ReduceSlotMillis computation incorrect (Siddharth Seth
|
|
via bobby)
|
|
|
|
MAPREDUCE-3977. LogAggregationService leaks log aggregator objects
|
|
(Jason Lowe via bobby)
|
|
|
|
MAPREDUCE-3975. Default value not set for Configuration parameter
|
|
mapreduce.job.local.dir (Eric Payne via bobby)
|
|
|
|
MAPREDUCE-3982. Fixed FileOutputCommitter to not err out for an 'empty-job'
|
|
whose tasks don't write any outputs. (Robert Joseph Evans via vinodkv)
|
|
|
|
MAPREDUCE-4005. Fixes broken AM container logs URL on ResourceManager
|
|
Application Info page. (Jason Lowe via sseth)
|
|
|
|
MAPREDUCE-4006. History server container log web UI sometimes combines
|
|
stderr/stdout/syslog contents together (Siddharth Seth via tgraves)
|
|
|
|
MAPREDUCE-4061. RM only has 1 AM launcher thread (tgraves via bobby)
|
|
|
|
Release 0.23.1 - 2012-02-17
|
|
|
|
INCOMPATIBLE CHANGES
|
|
|
|
NEW FEATURES
|
|
|
|
MAPREDUCE-3121. NodeManager should handle disk-failures (Ravi Gummadi via mahadev)
|
|
|
|
MAPREDUCE-2863. Support web services for YARN and MR components. (Thomas
|
|
Graves via vinodkv)
|
|
|
|
MAPREDUCE-3251. Network ACLs can prevent some clients to talk to MR ApplicationMaster
|
|
(Anupam Seth via mahadev)
|
|
|
|
IMPROVEMENTS
|
|
|
|
MAPREDUCE-3297. Moved log related components into yarn-common so that
|
|
HistoryServer and clients can use them without depending on the
|
|
yarn-server-nodemanager module. (Siddharth Seth via vinodkv)
|
|
|
|
MAPREDUCE-3336. Replaced guice internal.Preconditions api usage with the
|
|
public Preconditions API. (Thomas Graves via vinodkv)
|
|
|
|
MAPREDUCE-3280. Removed the unnecessary job user-name configuration in
|
|
mapred-site.xml. (vinodkv)
|
|
|
|
MAPREDUCE-3370. Fixed MiniMRYarnCluster and related tests to not use
|
|
a hard-coded path for the mr-app jar. (Ahmed Radwan via vinodkv)
|
|
|
|
MAPREDUCE-3325. Improvements to CapacityScheduler doc. (Thomas Graves
|
|
via mahadev)
|
|
|
|
MAPREDUCE-3341. Enhance logging of initalized queue limit values.
|
|
(Anupam Seth via mahadev)
|
|
|
|
MAPREDUCE-3243. Invalid tracking URL for streaming jobs (Jonathan Eagles
|
|
via mahadev)
|
|
|
|
MAPREDUCE-3331. Improvement to single node cluster setup documentation for
|
|
0.23 (Anupam Seth via mahadev)
|
|
|
|
MAPREDUCE-3102. Changed NodeManager to fail fast when LinuxContainerExecutor
|
|
has wrong configuration or permissions. (Hitesh Shah via vinodkv)
|
|
|
|
MAPREDUCE-3415. improve MiniMRYarnCluster & DistributedShell JAR resolution.
|
|
(tucu)
|
|
|
|
MAPREDUCE-3373. Hadoop scripts unconditionally source
|
|
"$bin"/../libexec/hadoop-config.sh. (Bruno Mahé via tomwhite)
|
|
|
|
MAPREDUCE-3372. HADOOP_PREFIX cannot be overridden.
|
|
(Bruno Mahé via tomwhite)
|
|
|
|
MAPREDUCE-3411. Performance Upgrade for jQuery (Jonathan Eagles via
|
|
mahadev)
|
|
|
|
MAPREDUCE-3371. Review and improve the yarn-api javadocs. (Ravi Prakash
|
|
via mahadev)
|
|
|
|
MAPREDUCE-3238. Small cleanup in SchedulerApp. (Todd Lipcon via mahadev)
|
|
|
|
MAPREDUCE-3413. RM web ui applications not sorted in any order by default.
|
|
(Jonathan Eagles via mahadev)
|
|
|
|
MAPREDUCE-3045. Fixed UI filters to not filter on hidden title-numeric
|
|
sort fields. (Jonathan Eagles via sseth)
|
|
|
|
MAPREDUCE-3448. TestCombineOutputCollector javac unchecked warning on mocked
|
|
generics (Jonathan Eagles via mahadev)
|
|
|
|
MAPREDUCE-3169 amendment. Deprecate MiniMRCluster. (Ahmed Radwan via
|
|
sseth)
|
|
|
|
MAPREDUCE-3369. Migrate MR1 tests to run on MR2 using the new interfaces
|
|
introduced in MAPREDUCE-3169. (Ahmed Radwan via tomwhite)
|
|
|
|
MAPREDUCE-3518. mapred queue -info <queue> -showJobs throws NPE.
|
|
(Jonathan Eagles via mahadev)
|
|
|
|
MAPREDUCE-3391. Making a trivial change to correct a log message in
|
|
DistributedShell app's AM. (Subroto Sanyal via vinodkv)
|
|
|
|
MAPREDUCE-3547. Added a bunch of unit tests for the the RM/NM webservices.
|
|
(Thomas Graves via acmurthy)
|
|
|
|
MAPREDUCE-3610. Remove use of the 'dfs.block.size' config for default block
|
|
size fetching. Use FS#getDefaultBlocksize instead. (Sho Shimauchi via harsh)
|
|
|
|
MAPREDUCE-3478. Cannot build against ZooKeeper 3.4.0. (Tom White via mahadev)
|
|
|
|
MAPREDUCE-3528. Fixed TaskHeartBeatHandler to use a new configuration
|
|
for the thread loop interval separate from task-timeout configuration
|
|
property. (Siddharth Seth via vinodkv)
|
|
|
|
MAPREDUCE-3312. Modified MR AM to not send a stop-container request for
|
|
a container that isn't launched at all. (Robert Joseph Evans via vinodkv)
|
|
|
|
MAPREDUCE-3382. Enhanced MR AM to use a proxy to ping the job-end
|
|
notification URL. (Ravi Prakash via vinodkv)
|
|
|
|
MAPREDUCE-3299. Added AMInfo table to the MR AM job pages to list all the
|
|
job-attempts when AM restarts and recovers. (Jonathan Eagles via vinodkv)
|
|
|
|
MAPREDUCE-3251. Network ACLs can prevent some clients to talk to MR AM.
|
|
Improved the earlier patch to not to JobHistoryServer repeatedly.
|
|
(Anupam Seth via vinodkv)
|
|
|
|
MAPREDUCE-3553. Add support for data returned when exceptions thrown from web
|
|
service apis to be in either xml or in JSON. (Thomas Graves via mahadev)
|
|
|
|
MAPREDUCE-3641. Making CapacityScheduler more conservative so as to
|
|
assign only one off-switch container in a single scheduling
|
|
iteration. (Arun C Murthy via vinodkv)
|
|
|
|
MAPREDUCE-3692. yarn-resourcemanager out and log files can get big. (eli)
|
|
|
|
MAPREDUCE-3710. Improved FileInputFormat to return better locality for the
|
|
last split. (Siddarth Seth via vinodkv)
|
|
|
|
MAPREDUCE-2765. DistCp Rewrite. (Mithun Radhakrishnan via mahadev)
|
|
|
|
MAPREDUCE-3737. The Web Application Proxy's is not documented very well.
|
|
(Robert Evans via mahadev)
|
|
|
|
MAPREDUCE-3699. Increased RPC handlers for all YARN servers to reasonable
|
|
values for working at scale. (Hitesh Shah via vinodkv)
|
|
|
|
MAPREDUCE-3693. Added mapreduce.admin.user.env to mapred-default.xml.
|
|
(Roman Shapshonik via acmurthy)
|
|
|
|
MAPREDUCE-3732. Modified CapacityScheduler to use only users with pending
|
|
requests for computing user-limits. (Arun C Murthy via vinodkv)
|
|
|
|
MAPREDUCE-3679. AM logs and others should not automatically refresh after every 1
|
|
second. (Vinod KV via mahadev)
|
|
|
|
MAPREDUCE-3754. Modified RM UI to filter applications based on state of the
|
|
applications. (vinodkv)
|
|
|
|
MAPREDUCE-3774. Moved yarn-default.xml to hadoop-yarn-common from
|
|
hadoop-server-common. (Mahadev Konar via vinodkv)
|
|
|
|
MAPREDUCE-3771. Un-deprecated the old mapred apis, port of MAPREDUCE-1735.
|
|
(acmurthy)
|
|
|
|
MAPREDUCE-3784. Fixed CapacityScheduler so that maxActiveApplications and
|
|
maxActiveApplicationsPerUser per queue are not too low for small
|
|
clusters. (Arun C Murthy via vinodkv)
|
|
|
|
OPTIMIZATIONS
|
|
|
|
MAPREDUCE-3567. Extraneous JobConf objects in AM heap. (Vinod Kumar
|
|
Vavilapalli via sseth)
|
|
|
|
MAPREDUCE-3399. Modifying ContainerLocalizer to send a heartbeat to NM
|
|
immediately after downloading a resource instead of always waiting for a
|
|
second. (Siddarth Seth via vinodkv)
|
|
|
|
MAPREDUCE-3568. Optimized Job's progress calculations in MR AM. (vinodkv)
|
|
|
|
MAPREDUCE-3569. TaskAttemptListener holds a global lock for all
|
|
task-updates. (Vinod Kumar Vavilapalli via sseth)
|
|
|
|
MAPREDUCE-3511. Removed a multitude of cloned/duplicate counters in the AM
|
|
thereby reducing the AM heap size and preventing full GCs. (vinodkv)
|
|
|
|
MAPREDUCE-3618. Fixed TaskHeartbeatHandler to not hold a global lock for all
|
|
task-updates. (Siddarth Seth via vinodkv)
|
|
|
|
MAPREDUCE-3512. Batching JobHistory flushing to DFS so that we don't flush
|
|
for every event slowing down AM. (Siddarth Seth via vinodkv)
|
|
|
|
MAPREDUCE-3718. Change default AM heartbeat interval to 1 second. (Hitesh
|
|
Shah via sseth)
|
|
|
|
MAPREDUCE-3360. Added information about lost/rebooted/decommissioned nodes
|
|
on the webapps. (Bhallamudi Venkata Siva Kamesh and Jason Lowe via vinodkv)
|
|
|
|
MAPREDUCE-3756. Made single shuffle limit configurable. (Hitesh Shah via
|
|
acmurthy)
|
|
|
|
MAPREDUCE-3811. Made jobclient-to-AM retries configurable. (sseth via
|
|
acmurthy)
|
|
|
|
BUG FIXES
|
|
|
|
MAPREDUCE-3221. Reenabled the previously ignored test in TestSubmitJob
|
|
and fixed bugs in it. (Devaraj K via vinodkv)
|
|
|
|
MAPREDUCE-3215. Reenabled and fixed bugs in the failing test
|
|
TestNoJobSetupCleanup. (Hitesh Shah via vinodkv)
|
|
|
|
MAPREDUCE-3219. Reenabled and fixed bugs in the failing test
|
|
TestDelegationToken. (Hitesh Shah via vinodkv)
|
|
|
|
MAPREDUCE-3217. Reenabled and fixed bugs in the failing ant test
|
|
TestAuditLogger. (Devaraj K via vinodkv)
|
|
|
|
MAPREDUCE-3291. App fail to launch due to delegation token not
|
|
found in cache (Robert Evans via mahadev)
|
|
|
|
MAPREDUCE-3344. o.a.h.mapreduce.Reducer since 0.21 blindly casts to
|
|
ReduceContext.ValueIterator. (Brock Noland via tomwhite)
|
|
|
|
MAPREDUCE-3342. Fixed JobHistoryServer to also show the job's queue
|
|
name. (Jonathan Eagles via vinodkv)
|
|
|
|
MAPREDUCE-3345. Fixed a race condition in ResourceManager that was causing
|
|
TestContainerManagerSecurity to fail sometimes. (Hitesh Shah via vinodkv)
|
|
|
|
MAPREDUCE-3333. Fixed bugs in ContainerLauncher of MR AppMaster due to
|
|
which per-container connections to NodeManager were lingering long enough
|
|
to hit the ulimits on number of processes. (vinodkv)
|
|
|
|
MAPREDUCE-3392. Fixed Cluster's getDelegationToken's API to return null
|
|
when there isn't a supported token. (John George via vinodkv)
|
|
|
|
MAPREDUCE-3379. Fixed LocalResourceTracker in NodeManager to remove deleted
|
|
cache entries correctly. (Siddharth Seth via vinodkv)
|
|
|
|
MAPREDUCE-3324. Not All HttpServer tools links (stacks,logs,config,metrics) are
|
|
accessible through all UI servers (Jonathan Eagles via mahadev)
|
|
|
|
MAPREDUCE-3355. Fixed MR AM's ContainerLauncher to handle node-command
|
|
timeouts correctly. (vinodkv)
|
|
|
|
MAPREDUCE-3407. Fixed pom files to refer to the correct MR app-jar needed
|
|
by the integration tests. (Hitesh Shah via vinodkv)
|
|
|
|
MAPREDUCE-3434. Nightly build broken (Hitesh Shah via mahadev)
|
|
|
|
MAPREDUCE-3447. mapreduce examples not working (mahadev)
|
|
|
|
MAPREDUCE-3444. trunk/0.23 builds broken (Hitesh Shah via mahadev)
|
|
|
|
MAPREDUCE-3454. [Gridmix] TestDistCacheEmulation is broken (Hitesh Shah
|
|
via mahadev)
|
|
|
|
MAPREDUCE-3408. yarn-daemon.sh unconditionnaly sets yarn.root.logger
|
|
(Bruno Mahe via mahadev)
|
|
|
|
MAPREDUCE-3329. Fixed CapacityScheduler to ensure maximum-capacity cannot
|
|
be lesser than capacity for any queue. (acmurthy)
|
|
|
|
MAPREDUCE-3464. mapreduce jsp pages missing DOCTYPE. (Dave Vronay via mattf)
|
|
|
|
MAPREDUCE-3265. Removed debug logs during job submission to LOG.debug to
|
|
cut down noise. (acmurthy)
|
|
|
|
MAPREDUCE-3433. Finding counters by legacy group name returns empty
|
|
counters. (tomwhite)
|
|
|
|
MAPREDUCE-3450. NM port info no longer available in JobHistory.
|
|
(Siddharth Seth via mahadev)
|
|
|
|
MAPREDUCE-3488. Streaming jobs are failing because the main class
|
|
isnt set in the pom files. (mahadev)
|
|
|
|
MAPREDUCE-3463. Second AM fails to recover properly when first AM is killed with
|
|
java.lang.IllegalArgumentException causing lost job. (Siddharth Seth via mahadev)
|
|
|
|
MAPREDUCE-3452. fifoscheduler web ui page always shows 0% used for the queue.
|
|
(Jonathan Eagles via mahadev)
|
|
|
|
MAPREDUCE-3443. JobClient and Job should function in the context of the
|
|
UGI which created them. (Mahadev Konar via sseth)
|
|
|
|
MAPREDUCE-3460. MR AM can hang if containers are allocated on a node
|
|
blacklisted by the AM. (Hitesh Shah and Robert Joseph Evans via sseth)
|
|
|
|
MAPREDUCE-3453. RM web ui application details page shows RM cluster about
|
|
information. (Jonathan Eagles via sseth)
|
|
|
|
MAPREDUCE-3479. JobClient#getJob cannot find local jobs. (tomwhite)
|
|
|
|
MAPREDUCE-3456. $HADOOP_PREFIX/bin/yarn should set defaults for
|
|
$HADOOP_*_HOME (Eric Payne via mahadev)
|
|
|
|
MAPREDUCE-3458. Fix findbugs warnings in hadoop-examples. (Devaraj K
|
|
via mahadev)
|
|
|
|
MAPREDUCE-3485. DISKS_FAILED -101 error code should be defined in same location as
|
|
ABORTED_CONTAINER_EXIT_STATUS. (Ravi Gummadi via mahadev)
|
|
|
|
MAPREDUCE-3496. Fixed client to print queue acls in consistent order.
|
|
(Jonathan Eagles via acmurthy)
|
|
|
|
MAPREDUCE-3147. Handle leaf queues with the same name properly. (Ravi Prakash via
|
|
mahadev)
|
|
|
|
MAPREDUCE-3327. RM web ui scheduler link doesn't show correct max value
|
|
for queues (Anupam Seth via mahadev)
|
|
|
|
MAPREDUCE-3513. Capacity Scheduler web UI has a spelling mistake for Memory.
|
|
(chackaravarthy via mahadev)
|
|
|
|
MAPREDUCE-3519. Fixed a deadlock in NodeManager LocalDirectories's handling
|
|
service. (Ravi Gummadi via vinodkv)
|
|
|
|
MAPREDUCE-3527. Fix minor API incompatibilities between 1.0 and 0.23.
|
|
(tomwhite)
|
|
|
|
MAPREDUCE-3328. mapred queue -list output inconsistent and missing child
|
|
queues. (Ravi Prakash via mahadev)
|
|
|
|
MAPREDUCE-3510. Capacity Scheduler inherited ACLs not displayed by mapred queue
|
|
-showacls (Jonathan Eagles via mahadev)
|
|
|
|
MAPREDUCE-3537. Fix race condition in DefaultContainerExecutor which led
|
|
to container localization occuring in wrong directories. (acmurthy)
|
|
|
|
MAPREDUCE-3542. Support "FileSystemCounter" legacy counter group name for
|
|
compatibility. (tomwhite)
|
|
|
|
MAPREDUCE-3426. Fixed MR AM in uber mode to write map intermediate outputs
|
|
in the correct directory to work properly in secure mode. (Hitesh Shah via
|
|
vinodkv)
|
|
|
|
MAPREDUCE-3541. Fix broken TestJobQueueClient test. (Ravi Prakash via
|
|
mahadev)
|
|
|
|
MAPREDUCE-3398. Fixed log aggregation to work correctly in secure mode.
|
|
(Siddharth Seth via vinodkv)
|
|
|
|
MAPREDUCE-3530. Fixed an NPE occuring during scheduling in the
|
|
ResourceManager. (Arun C Murthy via vinodkv)
|
|
|
|
MAPREDUCE-3484. Fixed JobEndNotifier to not get interrupted before completing
|
|
all its retries. (Ravi Prakash via vinodkv)
|
|
|
|
MAPREDUCE-3531. Fixed a race in ContainerTokenSecretManager. (Robert Joseph
|
|
Evans via sseth)
|
|
|
|
MAPREDUCE-3560. TestRMNodeTransitions is failing on trunk.
|
|
(Siddharth Seth via mahadev)
|
|
|
|
MAPREDUCE-3487. Fixed JobHistory web-UI to display links to single task's
|
|
counters' page. (Jason Lowe via vinodkv)
|
|
|
|
MAPREDUCE-3564. Fixed failures in TestStagingCleanup and TestJobEndNotifier
|
|
tests. (Siddharth Seth via vinodkv)
|
|
|
|
MAPREDUCE-3422. Counter display names are not being picked up. (Jonathan
|
|
Eagles via sseth)
|
|
|
|
MAPREDUCE-3366. Mapreduce component should use consistent directory structure
|
|
layout as HDFS/common (Eric Yang via mahadev)
|
|
|
|
MAPREDUCE-3387. Fixed AM's tracking URL to always go through the proxy, even
|
|
before the job started, so that it works properly with oozie throughout
|
|
the job execution. (Robert Joseph Evans via vinodkv)
|
|
|
|
MAPREDUCE-3579. ConverterUtils shouldn't include a port in a path from a url
|
|
without a port. (atm via harsh)
|
|
|
|
MAPREDUCE-3563. Fixed LocalJobRunner to work correctly with new mapreduce
|
|
apis. (acmurthy)
|
|
|
|
MAPREDUCE-3376. Fixed Task to ensure it passes reporter to combiners using
|
|
old MR api. (Subroto Sanyal via acmurthy)
|
|
|
|
MAPREDUCE-3339. Fixed MR AM to stop considering node blacklisting after the
|
|
number of nodes blacklisted crosses a threshold. (Siddharth Seth via vinodkv)
|
|
|
|
MAPREDUCE-3588. Fixed bin/yarn which was broken by MAPREDUCE-3366 so that
|
|
yarn daemons can start. (Arun C Murthy via vinodkv)
|
|
|
|
MAPREDUCE-3586. Modified CompositeService to avoid duplicate stop operations
|
|
thereby solving race conditions in MR AM shutdown. (vinodkv)
|
|
|
|
MAPREDUCE-3604. Fixed streaming to use new mapreduce.framework.name to
|
|
check for local mode. (acmurthy)
|
|
|
|
MAPREDUCE-3521. Fixed streaming to ensure it doesn't silently ignore
|
|
unknown arguments. (Robert Evans via acmurthy)
|
|
|
|
MAPREDUCE-3522. Ensure queues inherit ACLs from parent if they aren't
|
|
explicitly specified. (Jonathan Eagles via acmurthy)
|
|
|
|
MAPREDUCE-3608. Fixed compile issue with MAPREDUCE-3522. (mahadev via
|
|
acmurthy)
|
|
|
|
MAPREDUCE-3490. Fixed MapReduce AM to count failed maps also towards Reduce
|
|
ramp up. (Sharad Agarwal and Arun C Murthy via vinodkv)
|
|
|
|
MAPREDUCE-1744. DistributedCache creates its own FileSytem instance when
|
|
adding a file/archive to the path. (Dick King via tucu)
|
|
|
|
MAPREDUCE-3529. TokenCache does not cache viewfs credentials correctly
|
|
(sseth)
|
|
|
|
MAPREDUCE-3595. Add missing TestCounters#testCounterValue test from branch
|
|
1 to 0.23 (Tom White via sseth)
|
|
|
|
MAPREDUCE-3566. Fixed MR AM to construct CLC only once across all tasks.
|
|
(vinodkv via acmurthy)
|
|
|
|
MAPREDUCE-3572. Moved AM event dispatcher to a separate thread for
|
|
performance reasons. (vinodkv via acmurthy)
|
|
|
|
MAPREDUCE-3615. Fix some ant test failures. (Thomas Graves via sseth)
|
|
|
|
MAPREDUCE-3326. Added detailed information about queue's to the
|
|
CapacityScheduler web-ui. (Jason Lowe via acmurthy)
|
|
|
|
MAPREDUCE-3548. Added more unit tests for MR AM & JHS web-services.
|
|
(Thomas Graves via acmurthy)
|
|
|
|
MAPREDUCE-3617. Removed wrong default value for
|
|
yarn.resourcemanager.principal and yarn.nodemanager.principal. (Jonathan
|
|
Eagles via acmurthy)
|
|
|
|
MAPREDUCE-3183. hadoop-assemblies/src/main/resources/assemblies/hadoop-mapreduce-dist.xml
|
|
missing license header. (Hitesh Shah via tucu).
|
|
|
|
MAPREDUCE-3003. Publish MR JARs to Maven snapshot repository. (tucu)
|
|
|
|
MAPREDUCE-3204. mvn site:site fails on MapReduce. (tucu)
|
|
|
|
MAPREDUCE-3014. Rename and invert logic of '-cbuild' profile to 'native' and off
|
|
by default. (tucu)
|
|
|
|
MAPREDUCE-3477. Hadoop site documentation cannot be built anymore. (jeagles via tucu)
|
|
|
|
MAPREDUCE-3500. MRJobConfig creates an LD_LIBRARY_PATH using the platform ARCH. (tucu)
|
|
|
|
MAPREDUCE-3389. MRApps loads the 'mrapp-generated-classpath' file with
|
|
classpath from the build machine. (tucu)
|
|
|
|
MAPREDUCE-3544. gridmix build is broken, requires hadoop-archives to be added as
|
|
ivy dependency. (tucu)
|
|
|
|
MAPREDUCE-3557. MR1 test fail to compile because of missing hadoop-archives dependency.
|
|
(tucu)
|
|
|
|
MAPREDUCE-3624. Remove unnecessary dependency on JDK's tools.jar. (mahadev
|
|
via acmurthy)
|
|
|
|
MAPREDUCE-3616. Thread pool for launching containers in MR AM not
|
|
expanding as expected. (vinodkv via sseth)
|
|
|
|
MAPREDUCE-3639. Fixed TokenCache to work with absent FileSystem canonical
|
|
service-names. (Siddharth Seth via vinodkv)
|
|
|
|
MAPREDUCE-3380. Token infrastructure for running clients which are not kerberos
|
|
authenticated. (mahadev)
|
|
|
|
MAPREDUCE-3648. TestJobConf failing. (Thomas Graves via mahadev)
|
|
|
|
MAPREDUCE-3651. TestQueueManagerRefresh fails. (Thomas Graves via mahadev)
|
|
|
|
MAPREDUCE-3645. TestJobHistory fails. (Thomas Graves via mahadev)
|
|
|
|
MAPREDUCE-3652. org.apache.hadoop.mapred.TestWebUIAuthorization.testWebUIAuthorization
|
|
fails. (Thomas Graves via mahadev)
|
|
|
|
MAPREDUCE-3625. CapacityScheduler web-ui display of queue's used capacity is broken.
|
|
(Jason Lowe via mahadev)
|
|
|
|
MAPREDUCE-3596. Fix scheduler to handle cleaned up containers, which NMs
|
|
may subsequently report as running. (Vinod Kumar Vavilapalli via sseth)
|
|
|
|
MAPREDUCE-3656. Fixed a race condition in MR AM which is failing the sort
|
|
benchmark consistently. (Siddarth Seth via vinodkv)
|
|
|
|
MAPREDUCE-3532. Modified NM to report correct http address when an ephemeral
|
|
web port is configured. (Bhallamudi Venkata Siva Kamesh via vinodkv)
|
|
|
|
MAPREDUCE-3404. Corrected MR AM to honor speculative configuration and enable
|
|
speculating either maps or reduces. (Eric Payne via vinodkv)
|
|
|
|
MAPREDUCE-3649. Job End notification gives an error on calling back.
|
|
(Ravi Prakash via mahadev)
|
|
|
|
MAPREDUCE-3657. State machine visualize build fails. (Jason Lowe
|
|
via mahadev)
|
|
|
|
MAPREDUCE-2450. Fixed a corner case with interrupted communication threads
|
|
leading to a long timeout in Task. (Rajesh Balamohan via acmurthy)
|
|
|
|
MAPREDUCE-3669. Allow clients to talk to MR HistoryServer using both
|
|
delegation tokens and kerberos. (mahadev via acmurthy)
|
|
|
|
MAPREDUCE-3684. LocalDistributedCacheManager does not shut down its thread
|
|
pool (tomwhite)
|
|
|
|
MAPREDUCE-3582. Move successfully passing MR1 tests to MR2 maven tree.
|
|
(ahmed via tucu)
|
|
|
|
MAPREDUCE-3698. Client cannot talk to the history server in secure mode.
|
|
(mahadev)
|
|
|
|
MAPREDUCE-3689. RM web UI doesn't handle newline in job name.
|
|
(Thomas Graves via mahadev)
|
|
|
|
MAPREDUCE-3549. write api documentation for web service apis for RM, NM,
|
|
mapreduce app master, and job history server (Thomas Graves via mahadev)
|
|
|
|
MAPREDUCE-3705. ant build fails on 0.23 branch. (Thomas Graves via
|
|
mahadev)
|
|
|
|
MAPREDUCE-3691. webservices add support to compress response.
|
|
(Thomas Graves via mahadev)
|
|
|
|
MAPREDUCE-3702. internal server error trying access application master
|
|
via proxy with filter enabled (Thomas Graves via mahadev)
|
|
|
|
MAPREDUCE-3646. Remove redundant URL info from "mapred job" output.
|
|
(Jonathan Eagles via mahadev)
|
|
|
|
MAPREDUCE-3681. Fixed computation of queue's usedCapacity. (acmurthy)
|
|
|
|
MAPREDUCE-3505. yarn APPLICATION_CLASSPATH needs to be overridable.
|
|
(ahmed via tucu)
|
|
|
|
MAPREDUCE-3714. Fixed EventFetcher and Fetcher threads to shut-down properly
|
|
so that reducers don't hang in corner cases. (vinodkv)
|
|
|
|
MAPREDUCE-3712. The mapreduce tar does not contain the hadoop-mapreduce-client-
|
|
jobclient-tests.jar. (mahadev)
|
|
|
|
MAPREDUCE-3717. JobClient test jar has missing files to run all the test programs.
|
|
(mahadev)
|
|
|
|
MAPREDUCE-3630. Fixes a NullPointer exception while running TeraGen - if a
|
|
map is asked to generate 0 records. (Mahadev Konar via sseth)
|
|
|
|
MAPREDUCE-3683. Fixed maxCapacity of queues to be product of parent
|
|
maxCapacities. (acmurthy)
|
|
|
|
MAPREDUCE-3713. Fixed the way head-room is allocated to applications by
|
|
CapacityScheduler so that it deducts current-usage per user and not
|
|
per-application. (Arun C Murthy via vinodkv)
|
|
|
|
MAPREDUCE-3721. Fixed a race in shuffle which caused reduces to hang.
|
|
(sseth via acmurthy)
|
|
|
|
MAPREDUCE-3733. Add Apache License Header to hadoop-distcp/pom.xml.
|
|
(mahadev)
|
|
|
|
MAPREDUCE-3735. Add distcp jar to the distribution (tar).
|
|
(mahadev)
|
|
|
|
MAPREDUCE-3720. Changed bin/mapred job -list to not print job-specific
|
|
information not available at RM. (vinodkv via acmurthy)
|
|
|
|
MAPREDUCE-3742. "yarn logs" command fails with ClassNotFoundException.
|
|
(Jason Lowe via mahadev)
|
|
|
|
MAPREDUCE-3703. ResourceManager should provide node lists in JMX output.
|
|
(Eric Payne via mahadev)
|
|
|
|
MAPREDUCE-3716. Fixing YARN+MR to allow MR jobs to be able to use
|
|
java.io.File.createTempFile to create temporary files as part of their
|
|
tasks. (Jonathan Eagles via vinodkv)
|
|
|
|
MAPREDUCE-3748. Changed a log in CapacityScheduler.nodeUpdate to debug.
|
|
(ramya via acmurthy)
|
|
|
|
MAPREDUCE-3764. Fixed resource usage metrics for queues and users.
|
|
(acmurthy)
|
|
|
|
MAPREDUCE-3749. ConcurrentModificationException in counter groups.
|
|
(tomwhite)
|
|
|
|
MAPREDUCE-3762. Fixed default CapacityScheduler configs. (mahadev via
|
|
acmurthy)
|
|
|
|
MAPREDUCE-3499. New MiniMR does not setup proxyuser configuration
|
|
correctly, thus tests using doAs do not work. (johnvijoe via tucu)
|
|
|
|
MAPREDUCE-3696. MR job via oozie does not work on hadoop 23.
|
|
(John George via mahadev)
|
|
|
|
MAPREDUCE-3427. Fix streaming unit tests broken after mavenization.
|
|
(Hitesh Shah via acmurthy)
|
|
|
|
MAPREDUCE-3640. Allow AMRecovery to work with partial JobHistory files.
|
|
(Arun C Murthy via sseth)
|
|
|
|
MAPREDUCE-3752. Modified application limits to include queue max-capacities
|
|
besides the usual user limits. (Arun C Murthy via vinodkv)
|
|
|
|
MAPREDUCE-3744. Fix the yarn logs command line. Improve error messages for
|
|
mapred job -logs. (Jason Lowe via sseth)
|
|
|
|
MAPREDUCE-3780. Fixed a bug where applications killed before getting
|
|
activated were not getting cleaned up properly. (Hitesh Shah via acmurthy)
|
|
|
|
MAPREDUCE-3708. Metrics: Incorrect Apps Submitted Count (Bhallamudi via
|
|
mahadev)
|
|
|
|
MAPREDUCE-3727. jobtoken location property in jobconf refers to wrong
|
|
jobtoken file (tucu)
|
|
|
|
MAPREDUCE-3711. Fixed MR AM recovery so that only single selected task
|
|
output is recovered and thus reduce the unnecessarily bloated recovery
|
|
time. (Robert Joseph Evans via vinodkv)
|
|
|
|
MAPREDUCE-3760. Changed active nodes list to not contain unhealthy nodes
|
|
on the webUI and metrics. (vinodkv)
|
|
|
|
MAPREDUCE-3417. Fixed job-access-controls to work with MR AM and
|
|
JobHistoryServer web-apps. (Jonathan Eagles via vinodkv)
|
|
|
|
MAPREDUCE-3803. Fix broken build of raid contrib due to HDFS-2864.
|
|
(Ravi Prakash via suresh)
|
|
|
|
MAPREDUCE-3791. can't build site in hadoop-yarn-server-common.
|
|
(mahadev)
|
|
|
|
MAPREDUCE-3723. TestAMWebServicesJobs & TestHSWebServicesJobs
|
|
incorrectly asserting tests (Bhallamudi Venkata Siva Kamesh
|
|
via mahadev)
|
|
|
|
MAPREDUCE-3795. "job -status" command line output is malformed.
|
|
(vinodkv via mahadev)
|
|
|
|
MAPREDUCE-3759. ClassCastException thrown in -list-active-trackers when
|
|
there are a few unhealthy nodes (vinodkv via mahadev)
|
|
|
|
MAPREDUCE-3775. Change MiniYarnCluster to escape special chars in testname.
|
|
(Hitesh Shah via mahadev)
|
|
|
|
MAPREDUCE-3765. FifoScheduler does not respect yarn.scheduler.fifo.minimum-
|
|
allocation-mb setting (Hitesh Shah via mahadev)
|
|
|
|
MAPREDUCE-3747. Initialize queue metrics upfront and added start/finish
|
|
time to RM Web-UI. (acmurthy)
|
|
|
|
MAPREDUCE-3814. Fixed MRV1 compilation. (Arun C Murthy via vinodkv)
|
|
|
|
MAPREDUCE-3810. Performance tweaks - reduced logging in AM and defined
|
|
hascode/equals for ResourceRequest & Priority. (vinodkv via acmurthy)
|
|
|
|
MAPREDUCE-3813. Added a cache for resolved racks. (vinodkv via acmurthy)
|
|
|
|
MAPREDUCE-3808. Fixed an NPE in FileOutputCommitter for jobs with maps
|
|
but no reduces. (Robert Joseph Evans via vinodkv)
|
|
|
|
MAPREDUCE-3804. yarn webapp interface vulnerable to cross scripting attacks
|
|
(Dave Thompson via bobby)
|
|
|
|
MAPREDUCE-3354. Changed scripts so that jobhistory server is started by
|
|
bin/mapred instead of bin/yarn. (Jonathan Eagles via acmurthy)
|
|
|
|
MAPREDUCE-3809. Ensure that there is no needless sleep in Task at the end
|
|
of the task. (sseth via acmurthy)
|
|
|
|
MAPREDUCE-3794. Support mapred.Task.Counter and mapred.JobInProgress.Counter
|
|
enums for compatibility (Tom White via mahadev)
|
|
|
|
MAPREDUCE-3697. Support binary compatibility for Counters after
|
|
MAPREDUCE-901. (mahadev via acmurthy)
|
|
|
|
MAPREDUCE-3709. TestDistributedShell is failing. (Hitesh Shah via
|
|
mahadev)
|
|
|
|
MAPREDUCE-3817. Fixed bin/mapred to allow running of distcp and archive
|
|
jobs. (Arpit Gupta via acmurthy)
|
|
|
|
MAPREDUCE-3709. TestDistributedShell is failing. (Hitesh Shah via
|
|
mahadev)
|
|
|
|
MAPREDUCE-3436. JobHistory webapp address should use the host configured
|
|
in the jobhistory address. (Ahmed Radwan via sseth)
|
|
|
|
MAPREDUCE-3815. Fixed MR AM to always use hostnames and never IPs when
|
|
requesting containers so that scheduler can give off data local containers
|
|
correctly. (Siddarth Seth via vinodkv)
|
|
|
|
MAPREDUCE-3833. Fixed a bug in reinitiaziling of queues. (Jason Lowe via
|
|
acmurthy)
|
|
|
|
MAPREDUCE-3826. Fixed a bug in RM web-ui which broke sorting. (Jonathan
|
|
Eagles via acmurthy)
|
|
|
|
MAPREDUCE-3823. Ensure counters are calculated only once after a job
|
|
finishes. (Vinod Kumar Vavilapalli via sseth)
|
|
|
|
MAPREDUCE-3827. Changed Counters to use ConcurrentSkipListMap for
|
|
performance. (vinodkv via acmurthy)
|
|
|
|
MAPREDUCE-3822. Changed FS counter computation to use all occurences of
|
|
the same FS scheme, instead of randomly using one. (Mahadev Konar via
|
|
sseth)
|
|
|
|
MAPREDUCE-3834. Changed MR AM to not add the same rack entry multiple times
|
|
into the container request table when multiple hosts for a split happen to
|
|
be on the same rack. (Siddarth Seth via vinodkv)
|
|
|
|
MAPREDUCE-3828. Ensure that urls in single-node mode are correct. (sseth
|
|
via acmurthy)
|
|
|
|
MAPREDUCE-3770. Zombie.getJobConf() results into NPE. (amarrk)
|
|
|
|
MAPREDUCE-3840. JobEndNotifier doesn't use the proxyToUse during connecting
|
|
(Ravi Prakash via bobby)
|
|
|
|
MAPREDUCE-3843. Job summary log file found missing on the RM host
|
|
(Anupam Seth via tgraves)
|
|
|
|
MAPREDUCE-3846. Addressed MR AM hanging issues during AM restart and then
|
|
the recovery. (vinodkv)
|
|
|
|
MAPREDUCE-3802. Added test to validate that AM can crash multiple times and
|
|
still can recover successfully after MAPREDUCE-3846. (vinodkv)
|
|
|
|
MAPREDUCE-3858. Task attempt failure during commit results in task never completing.
|
|
(Tom White via mahadev)
|
|
|
|
MAPREDUCE-3856. Instances of RunningJob class givs incorrect job tracking
|
|
urls when mutiple jobs are submitted from same client jvm. (Eric Payne via
|
|
sseth)
|
|
|
|
MAPREDUCE-3880. Changed LCE binary to be 32-bit. (acmurthy)
|
|
|
|
Release 0.23.0 - 2011-11-01
|
|
|
|
INCOMPATIBLE CHANGES
|
|
|
|
MAPREDUCE-2455. Remove deprecated JobTracker.State in favour of
|
|
JobTrackerStatus. (tomwhite)
|
|
|
|
MAPREDUCE-2430. Remove mrunit contrib. (nigel via eli)
|
|
|
|
MAPREDUCE-2606. Remove IsolationRunner. (Alejandro Abdelnur via eli)
|
|
|
|
NEW FEATURES
|
|
|
|
MAPREDUCE-2682. Add "mapred classpath" command to print classpath
|
|
for MR applications. (vinodkv via acmurthy)
|
|
|
|
MAPREDUCE-2107. [Gridmix] Total heap usage emulation in Gridmix.
|
|
(Amar Kamat and Ravi Gummadi via amarrk)
|
|
|
|
MAPREDUCE-2106. [Gridmix] Cumulative CPU usage emulation in Gridmix.
|
|
(amarrk)
|
|
|
|
MAPREDUCE-2543. [Gridmix] High-Ram feature emulation in Gridmix. (amarrk)
|
|
|
|
MAPREDUCE-2408. [Gridmix] Compression emulation in Gridmix. (amarrk)
|
|
|
|
MAPREDUCE-2473. Add "mapred groups" command to query the server-side groups
|
|
resolved for a user. (Aaron T. Myers via todd)
|
|
|
|
MAPREDUCE-461. Enable ServicePlugins for the JobTracker.
|
|
(Fredrik Hedberg via tomwhite)
|
|
|
|
MAPREDUCE-2521. Create RPM and Debian packages for MapReduce. Changes
|
|
deployment layout to be consistent across the binary tgz, rpm, and deb.
|
|
(Eric Yang via omalley)
|
|
|
|
MAPREDUCE-2323. Add metrics to the fair scheduler. (todd)
|
|
|
|
MAPREDUCE-2037. Capture intermediate progress, CPU and memory usage for
|
|
tasks. (Dick King via acmurthy)
|
|
|
|
MAPREDUCE-279. MapReduce 2.0. Merging MR-279 branch into trunk. Contributed by
|
|
Arun C Murthy, Christopher Douglas, Devaraj Das, Greg Roelofs, Jeffrey
|
|
Naisbitt, Josh Wills, Jonathan Eagles, Krishna Ramachandran, Luke Lu, Mahadev
|
|
Konar, Robert Evans, Sharad Agarwal, Siddharth Seth, Thomas Graves, and Vinod
|
|
Kumar Vavilapalli.
|
|
|
|
MAPREDUCE-2930. Added the ability to be able to generate graphs from the
|
|
state-machine definitions. (Binglin Chang via vinodkv)
|
|
|
|
MAPREDUCE-2719. Add a simple, DistributedShell, application to illustrate
|
|
alternate frameworks on YARN. (Hitesh Shah via acmurthy)
|
|
|
|
MAPREDUCE-3104. Implemented Application-acls. (vinodkv)
|
|
|
|
MAPREDUCE-2708. Designed and implemented MR Application Master recovery to
|
|
make MR AMs resume their progress after restart. (Sharad Agarwal via vinodkv)
|
|
|
|
MAPREDUCE-2858. Added a WebApp Proxy for applications. (Robert Evans via
|
|
acmurthy)
|
|
|
|
IMPROVEMENTS
|
|
|
|
MAPREDUCE-2187. Reporter sends progress during sort/merge. (Anupam Seth via
|
|
acmurthy)
|
|
|
|
MAPREDUCE-2365. Add counters to track bytes (read,written) via
|
|
File(Input,Output)Format. (Siddharth Seth via acmurthy)
|
|
|
|
MAPREDUCE-2680. Display queue name in job client CLI. (acmurthy)
|
|
|
|
MAPREDUCE-2679. Minor changes to sync trunk with MR-279 branch. (acmurthy)
|
|
|
|
MAPREDUCE-2400. Remove Cluster's dependency on JobTracker via a
|
|
ServiceProvider for the actual implementation. (tomwhite via acmurthy)
|
|
|
|
MAPREDUCE-2596. [Gridmix] Summarize Gridmix runs. (amarrk)
|
|
|
|
MAPREDUCE-2563. [Gridmix] Add High-Ram emulation system tests to
|
|
Gridmix. (Vinay Kumar Thota via amarrk)
|
|
|
|
MAPREDUCE-2104. [Rumen] Add Cpu, Memory and Heap usages to
|
|
TraceBuilder's output. (amarrk)
|
|
|
|
MAPREDUCE-2554. [Gridmix] Add distributed cache emulation system tests
|
|
to Gridmix. (Vinay Kumar Thota via amarrk)
|
|
|
|
MAPREDUCE-2543. [Gridmix] High-Ram feature emulation testcase. (amarrk)
|
|
|
|
MAPREDUCE-2469. Task counters should also report the total heap usage of
|
|
the task. (Ravi Gummadi and Amar Ramesh Kamat via amarrk)
|
|
|
|
MAPREDUCE-2544. [Gridmix] Add compression emulation system tests to
|
|
Gridmix. (Vinay Kumar Thota via amarrk)
|
|
|
|
MAPREDUCE-2517. [Gridmix] Add system tests to Gridmix.
|
|
(Vinay Kumar Thota via amarrk)
|
|
|
|
MAPREDUCE-2492. The new MapReduce API should make available task's
|
|
progress to the task. (amarrk)
|
|
|
|
MAPREDUCE-2153. Bring in more job configuration properties in to the trace
|
|
file. (Rajesh Balamohan via amarrk)
|
|
|
|
MAPREDUCE-1461. Feature to instruct rumen-folder utility to skip jobs worth
|
|
of specific duration. (Rajesh Balamohan via amarrk)
|
|
|
|
MAPREDUCE-2172. Added test-patch.properties required by test-patch.sh
|
|
(nigel)
|
|
|
|
MAPREDUCE-2156. Raid-aware FSCK. (Patrick Kling via dhruba)
|
|
|
|
MAPREDUCE-2215. A more elegant FileSystem#listCorruptFileBlocks API
|
|
(RAID changes) (Patrick Kling via hairong)
|
|
|
|
MAPREDUCE-1831. BlockPlacement policy for HDFS-RAID.
|
|
(Scott Chen via dhruba)
|
|
|
|
MAPREDUCE-1906. Lower minimum heartbeat interval for TaskTracker
|
|
(Scott Carey and Todd Lipcon via todd)
|
|
|
|
MAPREDUCE-1382. MRAsyncDiscService should tolerate missing local.dir.
|
|
(Zheng Shao and tomwhite via tomwhite)
|
|
|
|
MAPREDUCE-2263. MapReduce side of HADOOP-6904: RPC compatibility.
|
|
(hairong)
|
|
|
|
MAPREDUCE-1706. Log RAID recoveries on HDFS. (schen)
|
|
|
|
MAPREDUCE-2334. Update BlockPlacementPolicyRaid for the new method
|
|
in BlockPlacementPolicy. (szetszwo)
|
|
|
|
MAPREDUCE-2254. Allow setting of end-of-record delimiter for
|
|
TextInputFormat (Ahmed Radwan via todd)
|
|
|
|
MAPREDUCE-1927. Unit test for HADOOP-6835 (concatenated gzip support).
|
|
(Greg Roelofs via tomwhite)
|
|
|
|
MAPREDUCE-2206. The task-cleanup tasks should be optional. (schen)
|
|
|
|
MAPREDUCE-2225. MultipleOutputs should not require the use of 'Writable'.
|
|
(Harsh J Chouraria via tomwhite)
|
|
|
|
MAPREDUCE-1811. Job.monitorAndPrintJob() should print status of the job
|
|
at completion. (Harsh J Chouraria via tomwhite)
|
|
|
|
MAPREDUCE-993. bin/hadoop job -events <jobid> <from-event-#> <#-of-events>
|
|
help message is confusing. (Harsh J Chouraria via tomwhite)
|
|
|
|
MAPREDUCE-2302. Add static factory methods in GaloisField. (schen)
|
|
|
|
MAPREDUCE-2351. mapred.job.tracker.history.completed.location should
|
|
support an arbitrary filesystem URI. (tomwhite)
|
|
|
|
MAPREDUCE-2239. BlockPlacementPolicyRaid should call getBlockLocations
|
|
only when necessary. (schen)
|
|
|
|
MAPREDUCE-2331. Add coverage of task graph servlet to fair scheduler system
|
|
test. (todd)
|
|
|
|
MAPREDUCE-2367. Allow using a file to exclude certain tests from build.
|
|
(todd)
|
|
|
|
MAPREDUCE-2202. Generalize CLITest structure and interfaces to faciliate
|
|
upstream adoption (e.g. for web or system testing). (cos)
|
|
|
|
MAPREDUCE-2420. JobTracker should be able to renew delegation token over
|
|
HTTP (Boris Shkolnik via jitendra)
|
|
|
|
MAPREDUCE-2474. Add docs to the new API Partitioner on how to access the
|
|
Job Configuration. (Harsh J Chouraria via todd)
|
|
|
|
MAPREDUCE-2475. Disable IPV6 for junit tests. (suresh srinivas via mahadev)
|
|
|
|
MAPREDUCE-2422. Removed unused internal methods from DistributedCache.
|
|
(tomwhite)
|
|
|
|
MAPREDUCE-2456. Log the reduce taskID and associated TaskTrackers with
|
|
failed fetch notifications in the JobTracker log.
|
|
(Jeffrey Naisbitt via cdouglas)
|
|
|
|
MAPREDUCE-869. Documentation for config to set map/reduce task environment
|
|
(Alejandro Abdelnur via todd)
|
|
|
|
MAPREDUCE-2410. Add entry to streaming FAQ about how streaming reducers
|
|
receive keys. (Harsh J Chouraria via todd)
|
|
|
|
MAPREDUCE-2499. MR part of HADOOP-7291. (eli)
|
|
|
|
MAPREDUCE-2497. Missing spaces in error messages. (eli)
|
|
|
|
MAPREDUCE-2502. JobSubmitter should use mapreduce.job.maps instead of
|
|
its deprecated equivalent. (eli via todd)
|
|
|
|
MAPREDUCE-2381. JobTracker instrumentation not consistent about error
|
|
handling. (Philip Zeyliger via tomwhite)
|
|
|
|
MAPREDUCE-2449. Allow for command line arguments when performing
|
|
"Run on Hadoop" action in Eclipse plugin. (Jeff Zemerick via todd)
|
|
|
|
MAPREDUCE-2483. Remove duplication of jars between Hadoop subprojects
|
|
from build artifacts. (Eric Yang via omalley)
|
|
|
|
MAPREDUCE-2372. TaskLogAppender mechanism shouldn't be set up in
|
|
log4j.properties (todd)
|
|
|
|
MAPREDUCE-2516. Rename webinterface.private.actions to
|
|
mapreduce.jobtracker.webinterface.trusted (Ari Rabkin via todd)
|
|
|
|
MAPREDUCE-2459. Cache HAR filesystem metadata. (Mac Yang via mahadev)
|
|
|
|
HADOOP-7259. Contrib modules should include the build.properties from
|
|
the enclosing hadoop directory. (omalley)
|
|
|
|
MAPREDUCE-2494. Order distributed cache deletions by LRU. (Robert Joseph
|
|
Evans via cdouglas)
|
|
|
|
MAPREDUCE-2452. Makes the cancellation of delegation tokens happen in a
|
|
separate thread. (ddas)
|
|
|
|
HADOOP-7106. Reorganize project SVN layout to "unsplit" the projects.
|
|
(todd, nigel)
|
|
|
|
MAPREDUCE-2249. Check the reflexive property of Counters objects when
|
|
comparing equality. (Devaraj K via todd)
|
|
|
|
MAPREDUCE-2623. Update ClusterMapReduceTestCase to use
|
|
MiniDFSCluster.Builder (Harsh J Chouraria via eli)
|
|
|
|
MAPREDUCE-2602. Allow setting of end-of-record delimiter for
|
|
TextInputFormat for the old API. (Ahmed Radwan via todd)
|
|
|
|
MAPREDUCE-2705. Permits parallel multiple task launches.
|
|
(Thomas Graves via ddas)
|
|
|
|
MAPREDUCE-2489. Jobsplits with random hostnames can make the queue
|
|
unusable (jeffrey naisbit via mahadev)
|
|
|
|
MAPREDUCE-2854. update INSTALL with config necessary run mapred on yarn.
|
|
(thomas graves via mahadev)
|
|
|
|
MAPREDUCE-2701. app/Job.java needs UGI for the user that launched it.
|
|
(Robert Evans via mahadev)
|
|
|
|
MAPREDUCE-2652. Enabled multiple NMs to be runnable on a single node by
|
|
making shuffle service port to be truely configurable. (Robert Evans via
|
|
vinodkv)
|
|
|
|
MAPREDUCE-2735. Add an applications summary log to ResourceManager.
|
|
(Thomas Graves via acmurthy)
|
|
|
|
MAPREDUCE-2697. Enhance CapacityScheduler to cap concurrently running
|
|
applications per-queue & per-user. (acmurthy)
|
|
Configuration changes:
|
|
add yarn.capacity-scheduler.maximum-am-resource-percent
|
|
|
|
MAPREDUCE-2774. Add startup message to ResourceManager & NodeManager on
|
|
startup. (Venu Gopala Rao via acmurthy)
|
|
|
|
MAPREDUCE-2655. Add audit logs to ResourceManager and NodeManager. (Thomas
|
|
Graves via acmurthy)
|
|
|
|
MAPREDUCE-2864. Normalize configuration variable names for YARN. (Robert
|
|
Evans via acmurthy)
|
|
|
|
MAPREDUCE-2690. Web-page for FifoScheduler. (Eric Payne via acmurthy)
|
|
|
|
MAPREDUCE-2711. Update TestBlockPlacementPolicyRaid for the new namesystem
|
|
and block management APIs. (szetszwo)
|
|
|
|
MAPREDUCE-2933. Change allocate call to return ContainerStatus for
|
|
completed containers rather than Container. (acmurthy)
|
|
|
|
MAPREDUCE-2675. Reformat JobHistory Server main page to be more
|
|
useful. (Robert Joseph Evans via vinodkv).
|
|
|
|
MAPREDUCE-2896. Simplify all apis to in
|
|
org.apache.hadoop.yarn.api.records.* to be get/set only. Added javadocs to
|
|
all public records. (acmurthy)
|
|
|
|
MAPREDUCE-2676. MR-279: JobHistory Job page needs reformatted. (Robert Evans via
|
|
mahadev)
|
|
|
|
MAPREDUCE-2899. Replace major parts of ApplicationSubmissionContext with a
|
|
ContainerLaunchContext (Arun Murthy via mahadev)
|
|
|
|
MAPREDUCE-2966. Added ShutDown hooks for MRV2 processes so that they can
|
|
gracefully exit. (Abhijit Suresh Shingate via vinodkv)
|
|
|
|
MAPREDUCE-2672. MR-279: JobHistory Server needs Analysis this job.
|
|
(Robert Evans via mahadev)
|
|
|
|
MAPREDUCE-2965. Streamlined the methods hashCode(), equals(), compareTo()
|
|
and toString() for all IDs. (Siddharth Seth via vinodkv)
|
|
|
|
MAPREDUCE-2726. Added job-file to the AM and JobHistoryServer web
|
|
interfaces. (Jeffrey Naisbitt via vinodkv)
|
|
|
|
MAPREDUCE-2880. Improve classpath-construction for mapreduce AM and
|
|
containers. (Arun C Murthy via vinodkv)
|
|
|
|
MAPREDUCE-3055. Simplified ApplicationAttemptId passing to
|
|
ApplicationMaster via environment variable. (vinodkv)
|
|
|
|
MAPREDUCE-3092. Removed a special comparator for JobIDs in JobHistory as
|
|
JobIDs are already comparable. (Devaraj K via vinodkv)
|
|
|
|
MAPREDUCE-3099. Add docs for setting up a single node MRv2 cluster.
|
|
(mahadev)
|
|
|
|
MAPREDUCE-3001. Added task-specific counters to AppMaster and JobHistory
|
|
web-UIs. (Robert Joseph Evans via vinodkv)
|
|
|
|
MAPREDUCE-3098. Fixed RM and MR AM to report YarnApplicationState and
|
|
application's FinalStatus separately. (Hitesh Shah via vinodkv)
|
|
|
|
MAPREDUCE-2889. Added documentation for writing new YARN applications.
|
|
(Hitesh Shah via acmurthy)
|
|
|
|
MAPREDUCE-3134. Added documentation the CapacityScheduler. (acmurthy)
|
|
|
|
MAPREDUCE-3013. Removed YarnConfiguration.YARN_SECURITY_INFO and its usage
|
|
as it doesn't affect security any more. (vinodkv)
|
|
|
|
MAPREDUCE-2907. Changed log level for various messages in ResourceManager
|
|
from INFO to DEBUG. (Ravi Prakash via vinodkv)
|
|
|
|
MAPREDUCE-2702. Added a new API in OutputCommitter for recovering
|
|
the outputs of tasks from a crashed job so as to support MR Application
|
|
Master recovery. (Sharad Agarwal and Arun C Murthy via vinodkv)
|
|
|
|
MAPREDUCE-2738. Added the missing cluster level statistics on the RM web
|
|
UI. (Robert Joseph Evans via vinodkv)
|
|
|
|
MAPREDUCE-2988. Reenabled TestLinuxContainerExecutor reflecting the
|
|
current NodeManager code. (Robert Joseph Evans via vinodkv)
|
|
|
|
MAPREDUCE-3161. Improved some javadocs and fixed some typos in
|
|
YARN. (Todd Lipcon via vinodkv)
|
|
|
|
MAPREDUCE-3148. Ported MAPREDUCE-2702 to old mapred api for aiding task
|
|
recovery. (acmurthy)
|
|
|
|
MAPREDUCE-3133. Running a set of methods in a Single Test Class.
|
|
(Jonathan Eagles via mahadev)
|
|
|
|
MAPREDUCE-3059. QueueMetrics do not have metrics for aggregate
|
|
containers-allocated and aggregate containers-released.
|
|
(Devaraj K via mahadev)
|
|
|
|
MAPREDUCE-3187. Add names for various unnamed threads in MR2.
|
|
(Todd Lipcon and Siddharth Seth via mahadev)
|
|
|
|
MAPREDUCE-3136. Added documentation for setting up Hadoop clusters in both
|
|
non-secure and secure mode for both HDFS & YARN. (acmurthy)
|
|
|
|
MAPREDUCE-3068. Added a whitelist of environment variables for containers
|
|
from the NodeManager and set MALLOC_ARENA_MAX for all daemons and
|
|
containers. (Chris Riccomini via acmurthy)
|
|
|
|
MAPREDUCE-3144. Augmented JobHistory with the information needed for
|
|
serving aggregated logs. (Siddharth Seth via vinodkv)
|
|
|
|
MAPREDUCE-3163. JobClient spews errors when killing MR2 job.
|
|
(mahadev)
|
|
|
|
MAPREDUCE-3239. Use new createSocketAddr API in MRv2 to give better
|
|
error messages on misconfig (Todd Lipcon via mahadev)
|
|
|
|
MAPREDUCE-2747. Cleaned up LinuxContainerExecutor binary sources and changed
|
|
the configuration to use yarn names. (Robert Joseph Evans via vinodkv)
|
|
|
|
MAPREDUCE-3205. Fix memory specifications to be physical rather than
|
|
virtual, allowing for a ratio between the two to be configurable. (todd
|
|
via acmurthy)
|
|
|
|
MAPREDUCE-2986. Fixed MiniYARNCluster to support multiple NodeManagers.
|
|
(Anupam Seth via vinodkv)
|
|
|
|
MAPREDUCE-2736. Remove unused contrib components dependent on MR1. (eli)
|
|
|
|
MAPREDUCE-2989. Modified JobHistory to link to task and AM logs from the
|
|
JobHistoryServer. (Siddharth Seth via vinodkv)
|
|
|
|
MAPREDUCE-3146. Added a MR specific command line to dump logs for a
|
|
given TaskAttemptID. (Siddharth Seth via vinodkv)
|
|
|
|
MAPREDUCE-3275. Added documentation for AM WebApp Proxy. (Robert Evans via
|
|
acmurthy)
|
|
|
|
MAPREDUCE-3322. Added a better index.html and an brief overview of YARN
|
|
architecture. (acmurthy)
|
|
|
|
OPTIMIZATIONS
|
|
|
|
MAPREDUCE-2026. Make JobTracker.getJobCounters() and
|
|
JobInProgress.getCounters() aquire locks in a shorter time period.
|
|
(Joydeep Sen Sarma via schen)
|
|
|
|
MAPREDUCE-2740. MultipleOutputs in new API creates needless
|
|
TaskAttemptContexts. (todd)
|
|
|
|
MAPREDUCE-901. Efficient framework counters. (llu via acmurthy)
|
|
|
|
MAPREDUCE-2629. Workaround a JVM class loading quirk which prevents
|
|
JIT compilation of inner classes methods in ReduceContextImpl.
|
|
|
|
BUG FIXES
|
|
|
|
MAPREDUCE-2603. Disable High-Ram emulation in system tests.
|
|
(Vinay Kumar Thota via amarrk)
|
|
|
|
MAPREDUCE-2539. Fixed NPE in getMapTaskReports in JobClient. (Robert Evans via
|
|
acmurthy)
|
|
|
|
MAPREDUCE-1978. Rumen TraceBuilder should provide recursive
|
|
input folder scanning.
|
|
|
|
MAPREDUCE-2416. Remove the restriction of specifying group names in
|
|
users-list file for Gridmix in RoundRobinUserResolver mode.
|
|
|
|
MAPREDUCE-2417. Fix Gridmix in RoundRobinUserResolver mode to
|
|
map testing/proxy users to unique users in a trace.
|
|
|
|
MAPREDUCE-2307. Exception thrown in Jobtracker logs, when the Scheduler
|
|
configured is FairScheduler. (Devaraj K via matei)
|
|
|
|
MAPREDUCE-2199. build is broken 0.22 branch creation. (cos)
|
|
|
|
MAPREDUCE-1752. Implement getFileBlockLocations in HarFilesystem.
|
|
(Patrick Kling via dhruba)
|
|
|
|
MAPREDUCE-2155. RaidNode should optionally use the mapreduce jobs to
|
|
fix missing blocks. (Patrick Kling via dhruba)
|
|
|
|
MAPREDUCE-1334. Fix TestIndexUpdater by ignoring _SUCCESS file in HDFS.
|
|
(Kay Kay via yhemanth)
|
|
|
|
MAPREDUCE-2232. Add missing methods to TestMapredGroupMappingServiceRefresh.
|
|
(Todd Lipcon via eli)
|
|
|
|
MAPREDUCE-2271. Fix TestSetupTaskScheduling failure on trunk.
|
|
(Liyin Liang via todd)
|
|
|
|
MAPREDUCE-2290. Fix compilation error in TestTaskCommit. (eli)
|
|
|
|
MAPREDUCE-2294. Fix compilation error in mumak. (todd)
|
|
|
|
MAPREDUCE-2300. Fix TestUmbilicalProtocolWithJobToken on trunk after
|
|
HADOOP-6904. (todd)
|
|
|
|
MAPREDUCE-2296. Fix references to misspelled method name
|
|
getProtocolSigature (todd)
|
|
|
|
MAPREDUCE-2311. Fix TestFairScheduler failure (schen)
|
|
|
|
MAPREDUCE-1996. API: Reducer.reduce() method detail misstatement.
|
|
(Harsh J Chouraria via tomwhite)
|
|
|
|
MAPREDUCE-2203. Wrong javadoc for TaskRunner's appendJobJarClasspaths
|
|
method. (Jingguo Yao via tomwhite)
|
|
|
|
MAPREDUCE-2074. Task should fail when symlink creation fails.
|
|
(Priyo Mustafi via tomwhite)
|
|
|
|
MAPREDUCE-1242. Chain APIs error misleading.
|
|
(Harsh J Chouraria via tomwhite)
|
|
|
|
MAPREDUCE-2379. Adds missing DistributedCache configurations in
|
|
mapred-default.xml (Todd Lipcon via amareshwari)
|
|
|
|
MAPREDUCE-2348. Disable mumak tests on trunk since they currently time out
|
|
(todd)
|
|
|
|
MAPREDUCE-2395. TestBlockFixer timing out on trunk. (Ramkumar Vadali via
|
|
todd)
|
|
|
|
MAPREDUCE-2426. Make TestFairSchedulerSystem fail with more verbose output
|
|
(todd)
|
|
|
|
MAPREDUCE-2448. NoSuchMethodError:
|
|
org.apache.hadoop.hdfs.TestDatanodeBlockScanner.corruptReplica(..) (eli)
|
|
|
|
MAPREDUCE-2460. Fix flaky test TestFairSchedulerSystem. (todd)
|
|
|
|
MAPREDUCE-2451. Log the details from health check script at the
|
|
JobTracker. (Thomas Graves via cdouglas)
|
|
|
|
MAPREDUCE-2467. HDFS-1052 changes break the raid contrib module in
|
|
MapReduce. (suresh srinivas via mahadev)
|
|
|
|
MAPREDUCE-2258. IFile reader closes stream and compressor in wrong order.
|
|
(todd via tomwhite)
|
|
|
|
MAPREDUCE-2518. The t flag is missing in distcp help message. (Wei Yongjun
|
|
via szetszwo)
|
|
|
|
MAPREDUCE-2514. Fix typo in TaskTracker ReinitTrackerAction log message.
|
|
(Jonathan Eagles via cdouglas)
|
|
|
|
MAPREDUCE-2490. Add logging to graylist and blacklist activity to aid
|
|
diagnosis of related issues. (Jonathan Eagles via cdouglas)
|
|
|
|
MAPREDUCE-2495. exit() the TaskTracker when the distributed cache cleanup
|
|
thread dies. (Robert Joseph Evans via cdouglas)
|
|
|
|
MAPREDUCE-2470. Fix NPE in RunningJobs::getCounters. (Robert Joseph Evans
|
|
via cdouglas)
|
|
|
|
MAPREDUCE-2536. Update FsShell -mv command usage in TestMRCLI. (Daryn
|
|
Sharp via szetszwo)
|
|
|
|
MAPREDUCE-2529. Add support for regex-based shuffle metric counting
|
|
exceptions. (Thomas Graves via cdouglas)
|
|
|
|
MAPREDUCE-2559. ant binary fails due to missing c++ lib dir. (eli)
|
|
|
|
MAPREDUCE-2573. Fix new findbugs warning introduced by MAPREDUCE-2494.
|
|
(Robert Joseph Evans via todd)
|
|
|
|
MAPREDUCE-2581. Spelling errors in log messages. (Tim Sell via eli)
|
|
|
|
MAPREDUCE-2588. Change raid to the new DataTransferProtocol API. (szetszwo)
|
|
|
|
MAPREDUCE-2576. Typo in comment in SimulatorLaunchTaskAction.java.
|
|
(Tim Sell via jghoman)
|
|
|
|
MAPREDUCE-2550. Fix bin/mapred to work properly from within a source
|
|
checkout (Eric Yang via todd)
|
|
|
|
MAPREDUCE-2620. Update RAID for HDFS-2087. (szetszwo)
|
|
|
|
MAPREDUCE-2624. Update RAID for HDFS-2107. (szetszwo)
|
|
|
|
MAPREDUCE-2670. Fixing spelling mistake in FairSchedulerServlet.java. (eli)
|
|
|
|
MAPREDUCE-2710. Update JobSubmitter.printTokens(..) for HDFS-2161.
|
|
(szetszwo)
|
|
|
|
MAPREDUCE-2409. DistributedCache maps files and archives to the same path,
|
|
despite semantic incompatibility. (Siddharth Seth via cdouglas)
|
|
|
|
MAPREDUCE-2575. TestMiniMRDFSCaching fails if test.build.dir is set
|
|
to something other than build/test (Thomas Graves via mahadev)
|
|
|
|
MAPREDUCE-2622. Remove the last remaining reference to the deprecated
|
|
configuration "io.sort.mb". (Harsh J Chouraria via todd)
|
|
|
|
MAPREDUCE-2732. Remove directly accessing FSNamesystem.LOG from
|
|
TestCopyFiles and TestDistCh. (szetszwo)
|
|
|
|
MAPREDUCE-2463. Job history files are not moved to done folder when job
|
|
history location is hdfs. (Devaraj K via szetszwo)
|
|
|
|
MAPREDUCE-2243. Close streams propely in a finally-block to avoid leakage
|
|
in CompletedJobStatusStore, TaskLog, EventWriter and TotalOrderPartitioner.
|
|
(Devaraj K via szetszwo)
|
|
|
|
MAPREDUCE-2741. Make ant build system work with hadoop-common JAR
|
|
generated by Maven. (Alejandro Abdelnur via tomwhite)
|
|
|
|
MAPREDUCE-2760. mapreduce.jobtracker.split.metainfo.maxsize typoed
|
|
in mapred-default.xml. (todd via eli)
|
|
|
|
MAPREDUCE-2797. Update mapreduce tests and RAID for HDFS-2239. (szetszwo)
|
|
|
|
MAPREDUCE-2805. Update RAID for HDFS-2241. (szetszwo)
|
|
|
|
MAPREDUCE-2837. Ported bug fixes from y-merge to prepare for MAPREDUCE-279
|
|
merge. (acmurthy)
|
|
|
|
MAPREDUCE-2541. Fixed a race condition in IndexCache.removeMap. (Binglin
|
|
Chang via acmurthy)
|
|
|
|
MAPREDUCE-2458. Rename sanitized pom.xml in build directory to work around IDE
|
|
bug (Luke Lu via mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Clear application notification if sent once
|
|
to NodeManager (mahadev)
|
|
|
|
MAPREDUCE-2433. YARNApplicationConstants hard code app master jar version (Luke
|
|
Lu via mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Implement restart for resource manager
|
|
phase 1 - Helper classes to store and restore the data structures. (mahadev)
|
|
|
|
MAPREDUCE-2414. Change MRv2 to use generic interfaces. (Siddharth Seth via
|
|
acmurthy)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Implement health-checks for the node -
|
|
server side(ResourceManager) changes. (vinodkv)
|
|
|
|
MAPREDUCE-2405: Implement uber-AppMaster (in-cluster LocalJobRunner for MRv2)
|
|
(Greg Roelofs via mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Implementing Containers' memory monitoring.
|
|
(vinodkv)
|
|
|
|
MAPREDUCE-2440. Name clashes in TypeConverter (luke via mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Add fail count to the command line of the
|
|
application master. (mahadev)
|
|
|
|
MAPREDUCE-2424. Polish uber-AppMaster: add uber-AM counters and GUI indicators.
|
|
(Greg Roelofs via mahadev)
|
|
|
|
MAPREDUCE-2405. Implement uber-AppMaster (in-cluster LocalJobRunner for MRv2).
|
|
(Greg Roelofs and Sharad Agarwal via mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix YarnRemoteException to give more
|
|
details. (Siddharth Seth via mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. WebApp for Job History (Krishna
|
|
Ramachandran via mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Install sanitized poms for downstream
|
|
sanity (Luke Lu via mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Add HistoryCleanerService to Job History
|
|
server. (Krishna Ramachandran via sharad)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Implement 'bin/mapred job -list' and
|
|
'bin/mapred job -list-active-trackers'. (acmurthy)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Implement 'bin/mapred queue [-info
|
|
[-showJobs]] [-list] and enhanced 'bin/mapred job -list' to show queue and
|
|
ApplicationMaster information. (acmurthy)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fixed computation of user-limits at
|
|
runtime. (acmurthy)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Added functionality to refresh queues at
|
|
runtime via the 'bin/yarn rmadmin' command. (acmurthy)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Added functionality to stop/start queues.
|
|
(acmurthy)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Introducing web-UI for NodeManager and
|
|
linking it from RM UI. (vinodkv)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix race condition in TestJobHistoryEvents
|
|
and TestJobHistoryParsing. (sharad)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Add Containers' logs' view to NM UI and
|
|
link it from AM UI. (vinodkv)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Add ACLs for queues and command-line
|
|
utilities for viewing them. (acmurthy)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Recovery of MR Application Master from
|
|
failures. (sharad)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Added support High-RAM applications in
|
|
CapacityScheduler. (acmurthy)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Completing the ZooKeeper Store for
|
|
ResourceManager state. (mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Reorient container localization to be
|
|
per-container rather than per-application. (cdouglas)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix file creation in
|
|
JobHistoryEventHandler. (sharad)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Disable ContainerMonitoring for non-linux
|
|
systems. (vinodkv)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix container launch w/ inconsistent
|
|
credential file naming. (cdouglas)
|
|
|
|
MAPREDUCE-2434. Metrics for ResourceManager. (Luke Lu via acmurthy)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. RM Restart Phase 2 - Completed the recovery
|
|
of components in the RM (mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix to send finish application event only
|
|
when the application is finished (mahadev)
|
|
|
|
MAPREDUCE-2462. Write job conf along with JobHistory, other minor improvements.
|
|
(Siddharth Seth via sharad)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Implement 'delay scheduling' for better
|
|
locality in CapacityScheduler and improved high-ram applications. (acmurthy)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Implement Job Acls in MR Application
|
|
Master. (sharad)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Moving userlogs out of container work-dir
|
|
into a separate directory structure. (vinodkv)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Completing RM Restart. Completed Phase 3 of
|
|
making sure events are logged and restored (mahadev)
|
|
|
|
MAPREDUCE-2468. Add metrics for NM Shuffle. (Luke Lu via cdouglas)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Adding user log handling for YARN. Making
|
|
NM put the user-logs on DFS and providing log-dump tools. (vinodkv)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fixing three tight-loops in RM that are
|
|
causing high cpu-usage. (vinodkv)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Replacing FileContext usage with FileSystem
|
|
to work around security authentication issues with FileContext against a secure
|
|
DFS. (vinodkv)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Client reconnect to restarted AM. (sharad)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix refreshProxy in ClientServiceDelegate.
|
|
(sharad)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix Null Pointer in TestUberAM. (sharad)
|
|
|
|
MAPREDUCE-2478. Improve history server. (Siddharth Seth via sharad)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Improved TestJobHistoryEvents and
|
|
TestJobHistoryParsing. (sharad)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix NM to use multiple disks for local
|
|
files and the userlogs. (vinodkv)
|
|
|
|
MAPREDUCE-2480: MR App should not depend on hard coded version of shuffle (luke
|
|
lu via mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Propagate error back to client in case of a
|
|
job submission failure (mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix assembly to add mapreduce shell scripts
|
|
to the assembly package. (mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix TestQueueMetrics. (Luke Lu via sharad)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Removal of stale application-log dirs from
|
|
NM local disks. (vinodkv)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Add license header and minor cleanup in
|
|
history server. (Siddharth Seth via sharad)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Minor fix for install instructions.
|
|
(mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix a race in MR task that was causing MR
|
|
containers to overwrite each other's job.xml. Also fix leaking attempt-dirs in
|
|
app-local-dir. (vinodkv)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Adding valid state to ASM on a finish when
|
|
its already completed and also disble UberAM. (mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fixed CS user limits. (acmurthy)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fixed reservation's bad interaction with
|
|
delay scheduling in CS. (acmurthy)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Support mapreduce old (0.20) APIs. (sharad)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Support fail-fast for MR jobs. (ddas)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix for clearing container requests on an
|
|
AM failure and add tostring methods to taskids and taskattemptids for better
|
|
grep support. (mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Speed up communication between MR AM and RM
|
|
by relying on a new config rather than AM_EXPIRY_INTERVAL which is too large.
|
|
(acmurthy)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix calculation of maximum capacity to use
|
|
parent's absolute-capacity rather than the leaf queue's absolute-capacity.
|
|
(acmurthy)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fixing a bug in JobIDPbImpl that's causing
|
|
AM to crash randomly. (vinodkv)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix calculation of maximum capacity in
|
|
ParentQueue to use its parent's absolute-capacity rather than its own
|
|
absolute-capacity. (acmurthy)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Launching bin/yarn and bin/mapred only
|
|
*once* in AM for constructing classpaths to avoid multiple forks and huge vmem
|
|
usage by AM. (vinodkv)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix CapacityScheduler to release unused
|
|
reservations on application completion. (acmurthy)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix CapacityScheduler (LeafQueue) to not
|
|
allocate DATA_LOCAL containers when they are not required on the rack.
|
|
(acmurthy)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Makes uber-task disabled by default (ddas)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Make logging and memory for AM configurable
|
|
for the user via command line (mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fixing a bug in previous patch (r1103657).
|
|
Now bin/yarn truly shouldn't be launched multiple times in a single AM.
|
|
(vinodkv)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fixing a bug to do with setting the staging
|
|
dir. (ddas)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fixed Composite Service to shutdown
|
|
services if an error occurs on starting any one of those (mahadev & chris)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix the tests to use jvm fork mode to avoid
|
|
errors in shutting down services (sidharth seth)
|
|
|
|
MAPREDUCE-2500. PB factories are not thread safe (Siddharth Seth via mahadev)
|
|
|
|
MAPREDUCE-2504. race in JobHistoryEventHandler stop (Siddharth Seth via mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix job hang if the AM launch fails.
|
|
(mahadev)
|
|
|
|
MAPREDUCE-2509. Fix NPE in UI for pending attempts. (Luke Lu via mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Add junit jar to lib in assembly (mahadev
|
|
and luke)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Distributed cache bug fix to pass Terasort.
|
|
(vinodkv)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix null pointer exception in kill task
|
|
attempt (mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Refactored RMContainerAllocator to release
|
|
unused containers. (sharad)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Changed Scheduler to return available limit
|
|
to AM in the allocate api. (acmurthy)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix nodemanager expiry to not throw OOM.
|
|
(mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Use DefaultContainerExecutor for
|
|
integration tests. (cdouglas)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix NPE in test case (mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix for regression on the scheduling of
|
|
reduces before maps are done (ddas)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix distributed-cache related bugs.
|
|
(vinodkv)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Ensure unused containers released by AM are
|
|
correctly counted for queue-capacity. (acmurthy)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix TestRuntimeEstimators (Siddharth Seth
|
|
via ddas)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix queue refresh to correctly record newly
|
|
added queues in CapacityScheduler. (acmurthy)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Added metrics for tracking reservations in
|
|
CapacityScheduler. (Luke Lu via acmurthy)
|
|
|
|
MAPREDUCE-2522. Security for JobHistory service. (Siddharth Seth via mahadev)
|
|
|
|
MAPREDUCE-2534. Fix CI breaking hard coded version in jobclient pom. (Luke Lu
|
|
via mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Add public cache. (cdouglas)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Made number of RPC server threads
|
|
configurable. (acmurthy)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Added acl check for RMAdmin. (acmurthy)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Adding job kill for any state that the job
|
|
is in with access control. (mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Add debug statements for AM not launching
|
|
(mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fixing race condition leader to hung jobs
|
|
in scheduler negotiator (mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Add debug config for delaying delete of
|
|
local files. (cdouglas)
|
|
|
|
MAPREDUCE-2527. Metrics for MRAppMaster (Luke lu via mahadev)
|
|
|
|
MAPREDUCE-2532. Metrics for NodeManager (Luke Lu via mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fixed an NPE during handling of unnecessary
|
|
reservations in CS. (acmurthy)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix for public dist cache to work with non
|
|
default hdfs (mahadev &ddas)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Making streaming -file option work. Also
|
|
minor fixes for successful compilation of contrib tests. (vinodkv)
|
|
|
|
MAPREDUCE-2536. Backporting changes to MR-279.
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Bugfix for using user staging directory for
|
|
history files (Siddharth Seth via mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. First fix for making basic speculative
|
|
execution work (ddas)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fixes for TestFail/Kill (ddas)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Set correct version of avro-maven-plugin
|
|
that is available in apache maven repositories. (vinodkv)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fixing and reneabling
|
|
TestContainerTokenSecretManager. (vinodkv)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Cleaning up configuration constants in
|
|
mapreduce modules. (vinodkv)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fixing NPE on speculator in MRAppMaster and
|
|
making job-history optional in tests to make test goal succeed. (vinodk and
|
|
sharadag).
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fixed NPE in CS by checking Application
|
|
state before scheduling and fixing synchronization in CS. (acmurthy)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Making pipes work with YARN. Changed pipes
|
|
to get log-locations from an environmental variable. (vinodkv)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Ensure 'lost' NodeManagers are dealt
|
|
appropriately, the containers are released correctly. (acmurthy)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Adding some more logging for AM expiry logs
|
|
(mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Reduce ramp up and zero maps support.
|
|
(sharad)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Allowing hdfs calls from streaming/pipes
|
|
tasks. (vinodkv)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Added ability to decommission nodes and
|
|
completed RM administration tools to achieve parity with JobTracker. (acmurthy)
|
|
|
|
MAPREDUCE-2551. Added JobSummaryLog. (Siddharth Seth via acmurthy)
|
|
|
|
MAPREDUCE-2552. Fixed NPE in CompletedJob in JobHistoryServer. (Siddharth Seth
|
|
via acmurthy)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix reduce slow start. (sharad)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fixed TestFifoScheduler. (acmurthy)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix build issue for using yarn.version
|
|
instead of hadoop-mapred.version (mahadev and giri)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fixes in the handling of KILL events in the
|
|
SUCCEEDED state for tasks in the application master (ddas)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix for NPE in TestRMNMRPCResponseId.
|
|
(mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix a deadlock in the resourcemanager.
|
|
(mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. NodeStatus.getNodeHealthStatus().setBlah
|
|
broken (Siddharth Seth)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix another NPE in TestRMNMRPCResponseId.
|
|
(mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix for NPE in TestNMExpiry (siddharth
|
|
seth)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Making each node aggregate all its
|
|
user-logs to a separate hdfs file. (vinodkv)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix calculation of max-capacity for a
|
|
queue, also fixed a bug in registration of NodeManagers. (acmurthy)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. More cleaning up constants, removing stale
|
|
code, and making conspicuous the envs that apps depend on to be provided by
|
|
YARN. (vinodkv)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix container size rounding in AM and
|
|
headroom in RM. (acmurthy and sharad)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Disable Job acls until fixed (mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix to report job status if the application
|
|
is KILLED/FAILED. (mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix a corner case in headroom computation -
|
|
now reservations are taken into account and headroom is computed much later to
|
|
account for allocations/reservations. (acmurthy)
|
|
|
|
MAPREDUCE-2537. The RM writes its log to
|
|
yarn-mapred-resourcemanager-<RM_Host>.out (Robert Evans via mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix logging for showing the state of job
|
|
(FAILED/KILLED/SUCCEEDED) when it completes (mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Re-enabled TestCapacityScheduler.
|
|
(acmurthy)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Support for min and max container capacity.
|
|
(acmurthy and sharad)
|
|
|
|
MAPREDUCE-2531. Fixed jobcontrol to downgrade JobID. (Robert Evans via acmurthy)
|
|
|
|
MAPREDUCE-2539. Fixed NPE in getMapTaskReports in JobClient. (Robert Evans via
|
|
acmurthy)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fixing the wrong config key used in
|
|
JobHistory that prevented configuring move-thread interval. (vinodkv)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fixed inconsistency in QueueACL enums.
|
|
(acmurthy)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix various issues with Web UI's. (Luke Lu)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix class cast exception in Task abort for
|
|
old mapreduce apis. (sharad)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Add deletion of distributed cache
|
|
resources. (cdouglas)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Disable aggregation of logs onto DFS till
|
|
JobHistoryServer starts serving logs. (vinodkv)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Cleanup redundant code in TaskAttemptImpl.
|
|
(sharad)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Work around broken signaling in public
|
|
cache. (cdouglas)
|
|
|
|
MAPREDUCE-2566. YarnConfiguration should reloadConfiguration if instantiated
|
|
with a non YarnConfiguration object. (Siddharth Seth)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fully resolve paths when launching
|
|
containers. (Siddharth Seth)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Re-enabling Uber-AM feature. (vinodkv)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fixed deadlock during expiring NMs.
|
|
(acmurthy)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Solving NPEs during
|
|
ContainerManager#StopContainer. Also removing the unused
|
|
ContainerManager#CleanupContainer api. (vinodkv)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Remove retries in dist cache so that NM's
|
|
do not shutdown (mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix classpath construction for Task.
|
|
(vinodkv via sharad)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Sending Job diagnostics from AM to RM and
|
|
redirect to history-url on job completion. (vinodkv and sharadag)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Added clover in pom dependency. (sharad)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Intermittent TestMRApp failures on faster
|
|
Linux desktop. (Luke lu via mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Including source files in release
|
|
distribution (Luke Lu via mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Intermittent TestMRApp failures on faster
|
|
Linux desktop (part 2) (Luke lu via mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Disable Uber AM. (mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Added few job diagnostic messages. (sharad)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Add ability to includes src files in
|
|
assembly target for maven (Luke Lu via mahadev)
|
|
|
|
MAPREDUCE-2582. Cleanup JobHistory event generation.(Siddharth Seth via sharad)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix rounding off problem in reduce ramp up.
|
|
(sharad)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix more rounding off problems in reduce
|
|
ramp up. Also fix a bug preventing the application of the cap on reduce ramp-up.
|
|
(Sharad Agarwal via vinodkv)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix to exclude images dir into the tar
|
|
distribution (Luke Lu via gkesavan)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Changes a couple of usages of FileContext
|
|
to FileSystem in YarnRunner to handle distributed cache path resolutions on
|
|
non-default filesystems. (ddas)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Display failed/killed attempts of the task
|
|
on MR AM UI separately. (vinodkv)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Miscellaneous UI fixes + source code
|
|
formatting for MR JobHistoryEventHandler. (vinodkv)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fixing broken link to logs for container on
|
|
NM web UI. (vinodkv)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fixing the bug which was causing FAILED
|
|
jobs to be displayed as COMPLETED on the RM UI. (vinodkv)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Job level node blacklisting. (sharad)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix NPE in history event handling
|
|
(Siddharth Seth via mahadev)
|
|
|
|
MAPREDUCE-2569. Ensure root queue allocated 100% capacity. (Jonathan Eagles via
|
|
cdouglas)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix ClassCastException in JobHistoryServer
|
|
for certain jobs. (Siddharth Seth via llu)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Changes for invoking rack resolution in the
|
|
RM and in the AM (ddas)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix concurrent modification exception in
|
|
the Capacity Scheduler (mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix stopContainer for setsid challenged
|
|
platforms. (llu)
|
|
|
|
MAPREDUCE-2587. Generate yarn version for UI. (Thomas Graves via lluts page to
|
|
the history server UI. (vinodkv)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Bug fix to set correct state on containers
|
|
so as to avoid duplicate containers from RM to AM. (vinodkv)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Hack until MAPREDUCE-2365 is fixed to make
|
|
PIG work with MRV2. (mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Changes a couple of usages of FileContext
|
|
to FileSystem in TaskAttemptImpl to handle distributed cache path resolutions on
|
|
non-default filesystems. (ddas)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix NPE when requesting attempts for
|
|
completed jobs. (Siddharth Seth via llu)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Improve logging for AM when requesting
|
|
containers to show the right ask and release fields (mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix race condition between multiple
|
|
localizers on a single node. (cdouglas via mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix RM app start/finish time and
|
|
diagnostics. (llu)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix to schedule reduces irrespective of the
|
|
headroom when all maps are done so as to avoid stall in reduce-scheduling when
|
|
slow-start is disabled. (Sharad Agarwal via vinodkv).
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Disabling locality-wait in
|
|
CapacityScheduler for now to prevent uber-slow scheduling for apps with no
|
|
data-locality constraints (sleep-job like). (vinodkv)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fixing scheduling deadlock in AM because of
|
|
incorrect headRoom values from RM. The bug happens when AM releases containers
|
|
and RM decrements current memory usage twice for all those containers. (vinodkv)
|
|
|
|
MAPREDUCE-2611. Fix counters, finish times etc. in job history. (Siddharth Seth
|
|
via llu)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix for ConcurrentModification exception
|
|
while iterating through tokens in a UGI in ContainerLauncherImpl. (ddas)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix for NPE in YarnChild that was causing
|
|
lots of tasks to fail. (vinodkv)
|
|
|
|
MAPREDUCE-2615. Make killJob go through AM and fix JobSummaryLog. (Siddharth
|
|
Seth via llu)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix class cast exception in release
|
|
reserved containers in capacity scheduler (mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix diagnostics display for more than 100
|
|
apps in RM. (llu)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix some invalid transitions in the RM.
|
|
(vinodkv via ddas)
|
|
|
|
MAPREDUCE-2618. Fix NPE in 0 map 0 reduce jobs. (Jeffrey Naisbitt via llu)
|
|
|
|
MAPREDUCE-2625. Add version info to nodemanager info page. (Jonathan Eagles via
|
|
llu)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. (1) Faster retries from AM to HistoryServer
|
|
(2) Correct diagnostics for containers. (vinodkv)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Major ASM cleanup. Streamlining classes,
|
|
interface and events. (vinodkv)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Reinstate resolve path fixes for viewfs.
|
|
(Siddharth Seth via llu)
|
|
|
|
MAPREDUCE-2633. Add a getCounter(Enum) method to the Counters record. (Josh
|
|
Wills via sharad)
|
|
|
|
MAPREDUCE-2645. Updates to MRv2 INSTALL and README documentation. (Josh Wills
|
|
via vinodkv)
|
|
|
|
MAPREDUCE-2628. Add compiled on date to NM and RM info/about page.
|
|
|
|
MAPREDUCE-2400. Remove Cluster's dependency on JobTracker via a ServiceProvider
|
|
for the actual implementation. (tomwhite via acmurthy)
|
|
|
|
MAPREDUCE-2663. Refactoring StateMachineFactory inner classes. (ahmed radwan via
|
|
mahadev)
|
|
|
|
MAPREDUCE-2678. minimum-user-limit-percent no longer honored. (naisbitt via
|
|
mahadev)
|
|
|
|
MAPREDUCE-2630. refreshQueues leads to NPEs when used w/FifoScheduler. (Josh
|
|
Wills via mahadev)
|
|
|
|
MAPREDUCE-2644. NodeManager fails to create containers when NM_LOG_DIR is not
|
|
explicitly set in the Configuration. (Josh Wills via vinodkv)
|
|
|
|
MAPREDUCE-2661. Fix TaskImpl to not access MapTaskImpl. (Ahmed Radwan via
|
|
sharad)
|
|
|
|
HADOOP-6929. Backport changes to MR-279 (mahadev and owen)
|
|
|
|
HADOOP-6929. Making Security Info abstract and not an interface (mahadev)
|
|
|
|
MAPREDUCE-2667. mapred job -kill leaves application in RUNNING state (thomas
|
|
graves via mahadev)
|
|
|
|
MAPREDUCE-2664. Implement JobCounters for Mtions as asynchronous. (vinodkv,
|
|
sharad and acmurthy)
|
|
|
|
MAPREDUCE-2773. server.api.records.NodeHealthStatus renamed but not updated in
|
|
client NodeHealthStatus.java (Thomas Graves via mahadev)
|
|
|
|
MAPREDUCE-2772. Fix MR-279 build after common mavenization. (Robert Joseph Evans
|
|
via llu)
|
|
|
|
MAPREDUCE-2772. Fix MR-279 build after common mavenization, part 2. (Thomas
|
|
Graves via llu)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Harmonize slf4j versions. (llu)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix NPE in FifoScheduler. (mahadev)
|
|
|
|
MAPREDUCE-2776. Fix some of the yarn findbug warnings. (Siddharth Seth via
|
|
mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix findbugs warnings in mr-client modules,
|
|
part 1 (mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix findbugs warnings in mr-client modules
|
|
part 2 (mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix findbugs warnings in mr-client modules
|
|
part 3 (mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix the poms to enable 0.23 snapshots for
|
|
hdfs/common from apache nightly builds (gkesavan)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix ivy conf to work with the hadoop common
|
|
trunk maven build changes. (Giridharan Kesavan)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Patch for findbugs warnings in Resource
|
|
Manager (Siddharth Seth via mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fix for running ant targets to use the
|
|
right set of common/test jars (gkesavan via mahadev)
|
|
|
|
MAPREDUCE-2782. Unit tests for CapacityScheduler. (acmurthy)
|
|
|
|
MAPREDUCE-2706. Log job submission failures. (Jeffrey Naisbitt via acmurthy)
|
|
|
|
MAPREDUCE-2781. mr279 RM application finishtime not set (Thomas Graves via
|
|
mahadev)
|
|
|
|
MAPREDUCE-279. Fix in MR-279 branch. Fixed CS locality wait factor. (acmurthy)
|
|
|
|
MAPREDUCE-2808. pull MAPREDUCE-2797 into mr279 branch (Thomas Graves via
|
|
mahadev)
|
|
|
|
MAPREDUCE-2639. Bug fixes in speculate.DataStatistics. (Josh Wills via acmurthy)
|
|
|
|
MAPREDUCE-2839. Fixed TokenCache to get delegation tokens using both new and old
|
|
apis. (Siddharth Seth via acmurthy)
|
|
|
|
MAPREDUCE-2727. Fix divide-by-zero error in SleepJob for sleepCount equals 0.
|
|
(Jeffrey Naisbitt via acmurthy)
|
|
|
|
MAPREDUCE-2839. Fixed TokenCache to get delegation tokens using both new
|
|
and old apis. (Siddharth Seth via acmurthy)
|
|
|
|
MAPREDUCE-2727. Fix divide-by-zero error in SleepJob for sleepCount equals
|
|
0. (Jeffrey Naisbitt via acmurthy)
|
|
|
|
MAPREDUCE-2860. Fix log4j logging in the maven test cases. (mahadev)
|
|
|
|
MAPREDUCE-2867. Remove Unused TestApplicaitonCleanup in resourcemanager/applicationsmanager.
|
|
(mahadev)
|
|
|
|
MAPREDUCE-2868. ant build broken in hadoop-mapreduce dir (mahadev, giri and arun via mahadev)
|
|
|
|
MAPREDUCE-2649. Handling of finished applications in RM. (Thomas Graves
|
|
via acmurthy)
|
|
|
|
MAPREDUCE-2838. Fix MapReduce builds to use new hadoop-common test jars.
|
|
(gkesavan via acmurthy)
|
|
|
|
MAPREDUCE-2859. Fix eclipse plugin contrib module compilation (gkesavan)
|
|
|
|
MAPREDUCE-2846. Fix missing synchronization in the task log management.
|
|
(omalley)
|
|
|
|
MAPREDUCE-2807. Fix AM restart and client redirection. (sharad)
|
|
|
|
MAPREDUCE-2877. Add missing Apache license header in some files in MR
|
|
and also add the rat plugin to the poms. (mahadev)
|
|
|
|
MAPREDUCE-2796. Set start times for MR applications for clients to see.
|
|
(Devaraj K via acmurthy)
|
|
|
|
MAPREDUCE-2879. Fix version for MR-279 to 0.23.0. (acmurthy)
|
|
|
|
MAPREDUCE-2881. Fix to include log4j 1.2.16 depenency (gkesavan)
|
|
|
|
MAPREDUCE-2885. Fix mapred-config.sh to look for hadoop-config.sh in
|
|
HADOOP_COMMON_HOME/libexec. (acmurthy)
|
|
|
|
MAPREDUCE-2893. Remove duplicate entry of YarnClientProtocolProvider in
|
|
ClientProtocolProvider services file. (Liang-Chi Hsieh via acmurthy)
|
|
|
|
MAPREDUCE-2891. Javadoc for AMRMProtocol and related records. (acmurthy)
|
|
|
|
MAPREDUCE-2898. Javadoc for ContainerManager protocol and related records.
|
|
(acmurthy)
|
|
|
|
MAPREDUCE-2904. Fixed bin/yarn to correctly include HDFS jars and
|
|
clean up of stale refs to pre-mavenized Hadoop Common and HDFS.
|
|
(Sharad Agarwal and Arun C. Murthy via acmurthy)
|
|
|
|
MAPREDUCE-2737. Update the progress of jobs on client side. (Siddharth Seth
|
|
and Mahadev Konar via mahadev)
|
|
|
|
MAPREDUCE-2886. Fix Javadoc warnings in MapReduce. (mahadev)
|
|
|
|
MAPREDUCE-2897. Javadoc for ClientRMProtocol protocol and related records.
|
|
(acmurthy)
|
|
|
|
MAPREDUCE-2916. Ivy build for MRv1 fails with bad organization for
|
|
common daemon. (mahadev)
|
|
|
|
MAPREDUCE-2917. Fixed corner case in container reservation which led to
|
|
starvation and hung jobs. (acmurthy)
|
|
|
|
MAPREDUCE-2756. Better error handling in JobControl for failed jobs.
|
|
(Robert Evans via acmurthy)
|
|
|
|
MAPREDUCE-2716. MRReliabilityTest job fails because of missing
|
|
job-file. (Jeffrey Naisbitt via vinodkv)
|
|
|
|
MAPREDUCE-2882. TestLineRecordReader depends on ant jars. (todd)
|
|
|
|
MAPREDUCE-2687. Fix NodeManager to use the right version of
|
|
LocalDirAllocator.getLocalPathToWrite. (mahadev & acmurthy)
|
|
|
|
MAPREDUCE-2800. Set final progress for tasks to ensure all task information
|
|
is correctly logged to JobHistory. (Siddharth Seth via acmurthy)
|
|
|
|
MAPREDUCE-2938. Log application submission failure in CapacityScheduler.
|
|
(acmurthy)
|
|
|
|
MAPREDUCE-2948. Hadoop streaming test failure, post MR-2767 (mahadev)
|
|
|
|
MAPREDUCE-2908. Fix all findbugs warnings. (vinodkv via acmurthy)
|
|
|
|
MAPREDUCE-2947. Fixed race condition in AuxiliaryServices. (vinodkv via
|
|
acmurthy)
|
|
|
|
MAPREDUCE-2844. Fixed display of nodes in UI. (Ravi Teja Ch N V via
|
|
acmurthy)
|
|
|
|
MAPREDUCE-2677. Fixed 404 for some links from HistoryServer. (Robert Evans
|
|
via acmurthy)
|
|
|
|
MAPREDUCE-2937. Ensure reason for application failure is displayed to the
|
|
user. (mahadev via acmurthy)
|
|
|
|
MAPREDUCE-2953. Fix a race condition on submission which caused client to
|
|
incorrectly assume application was gone by making submission synchronous
|
|
for RMAppManager. (Thomas Graves via acmurthy)
|
|
|
|
MAPREDUCE-2963. Fix hang in TestMRJobs. (Siddharth Seth via acmurthy)
|
|
|
|
MAPREDUCE-2954. Fixed a deadlock in NM caused due to wrong synchronization
|
|
in protocol buffer records. (Siddharth Seth via vinodkv)
|
|
|
|
MAPREDUCE-2975. Fixed YARNRunner to use YarnConfiguration rather than
|
|
Configuration. (mahadev via acmurthy)
|
|
|
|
MAPREDUCE-2971. ant build mapreduce fails protected access jc.displayJobList
|
|
(jobs) (Thomas Graves via mahadev)
|
|
|
|
MAPREDUCE-2691. Finishing up the cleanup of distributed cache file resources
|
|
and related tests. (Siddharth Seth via vinodkv)
|
|
|
|
MAPREDUCE-2749. Ensure NM registers with RM after starting all its services
|
|
correctly. (Thomas Graves via acmurthy)
|
|
|
|
MAPREDUCE-2979. Removed the needless ClientProtocolProvider configuration
|
|
from the hadoop-mapreduce-client-core module. (Siddharth Seth via vinodkv)
|
|
|
|
MAPREDUCE-2985. Fixed findbugs warnings in ResourceLocalizationService.
|
|
(Thomas Graves via acmurthy)
|
|
|
|
MAPREDUCE-2874. Fix formatting of ApplicationId in web-ui. (Eric Payne via
|
|
acmurthy)
|
|
|
|
MAPREDUCE-2995. Better handling of expired containers in MapReduce
|
|
ApplicationMaster. (vinodkv via acmurthy)
|
|
|
|
MAPREDUCE-2995. Fixed race condition in ContainerLauncher. (vinodkv via
|
|
acmurthy)
|
|
|
|
MAPREDUCE-2949. Fixed NodeManager to shut-down correctly if a service
|
|
startup fails. (Ravi Teja via vinodkv)
|
|
|
|
MAPREDUCE-3005. Fix both FifoScheduler and CapacityScheduler to correctly
|
|
enforce locality constraints. (acmurthy)
|
|
|
|
MAPREDUCE-3007. Fixed Yarn Mapreduce client to be able to connect to
|
|
JobHistoryServer in secure mode. (vinodkv)
|
|
|
|
MAPREDUCE-2987. Fixed display of logged user on RM Web-UI. (Thomas Graves
|
|
via acmurthy)
|
|
|
|
MAPREDUCE-3006. Fixed MapReduce AM to exit only after properly writing out
|
|
history file. (vinodkv)
|
|
|
|
MAPREDUCE-2925. Fixed Yarn+MR client code to behave saner with completed
|
|
jobs. (Devaraj K via vinodkv)
|
|
|
|
MAPREDUCE-3030. Fixed a bug in NodeId.equals() that was causing RM to
|
|
reject all NMs. (Devaraj K via vinodkv)
|
|
|
|
MAPREDUCE-3042. Fixed default ResourceTracker address. (Chris Riccomini
|
|
via acmurthy)
|
|
|
|
MAPREDUCE-3038. job history server not starting because conf() missing
|
|
HsController (Jeffrey Naisbitt via mahadev)
|
|
|
|
MAPREDUCE-3004. Fix ReduceTask to not assume 'local' mode in YARN. (Hitesh
|
|
Shah via acmurthy)
|
|
|
|
MAPREDUCE-3017. The Web UI shows FINISHED for killed/successful/failed jobs.
|
|
(mahadev)
|
|
|
|
MAPREDUCE-3040. Fixed extra copy of Configuration in
|
|
YarnClientProtocolProvider and ensured MiniMRYarnCluster sets JobHistory
|
|
configuration for tests. (acmurthy)
|
|
|
|
MAPREDUCE-3018. Fixed -file option for streaming. (mahadev via acmurthy)
|
|
|
|
MAPREDUCE-3036. Fixed metrics for reserved resources in CS. (Robert Evans
|
|
via acmurthy)
|
|
|
|
MAPREDUCE-2998. Fixed a bug in TaskAttemptImpl which caused it to fork
|
|
bin/mapred too many times. (vinodkv via acmurthy)
|
|
|
|
MAPREDUCE-3023. Fixed clients to display queue state correctly. (Ravi
|
|
Prakash via acmurthy)
|
|
|
|
MAPREDUCE-2970. Fixed NPEs in corner cases with different configurations
|
|
for mapreduce.framework.name. (Venu Gopala Rao via vinodkv)
|
|
|
|
MAPREDUCE-3062. Fixed default RMAdmin address. (Chris Riccomini
|
|
via acmurthy)
|
|
|
|
MAPREDUCE-3066. Fixed default ResourceTracker address for the NodeManager.
|
|
(Chris Riccomini via acmurthy)
|
|
|
|
MAPREDUCE-3044. Pipes jobs stuck without making progress. (mahadev)
|
|
|
|
MAPREDUCE-2754. Fixed MR AM stdout, stderr and syslog to redirect to
|
|
correct log-files. (Ravi Teja Ch N V via vinodkv)
|
|
|
|
MAPREDUCE-3073. Fixed build issues in MR1. (mahadev via acmurthy)
|
|
|
|
MAPREDUCE-2691. Increase threadpool size for launching containers in
|
|
MapReduce ApplicationMaster. (vinodkv via acmurthy)
|
|
|
|
|
|
MAPREDUCE-2990. Fixed display of NodeHealthStatus. (Subroto Sanyal via
|
|
acmurthy)
|
|
|
|
MAPREDUCE-3053. Better diagnostic message for unknown methods in ProtoBuf
|
|
RPCs. (vinodkv via acmurthy)
|
|
|
|
MAPREDUCE-2952. Fixed ResourceManager/MR-client to consume diagnostics
|
|
for AM failures in a couple of corner cases. (Arun C Murthy via vinodkv)
|
|
|
|
MAPREDUCE-3064. 27 unit test failures with Invalid
|
|
"mapreduce.jobtracker.address" configuration value for
|
|
JobTracker: "local" (Venu Gopala Rao via mahadev)
|
|
|
|
MAPREDUCE-3090. Fix MR AM to use ApplicationAttemptId rather than
|
|
(ApplicationId, startCount) consistently. (acmurthy)
|
|
|
|
MAPREDUCE-2646. Fixed AMRMProtocol to return containers based on
|
|
priority. (Sharad Agarwal and Arun C Murthy via vinodkv)
|
|
|
|
MAPREDUCE-3031. Proper handling of killed containers to prevent stuck
|
|
containers/AMs on an external kill signal. (Siddharth Seth via vinodkv)
|
|
|
|
MAPREDUCE-2984. Better error message for displaying completed containers.
|
|
(Devaraj K via acmurthy)
|
|
|
|
MAPREDUCE-3071. app master configuration web UI link under the Job menu
|
|
opens up application menu. (thomas graves via mahadev)
|
|
|
|
MAPREDUCE-3067. Ensure exit-code is set correctly for containers. (Hitesh
|
|
Shah via acmurthy)
|
|
|
|
MAPREDUCE-2999. Fix YARN webapp framework to properly filter servlet
|
|
paths. (Thomas Graves via vinodkv)
|
|
|
|
MAPREDUCE-3095. fairscheduler ivy including wrong version for hdfs.
|
|
(John George via mahadev)
|
|
|
|
MAPREDUCE-3054. Unable to kill submitted jobs. (mahadev)
|
|
|
|
MAPREDUCE-3021. Change base urls for RM web-ui. (Thomas Graves via
|
|
acmurthy)
|
|
|
|
MAPREDUCE-3041. Fixed ClientRMProtocol to provide min/max resource
|
|
capabilities along-with new ApplicationId for application submission.
|
|
(Hitesh Shah via acmurthy)
|
|
|
|
MAPREDUCE-2843. Fixed the node-table to be completely displayed and making
|
|
node entries on RM UI to be sortable. (Abhijit Suresh Shingate via vinodkv)
|
|
|
|
MAPREDUCE-3110. Fixed TestRPC failure. (vinodkv)
|
|
|
|
MAPREDUCE-3078. Ensure MapReduce AM reports progress correctly for
|
|
displaying on the RM Web-UI. (vinodkv via acmurthy)
|
|
|
|
MAPREDUCE-3114. Fixed invalid ApplicationURL on RM WebUI. (Subroto Sanyal
|
|
via vinodkv)
|
|
|
|
MAPREDUCE-2791. Added missing info on 'job -status' output. (Devaraj K via
|
|
acmurthy)
|
|
|
|
MAPREDUCE-2996. Add uber-ness information to JobHistory. (Jonathan Eagles
|
|
via acmurthy)
|
|
|
|
MAPREDUCE-3050. Add ability to get resource usage information for
|
|
applications and nodes. (Robert Evans via acmurthy)
|
|
|
|
MAPREDUCE-3113. Ensure bin/yarn and bin/yarn-daemon.sh identify the root
|
|
of the install properly. (Xie Xianshan via acmurthy)
|
|
|
|
MAPREDUCE-2792. Replace usage of node ip-addresses with hostnames.
|
|
(vinodkv via acmurthy)
|
|
|
|
MAPREDUCE-3112. Fixed recursive sourcing of HADOOP_OPTS environment
|
|
variable. (Eric Yang)
|
|
|
|
MAPREDUCE-3056. Changed the default staging directory to not include
|
|
user.name to prevent issues with non-secure mode. (Devaraj K via vinodkv)
|
|
|
|
MAPREDUCE-2913. Fixed TestMRJobs.testFailingMapper to assert the correct
|
|
TaskCompletionEventStatus. (Jonathan Eagles via vinodkv)
|
|
|
|
MAPREDUCE-2794. [MR-279] Incorrect metrics value for AvailableGB per
|
|
queue per user. (John George via mahadev)
|
|
|
|
MAPREDUCE-2783. Fixing RM web-UI to show no tracking-URL when AM
|
|
crashes. (Eric Payne via vinodkv)
|
|
|
|
MAPREDUCE-3141. Fix the broken MRAppMaster to work over YARN in security
|
|
mode.(vinodkv)
|
|
|
|
MAPREDUCE-2751. Modified NodeManager to stop leaving around local files
|
|
after application finishes. (Siddharth Seth via vinodkv)
|
|
|
|
MAPREDUCE-3033. Ensure Master interface pays attention to classic v/s yarn
|
|
frameworks. (Hitesh Shah via acmurthy)
|
|
|
|
MAPREDUCE-2802. Ensure JobHistory filenames have jobId. (Jonathan Eagles
|
|
via acmurthy)
|
|
|
|
MAPREDUCE-2876. Use a different config for ContainerAllocationExpirer.
|
|
(Anupam Seth via acmurthy)
|
|
|
|
MAPREDUCE-3153. Fix TestFileOutputCommitter which was broken by
|
|
MAPREDUCE-2702. (mahadev via acmurthy)
|
|
|
|
MAPREDUCE-3123. Fix NM to quote symlink names to escape special
|
|
characters. (Hitesh Shah via acmurthy)
|
|
|
|
MAPREDUCE-3154. Fix JobSubmitter to check for output specs before copying
|
|
job submission files to fail fast. (Abhijit Suresh Shingate via acmurthy)
|
|
|
|
MAPREDUCE-3158. Fix test failures in MRv1 due to default framework being
|
|
set to yarn. (Hitesh Shah via acmurthy)
|
|
|
|
MAPREDUCE-3167. container-executor is not being packaged with the assembly
|
|
target. (mahadev)
|
|
|
|
MAPREDUCE-3020. Fixed TaskAttemptImpl to log the correct node-address for
|
|
a finished Reduce task. (Chackaravarthy via vinodkv)
|
|
|
|
MAPREDUCE-2668. Fixed AuxServices to send a signal on application-finish
|
|
to all the services. (Thomas Graves via vinodkv)
|
|
|
|
MAPREDUCE-3126. Fixed a corner case in CapacityScheduler where headroom
|
|
wasn't updated on changes to cluster size. (acmurthy)
|
|
|
|
MAPREDUCE-3140. Fixed the invalid JobHistory URL for failed
|
|
applications. (Subroto Sanyal via vinodkv)
|
|
|
|
MAPREDUCE-3125. Modified TaskImpl to consider only non-failed, non-killed
|
|
task-attempts for obtaining task's progress. (Hitesh Shah via vinodkv)
|
|
|
|
MAPREDUCE-2666. Retrieve shuffle port number from JobHistory on MR AM
|
|
restart. (Jonathan Eagles via acmurthy)
|
|
|
|
MAPREDUCE-2789. Complete schedulingInfo on CLI. (Eric Payne via acmurthy)
|
|
|
|
MAPREDUCE-3170. Fixed job output commit for deep hierarchies. (Hitesh Shah
|
|
via acmurthy)
|
|
|
|
MAPREDUCE-3124. Fixed location of native libs i.e. libhadoop.so for
|
|
containers. (John George via acmurthy)
|
|
|
|
MAPREDUCE-3057. Job History Server goes of OutOfMemory with 1200 Jobs
|
|
and Heap Size set to 10 GB. (Eric Payne via mahadev)
|
|
|
|
MAPREDUCE-2840. mr279 TestUberAM.testSleepJob test fails. (jonathan eagles
|
|
via mahadev)
|
|
|
|
MAPREDUCE-3190. Ensure bin/yarn fails early with a clear error message
|
|
when HADOOP_COMMON_HOME or HADOOP_HDFS_HOME are not set. (todd & acmurthy
|
|
via acmurthy)
|
|
|
|
MAPREDUCE-3189. Add link decoration back to MR2's CSS. (Todd Lipcon via
|
|
mahadev)
|
|
|
|
MAPREDUCE-3127. Changed default value of yarn.resourcemanager.acl.enable
|
|
to true and added some more documentation. (acmurthy)
|
|
|
|
MAPREDUCE-3032. Fixed TaskAttemptImpl so that JobHistory can have error
|
|
information about failed tasks. (Devaraj K via vinodkv)
|
|
|
|
MAPREDUCE-3196. TestLinuxContainerExecutorWithMocks fails on Mac OSX.
|
|
(Arun Murthy via mahadev)
|
|
|
|
MAPREDUCE-3197. TestMRClientService failing on building clean checkout of
|
|
branch 0.23 (mahadev)
|
|
|
|
MAPREDUCE-2762. Cleanup MR staging directory on completion. (mahadev via
|
|
acmurthy)
|
|
|
|
MAPREDUCE-3165. Ensure logging options are set correctly for MR AM and
|
|
tasks. (todd via acmurthy)
|
|
|
|
MAPREDUCE-3203. Fix some javac warnings in MRAppMaster. (mahadev)
|
|
|
|
MAPREDUCE-3199. Fixed pom files to include correct log4j configuration for
|
|
tests. (vinodkv)
|
|
|
|
MAPREDUCE-3162. Separated application-init and container-init event types
|
|
in NodeManager's Application state machine. (Todd Lipcon via vinodkv)
|
|
|
|
MAPREDUCE-3176. Fixed ant mapreduce tests that are timing out because
|
|
of wrong framework name. (Hitesh Shah via vinodkv)
|
|
|
|
MAPREDUCE-3181. Fixed MapReduce runtime to load yarn-default.xml and
|
|
yarn-site.xml. (acmurthy)
|
|
|
|
MAPREDUCE-2788. Normalize resource requests in FifoScheduler
|
|
appropriately. (Ahmed Radwan via acmurthy)
|
|
|
|
MAPREDUCE-2693. Fix NPE in job-blacklisting. (Hitesh Shah via acmurthy)
|
|
|
|
MAPREDUCE-3208. Fix NPE task/container log appenders. (liangzhwa via
|
|
acmurthy)
|
|
|
|
MAPREDUCE-3212. Fix usage/help message for bin/yarn. (Bhallamudi Venkata
|
|
Siva Kamesh via acmurthy)
|
|
|
|
MAPREDUCE-3179. Ensure failed tests exit with right error code. (Jonathan
|
|
Eagles via acmurthy)
|
|
|
|
MAPREDUCE-3188. Ensure correct shutdown in services. (todd via acmurthy)
|
|
|
|
MAPREDUCE-3226. Fix shutdown of fetcher threads. (vinodkv via acmurthy)
|
|
|
|
MAPREDUCE-3070. Fix NodeManager to use ephemeral ports by default.
|
|
(Devaraj K via acmurthy)
|
|
|
|
MAPREDUCE-3242. Trunk compilation broken with bad interaction from
|
|
MAPREDUCE-3070 and MAPREDUCE-3239. (mahadev)
|
|
|
|
MAPREDUCE-3058. Fixed MR YarnChild to report failure when task throws an
|
|
error and thus prevent a hanging task and job. (vinodkv)
|
|
|
|
MAPREDUCE-3087. Fixed the mapreduce classpath to correctly include the
|
|
generated-classpath file needed for tests. (Ravi Prakash via vinodkv)
|
|
|
|
MAPREDUCE-3233. Fixed a bug in MR Job so as to be able to restart the
|
|
application on AM crash. (Mahadev Konar via vinodkv)
|
|
|
|
MAPREDUCE-3028. Added job-end notification support. (Ravi Prakash via
|
|
acmurthy)
|
|
|
|
MAPREDUCE-3249. Ensure shuffle-port is correctly used duringMR AM recovery.
|
|
(vinodkv via acmurthy)
|
|
|
|
MAPREDUCE-3252. Fix map tasks to not rewrite data an extra time when
|
|
map output fits in spill buffer. (todd)
|
|
|
|
MAPREDUCE-3159. Ensure DefaultContainerExecutor doesn't delete application
|
|
directories during app-init. (todd via acmurthy)
|
|
|
|
MAPREDUCE-3248. Fixed log4j properties. (vinodkv via acmurthy)
|
|
|
|
MAPREDUCE-2746. Yarn servers can't communicate with each other with
|
|
hadoop.security.authorization set to true (acmurthy via mahadev)
|
|
|
|
MAPREDUCE-2821. Added missing fields (resourcePerMap & resourcePerReduce)
|
|
to JobSummary logs. (mahadev via acmurthy)
|
|
|
|
MAPREDUCE-3253. Fixed ContextFactory to clone JobContext correctly.
|
|
(acmurthy)
|
|
|
|
MAPREDUCE-3263. Fixed the MAPREDUCE-3028 commit which broke MR1. (Hitesh
|
|
Shah via acmurthy)
|
|
|
|
MAPREDUCE-3269. Fixed log4j properties to correctly set logging options
|
|
for JobHistoryServer vis-a-vis JobSummary logs. (mahadev via acmurthy)
|
|
|
|
MAPREDUCE-2977. Fix ResourceManager to renew HDFS delegation tokens for
|
|
applications. (acmurthy)
|
|
|
|
MAPREDUCE-3250. When AM restarts, client keeps reconnecting to the new AM
|
|
and prints a lots of logs. (vinodkv via mahadev)
|
|
|
|
MAPREDUCE-3254. Fixed streaming to set the job.jar by using the right
|
|
JobConf ctor. (acmurthy)
|
|
|
|
MAPREDUCE-3264. mapreduce.job.user.name needs to be set automatically.
|
|
(acmurthy via mahadev)
|
|
|
|
MAPREDUCE-3175. Add authorization to admin web-pages such as /stacks, /jmx
|
|
etc. (Jonathan Eagles via acmurthy)
|
|
|
|
MAPREDUCE-3257. Added authorization checks for the protocol between
|
|
ResourceManager and ApplicationMaster. (vinodkv via acmurthy)
|
|
|
|
MAPREDUCE-3259. Added java.library.path of NodeManager to
|
|
ContainerLocalizer in LinuxContainerExecutor. (Kihwal Lee via acmurthy)
|
|
|
|
MAPREDUCE-3279. Fixed TestJobHistoryParsing which assumed user name to be
|
|
mapred all the time. (Siddharth Seth via acmurthy)
|
|
|
|
MAPREDUCE-3240. Fixed NodeManager to be able to forcefully cleanup its
|
|
containers (process-trees) irrespective of whether the container succeeded,
|
|
or killed. (Hitesh Shah via vinodkv)
|
|
|
|
MAPREDUCE-3281. Fixed a bug in TestLinuxContainerExecutorWithMocks. (vinodkv)
|
|
|
|
MAPREDUCE-3228. Fixed MR AM to timeout RPCs to bad NodeManagers. (vinodkv
|
|
via acmurthy)
|
|
|
|
MAPREDUCE-3284. Moved JobQueueClient to hadoop-mapreduce-client-core.
|
|
(acmurthy)
|
|
|
|
MAPREDUCE-3282. bin/mapred job -list throws exception. (acmurthy via
|
|
mahadev)
|
|
|
|
MAPREDUCE-3186. User jobs are getting hanged if the Resource manager
|
|
process goes down and comes up while job is getting executed.
|
|
(Eric Payne via mahadev)
|
|
|
|
MAPREDUCE-3209. Jenkins reports 160 FindBugs warnings (mahadev)
|
|
|
|
MAPREDUCE-3258. Fixed AM & JobHistory web-ui to display counters properly.
|
|
(Siddharth Seth via acmurthy)
|
|
|
|
MAPREDUCE-3290. Fixed a NPE in ClientRMService. (acmurthy)
|
|
|
|
MAPREDUCE-3185. RM Web UI does not sort the columns in some cases.
|
|
(Jonathan Eagles via mahadev)
|
|
|
|
MAPREDUCE-3292. In secure mode job submission fails with Provider
|
|
org.apache.hadoop.mapreduce.security.token.JobTokenIndentifier$Renewer
|
|
not found. (mahadev)
|
|
|
|
MAPREDUCE-3296. Fixed the remaining nine FindBugs warnings. (vinodkv)
|
|
|
|
MAPREDUCE-2775. Fixed ResourceManager and NodeManager to force a
|
|
decommissioned node to shutdown. (Devaraj K via vinodkv)
|
|
|
|
MAPREDUCE-3304. Fixed intermittent test failure due to a race in
|
|
TestRMContainerAllocator#testBlackListedNodes. (Ravi Prakash via acmurthy)
|
|
|
|
MAPREDUCE-3306. Fixed a bug in NodeManager ApplicationImpl that was causing
|
|
NodeManager to crash. (vinodkv)
|
|
|
|
MAPREDUCE-3256. Added authorization checks for the protocol between
|
|
NodeManager and ApplicationMaster. (vinodkv via acmurthy)
|
|
|
|
MAPREDUCE-3274. Fixed a race condition in MRAppMaster that was causing a
|
|
task-scheduling deadlock. (Robert Joseph Evans via vinodkv)
|
|
|
|
MAPREDUCE-3313. Fixed initialization of ClusterMetrics which was failing
|
|
TestResourceTrackerService sometimes. (Hitesh Shah via vinodkv)
|
|
|
|
MAPREDUCE-2766. Fixed NM to set secure permissions for files and directories
|
|
in distributed-cache. (Hitesh Shah via vinodkv)
|
|
|
|
MAPREDUCE-2696. Fixed NodeManager to cleanup logs in a thread when logs'
|
|
aggregation is not enabled. (Siddharth Seth via vinodkv)
|
|
|
|
MAPREDUCE-3262. Fixed Container's state-machine in NodeManager to handle
|
|
a couple of events in failure states correctly. (Hitesh Shah and Siddharth
|
|
Seth via vinodkv)
|
|
|
|
MAPREDUCE-3035. Fixed MR JobHistory to ensure rack information is present.
|
|
(chakravarthy via acmurthy)
|
|
|
|
MAPREDUCE-3321. Disabled a few MR tests for 0.23. (Hitesh Shah via
|
|
acmurthy)
|
|
|
|
MAPREDUCE-3220. Fixed TestCombineOutputCollector. (Devaraj K via acmurthy)
|
|
|
|
MAPREDUCE-3103. Implement Job ACLs for MRAppMaster.
|
|
(mahadev)
|
|
|
|
MAPREDUCE-3241. [Rumen] Fix Rumen to ignore the AMStartedEvent. (amarrk)
|
|
|
|
MAPREDUCE-3166. [Rumen] Make Rumen use job history api instead of relying
|
|
on current history file name format. (Ravi Gummadi)
|
|
|
|
MAPREDUCE-3157. [Rumen] Fix TraceBuilder to handle 0.20 history file
|
|
names also. (Ravi Gummadi)
|
|
|
|
MAPREDUCE-3081. Fix vaidya startup script. (gkesavan via suhas).
|
|
|
|
MAPREDUCE-2764. Fix renewal of dfs delegation tokens. (Owen via jitendra)
|
|
|
|
MAPREDUCE-3192. Fix Javadoc warning in JobClient.java and Cluster.java.
|
|
(jitendra)
|
|
|
|
MAPREDUCE-3237. Move LocalJobRunner to hadoop-mapreduce-client-core.
|
|
(tomwhite via acmurthy)
|
|
|
|
MAPREDUCE-3316. Rebooted link is not working properly.
|
|
(Bhallamudi Venkata Siva Kamesh via mahadev)
|
|
|
|
MAPREDUCE-3317. Rumen TraceBuilder is emiting null as hostname.
|
|
(Ravi Gummadi via mahadev)
|
|
|
|
MAPREDUCE-3332. contrib/raid compile breaks due to changes in hdfs/protocol/datatransfer/
|
|
Sender#writeBlock related to checksum handling (Hitesh Shah via mahadev)
|
|
|
|
MAPREDUCE-3337. Added missing license headers. (acmurthy)
|
|
|
|
Release 0.22.1 - Unreleased
|
|
|
|
INCOMPATIBLE CHANGES
|
|
|
|
NEW FEATURES
|
|
|
|
IMPROVEMENTS
|
|
|
|
OPTIMIZATIONS
|
|
|
|
BUG FIXES
|
|
|
|
MAPREDUCE-3837. Job tracker is not able to recover jobs after crash.
|
|
(Mayank Bansal via shv)
|
|
|
|
Release 0.22.0 - 2011-11-29
|
|
|
|
INCOMPATIBLE CHANGES
|
|
|
|
MAPREDUCE-1866. Removes deprecated class
|
|
org.apache.hadoop.streaming.UTF8ByteArrayUtils. (amareshwari)
|
|
|
|
MAPREDUCE-1664. Changes the behaviour of the combination of job-acls
|
|
when they function together with queue-acls. (Ravi Gummadi via vinodkv)
|
|
|
|
MAPREDUCE-2994. Fixed a bug in ApplicationID parsing that affects RM
|
|
UI. (Devaraj K via vinodkv)
|
|
|
|
MAPREDUCE-1788. o.a.h.mapreduce.Job shouldn't make a copy of the JobConf.
|
|
(Arun Murthy via mahadev)
|
|
|
|
NEW FEATURES
|
|
|
|
MAPREDUCE-1804. Stress-test tool for HDFS introduced in HDFS-708.
|
|
(Joshua Harlow via shv)
|
|
|
|
MAPREDUCE-220. Collect cpu and memory statistics per task. (Scott Chen via
|
|
acmurthy)
|
|
|
|
MAPREDUCE-1970. Reed-Solomon code implementation for HDFS RAID.
|
|
(Scott Chen via dhruba)
|
|
|
|
MAPREDUCE-2169. Integrated Reed-Solomon code with RaidNode. (Ramkumar
|
|
Vadali via schen)
|
|
|
|
MAPREDUCE-2936. Contrib Raid compilation broken after HDFS-1620. (vinodkv)
|
|
|
|
IMPROVEMENTS
|
|
|
|
MAPREDUCE-2141. Add an "extra data" field to Task for use by Mesos. (matei)
|
|
|
|
MAPREDUCE-2140. Regenerate fair scheduler design doc PDF. (matei)
|
|
|
|
MAPREDUCE-1546. Redirect all job pages to corresponding history page
|
|
if job is not in JT memory. (Scott Chen via sharad)
|
|
|
|
MAPREDUCE-1092. Enable assertions for unit tests. (Eli Collins via
|
|
cdouglas)
|
|
|
|
MAPREDUCE-1680. Add a metric recording JobTracker heartbeats processed.
|
|
(Dick King via cdouglas)
|
|
|
|
MAPREDUCE-1761. FairScheduler allows separate configuration of node
|
|
and rack locality wait time (Scott Chen via dhruba)
|
|
|
|
MAPREDUCE-1539. authorization checks for inter-server protocol
|
|
(based on HADOOP-6600) (Boris Shkolnik via shv)
|
|
|
|
MAPREDUCE-1798. Names the configuration keys for the Kerberos
|
|
principals better. (Boris Shkolnik via ddas)
|
|
|
|
MAPREDUCE-1773. streaming doesn't support jobclient.output.filter.
|
|
(Amareshwari Sriramadasu via vinodkv)
|
|
|
|
MAPREDUCE-1785. Add streaming config option for not emitting the key.
|
|
(Eli Collins via sharad)
|
|
|
|
MAPREDUCE-572. If #link is missing from uri format of -cacheArchive
|
|
then streaming does not throw error. (Amareshwari Sriramadasu via
|
|
vinodkv)
|
|
|
|
MAPREDUCE-1545. Add timestamps for first task type launched in job summary.
|
|
(Luke Lu via cdouglas)
|
|
|
|
MAPREDUCE-1543. Add an audit log for authentication events. (Amar Kamat and
|
|
Luke Lu via cdouglas)
|
|
|
|
MAPREDUCE-1762. Add ability to set values of task counters. (Scott Chen via
|
|
cdouglas)
|
|
|
|
MAPREDUCE-1533. Reduce overhead of logging and string manipulation during
|
|
heartbeat processing. (Amar Kamat and Dick King via cdouglas)
|
|
|
|
MAPREDUCE-1516. JobTracker issues delegation tokens only if the user's
|
|
authentication is Kerberos. (Jitendra Pandey via ddas)
|
|
|
|
MAPREDUCE-647. Update distcp forrest documentation to reflect the changes
|
|
of HADOOP-5472, MAPREDUCE-642 and HADOOP-5620. (Rodrigo Schmidt via
|
|
szetszwo)
|
|
|
|
MAPREDUCE-1851. Documents configuration parameters in streaming.
|
|
(amareshwari)
|
|
|
|
MAPREDUCE-1868. Add a read and connection timeout to JobClient while
|
|
pulling tasklogs. (Krishna Ramachandran via acmurthy)
|
|
|
|
MAPREDUCE-1778. Ensure failure to setup CompletedJobStatusStore is not
|
|
silently ignored by the JobTracker. (Krishna Ramachandran via acmurthy)
|
|
|
|
MAPREDUCE-1850. Includes job submit host information (name and ip) in
|
|
jobconf and jobdetails display (Krishna Ramachandran via amareshwari)
|
|
|
|
MAPREDUCE-1893. Slive with multiple reducers. (shv)
|
|
|
|
MAPREDUCE-1248. Fixes redudant memory copying in StreamKeyValUtil.
|
|
(Ruibang He via amareshwari)
|
|
|
|
MAPREDUCE-1840. Enhancements to Gridmix benchmark simulating user
|
|
diversity, queue replay, and task duration for JobTracker load testing.
|
|
Also includes compatibility with security enhancements, and scalability
|
|
improvements. (Amar Kamat, Rahul Singh, Hong Tang, and cdouglas)
|
|
|
|
MAPREDUCE-1848. Put number of speculative, data local, rack local
|
|
tasks in JobTracker metrics. (Scott Chen via dhruba)
|
|
|
|
MAPREDUCE-1935. Makes the Distcp to work in a secure environment.
|
|
(Boris Shkolnik via ddas)
|
|
|
|
MAPREDUCE-1945. The MapReduce component for HADOOP-6632.
|
|
(Kan Zhang & Jitendra Pandey via ddas)
|
|
|
|
MAPREDUCE-1936. Modify Gridmix3 to support more tunable parameters for
|
|
stress submission and sleep jobs. (Hong Tang via cdouglas)
|
|
|
|
MAPREDUCE-1733. Makes pipes applications secure. (Jitendra Pandey via ddas)
|
|
|
|
MAPREDUCE-1566. Adds a configuration attribute using which job clients can
|
|
specify a credentials file. The tokens from there will be passed to the job.
|
|
(Jitendra Pandey and Owen O'Malley via ddas)
|
|
|
|
MAPREDUCE-1624. Documents the job credentials and associated details to do
|
|
with delegation tokens (on the client side).
|
|
(Jitendra Pandey and Devaraj Das via ddas)
|
|
|
|
MAPREDUCE-1834. TestSimulatorDeterministicReplay timesout on trunk.
|
|
(Hong Tang via mahadev)
|
|
|
|
MAPREDUCE-1993. Fixes test failure
|
|
TestTrackerDistributedCacheManagerWithLinuxTaskController. (Devaraj Das
|
|
via amareshwari)
|
|
|
|
MAPREDUCE-1523. Making Mumak work with Capacity-Scheduler (Anirban Das
|
|
via mahadev)
|
|
|
|
MAPREDUCE-1920. Enables completed jobstatus store by default. (Tom White
|
|
via amareshwari)
|
|
|
|
MAPREDUCE-1881. Improve TaskTrackerInstrumentation to enable collection of
|
|
advanced metrics. (Matei Zaharia via acmurthy)
|
|
|
|
MAPREDUCE-1548. Hadoop archives preserve times and other properties from
|
|
original files. (Rodrigo Schmidt via dhruba)
|
|
|
|
MAPREDUCE-1517. Supports streaming job to run in the background. (Bochun Bai
|
|
via amareshwari)
|
|
|
|
MAPREDUCE-1819. RaidNode is now smarter in submitting Raid jobs. (Ramkumar
|
|
Vadali via schen)
|
|
|
|
MAPREDUCE-2132. A command line option in RaidShell to fix blocks using raid
|
|
|
|
MAPREDUCE-2147. Remove redundant lines in JobInProgress ctor.
|
|
(Harsh J Chouraria via cutting)
|
|
|
|
HADOOP-7007. Update the hudson-test-patch ant target to work with the
|
|
latest test-patch.sh script (gkesavan)
|
|
|
|
MAPREDUCE-1818. RaidNode can specify scheduling parameters. (Ramkumar
|
|
Vadali via schen)
|
|
|
|
MAPREDUCE-2051. Contribute a fair scheduler preemption system test.
|
|
(Todd Lipcon via tomwhite)
|
|
|
|
MAPREDUCE-1892. RaidNode can allow layered policies more efficiently.
|
|
(Ramkumar Vadali via schen)
|
|
|
|
MAPREDUCE-1592. Generate Eclipse's .classpath file from Ivy config.
|
|
(tomwhite via nigel)
|
|
|
|
MAPREDUCE-2073. TestTrackerDistributedCacheManager should be up-front
|
|
about requirements on build environment. (Todd Lipcon via tomwhite)
|
|
|
|
MAPREDUCE-2093. Herriot JT and TT clients should vend statistics. (cos)
|
|
|
|
MAPREDUCE-2167. Faster directory traversal for raid node. (Ramkumar Vadali
|
|
via schen)
|
|
|
|
MAPREDUCE-1931. Gridmix forrest documentation . (Ranjit Mathew via vinodkv).
|
|
|
|
MAPREDUCE-2184. Port DistRaid.java to new mapreduce API. (Ramkumar Vadali
|
|
via schen)
|
|
|
|
MAPREDUCE-1878. Add MRUnit documentation. (Aaron Kimball via tomwhite)
|
|
|
|
MAPREDUCE-2180. Add coverage of fair scheduler servlet to system test (todd)
|
|
|
|
MAPREDUCE-2250. Fix logging in raid code. (Ramkumar Vadali via schen)
|
|
|
|
MAPREDUCE-2260. Remove auto-generated native build files. (rvs via eli)
|
|
|
|
MAPREDUCE-2314. configure files that are generated as part of the released
|
|
tarball need to have executable bit set (rvs via cos)
|
|
|
|
MAPREDUCE-1159. Limit Job name in web UI to be 80 char long. (Harsh J
|
|
Chouraria via szetszwo)
|
|
|
|
MAPREDUCE-2337. Remove dependence of public MapReduce API on classes in
|
|
server package. (tomwhite)
|
|
|
|
MAPREDUCE-2383. Improve documentation of DistributedCache methods (Harsh J
|
|
Chouraria via todd)
|
|
|
|
MAPREDUCE-2222. Ivy resolve force mode should be turned off by default.
|
|
(Luke Lu via tomwhite)
|
|
|
|
MAPREDUCE-2103. task-controller shouldn't require o-r permissions.
|
|
(todd via eli)
|
|
|
|
MAPREDUCE-2505. Explain how to use ACLs in the fair scheduler.
|
|
(matei via eli)
|
|
|
|
MAPREDUCE-3138. Add a utility to help applications bridge changes in
|
|
Context Objects APIs due to MAPREDUCE-954. (omalley via acmurthy)
|
|
|
|
OPTIMIZATIONS
|
|
|
|
MAPREDUCE-1354. Enhancements to JobTracker for better performance and
|
|
scalability. (Arun C. Murthy & Richard King via acmurthy)
|
|
|
|
MAPREDUCE-1829. JobInProgress.findSpeculativeTask should use min() to
|
|
find the candidate instead of sort(). (Scott Chen via vinodkv)
|
|
|
|
BUG FIXES
|
|
|
|
MAPREDUCE-1845. FairScheduler.tasksToPreempt() can return negative number.
|
|
(Scott Chen via matei)
|
|
|
|
MAPREDUCE-1707. TaskRunner can get NPE in getting ugi from TaskTracker.
|
|
(Vinod Kumar Vavilapalli)
|
|
|
|
MAPREDUCE-1532. Ensures that delegation tokens is obtained as the
|
|
actual user when the proxy-user is used for submitting jobs. Also
|
|
refactors the DelegationTokenToRenew class. (ddas)
|
|
|
|
MAPREDUCE-1558. Fixes MRAdmin to look up the principal of the
|
|
JobTracker and use that in the RefreshUserToGroupsMapping protocol and
|
|
RefreshAuthorizationPolicyProtocol. (Boris Shkolnik via ddas)
|
|
|
|
MAPREDUCE-1662. Remove unused methods from TaskRunner. (Amareshwari
|
|
Sriramadasu via cdouglas)
|
|
|
|
MAPREDUCE-1617. Use IPv4 stack for unit tests. (Amar Kamat and Luke Lu via
|
|
cdouglas)
|
|
|
|
MAPREDUCE-1599. Fixes MRBench so that it reuses tokens across jobs
|
|
correctly. (Jitendra Nath Pandey via ddas)
|
|
|
|
MAPREDUCE-1836. Refresh for proxy superuser config (mr part for HDFS-1096).
|
|
(Boris Shkolnik via shv)
|
|
|
|
MAPREDUCE-1505. Create RPC client on job submission, not in cstr of Job
|
|
instance. (Dick King via cdouglas)
|
|
|
|
MAPREDUCE-1813. NPE in PipeMapred.MRErrorThread. (Ravi Gummadi via vinodkv)
|
|
|
|
MAPREDUCE-1225. Fixes DistributedCache to check if the file is fresh or not,
|
|
for the first localization also. (Zhong Wang via amareshwari)
|
|
|
|
MAPREDUCE-1559. Fixes the token renewer to use the JobTracker's
|
|
credentials for talking to the NameNode. (ddas)
|
|
|
|
MAPREDUCE-1492. Delete obsolete har files used on the parity files
|
|
of hdfs raid. (Rodrigo Schmidt via dhruba)
|
|
|
|
MAPREDUCE-1857. Removes unused configuration parameters in streaming.
|
|
(amareshwari)
|
|
|
|
MAPREDUCE-1887. MRAsyncDiskService now properly absolutizes volume root
|
|
paths. (Aaron Kimball via zshao)
|
|
|
|
MAPREDUCE-1863. Fix NPE in Rumen when processing null CDF for failed task
|
|
attempts. (Amar Kamat via cdouglas)
|
|
|
|
MAPREDUCE-1864. Removes uninitialized/unused variables in
|
|
org.apache.hadoop.streaming.PipeMapRed. (amareshwari)
|
|
|
|
MAPREDUCE-1888. Fixes Streaming to override output key and value types,
|
|
only if mapper/reducer is a command. (Ravi Gummadi via amareshwari)
|
|
|
|
MAPREDUCE-577. Fixes duplicate records in StreamXmlRecordReader.
|
|
(Ravi Gummadi via amareshwari)
|
|
|
|
MAPREDUCE-1894. Fixed a bug in DistributedRaidFileSystem.readFully()
|
|
that was causing it to loop infinitely. (Ramkumar Vadali via dhruba)
|
|
|
|
MAPREDUCE-1838. Reduce the time needed for raiding a bunch of files
|
|
by randomly assigning files to map tasks. (Ramkumar Vadali via dhruba)
|
|
|
|
MAPREDUCE-1820. Fix InputSampler to clone sampled keys. (Alex Kozlov via
|
|
cdouglas)
|
|
|
|
MAPREDUCE-1528. Incorporates the changes to the credentials API done in
|
|
HADOOP-6845. Also, introduces Credentials in JobConf, and in JobContext.
|
|
(Jitendra Pandey and Arun Murthy via ddas)
|
|
|
|
MAPREDUCE-1865. Rumen should also support jobhistory files generated using
|
|
trunk. (Amar Kamat via amareshwari)
|
|
|
|
MAPREDUCE-1621. Fixes NPE in TextOutputReader.getLastOutput if it has never
|
|
read any output. (amareshwari)
|
|
|
|
MAPREDUCE-1911. Fixes errors in -info message in streaming. (amareshwari)
|
|
|
|
MAPREDUCE-1772. Corrects errors in streaming documentation in forrest.
|
|
(amareshwari)
|
|
|
|
MAPREDUCE-1925. Fix failing TestRumenJobTraces. (Ravi Gummadi via cdouglas)
|
|
|
|
MAPREDUCE-1718. Fixes a bug in the construction of jobconf key for the
|
|
mapping that the tasks use at runtime for looking up delegation tokens.
|
|
(Boris Shkolnik via ddas)
|
|
|
|
MAPREDUCE-1701. Fixes a problem to do with exception handling in
|
|
delegation-token renewals. (Boris Shkolnik via ddas)
|
|
|
|
MAPREDUCE-1686. Fixes StreamUtil.goodClassOrNull to find classes without
|
|
package names. (Paul Burkhardt via amareshwari)
|
|
|
|
MAPREDUCE-1288. Fixes TrackerDistributedCacheManager to take into account
|
|
the owner of the localized file in the mapping from cache URIs to
|
|
CacheStatus objects. (ddas)
|
|
|
|
MAPREDUCE-1982. Fixes Rumen's TraceBuilder to extract job name from either
|
|
of configuration properties "mapreduce.job.name" and "mapred.job.name".
|
|
(Ravi Gummadi via amareshwari)
|
|
|
|
MAPREDUCE-1958. The MapReduce part corresponding to the HADOOP-6873.
|
|
(Boris Shkolnik & Owen O'Malley via ddas)
|
|
|
|
MAPREDUCE-1900. TaskTracker and JobTracker closes FileSystems, opened on
|
|
behalf of users that it no longer requires. (Kan Zhang and ddas via ddas)
|
|
|
|
MAPREDUCE-1992. Fixes a problem to do with bringing up the JobTracker in
|
|
unsecure mode. (Kan Zhang via ddas)
|
|
|
|
MAPREDUCE-1999. Fixes ClientProtocol to use the correct
|
|
DelegationTokenSelector. (Jitendra Pandey via ddas)
|
|
|
|
MAPREDUCE-1780. AccessControlList.toString() is used for serialization of
|
|
ACL in JobStatus.java. (Ravi Gummadi via vinodkv)
|
|
|
|
MAPREDUCE-1961. Fix ConcurrentModificationException in Gridmix during
|
|
shutdown. (Hong Tang via cdouglas)
|
|
|
|
MAPREDUCE-2000. Fix parsing of JobHistory lines in Rumen when quotes are
|
|
escaped. (Hong Tang via cdouglas)
|
|
|
|
MAPREDUCE-2022. Fixes compilation errors in TestSubmitJob. (amareshwari)
|
|
|
|
MAPREDUCE-1670. RAID policies should not scan their own destination path.
|
|
(Ramkumar Vadali via dhruba)
|
|
|
|
MAPREDUCE-1668. RaidNode Hars a directory only if all its parity files
|
|
have been created. (Ramkumar Vadali via dhruba)
|
|
|
|
MAPREDUCE-2021. Fixes duplicate hostnames in CombineFileInputFormat's
|
|
split locations. (amareshwari)
|
|
|
|
MAPREDUCE-1375. Fixes flaky test TestFileArgs. (Todd Lipcon via
|
|
amareshwari)
|
|
|
|
MAPREDUCE-2023. TestDFSIO should not stop reading if curSize != bufferSize.
|
|
(Hong Tang via szetszwo)
|
|
|
|
MAPREDUCE-2031. Fixes test failures TestTaskLauncher and
|
|
TestTaskTrackerLocalization. (Ravi Gummadi via amareshwari)
|
|
|
|
MAPREDUCE-2046. Fixes CombineFileInputFormat to allow splits with size
|
|
less than DFS block size. (dhruba borthakur via amareshwari)
|
|
|
|
MAPREDUCE-1975. Fixes unnecessary InterruptedException log in gridmix.
|
|
(Ravi Gummadi via amareshwari)
|
|
|
|
MAPREDUCE-1597. Fixes CombineFileInputFormat to work with non-splittable
|
|
files. (amareshwari)
|
|
|
|
MAPREDUCE-2032. Fixes TestJobCleanup to cleanup test directory in
|
|
tearDown. (Dick King via amareshwari)
|
|
|
|
MAPREDUCE-1979. Fixes "Output directory already exists" error in gridmix
|
|
when gridmix.output.directory is not defined. (Ravi Gummadi via
|
|
amareshwari)
|
|
|
|
MAPREDUCE-1918. Adds documentation to Rumen. (Amar Kamat via amareshwari)
|
|
|
|
MAPREDUCE-2078. Fixes TraceBuilder to generate traces when a globbed job
|
|
history path is given. (Amar Kamat via amareshwari)
|
|
|
|
MAPREDUCE-1989. Fixes error message in gridmix when user resolver is set
|
|
and no user list is given. (Ravi Gummadi via amareshwari)
|
|
|
|
MAPREDUCE-2067. Distinct minicluster services (e.g. NN and JT) overwrite
|
|
each other's service policies. (Aaron T. Myers via tomwhite)
|
|
|
|
MAPREDUCE-2029. DistributedRaidFileSystem removes itself from FileSystem
|
|
cache when it is closed. (Ramkumar Vadali via dhruba)
|
|
|
|
MAPREDUCE-1816. HAR files used for RAID parity-bite have configurable
|
|
partfile size. (Ramkumar Vadali via dhruba)
|
|
|
|
MAPREDUCE-2082. Fixes Pipes to create the jobtoken file in the right
|
|
place. (Jitendra Pandey via ddas)
|
|
|
|
MAPREDUCE-2095. Fixes Gridmix to run from compressed traces. (Ranjit
|
|
Mathew via amareshwari)
|
|
|
|
MAPREDUCE-1908. DistributedRaidFileSystem now handles ChecksumException
|
|
correctly. (Ramkumar Vadali via schen)
|
|
|
|
MAPREDUCE-2126. JobQueueJobInProgressListener's javadoc is inconsistent
|
|
with source code. (Jingguo Yao via tomwhite)
|
|
|
|
MAPREDUCE-2143. HarFileSystem is able to handle spaces in pathnames.
|
|
(Ramkumar Vadali via dhruba)
|
|
|
|
MAPREDUCE-1867. Remove unused methods in
|
|
org.apache.hadoop.streaming.StreamUtil. (amareshwari via tomwhite)
|
|
|
|
MAPREDUCE-2146. Raid does not affect access time of a source file.
|
|
(Ramkumar Vadali via dhruba)
|
|
|
|
MAPREDUCE-2150. RaidNode periodically fixes corrupt blocks. (Ramkumar Vadali via
|
|
schen)
|
|
|
|
MAPREDUCE-2099. RaidNode recreates outdated parity HARs. (Ramkumar Vadali
|
|
via schen)
|
|
|
|
MAPREDUCE-2173. Fix race condition in TestBlockFixer that was
|
|
causing intermittent failure (Patrick Kling via dhruba)
|
|
|
|
MAPREDUCE-2142. Refactor RaidNode so that the map-reduce component is
|
|
clearly separated out. (Patrick Kling via dhruba)
|
|
|
|
MAPREDUCE-2179. Fix RaidBlockSender compilation failure. (Ramkumar Vadali
|
|
via schen)
|
|
|
|
MAPREDUCE-2034. TestSubmitJob triggers NPE instead of permissions error.
|
|
(Todd Lipcon via tomwhite)
|
|
|
|
MAPREDUCE-2195. New property for local conf directory in
|
|
system-test-mapreduce.xml file. (cos)
|
|
|
|
MAPREDUCE-1783. FairScheduler initializes tasks only when the job can be
|
|
run. (Ramkumar Vadali via schen)
|
|
|
|
MAPREDUCE-2224. Fix synchronization bugs in JvmManager. (todd)
|
|
|
|
MAPREDUCE-714. JobConf.findContainingJar unescapes unnecessarily on linux (todd)
|
|
|
|
MAPREDUCE-2096. Secure local filesystem IO from symlink vulnerabilities (todd)
|
|
|
|
MAPREDUCE-2234. If Localizer can't create task log directory, it should fail
|
|
on the spot. (todd)
|
|
|
|
MAPREDUCE-2219. JobTracker should not try to remove mapred.system.dir
|
|
during startup. (todd)
|
|
|
|
MAPREDUCE-2207. Task-cleanup task should not be scheduled on the node that
|
|
the task just failed. (Liyin Liang via schen)
|
|
|
|
MAPREDUCE-2084. Remove deprecate annotation for package file. The package
|
|
classes themselves are already deprecated. This removes an Eclipse error.
|
|
(tomwhite via nigel)
|
|
|
|
MAPREDUCE-2248. DistributedRaidFileSystem should unraid only the corrupt
|
|
block (Ramkumar Vadali via schen)
|
|
|
|
MAPREDUCE-1085. For tasks, "ulimit -v -1" is being run when user doesn't
|
|
specify a ulimit (todd)
|
|
|
|
MAPREDUCE-2282. Fix TestMRServerPorts for the changes in
|
|
TestHDFSServerPorts. (shv via szetszwo)
|
|
|
|
MAPREDUCE-2238. Fix permissions handling to avoid leaving undeletable
|
|
directories in local dirs. (todd)
|
|
|
|
MAPREDUCE-2277. TestCapacitySchedulerWithJobTracker needs to wait for jobs
|
|
to complete before testing status. (todd)
|
|
|
|
MAPREDUCE-2253. Servlets should specify content type (todd)
|
|
|
|
MAPREDUCE-2283. Add timeout for Raid Tests (Ramkumar Vadali via schen)
|
|
|
|
MAPREDUCE-1754. Replace mapred.persmissions.supergroup with an
|
|
acl : mapreduce.cluster.administrators (Amareshwari Sriramadasu via shv)
|
|
|
|
MAPREDUCE-2256. FairScheduler fairshare preemption from multiple pools may
|
|
preempt all tasks from one pool causing that pool to go below fairshare.
|
|
(Priyo Mustafi via shv)
|
|
|
|
MAPREDUCE-2281. MR part of HADOOP-6642. (Chris Douglas, Po Cheung via shv)
|
|
|
|
MAPREDUCE-2200. TestUmbilicalProtocolWithJobToken is failing without Krb
|
|
evironment: needs to be conditional. (cos)
|
|
|
|
MAPREDUCE-2077. Resolve name clash in the deprecated
|
|
o.a.h.util.MemoryCalculatorPlugin (Luke Lu via shv)
|
|
|
|
MAPREDUCE-2188. The new API MultithreadedMapper doesn't initialize
|
|
RecordReader. (Owen O'Malley via shv)
|
|
|
|
MAPREDUCE-1915. Fix IndexOutOfBoundsException in IndexCache.
|
|
(Priyo Mustafi via shv)
|
|
|
|
MAPREDUCE-1974. Fix multiple preemtions of the same task in FairScheduler.
|
|
(Scott Chen via shv)
|
|
|
|
MAPREDUCE-2304. Fix TestMRCLI to allow hostname with a hyphen (-).
|
|
(Priyo Mustafi via shv)
|
|
|
|
MAPREDUCE-1825. jobqueue_details.jsp and FairSchedulerServelet should not
|
|
call finishedMaps and finishedReduces when job is not initialized.
|
|
(Scott Chen via shv)
|
|
|
|
MAPREDUCE-2285. MiniMRCluster does not start after ant test-patch
|
|
(todd)
|
|
|
|
MAPREDUCE-2315. javadoc is failing in nightly build (todd)
|
|
|
|
MAPREDUCE-2054. Hierarchical queue implementation broke dynamic queue
|
|
addition in Dynamic Scheduler. (Thomas Sandholm via tomwhite)
|
|
|
|
MAPREDUCE-2272. Job ACL file should not be executable
|
|
(Harsh J Chouraria via todd)
|
|
|
|
MAPREDUCE-2241. ClusterWithLinuxTaskController should accept relative path
|
|
on the command line. (todd)
|
|
|
|
MAPREDUCE-2251. Remove unused mapreduce.job.userhistorylocation config.
|
|
(Harsh J Chouraria via todd)
|
|
|
|
MAPREDUCE-2284. TestLocalRunner.testMultiMaps times out (todd)
|
|
|
|
MAPREDUCE-2336. Tool-related packages should be in the Tool javadoc group.
|
|
(tomwhite)
|
|
|
|
MAPREDUCE-2394. JUnit output format doesn't propagate into raid contrib
|
|
build. (todd)
|
|
|
|
MAPREDUCE-2392. TaskTracker shutdown in the tests sometimes take 60s.
|
|
(tomwhite)
|
|
|
|
MAPREDUCE-2437. SLive uses only part* files to generating the final report.
|
|
(shv)
|
|
|
|
MAPREDUCE-2428. start-mapred.sh script fails if HADOOP_HOME is not set.
|
|
(tomwhite via eli)
|
|
|
|
MAPREDUCE-2445. Fix TestMiniMRWithDFSWithDistinctUsers to be a valid test.
|
|
(todd)
|
|
|
|
MAPREDUCE-2457. Job submission should inject group.name on the JobTracker
|
|
(Alejandro Abdelnur via todd)
|
|
|
|
MAPREDUCE-2472. Extra whitespace in mapred.child.java.opts breaks JVM
|
|
initialization. (Aaron T. Myers via todd)
|
|
|
|
MAPREDUCE-2222. Ivy resolve force mode should be turned off by default.
|
|
(Luke Lu via tomwhite)
|
|
|
|
MAPREDUCE-2486. Incorrect snapshot dependency published in .pom files
|
|
(todd)
|
|
|
|
MAPREDUCE-2327. MapTask doesn't need to put username information in
|
|
SpillRecord. (todd via tomwhite)
|
|
|
|
MAPREDUCE-2515. MapReduce code references some deprecated options
|
|
(Ari Rabkin via todd)
|
|
|
|
MAPREDUCE-2487. ChainReducer uses MAPPER_BY_VALUE instead of
|
|
REDUCER_BY_VALUE. (Devaraj K via todd)
|
|
|
|
MAPREDUCE-2185. Fix infinite loop at creating splits using
|
|
CombineFileInputFormat. (Ramkumar Vadali via schen)
|
|
|
|
MAPREDUCE-2571. CombineFileInputFormat.getSplits throws a
|
|
java.lang.ArrayStoreException. (Bochun Bai via todd)
|
|
|
|
MAPREDUCE-2767. Remove Linux task-controller. (Milind Bhandarkar via shv)
|
|
|
|
MAPREDUCE-2991. queueinfo.jsp fails to show queue status for Capacity
|
|
scheduler if queue names contain special symbols. (Priyo Mustafi via shv)
|
|
|
|
MAPREDUCE-2779. JobSplitWriter.java can't handle large job.split file.
|
|
(Ming Ma via shv)
|
|
|
|
MAPREDUCE-2531. Fixed jobcontrol to downgrade JobID. (Robert Evans via
|
|
acmurthy)
|
|
|
|
MAPREDUCE-3139. SlivePartitioner generates negative partitions. (jghoman)
|
|
|
|
Release 0.21.1 - Unreleased
|
|
|
|
NEW FEATURES
|
|
|
|
MAPREDUCE-2040. Forrest Documentation for Dynamic Priority Scheduler.
|
|
(Thomas Sandholm via tomwhite)
|
|
|
|
BUG FIXES
|
|
|
|
MAPREDUCE-1897. trunk build broken on compile-mapred-test (cos)
|
|
|
|
MAPREDUCE-1280. Update Eclipse plugin to the new eclipse.jdt API.
|
|
(Alex Kozlov via szetszwo)
|
|
|
|
MAPREDUCE-1984. herriot TestCluster fails because exclusion is not there
|
|
(Balaji Rajagopalan via cos)
|
|
|
|
MAPREDUCE-2090. Clover build doesn't generate per-test coverage. (cos)
|
|
|
|
MAPREDUCE-2134. ant binary-system is broken in mapreduce project. (cos)
|
|
|
|
MAPREDUCE-1905. Fixes Context.setStatus() and progress() apis.
|
|
(amareshwari)
|
|
|
|
MAPREDUCE-1809. Ant build changes for Streaming system tests in contrib
|
|
projects. (Vinay Kumar Thota via amareshwari)
|
|
|
|
MAPREDUCE-2223. TestMRCLI might fail on Ubuntu with default /etc/hosts
|
|
(cos)
|
|
|
|
MAPREDUCE-2228. Remove java5 dependencies from build. (cos)
|
|
|
|
MAPREDUCE-1929. Allow artifacts to be published to the staging Apache Nexus
|
|
Maven Repository. (tomwhite)
|
|
|
|
MAPREDUCE-2317. Fix a NPE in HadoopArchives. (Devaraj K via szetszwo)
|
|
|
|
MAPREDUCE-2127. mapreduce trunk builds are filing on hudson.
|
|
(Bruno Mahé via eli)
|
|
|
|
Release 0.21.0 - 2010-08-13
|
|
|
|
INCOMPATIBLE CHANGES
|
|
|
|
MAPREDUCE-516. Fix the starvation problem in the Capacity Scheduler
|
|
when running High RAM Jobs. (Arun Murthy via yhemanth)
|
|
|
|
MAPREDUCE-358. Change org.apache.hadoop.examples. AggregateWordCount
|
|
and org.apache.hadoop.examples.AggregateWordHistogram to use new
|
|
mapreduce api. (Amareshwari Sriramadasu via sharad)
|
|
|
|
MAPREDUCE-245. Change Job and jobcontrol classes to use the List interface
|
|
rather than ArrayList in APIs. (Tom White via cdouglas)
|
|
|
|
MAPREDUCE-766. Enhanced list-blacklisted-trackers to display reasons
|
|
for blacklisting a node. (Sreekanth Ramakrishnan via yhemanth)
|
|
|
|
MAPREDUCE-817. Add a cache for retired jobs with minimal job info and
|
|
provide a way to access history file url. (sharad)
|
|
|
|
MAPREDUCE-711. Moved Distributed Cache from Common to Map/Reduce
|
|
project. (Vinod Kumar Vavilapalli via yhemanth)
|
|
|
|
MAPREDUCE-895. Per the contract elucidated in HADOOP-6201, throw
|
|
FileNotFoundException from FileSystem::listStatus rather than returning
|
|
null. (Jakob Homan via cdouglas)
|
|
|
|
MAPREDUCE-479. Provide full task id to map output servlet rather than the
|
|
reduce id, only. (Jiaqi Tan via cdouglas)
|
|
|
|
MAPREDUCE-873. Simplify job recovery. Incomplete jobs are resubmitted on
|
|
jobtracker restart. Removes a public constructor in JobInProgress. (sharad)
|
|
|
|
HADOOP-6230. Moved process tree and memory calculator related classes from
|
|
Common to Map/Reduce. (Vinod Kumar Vavilapalli via yhemanth)
|
|
|
|
MAPREDUCE-157. Refactor job history APIs and change the history format to
|
|
JSON. (Jothi Padmanabhan via sharad)
|
|
|
|
MAPREDUCE-849. Rename configuration properties. (Amareshwari Sriramadasu
|
|
via sharad)
|
|
|
|
MAPREDUCE-1287. Only call the partitioner with more than one reducer.
|
|
(cdouglas)
|
|
|
|
MAPREDUCE-1385. Use the new UserGroupInformation from HADOOP-6299.
|
|
(ddas via omalley)
|
|
|
|
MAPREDUCE-1493. Authorization for job-history pages. (vinodkv)
|
|
|
|
MAPREDUCE-1607. Task controller may not set permissions for a
|
|
task cleanup attempt's log directory (Amareshwari Sriramadasu via vinodkv)
|
|
|
|
MAPREDUCE-1683. Remove JNI calls from ClusterStatus cstr. (Arun Murthy and
|
|
Luke Lu via cdouglas)
|
|
|
|
MAPREDUCE-1855. Makes the refresh methods (for groups and proxy users)
|
|
independent of the client side configuration. (Boris Shkolnik via ddas)
|
|
|
|
NEW FEATURES
|
|
|
|
MAPREDUCE-1774. Large-scale Automated Framework (Sharad Agarwal, Sreekanth
|
|
Ramakrishnan, Konstantin Boudnik, at all via cos)
|
|
|
|
MAPREDUCE-706. Support for FIFO pools in the fair scheduler.
|
|
(Matei Zaharia)
|
|
|
|
MAPREDUCE-546. Provide sample fair scheduler config file in conf/ and use
|
|
it by default if no other config file is specified. (Matei Zaharia)
|
|
|
|
MAPREDUCE-551. Preemption support in the Fair Scheduler. (Matei Zaharia)
|
|
|
|
MAPREDUCE-567. Add a new example MR that always fails. (Philip Zeyliger
|
|
via tomwhite)
|
|
|
|
MAPREDUCE-211. Provides ability to run a health check script on the
|
|
tasktracker nodes and blacklist nodes if they are unhealthy.
|
|
(Sreekanth Ramakrishnan via yhemanth)
|
|
|
|
MAPREDUCE-637. Add an example, distbbp, which able to compute the n th bit
|
|
of Pi for some large n. (szetszwo)
|
|
|
|
MAPREDUCE-532. Provide a way to limit the number of used slots
|
|
per queue in the capacity scheduler.
|
|
(Rahul Kumar Singh via yhemanth)
|
|
|
|
MAPREDUCE-467. Provide ability to collect statistics about total tasks
|
|
and succeeded tasks in different time windows. (sharad)
|
|
|
|
MAPREDUCE-740. Log a job-summary at the end of a job, while allowing it
|
|
to be configured to use a custom appender if desired. (acmurthy)
|
|
|
|
MAPREDUCE-814. Provide a way to configure completed job history files
|
|
to be on HDFS. (sharad)
|
|
|
|
MAPREDUCE-800. MRUnit should support the new API. (Aaron Kimball via
|
|
tomwhite)
|
|
|
|
MAPREDUCE-798. MRUnit should be able to test a succession of MapReduce
|
|
passes. (Aaron Kimball via tomwhite)
|
|
|
|
MAPREDUCE-768. Provide an option to dump jobtracker configuration in JSON
|
|
format to standard output. (V.V.Chaitanya Krishna via yhemanth)
|
|
|
|
MAPREDUCE-824. Add support for a hierarchy of queues in the capacity
|
|
scheduler. (Rahul Kumar Singh via yhemanth)
|
|
|
|
MAPREDUCE-751. Add Rumen, a tool for extracting statistics from job tracker
|
|
logs and generating job traces for simulation and analysis. (Dick King via
|
|
cdouglas)
|
|
|
|
MAPREDUCE-830. Add support for splittable compression to TextInputFormats.
|
|
(Abdul Qadeer via cdouglas)
|
|
|
|
MAPREDUCE-861. Add support for hierarchical queues in the Map/Reduce
|
|
framework. (Rahul Kumar Singh via yhemanth)
|
|
|
|
MAPREDUCE-776. Add Gridmix, a benchmark processing Rumen traces to simulate
|
|
a measured mix of jobs on a cluster. (cdouglas)
|
|
|
|
MAPREDUCE-862. Enhance JobTracker UI to display hierarchical queues.
|
|
(V.V.Chaitanya Krishna via yhemanth)
|
|
|
|
MAPREDUCE-777. Brand new apis to track and query jobs as a
|
|
replacement for JobClient. (Amareshwari Sriramadasu via acmurthy)
|
|
|
|
MAPREDUCE-775. Add native and streaming support for Vertica as an input
|
|
or output format taking advantage of parallel read and write properties of
|
|
the DBMS. (Omer Trajman via ddas)
|
|
|
|
MAPREDUCE-679. XML-based metrics as JSP servlet for JobTracker.
|
|
(Aaron Kimball via tomwhite)
|
|
|
|
MAPREDUCE-980. Modify JobHistory to use Avro for serialization. (cutting)
|
|
|
|
MAPREDUCE-728. Add Mumak, a Hadoop map/reduce simulator. (Arun C Murthy,
|
|
Tamas Sarlos, Anirban Dasgupta, Guanying Wang, and Hong Tang via cdouglas)
|
|
|
|
MAPREDUCE-1383. Automates fetching of delegation tokens in File*Formats
|
|
Distributed Cache and Distcp. Also, provides a config
|
|
mapreduce.job.hdfs-servers that the jobs can populate with a comma
|
|
separated list of namenodes. The job client automatically fetches
|
|
delegation tokens from those namenodes. (Boris Shkolnik via ddas)
|
|
|
|
MAPREDUCE-698. Per-pool task limits for the fair scheduler.
|
|
(Kevin Peterson via matei)
|
|
|
|
MAPREDUCE-1026. Does mutual authentication of the shuffle
|
|
transfers using a shared JobTracker generated key.
|
|
(Boris Shkolnik via ddas)
|
|
|
|
MAPREDUCE-744. Introduces the notion of a public distributed cache.
|
|
(Devaraj Das)
|
|
|
|
MAPREDUCE-1338. Introduces the notion of token cache using which
|
|
tokens and secrets can be sent by the Job client to the JobTracker.
|
|
(Boris Shkolnik via ddas)
|
|
|
|
HDFS-503. This patch implements an optional layer over HDFS that
|
|
implements offline erasure-coding. It can be used to reduce the
|
|
total storage requirements of HDFS. (dhruba)
|
|
|
|
MAPREDUCE-1432. Adds hooks in the jobtracker and tasktracker
|
|
for loading the tokens in the user's ugi. This is required
|
|
for the copying of files from the hdfs. (ddas)
|
|
|
|
MAPREDUCE-1335. Adds SASL Kerberos/Digest authentication in MapReduce.
|
|
(Kan Zhang via ddas)
|
|
|
|
MAPREDUCE-1464. Makes a compatible change in JobTokenIdentifier to
|
|
account for HADOOP-6510. (Jitendra Nath Pandey via ddas)
|
|
|
|
MAPREDUCE-1433. Add a delegation token for MapReduce. (omalley)
|
|
|
|
MAPREDUCE-1307. Introduces the Job level ACLs feature.
|
|
(Vinod Kumar Vavilapalli via ddas)
|
|
|
|
MAPREDUCE-1430. JobTracker automatically renews delegation tokens for jobs.
|
|
(Boris Shkolnik via ddas)
|
|
|
|
MAPREDUCE-1455. Introduces job-level authorization for mapreduce servlets.
|
|
(Ravi Gummadi via vinodkv)
|
|
|
|
IMPROVEMENTS
|
|
|
|
MAPREDUCE-463. Makes job setup and cleanup tasks as optional.
|
|
(Amareshwari Sriramadasu via sharad)
|
|
|
|
MAPREDUCE-502. Allow jobtracker to be configured with zero completed jobs
|
|
in memory. (Amar Kamat via sharad)
|
|
|
|
MAPREDUCE-416. Moves the history file to a "done" folder whenever a job
|
|
completes. (Amar Kamat via ddas)
|
|
|
|
MAPREDUCE-646. Increase srcfilelist replication number in dictcp job.
|
|
(Ravi Gummadi via szetszwo)
|
|
|
|
HADOOP-6106. Updated hadoop-core and test jars from hudson trunk
|
|
build #12. (Giridharan Kesavan)
|
|
|
|
MAPREDUCE-642. A option to distcp that allows preserving the full
|
|
source path of a file in the specified destination directory.
|
|
(Rodrigo Schmidt via dhruba)
|
|
|
|
MAPREDUCE-686. Move TestSpeculativeExecution.Fake* into a separate class
|
|
so that it can be used by other tests. (Jothi Padmanabhan via sharad)
|
|
|
|
MAPREDUCE-625. Modify TestTaskLimits to improve execution time.
|
|
(Jothi Padmanabhan via sharad)
|
|
|
|
MAPREDUCE-465. Deprecate o.a.h.mapred.lib.MultithreadedMapRunner and add
|
|
test for o.a.h.mapreduce.lib.MultithreadedMapper.
|
|
(Amareshwari Sriramadasu via sharad)
|
|
|
|
MAPREDUCE-701. Improves the runtime of the TestRackAwareTaskPlacement
|
|
by making it a unit test. (Jothi Padmanabhan via ddas)
|
|
|
|
MAPREDUCE-371. Change KeyFieldBasedComparator and KeyFieldBasedPartitioner
|
|
to use new api. (Amareshwari Sriramadasu via sharad)
|
|
|
|
MAPREDUCE-623. Resolve javac warnings in mapreduce. (Jothi Padmanabhan
|
|
via sharad)
|
|
|
|
MAPREDUCE-655. Change KeyValueLineRecordReader and KeyValueTextInputFormat
|
|
to use new mapreduce api. (Amareshwari Sriramadasu via sharad)
|
|
|
|
MAPREDUCE-632. Merge TestCustomOutputCommitter with
|
|
TestCommandLineJobSubmission. (Jothi Padmanabhan via sharad)
|
|
|
|
MAPREDUCE-627. Improves execution time of TestTrackerBlacklistAcrossJobs.
|
|
(Jothi Padmanabhan via ddas)
|
|
|
|
MAPREDUCE-630. Improves execution time of TestKillCompletedJob.
|
|
(Jothi Padmanabhan via ddas)
|
|
|
|
MAPREDUCE-626. Improves the execution time of TestLostTracker.
|
|
(Jothi Padmanabhan via ddas)
|
|
|
|
MAPREDUCE-353. Makes the shuffle read and connection timeouts
|
|
configurable. (Ravi Gummadi via ddas)
|
|
|
|
MAPREDUCE-739. Allow relative paths to be created in archives. (Mahadev
|
|
Konar via cdouglas)
|
|
|
|
MAPREDUCE-772. Merge HADOOP-4010 changes to LineRecordReader into mapreduce
|
|
package. (Abdul Qadeer via cdouglas)
|
|
|
|
MAPREDUCE-785. Separate sub-test of TestReduceFetch to be included in
|
|
MR-670. (Jothi Padmanabhan via cdouglas)
|
|
|
|
MAPREDUCE-784. Modify TestUserDefinedCounters to use LocalJobRunner
|
|
instead of MiniMR. (Jothi Padmanabhan via sharad)
|
|
|
|
HADOOP-6160. Fix releaseaudit target to run on specific directories.
|
|
(gkesavan)
|
|
|
|
MAPREDUCE-782. Use PureJavaCrc32 in SpillRecord. (Todd Lipcon via
|
|
szetszwo)
|
|
|
|
MAPREDUCE-369. Change org.apache.hadoop.mapred.lib.MultipleInputs to
|
|
use new api. (Amareshwari Sriramadasu via sharad)
|
|
|
|
MAPREDUCE-373. Change org.apache.hadoop.mapred.lib.FieldSelectionMapReduce
|
|
to use new api. (Amareshwari Sriramadasu via sharad)
|
|
|
|
MAPREDUCE-628. Improves the execution time of TestJobInProgress.
|
|
(Jothi Padmanabhan via ddas)
|
|
|
|
MAPREDUCE-793. Creates a new test that consolidates a few tests to
|
|
include in the commit-test list. (Jothi Padmanabhan via ddas)
|
|
|
|
MAPREDUCE-797. Adds combiner support to MRUnit MapReduceDriver.
|
|
(Aaron Kimball via johan)
|
|
|
|
MAPREDUCE-656. Change org.apache.hadoop.mapred.SequenceFile* classes
|
|
to use new mapreduce api. (Amareshwari Sriramadasu via sharad)
|
|
|
|
MAPREDUCE-670. Creates ant target for 10 mins patch test build.
|
|
(Jothi Padmanabhan via gkesavan)
|
|
|
|
MAPREDUCE-375. Change org.apache.hadoop.mapred.lib.NLineInputFormat
|
|
and org.apache.hadoop.mapred.MapFileOutputFormat to use new api.
|
|
(Amareshwari Sriramadasu via ddas)
|
|
|
|
MAPREDUCE-779. Added node health failure counts into
|
|
JobTrackerStatistics. (Sreekanth Ramakrishnan via yhemanth)
|
|
|
|
MAPREDUCE-842. Setup secure permissions for localized job files,
|
|
intermediate outputs and log files on tasktrackers.
|
|
(Vinod Kumar Vavilapalli via yhemanth)
|
|
|
|
MAPREDUCE-478. Allow map and reduce jvm parameters, environment variables
|
|
and ulimit to be set separately.
|
|
Configuration changes:
|
|
add mapred.map.child.java.opts
|
|
add mapred.reduce.child.java.opts
|
|
add mapred.map.child.env
|
|
add mapred.reduce.child.ulimit
|
|
add mapred.map.child.env
|
|
add mapred.reduce.child.ulimit
|
|
deprecated mapred.child.java.opts
|
|
deprecated mapred.child.env
|
|
deprecated mapred.child.ulimit
|
|
(acmurthy)
|
|
|
|
MAPREDUCE-767. Remove the dependence on the CLI 2.0 snapshot.
|
|
(Amar Kamat via omalley)
|
|
|
|
MAPREDUCE-712. Minor efficiency tweaks to RandomTextWriter. (cdouglas)
|
|
|
|
MAPREDUCE-870. Remove the job retire thread and the associated
|
|
config parameters. (sharad)
|
|
|
|
MAPREDUCE-874. Rename the PiEstimator example to QuasiMonteCarlo.
|
|
(szetszwo)
|
|
|
|
MAPREDUCE-336. Allow logging level of map/reduce tasks to be configurable.
|
|
Configuration changes:
|
|
add mapred.map.child.log.level
|
|
add mapred.reduce.child.log.level
|
|
(acmurthy)
|
|
|
|
MAPREDUCE-355. Update mapred.join package to use the new API. (Amareshwari
|
|
Sriramadasu via cdouglas)
|
|
|
|
HADOOP-6184. Updated hadoop common and test jars to get the new API
|
|
in Configuration for dumping in JSON format from Hudson trunk build #68.
|
|
(yhemanth)
|
|
|
|
MAPREDUCE-476. Extend DistributedCache to work locally (LocalJobRunner).
|
|
(Philip Zeyliger via tomwhite)
|
|
|
|
MAPREDUCE-825. JobClient completion poll interval of 5s causes slow tests
|
|
in local mode. (Aaron Kimball via tomwhite)
|
|
|
|
MAPREDUCE-910. Support counters in MRUnit. (Aaron Kimball via cdouglas)
|
|
|
|
MAPREDUCE-788. Update gridmix2 to use the new API (Amareshwari Sriramadasu
|
|
via cdouglas)
|
|
|
|
MAPREDUCE-875. Make DBRecordReader execute queries lazily. (Aaron Kimball
|
|
via enis)
|
|
|
|
MAPREDUCE-318. Modularizes the shuffle code. (Jothi Padmanabhan and
|
|
Arun Murthy via ddas)
|
|
|
|
MAPREDUCE-936. Allow a load difference for fairshare scheduler.
|
|
(Zheng Shao via dhruba)
|
|
|
|
MAPREDUCE-370. Update MultipleOutputs to use the API, merge funcitonality
|
|
of MultipleOutputFormat. (Amareshwari Sriramadasu via cdouglas)
|
|
|
|
MAPREDUCE-898. Changes DistributedCache to use the new API.
|
|
(Amareshwari Sriramadasu via ddas)
|
|
|
|
MAPREDUCE-144. Includes dump of the process tree in task diagnostics when
|
|
a task is killed due to exceeding memory limits.
|
|
(Vinod Kumar Vavilapalli via yhemanth)
|
|
|
|
MAPREDUCE-945. Modifies MRBench and TestMapRed to use ToolRunner so that
|
|
options such as queue name can be passed via command line.
|
|
(Sreekanth Ramakrishnan via yhemanth)
|
|
|
|
MAPREDUCE-963. Deprecate o.a.h.mapred.FileAlreadyExistsException and
|
|
replace it with o.a.h.fs.FileAlreadyExistsException. (Boris Shkolnik
|
|
via szetszwo)
|
|
|
|
MAPREDUCE-960. Remove an unnecessary intermediate copy and obsolete API
|
|
from KeyValueLineRecordReader. (cdouglas)
|
|
|
|
MAPREDUCE-930. Modify Rumen to resolve paths in the canonical way, rather
|
|
than defaulting to the local filesystem. (cdouglas)
|
|
|
|
MAPREDUCE-944. Extend the LoadManager API of the fair-share scheduler
|
|
to support regulating tasks for a job based on resources currently in use
|
|
by that job. (dhruba)
|
|
|
|
MAPREDUCE-973. Move FailJob and SleepJob from examples to test. (cdouglas
|
|
via omalley)
|
|
|
|
MAPREDUCE-966. Modify Rumen to clean up interfaces and simplify integration
|
|
with other tools. (Hong Tang via cdouglas)
|
|
|
|
MAPREDUCE-856. Setup secure permissions for distributed cache files.
|
|
(Vinod Kumar Vavilapalli via yhemanth)
|
|
|
|
MAPREDUCE-885. More efficient SQL queries for DBInputFormat. (Aaron Kimball
|
|
via enis)
|
|
|
|
MAPREDUCE-284. Enables ipc.client.tcpnodelay in Tasktracker's Child.
|
|
(Ravi Gummadi via sharad)
|
|
|
|
MAPREDUCE-916. Split the documentation to match the project split.
|
|
(Corinne Chandel via omalley)
|
|
|
|
MAPREDUCE-649. Validate a copy by comparing the source and destination
|
|
checksums in distcp. Also adds an intra-task retry mechanism for errors
|
|
detected during the copy. (Ravi Gummadi via cdouglas)
|
|
|
|
MAPREDUCE-654. Add a -dryrun option to distcp printing a summary of the
|
|
file data to be copied, without actually performing the copy. (Ravi Gummadi
|
|
via cdouglas)
|
|
|
|
MAPREDUCE-664. Display the number of files deleted by distcp when the
|
|
-delete option is specified. (Ravi Gummadi via cdouglas)
|
|
|
|
MAPREDUCE-781. Let the name of distcp jobs be configurable. (Venkatesh S
|
|
via cdouglas)
|
|
|
|
MAPREDUCE-975. Add an API in job client to get the history file url for
|
|
a given job id. (sharad)
|
|
|
|
MAPREDUCE-905. Add Eclipse launch tasks for MapReduce. (Philip Zeyliger
|
|
via tomwhite)
|
|
|
|
MAPREDUCE-277. Makes job history counters available on the job history
|
|
viewers. (Jothi Padmanabhan via ddas)
|
|
|
|
MAPREDUCE-893. Provides an ability to refresh queue configuration
|
|
without restarting the JobTracker.
|
|
(Vinod Kumar Vavilapalli and Rahul Kumar Singh via yhemanth)
|
|
|
|
MAPREDUCE-1011. Add build.properties to svn and git ignore. (omalley)
|
|
|
|
MAPREDUCE-954. Change Map-Reduce context objects to be interfaces.
|
|
(acmurthy)
|
|
|
|
MAPREDUCE-639. Change Terasort example to reflect the 2009 updates.
|
|
(omalley)
|
|
|
|
MAPREDUCE-1063. Document gridmix benchmark. (cdouglas)
|
|
|
|
MAPREDUCE-931. Use built-in interpolation classes for making up task
|
|
runtimes in Rumen. (Dick King via cdouglas)
|
|
|
|
MAPREDUCE-1012. Mark Context interfaces as public evolving. (Tom White via
|
|
cdouglas)
|
|
|
|
MAPREDUCE-971. Document use of distcp when copying to s3, managing timeouts
|
|
in particular. (Aaron Kimball via cdouglas)
|
|
|
|
HDFS-663. DFSIO for append. (shv)
|
|
|
|
HDFS-641. Move all of the components that depend on map/reduce to
|
|
map/reduce. (omalley)
|
|
|
|
HADOOP-5107. Use Maven ant tasks to publish artifacts. (Giridharan Kesavan
|
|
via omalley)
|
|
|
|
MAPREDUCE-1229. Allow customization of job submission policy in Mumak.
|
|
(Hong Tang via cdouglas)
|
|
|
|
MAPREDUCE-1317. Reduce the memory footprint of Rumen objects by interning
|
|
host Strings. (Hong Tang via cdouglas)
|
|
|
|
MAPREDUCE-1097. Add support for Vertica 3.5 to its contrib module. (Omer
|
|
Trajman via cdouglas)
|
|
|
|
MAPREDUCE-1627. HadoopArchives should not uses a method in DistCp.
|
|
(szetszwo)
|
|
|
|
MAPREDUCE-1198. Alternatively schedule different types of tasks in
|
|
fair share scheduler. (Scott Chen via matei)
|
|
|
|
MAPREDUCE-707. Provide a jobconf property for explicitly assigning a job to
|
|
a pool in the Fair Scheduler. (Alan Heirich via matei)
|
|
|
|
MAPREDUCE-947. Added commitJob and abortJob apis to OutputCommitter.
|
|
Enhanced FileOutputCommitter to create a _SUCCESS file for successful
|
|
jobs. (Amar Kamat & Jothi Padmanabhan via acmurthy)
|
|
|
|
MAPREDUCE-1103. Added more metrics to Jobtracker. (sharad)
|
|
|
|
MAPREDUCE-1048. Add occupied/reserved slot usage summary on jobtracker UI.
|
|
(Amareshwari Sriramadasu and Hemanth Yamijala via sharad)
|
|
|
|
MAPREDUCE-1090. Modified log statement in TaskMemoryManagerThread to
|
|
include task attempt id. (yhemanth)
|
|
|
|
MAPREDUCE-1189. Reduce ivy console output to ovservable level (cos)
|
|
|
|
MAPREDUCE-1167. ProcfsBasedProcessTree collects rss memory information.
|
|
(Scott Chen via dhruba)
|
|
|
|
MAPREDUCE-1231. Added a new DistCp option, -skipcrccheck, so that the CRC
|
|
check during setup can be skipped. (Jothi Padmanabhan via szetszwo)
|
|
|
|
MAPREDUCE-1190. Add package documentation for BBP example.
|
|
(Tsz Wo (Nicholas) Sze via cdouglas)
|
|
|
|
MAPREDUCE-1119. When tasks fail to report status, show tasks's stack dump
|
|
before killing. (Aaron Kimball via tomwhite)
|
|
|
|
MAPREDUCE-1185. Redirect running job url to history url if job is already
|
|
retired. (Amareshwari Sriramadasu and Sharad Agarwal via sharad)
|
|
|
|
MAPREDUCE-1050. Introduce a mock object testing framework. (tomwhite)
|
|
|
|
MAPREDUCE-1084. Implementing aspects development and fault injeciton
|
|
framework for MapReduce. (Sreekanth Ramakrishnan via cos)
|
|
|
|
MAPREDUCE-1209. Move common specific part of the test TestReflectionUtils
|
|
out of mapred into common. (Todd Lipcon via tomwhite)
|
|
|
|
MAPREDUCE-967. TaskTracker does not need to fully unjar job jars.
|
|
(Todd Lipcon via tomwhite)
|
|
|
|
MAPREDUCE-1083. Changes in MapReduce so that group information of users
|
|
can be refreshed in the JobTracker via command line.
|
|
(Boris Shkolnik via ddas)
|
|
|
|
MAPREDUCE-181. Changes the job submission process to be secure.
|
|
(Devaraj Das)
|
|
|
|
MAPREDUCE-1250. Refactors the JobToken to use Common's Token interface.
|
|
(Kan Zhang via ddas)
|
|
|
|
MAPREDUCE-896. Enhance tasktracker to cleanup files that might have
|
|
been created by user tasks with non-writable permissions.
|
|
(Ravi Gummadi via yhemanth)
|
|
|
|
MAPREDUCE-372. Change org.apache.hadoop.mapred.lib.ChainMapper/Reducer
|
|
to use new mapreduce api. (Amareshwari Sriramadasu via sharad)
|
|
|
|
MAPREDUCE-1295. Add a tool in Rumen for folding and manipulating job
|
|
traces. (Dick King via cdouglas)
|
|
|
|
MAPREDUCE-1302. TrackerDistributedCacheManager deletes file
|
|
asynchronously, thus reducing task initialization delays.
|
|
(Zheng Shao via dhruba)
|
|
|
|
MAPREDUCE-1218. TaskTrackers send cpu and memory usage of
|
|
node to JobTracker. (Scott Chen via dhruba)
|
|
|
|
MAPREDUCE-847. Fix Releaseaudit warning count to zero
|
|
(Giridharan Kesavan)
|
|
|
|
MAPREDUCE-1337. Use generics in StreamJob to improve readability of that
|
|
class. (Kay Kay via cdouglas)
|
|
|
|
MAPREDUCE-361. Port terasort example to the new mapreduce API. (Amareshwari
|
|
Sriramadasu via cdouglas)
|
|
|
|
MAPREDUCE-1367. LocalJobRunner should support parallel mapper execution.
|
|
(Aaron Kimball via tomwhite)
|
|
|
|
MAPREDUCE-64. Eliminate io.sort.record.percent from MapTask configuration.
|
|
(cdouglas)
|
|
|
|
MAPREDUCE-1440. Replace the long user name in MapReduce with the local
|
|
name. (omalley)
|
|
|
|
MAPREDUCE-1470. Move delegation tokens from HDFS to Common so that
|
|
MapReduce can use them too. (omalley)
|
|
|
|
MAPREDUCE-1425. Reduce memory usage by archive. (mahadev via szetszwo)
|
|
|
|
MAPREDUCE-1441. Trim whitespace from directory lists pulled from the
|
|
configuration. (Todd Lipcon via cdouglas)
|
|
|
|
MAPREDUCE-1309. Refactor Rumen trace generator to improve code structure
|
|
and add extensible support for log formats. (Dick King via cdouglas)
|
|
|
|
MAPREDUCE-1503. Delegation token renewing and cancelling should provide
|
|
meaningful exceptions when there are failures instead of returning
|
|
false. (omalley)
|
|
|
|
HADOOP-6579. Upgrade commons-codec library to 1.4. (omalley)
|
|
|
|
MAPREDUCE-1423. Improve performance of CombineFileInputFormat when multiple
|
|
pools are configured. (Dhruba Borthakur via zshao)
|
|
|
|
MAPREDUCE-1454. Quote user supplied strings in Tracker servlets. (cdouglas)
|
|
|
|
MAPREDUCE-1408. Add customizable job submission policies to Gridmix. (Rahul
|
|
Singh via cdouglas)
|
|
|
|
MAPREDUCE-1527. Better warning logged when mapred.queue.names is
|
|
overshadowed by mapred-queues.xml. (Hong Tang via acmurthy)
|
|
|
|
MAPREDUCE-1403. Save the size and number of distributed cache artifacts in
|
|
the configuration. (Arun Murthy via cdouglas)
|
|
|
|
MAPREDUCE-1482. Truncate state string and diagnostic information in
|
|
TaskStatus. (Amar Kamat via szetszwo)
|
|
|
|
MAPREDUCE-1593. [Rumen] Improvements to random seed generation (tamas via
|
|
mahadev)
|
|
|
|
MAPREDUCE-1460. Oracle support in DataDrivenDBInputFormat.
|
|
(Aaron Kimball via tomwhite)
|
|
|
|
MAPREDUCE-1569. Pass configuration through mocked contexts in MRUnit.
|
|
(Chris White via cdouglas)
|
|
|
|
MAPREDUCE-1590. Move HarFileSystem from Hadoop Common to Mapreduce tools.
|
|
(mahadev)
|
|
|
|
MAPREDUCE-1629. Get rid of fakeBlockLocations() on HarFileSystem, since
|
|
it's not used (mahadev)
|
|
|
|
MAPREDUCE-1489. DataDrivenDBInputFormat should not query the database
|
|
when generating only one split. (Aaron Kimball via tomwhite)
|
|
|
|
MAPREDUCE-1514. Add documentation on replication, permissions, new options,
|
|
limitations and internals of har. (mahadev via szetszwo)
|
|
|
|
MAPREDUCE-1428. Make block size and the size of archive created files
|
|
configurable. (mahadev via szetszwo)
|
|
|
|
MAPREDUCE-1656. JobStory should provide queue info. (hong via mahadev)
|
|
|
|
MAPREDUCE-1466. Record number of files processed in FileInputFormat in the
|
|
Configuration for offline analysis. (Luke Lu and Arun Murthy via cdouglas)
|
|
|
|
MAPREDUCE-1538. TrackerDistributedCacheManager manages the
|
|
number of files. (Scott Chen via dhruba)
|
|
|
|
MAPREDUCE-1673. Scripts to start and stop RaidNode.
|
|
(Rodrigo Schmidt via dhruba)
|
|
|
|
MAPREDUCE-1659. RaidNode writes temp files on configured tmp directory and
|
|
add random numbers to their names to avoid conflicts
|
|
(Rodrigo Schmidt via dhruba)
|
|
|
|
MAPREDUCE-1221. Allow admins to control physical memory limits per-task
|
|
and per-node. (Scott Chen via acmurthy)
|
|
|
|
MAPREDUCE-1065. Update mapred tutorial to use the new API. (Aaron Kimball
|
|
via cdouglas)
|
|
|
|
MAPREDUCE-1304. Add a task counter tracking time spent in GC. (Aaron
|
|
Kimball via cdouglas)
|
|
|
|
MAPREDUCE-1570. Add grouping comparators to MRUnit. (Chris White via
|
|
cdouglas)
|
|
|
|
MAPREDUCE-1650. Exclude Private elements from generated MapReduce
|
|
Javadoc. (tomwhite)
|
|
|
|
MAPREDUCE-1625. Improve grouping of packages in Javadoc. (tomwhite)
|
|
|
|
MAPREDUCE-1417. Forrest documentation should be updated to reflect
|
|
the changes in MAPREDUCE-744. (Ravi Gummadi via vinodkv)
|
|
|
|
MAPREDUCE-1568. TrackerDistributedCacheManager should clean up cache
|
|
in a background thread. (Scott Chen via zshao)
|
|
|
|
MAPREDUCE-1749. Move configuration strings out of JobContext so that it
|
|
can be made public stable. (omalley)
|
|
|
|
MAPREDUCE-1623. Apply audience and stability notations to Hadoop
|
|
Map-Reduce. (tomwhite via acmurthy)
|
|
|
|
MAPREDUCE-1751. Change MapReduce to depend on Hadoop 'common' artifacts
|
|
instead of 'core'. (tomwhite)
|
|
|
|
MAPREDUCE-1535. Replace usage of FileStatus#isDir(). (Eli Collins via
|
|
tomwhite)
|
|
|
|
MAPREDUCE-1832. Allow file sizes less than 1MB in DFSIO benchmark. (shv)
|
|
|
|
MAPREDUCE-1404. Move Cluster-Setup and Single-Node-Setup Docs from
|
|
MapReduce to Common. (tomwhite)
|
|
|
|
MAPREDUCE-1697. Document the behavior of -file option and deprecate it
|
|
in favour of -files option in streaming. (Amareshwari Sriramadasu
|
|
via vinodkv)
|
|
|
|
MAPREDUCE-1033. Resolve location of scripts and configuration files after
|
|
project split. (tomwhite)
|
|
|
|
MAPREDUCE-1018. Document changes to the memory management and scheduling
|
|
model. (Hemanth Yamijala via vinodkv)
|
|
|
|
MAPREDUCE-1896. [Herriot] New property for multi user list. (Vinay Thota
|
|
via cos)
|
|
|
|
MAPREDUCE-1812. New properties for suspend and resume process. (Vinay
|
|
Thota via cos)
|
|
|
|
OPTIMIZATIONS
|
|
|
|
MAPREDUCE-270. Fix the tasktracker to optionally send an out-of-band
|
|
heartbeat on task-completion for better job-latency. (acmurthy)
|
|
Configuration changes:
|
|
add mapreduce.tasktracker.outofband.heartbeat
|
|
|
|
MAPREDUCE-1186. Modified code in distributed cache to set permissions
|
|
only on required set of localized paths.
|
|
(Amareshwari Sriramadasu via yhemanth)
|
|
|
|
MAPREDUCE-1501. FileInputFormat supports multi-level, recursive
|
|
directory listing. (Zheng Shao via dhruba)
|
|
|
|
MAPREDUCE-1556. upgrade to Avro 1.3.0. (cutting via tomwhite)
|
|
|
|
MAPREDUCE-1613. Install/deploy source jars to Maven repo
|
|
(Patrick Angeles via ddas)
|
|
|
|
MAPREDUCE-1610. Forrest documentation should be updated to reflect
|
|
the changes in MAPREDUCE-856. (Ravi Gummadi via vinodkv)
|
|
|
|
MAPREDUCE-1853. Adds caching for TaskAttemptContext in MultipleOutputs.
|
|
(Torsten Curdt via amareshwari)
|
|
|
|
BUG FIXES
|
|
|
|
MAPREDUCE-878. Rename fair scheduler design doc to
|
|
fair-scheduler-design-doc.tex and add Apache license header (matei)
|
|
|
|
HADOOP-4687. MapReduce is split from Hadoop Core. It is a subproject under
|
|
Hadoop (Owen O'Malley)
|
|
|
|
HADOOP-6096. Fix Eclipse project and classpath files following project
|
|
split. (tomwhite)
|
|
|
|
MAPREDUCE-419. Reconcile mapred.userlog.limit.kb defaults in configuration
|
|
and code. (Philip Zeyliger via cdouglas)
|
|
|
|
MAPREDUCE-2. Fixes a bug in KeyFieldBasedPartitioner in handling empty
|
|
keys. (Amar Kamat via sharad)
|
|
|
|
MAPREDUCE-130. Delete the jobconf copy from the log directory of the
|
|
JobTracker when the job is retired. (Amar Kamat via sharad)
|
|
|
|
MAPREDUCE-657. Fix hardcoded filesystem problem in CompletedJobStatusStore.
|
|
(Amar Kamat via sharad)
|
|
|
|
MAPREDUCE-179. Update progress in new RecordReaders. (cdouglas)
|
|
|
|
MAPREDUCE-658. Replace NPE in distcp with a meaningful error message when
|
|
the source path does not exist. (Ravi Gummadi via cdouglas)
|
|
|
|
MAPREDUCE-671. Update ignore list to include untracked, generated
|
|
build artifacts and config files. (cdouglas)
|
|
|
|
MAPREDUCE-433. Use more reliable counters in TestReduceFetch. (cdouglas)
|
|
|
|
MAPREDUCE-124. Fix a bug in failure handling of abort task of
|
|
OutputCommiter. (Amareshwari Sriramadasu via sharad)
|
|
|
|
MAPREDUCE-694. Fix to add jsp-api jars to capacity-scheduler classpath.
|
|
(Giridharan Kesavan)
|
|
|
|
MAPREDUCE-702. Fix eclipse-plugin jar target (Giridharan Kesavan)
|
|
|
|
MAPREDUCE-522. Replace TestQueueCapacities with simpler test case to
|
|
test integration between capacity scheduler and MR framework.
|
|
(Sreekanth Ramakrishnan via yhemanth)
|
|
|
|
MAPREDUCE-683. Fixes an initialization problem in the JobHistory.
|
|
The initialization of JobHistoryFilesManager is now done in the
|
|
JobHistory.init call. (Amar Kamat via ddas)
|
|
|
|
MAPREDUCE-708. Fixes a bug to allow updating the reason for
|
|
blacklisting a node on the JobTracker UI.
|
|
(Sreekanth Ramakrishnan via yhemanth)
|
|
|
|
MAPREDUCE-709. Fixes message displayed for a blacklisted node where
|
|
the reason for blacklisting is due to the health check script
|
|
timing out. (Sreekanth Ramakrishnan via yhemanth)
|
|
|
|
MAPREDUCE-676. Existing diagnostic rules fail for MAP ONLY jobs.
|
|
(Suhas Gogate via tomwhite)
|
|
|
|
MAPREDUCE-722. Fixes a bug with tasktracker reservations for
|
|
high memory jobs in capacity scheduler.
|
|
(Vinod Kumar Vavilapalli via yhemanth)
|
|
|
|
HADOOP-6090. Updates gridmix script to use new mapreduce api output
|
|
format. (Amareshwari Sriramadasu via sharad)
|
|
|
|
MAPREDUCE-732. Removed spurious log statements in the node
|
|
blacklisting logic. (Sreekanth Ramakrishnan via yhemanth)
|
|
|
|
MAPREDUCE-734. Fix a ConcurrentModificationException in unreserving
|
|
unused reservations for a job when it completes.
|
|
(Arun Murthy and Sreekanth Ramakrishnan via yhemanth)
|
|
|
|
MAPREDUCE-733. Fix a RuntimeException while unreserving trackers
|
|
that are blacklisted for a job.
|
|
(Arun Murthy and Sreekanth Ramakrishnan via yhemanth)
|
|
|
|
MAPREDUCE-677. Fix timeout in TestNodeRefresh. (Amar Kamat via
|
|
sharad)
|
|
|
|
MAPREDUCE-153. Fix timeout in TestJobInProgressListener. (Amar
|
|
Kamat via sharad)
|
|
|
|
MAPREDUCE-742. Fix output messages and java comments in the Pi related
|
|
examples. (szetszwo)
|
|
|
|
MAPREDUCE-565. Fix partitioner to work with new API. (Owen O'Malley via
|
|
cdouglas)
|
|
|
|
MAPREDUCE-680. Fix so MRUnit can handle reuse of Writable objects.
|
|
(Aaron Kimball via johan)
|
|
|
|
MAPREDUCE-18. Puts some checks for cross checking whether a reduce
|
|
task gets the correct shuffle data. (Ravi Gummadi via ddas)
|
|
|
|
MAPREDUCE-771. Fix scheduling of setup and cleanup tasks to use
|
|
free slots instead of tasks for scheduling. (yhemanth)
|
|
|
|
MAPREDUCE-717. Fixes some corner case issues in speculative
|
|
execution heuristics. (Devaraj Das)
|
|
|
|
MAPREDUCE-716. Make DBInputFormat work with Oracle. (Aaron Kimball
|
|
via tomwhite)
|
|
|
|
MAPREDUCE-735. Fixes a problem in the KeyFieldHelper to do with
|
|
the end index for some inputs (Amar Kamat via ddas)
|
|
|
|
MAPREDUCE-682. Removes reservations on tasktrackers which are
|
|
blacklisted. (Sreekanth Ramakrishnan via yhemanth)
|
|
|
|
MAPREDUCE-743. Fixes a problem to do with progress reporting
|
|
in the map phase. (Ravi Gummadi via ddas)
|
|
|
|
MAPREDUCE-765. Eliminate the deprecated warnings introduced by H-5438.
|
|
(He Yongqiang via szetszwo)
|
|
|
|
MAPREDUCE-383. Fix a bug in Pipes combiner due to bytes count not
|
|
getting reset after the spill. (Christian Kunz via sharad)
|
|
|
|
MAPREDUCE-809. Fix job-summary logs to correctly record status of FAILED
|
|
and KILLED jobs. (acmurthy)
|
|
|
|
MAPREDUCE-792. Fix unchecked warnings in DBInputFormat. (Aaron Kimball
|
|
via szetszwo)
|
|
|
|
MAPREDUCE-760. Fix a timing issue in TestNodeRefresh. (Amar Kamat via
|
|
sharad)
|
|
|
|
MAPREDUCE-40. Keep memory management backwards compatible for job
|
|
configuration parameters and limits. (Rahul Kumar Singh via yhemanth)
|
|
|
|
MAPREDUCE-587. Fixes a OOM issue in TestStreamingExitStatus.
|
|
(Amar Kamat via ddas)
|
|
|
|
MAPREDUCE-408. Fixes an assertion problem in TestKillSubProcesses
|
|
(Ravi Gummadi via ddas)
|
|
|
|
MAPREDUCE-659. Fix gridmix2 compilation. (Giridharan Kesavan)
|
|
|
|
MAPREDUCE-796. Fixes a ClassCastException in an exception log in
|
|
MultiThreadedMapRunner. (Amar Kamat via ddas)
|
|
|
|
MAPREDUCE-808. Fixes a serialization problem in TypedBytes.
|
|
(Klaas Bosteels via ddas)
|
|
|
|
MAPREDUCE-845. Fix a findbugs heap size problem in build.xml and add
|
|
a new property findbugs.heap.size. (Lee Tucker via szetszwo)
|
|
|
|
MAPREDUCE-838. Fixes a problem in the way commit of task outputs
|
|
happens. The bug was that even if commit failed, the task would
|
|
be declared as successful. (Amareshwari Sriramadasu via ddas)
|
|
|
|
MAPREDUCE-813. Updates Streaming and M/R tutorial documents.
|
|
(Corinne Chandel via ddas)
|
|
|
|
MAPREDUCE-805. Fixes some deadlocks in the JobTracker due to the fact
|
|
the JobTracker lock hierarchy wasn't maintained in some JobInProgress
|
|
method calls. (Amar Kamat via ddas)
|
|
|
|
MAPREDUCE-799. Fixes so all of the MRUnit self-tests run.
|
|
(Aaron Kimball via johan)
|
|
|
|
MAPREDUCE-848. Fixes a problem to do with TestCapacityScheduler
|
|
failing (Amar Kamat via ddas)
|
|
|
|
MAPREDUCE-840. DBInputFormat leaves open transaction.
|
|
(Aaron Kimball via tomwhite)
|
|
|
|
MAPREDUCE-859. Adds Avro and its dependencies required by Hadoop
|
|
common. (Ravi Gummadi via sharad)
|
|
|
|
MAPREDUCE-867. Fix ivy conf to look for avro jar from maven repo.
|
|
(Giridharan Kesavan)
|
|
|
|
MAPREDUCE-877. Added avro as a dependency to contrib ivy settings.
|
|
(Tsz Wo (Nicholas) Sze via yhemanth)
|
|
|
|
MAPREDUCE-852. In build.xml, remove the Main-Class, which is incorrectly
|
|
set in tools, and rename the target "tools-jar" to "tools". (szetszwo)
|
|
|
|
MAPREDUCE-773. Sends progress reports for compressed gzip inputs in maps.
|
|
Fixes a native direct buffer leak in LineRecordReader classes.
|
|
(Hong Tang and ddas)
|
|
|
|
MAPREDUCE-832. Reduce number of warning messages printed when
|
|
deprecated memory variables are used. (Rahul Kumar Singh via yhemanth)
|
|
|
|
MAPREDUCE-745. Fixes a testcase problem to do with generation of JobTracker
|
|
IDs. (Amar Kamat via ddas)
|
|
|
|
MAPREDUCE-834. Enables memory management on tasktrackers when old
|
|
memory management parameters are used in configuration.
|
|
(Sreekanth Ramakrishnan via yhemanth)
|
|
|
|
MAPREDUCE-818. Fixes Counters#getGroup API. (Amareshwari Sriramadasu
|
|
via sharad)
|
|
|
|
MAPREDUCE-807. Handles the AccessControlException during the deletion of
|
|
mapred.system.dir in the JobTracker. The JobTracker will bail out if it
|
|
encounters such an exception. (Amar Kamat via ddas)
|
|
|
|
MAPREDUCE-430. Fix a bug related to task getting stuck in case of
|
|
OOM error. (Amar Kamat via ddas)
|
|
|
|
MAPREDUCE-871. Fix ownership of Job/Task local files to have correct
|
|
group ownership according to the egid of the tasktracker.
|
|
(Vinod Kumar Vavilapalli via yhemanth)
|
|
|
|
MAPREDUCE-911. Fix a bug in TestTaskFail related to speculative
|
|
execution. (Amareshwari Sriramadasu via sharad)
|
|
|
|
MAPREDUCE-687. Fix an assertion in TestMiniMRMapRedDebugScript.
|
|
(Amareshwari Sriramadasu via sharad)
|
|
|
|
MAPREDUCE-924. Fixes the TestPipes testcase to use Tool.
|
|
(Amareshwari Sriramadasu via sharad)
|
|
|
|
MAPREDUCE-903. Add Avro jar to eclipse classpath.
|
|
(Philip Zeyliger via tomwhite)
|
|
|
|
MAPREDUCE-943. Removes a testcase in TestNodeRefresh that doesn't make
|
|
sense in the new Job recovery model. (Amar Kamat via ddas)
|
|
|
|
MAPREDUCE-764. TypedBytesInput's readRaw() does not preserve custom type
|
|
codes. (Klaas Bosteels via tomwhite)
|
|
|
|
HADOOP-6243. Fixes a NullPointerException in handling deprecated keys.
|
|
(Sreekanth Ramakrishnan via yhemanth)
|
|
|
|
MAPREDUCE-968. NPE in distcp encountered when placing _logs directory on
|
|
S3FileSystem. (Aaron Kimball via tomwhite)
|
|
|
|
MAPREDUCE-826. harchive doesn't use ToolRunner / harchive returns 0 even
|
|
if the job fails with exception (koji Noguchi via mahadev)
|
|
|
|
MAPREDUCE-839. unit test TestMiniMRChildTask fails on mac os-x (hong tang
|
|
via mahadev)
|
|
|
|
MAPREDUCE-112. Add counters for reduce input, output records to the new API.
|
|
(Jothi Padmanabhan via cdouglas)
|
|
|
|
MAPREDUCE-648. Fix two distcp bugs: (1) it should not launch a job if all
|
|
src paths are directories, and (2) it does not skip copying when updating
|
|
a single file. (Ravi Gummadi via szetszwo)
|
|
|
|
MAPREDUCE-946. Fix a regression in LineRecordReader where the
|
|
maxBytesToConsume parameter is not set correctly. (cdouglas)
|
|
|
|
MAPREDUCE-977. Missing jackson jars from Eclipse template. (tomwhite)
|
|
|
|
MAPREDUCE-988. Fix a packaging issue in the contrib modules. (Hong Tang via
|
|
cdouglas)
|
|
|
|
MAPREDUCE-971. distcp does not always remove distcp.tmp.dir. (Aaron Kimball
|
|
via tomwhite)
|
|
|
|
MAPREDUCE-995. Fix a bug in JobHistory where tasks completing after the job
|
|
is closed cause a NPE. (Jothi Padmanabhan via cdouglas)
|
|
|
|
MAPREDUCE-953. Fix QueueManager to dump queue configuration in JSON format.
|
|
(V.V. Chaitanya Krishna via yhemanth)
|
|
|
|
MAPREDUCE-645. Prevent distcp from running a job when the destination is a
|
|
file, but the source is not. (Ravi Gummadi via cdouglas)
|
|
|
|
MAPREDUCE-1002. Flushed writer in JobQueueClient so queue information is
|
|
printed correctly. (V.V. Chaitanya Krishna via yhemanth)
|
|
|
|
MAPREDUCE-1003. Fix compilation problem in eclipse plugin when
|
|
eclipse.home is set. (Ravi Gummadi via yhemanth)
|
|
|
|
MAPREDUCE-941. Vaidya script fails on Solaris. (Chad Metcalf
|
|
via tomwhite)
|
|
|
|
MAPREDUCE-912. Add and standardize Apache license headers. (Chad Metcalf
|
|
via cdouglas)
|
|
|
|
MAPREDUCE-1022. Fix compilation of vertica testcases. (Vinod Kumar
|
|
Vavilapalli via acmurthy)
|
|
|
|
MAPREDUCE-1000. Handle corrupt history files in JobHistory.initDone().
|
|
(Jothi Padmanabhan via sharad)
|
|
|
|
MAPREDUCE-1028. Fixed number of slots occupied by cleanup tasks to one
|
|
irrespective of slot size for the job.
|
|
(Ravi Gummadi via yhemanth)
|
|
|
|
MAPREDUCE-964. Fixed start and finish times of TaskStatus to be
|
|
consistent, thereby fixing inconsistencies in metering tasks.
|
|
(Sreekanth Ramakrishnan via yhemanth)
|
|
|
|
MAPREDUCE-1076. Deprecate ClusterStatus and add javadoc in ClusterMetrics.
|
|
(Amareshwari Sriramadasu via sharad)
|
|
|
|
MAPREDUCE-979. Fixed JobConf APIs related to memory parameters to return
|
|
values of new configuration variables when deprecated variables are
|
|
disabled. (Sreekanth Ramakrishnan via yhemanth)
|
|
|
|
MAPREDUCE-1030. Modified scheduling algorithm to return a map and reduce
|
|
task per heartbeat in the capacity scheduler.
|
|
(Rahul Kumar Singh via yhemanth)
|
|
|
|
MAPREDUCE-1071. Use DataInputStream rather than FSDataInputStream in the
|
|
JobHistory EventReader. (Hong Tang via cdouglas)
|
|
|
|
MAPREDUCE-986. Fix Rumen to work with truncated task lines. (Dick King via
|
|
cdouglas)
|
|
|
|
MAPREDUCE-1029. Fix failing TestCopyFiles by restoring the unzipping of
|
|
HDFS webapps from the hdfs jar. (Aaron Kimball and Jothi Padmanabhan via
|
|
cdouglas)
|
|
|
|
MAPREDUCE-769. Make findbugs and javac warnings to zero.
|
|
(Amareshwari Sriramadasu via sharad)
|
|
|
|
MAPREDUCE-1104. Initialize RecoveryManager in JobTracker cstr called by
|
|
Mumak. (Hong Tang via cdouglas)
|
|
|
|
MAPREDUCE-1061. Add unit test validating byte specifications for gridmix
|
|
jobs. (cdouglas)
|
|
|
|
MAPREDUCE-1077. Fix Rumen so that truncated tasks do not mark the job as
|
|
successful. (Dick King via cdouglas)
|
|
|
|
MAPREDUCE-1041. Make TaskInProgress::taskStatuses map package-private.
|
|
(Jothi Padmanabhan via cdouglas)
|
|
|
|
MAPREDUCE-1070. Prevent a deadlock in the fair scheduler servlet.
|
|
(Todd Lipcon via cdouglas)
|
|
|
|
MAPREDUCE-1086. Setup Hadoop logging environment for tasks to point to
|
|
task related parameters. (Ravi Gummadi via yhemanth)
|
|
|
|
MAPREDUCE-1105. Remove max limit configuration in capacity scheduler in
|
|
favor of max capacity percentage thus allowing the limit to go over
|
|
queue capacity. (Rahul Kumar Singh via yhemanth)
|
|
|
|
MAPREDUCE-1016. Make the job history log format JSON. (cutting)
|
|
|
|
MAPREDUCE-1038. Weave Mumak aspects only if related files have changed.
|
|
(Aaron Kimball via cdouglas)
|
|
|
|
MAPREDUCE-1163. Remove unused, hard-coded paths from libhdfs. (Allen
|
|
Wittenauer via cdouglas)
|
|
|
|
MAPREDUCE-962. Fix a NullPointerException while killing task process
|
|
trees. (Ravi Gummadi via yhemanth)
|
|
|
|
MAPREDUCE-1177. Correct setup/cleanup inversion in
|
|
JobTracker::getTaskReports. (Vinod Kumar Vavilapalli via cdouglas)
|
|
|
|
MAPREDUCE-1178. Fix ClassCastException in MultipleInputs by adding
|
|
a DelegatingRecordReader. (Amareshwari Sriramadasu and Jay Booth
|
|
via sharad)
|
|
|
|
MAPREDUCE-1068. Fix streaming job to show proper message if file is
|
|
is not present. (Amareshwari Sriramadasu via sharad)
|
|
|
|
MAPREDUCE-1147. Add map output counters to new API. (Amar Kamat via
|
|
cdouglas)
|
|
|
|
MAPREDUCE-915. The debug scripts are run as the job user. (ddas)
|
|
|
|
MAPREDUCE-1007. Fix NPE in CapacityTaskScheduler.getJobs().
|
|
(V.V.Chaitanya Krishna via sharad)
|
|
|
|
MAPREDUCE-28. Refactor TestQueueManager and fix default ACLs.
|
|
(V.V.Chaitanya Krishna and Rahul K Singh via sharad)
|
|
|
|
MAPREDUCE-1182. Fix overflow in reduce causing allocations to exceed the
|
|
configured threshold. (cdouglas)
|
|
|
|
MAPREDUCE-1239. Fix contrib components build dependencies.
|
|
(Giridharan Kesavan and omalley)
|
|
|
|
MAPREDUCE-787. Fix JobSubmitter to honor user given symlink path.
|
|
(Amareshwari Sriramadasu via sharad)
|
|
|
|
MAPREDUCE-1249. Update config default value for socket read timeout to
|
|
match code default. (Amareshwari Sriramadasu via cdouglas)
|
|
|
|
MAPREDUCE-1161. Remove ineffective synchronization in NotificationTestCase.
|
|
(Owen O'Malley via cdouglas)
|
|
|
|
MAPREDUCE-1244. Fix eclipse-plugin's build dependencies. (gkesavan)
|
|
|
|
MAPREDUCE-1075. Fix JobTracker to not throw an NPE for a non-existent
|
|
queue. (V.V.Chaitanya Krishna via yhemanth)
|
|
|
|
MAPREDUCE-754. Fix NPE in expiry thread when a TT is lost. (Amar Kamat
|
|
via sharad)
|
|
|
|
MAPREDUCE-1074. Document Reducer mark/reset functionality. (Jothi
|
|
Padmanabhan via cdouglas)
|
|
|
|
MAPREDUCE-1267. Fix typo in mapred-default.xml. (Todd Lipcon via cdouglas)
|
|
|
|
MAPREDUCE-952. Remove inadvertently reintroduced Task.Counter enum. (Jothi
|
|
Padmanabhan via cdouglas)
|
|
|
|
MAPREDUCE-1230. Fix handling of null records in VerticaInputFormat. (Omer
|
|
Trajman via cdouglas)
|
|
|
|
MAPREDUCE-1171. Allow shuffle retries and read-error reporting to be
|
|
configurable. (Amareshwari Sriramadasu via acmurthy)
|
|
|
|
MAPREDUCE-879. Fix broken unit test TestTaskTrackerLocalization on MacOS.
|
|
(Sreekanth Ramakrishnan via yhemanth)
|
|
|
|
MAPREDUCE-1124. Fix imprecise byte counts in Gridmix. (cdouglas)
|
|
|
|
MAPREDUCE-1222. Add an option to exclude numeric IP addresses in topologies
|
|
processed by Mumak. (Hong Tang via cdouglas)
|
|
|
|
MAPREDUCE-1284. Fix fts_open() call in task-controller that was failing
|
|
LinuxTaskController unit tests. (Ravi Gummadi via yhemanth)
|
|
|
|
MAPREDUCE-1143. Fix running task counters to be updated correctly
|
|
when speculative attempts are running for a TIP.
|
|
(Rahul Kumar Singh via yhemanth)
|
|
|
|
MAPREDUCE-1241. Use a default queue configuration in JobTracker when
|
|
mapred-queues.xml is unavailable. (Todd Lipcon via cdouglas)
|
|
|
|
MAPREDUCE-1301. Fix set up of permission checking script used in
|
|
localization tests. (Amareshwari Sriramadasu via yhemanth)
|
|
|
|
MAPREDUCE-1286. Remove quoting from client opts in TaskRunner. (Yuri
|
|
Pradkin via cdouglas)
|
|
|
|
MAPREDUCE-1059. Use distcp.bytes.per.map when adding sync markers in
|
|
distcp. (Aaron Kimball via cdouglas)
|
|
|
|
MAPREDUCE-1009. Update forrest documentation describing hierarchical
|
|
queues. (Vinod Kumar Vavilapalli via yhemanth)
|
|
|
|
MAPREDUCE-1342. Fixed deadlock in global blacklisting of tasktrackers.
|
|
(Amareshwari Sriramadasu via acmurthy)
|
|
|
|
MAPREDUCE-1316. Fixes a memory leak of TaskInProgress instances in
|
|
the jobtracker. (Amar Kamat via yhemanth)
|
|
|
|
MAPREDUCE-1359. TypedBytes TestIO doesn't mkdir its test dir first.
|
|
(Anatoli Fomenko via cos)
|
|
|
|
MAPREDUCE-1314. Correct errant mapreduce.x.mapreduce.x replacements from
|
|
bulk change. (Amareshwari Sriramadasu via cdouglas)
|
|
|
|
MAPREDUCE-1365. Restore accidentally renamed test in
|
|
TestTaskTrackerBloacklisting. (Amareshwari Sriramadasu via cdouglas)
|
|
|
|
MAPREDUCE-1406. Fix spelling of JobContext.MAP_COMBINE_MIN_SPILLS.
|
|
(cdouglas)
|
|
|
|
MAPREDUCE-1369. JUnit tests should never depend on anything in conf
|
|
(Anatoli Fomenko via cos)
|
|
|
|
MAPREDUCE-1412. Fix timer granularity issue causing failures in
|
|
TestTaskTrackerBlacklisting. (cdouglas)
|
|
|
|
MAPREDUCE-1448. Respect --config option in Mumak script. (Hong Tang via
|
|
cdouglas)
|
|
|
|
MAPREDUCE-1251. c++ utils doesn't compile. (Eli Collins via tomwhite)
|
|
|
|
MAPREDUCE-1522. FileInputFormat may use the default FileSystem for the
|
|
input path. (Tsz Wo (Nicholas), SZE via cdouglas)
|
|
|
|
MAPREDUCE-1407. Update javadoc in mapreduce.{Mapper,Reducer} to match
|
|
actual usage. (Benoit Sigoure via cdouglas)
|
|
|
|
MAPREDUCE-1258. Fix fair scheduler event log not logging job info.
|
|
(matei)
|
|
|
|
MAPREDUCE-1089. Fix NPE in fair scheduler preemption when tasks are
|
|
scheduled but not running. (Todd Lipcon via matei)
|
|
|
|
MAPREDUCE-1014. Fix the libraries for common and hdfs. (omalley)
|
|
|
|
MAPREDUCE-1111. JT Jetty UI not working if we run mumak.sh
|
|
off packaged distribution directory. (hong tang via mahadev)
|
|
|
|
MAPREDUCE-1133. Eclipse .classpath template has outdated jar files and is
|
|
missing some new ones. (cos)
|
|
|
|
MAPREDUCE-1098. Fixed the distributed-cache to not do i/o while holding a
|
|
global lock. (Amareshwari Sriramadasu via acmurthy)
|
|
|
|
MAPREDUCE-1158. Fix JT running maps and running reduces metrics.
|
|
(sharad)
|
|
|
|
MAPREDUCE-1160. Reduce verbosity of log lines in some Map/Reduce classes
|
|
to avoid filling up jobtracker logs on a busy cluster.
|
|
(Ravi Gummadi and Hong Tang via yhemanth)
|
|
|
|
MAPREDUCE-1153. Fix tasktracker metrics when trackers are decommissioned.
|
|
(sharad)
|
|
|
|
MAPREDUCE-1128. Fix MRUnit to prohibit iterating over values twice. (Aaron
|
|
Kimball via cdouglas)
|
|
|
|
MAPREDUCE-665. Move libhdfs to HDFS subproject. (Eli Collins via dhruba)
|
|
|
|
MAPREDUCE-1196. Fix FileOutputCommitter to use the deprecated cleanupJob
|
|
api correctly. (acmurthy)
|
|
|
|
MAPREDUCE-1244. Fix eclipse-plugin's build dependencies. (gkesavan)
|
|
|
|
MAPREDUCE-1140. Fix DistributedCache to not decrement reference counts for
|
|
unreferenced files in error conditions.
|
|
(Amareshwari Sriramadasu via yhemanth)
|
|
|
|
MAPREDUCE-1245. Fix TestFairScheduler failures by instantiating lightweight
|
|
Jobtracker. (sharad)
|
|
|
|
MAPREDUCE-1260. Update Eclipse configuration to match changes to Ivy
|
|
configuration. (Edwin Chan via cos)
|
|
|
|
MAPREDUCE-1152. Distinguish between failed and killed tasks in
|
|
JobTrackerInstrumentation. (Sharad Agarwal via cdouglas)
|
|
|
|
MAPREDUCE-1285. In DistCp.deleteNonexisting(..), get class from the
|
|
parameter instead of using FileStatus.class. (Peter Romianowski via
|
|
szetszwo)
|
|
|
|
MAPREDUCE-1294. Build fails to pull latest hadoop-core-* artifacts (cos)
|
|
|
|
MAPREDUCE-1213. TaskTrackers restart is faster because it deletes
|
|
distributed cache directory asynchronously. (Zheng Shao via dhruba)
|
|
|
|
MAPREDUCE-1265. The task attempt error log prints the name of the
|
|
tasktracker machine. (Scott Chen via dhruba)
|
|
|
|
MAPREDUCE-1201. ProcfsBasedProcessTree collects CPU usage information.
|
|
(Scott Chen via dhruba)
|
|
|
|
MAPREDUCE-1326. fi tests don't use fi-site.xml (cos)
|
|
|
|
MAPREDUCE-1165. Replace non-portable function name with C99 equivalent.
|
|
(Allen Wittenauer via cdouglas)
|
|
|
|
MAPREDUCE-1331. Fixes a typo in a testcase (Devaraj Das)
|
|
|
|
MAPREDUCE-1293. AutoInputFormat doesn't work with non-default FileSystems.
|
|
(Andrew Hitchcock via tomwhite)
|
|
|
|
MAPREDUCE-1131. Using profilers other than hprof can cause JobClient to
|
|
report job failure. (Aaron Kimball via tomwhite)
|
|
|
|
MAPREDUCE-1155. Streaming tests swallow exceptions.
|
|
(Todd Lipcon via tomwhite)
|
|
|
|
MAPREDUCE-1212. Mapreduce contrib project ivy dependencies are not included
|
|
in binary target. (Aaron Kimball via tomwhite)
|
|
|
|
MAPREDUCE-1388. Move the HDFS RAID package from HDFS to MAPREDUCE.
|
|
(Eli Collins via dhruba)
|
|
|
|
MAPREDUCE-1322. Defines default value for staging directory to be user
|
|
based fixing a failing streaming test.
|
|
(Devaraj Das and Amareshwari Sriramadasu via yhemanth)
|
|
|
|
MAPREDUCE-899. Modified LinuxTaskController to check that task-controller
|
|
has right permissions and ownership before performing any actions.
|
|
(Amareshwari Sriramadasu via yhemanth)
|
|
|
|
MAPREDUCE-1443. DBInputFormat can leak connections.
|
|
(Aaron Kimball via tomwhite)
|
|
|
|
MAPREDUCE-1457. Fixes JobTracker to get the FileSystem object within
|
|
getStagingAreaDir within a privileged block. Fixes Child.java to use the
|
|
appropriate UGIs while getting the TaskUmbilicalProtocol proxy and
|
|
while executing the task. (Jakob Homan via ddas)
|
|
|
|
MAPREDUCE-1399. The archive command shows a null error message (nicholas
|
|
via mahadev)
|
|
|
|
MAPREDUCE-1305. Improve efficiency of distcp -delete. (Peter Romianowski
|
|
via cdouglas)
|
|
|
|
MAPREDUCE-1474. Update forrest documentation for Hadoop Archives. (Mahadev
|
|
Konar via cdouglas)
|
|
|
|
MAPREDUCE-1400. Use tr rather than sed to effect literal substitution in
|
|
the build script. (Allen Wittenauer via cdouglas)
|
|
|
|
MAPREDUCE-1358. Avoid false positives in OutputLogFilter. (Todd Lipcon via
|
|
cdouglas)
|
|
|
|
MAPREDUCE-1490. Fix a NullPointerException that could occur during
|
|
instantiation and initialization of the DistributedRaidFileSystem.
|
|
(Rodrigo Schmidt via dhruba)
|
|
|
|
MAPREDUCE-1476. Fix the M/R framework to not call commit for special
|
|
tasks like job setup/cleanup and task cleanup.
|
|
(Amareshwari Sriramadasu via yhemanth)
|
|
|
|
MAPREDUCE-1398. Fix TaskLauncher to stop waiting for slots on a TIP that
|
|
is killed / failed.
|
|
(Amareshwari Sriramadasu via yhemanth)
|
|
|
|
MAPREDUCE-1491. The parity files created by the RAID are combined
|
|
using Hadoop Archive Files (HAR). (Rodrigo Schmidt via dhruba)
|
|
|
|
MAPREDUCE-1378. URL encode link in jobhistory.jsp to avoid errors caused by
|
|
unescaped characters. (E. Sammer via cdouglas)
|
|
|
|
MAPREDUCE-1519. RaidNode fails to create new parity file
|
|
if an older version already exists. (Rodrigo Schmidt via dhruba)
|
|
|
|
MAPREDUCE-1537. Fixes a compilation problem in a testcase after commit
|
|
HDFS-984. (Jitendra Nath Pandey via ddas)
|
|
|
|
MAPREDUCE-1537. The patch makes the job client call the getDelegationToken
|
|
only when security is enabled. (Jitendra Nath Pandey via ddas)
|
|
|
|
MAPREDUCE-1510. RAID should regenerate parity files if they get deleted.
|
|
(Rodrigo Schmidt via dhruba)
|
|
|
|
MAPREDUCE-1421. Fix the LinuxTaskController tests failing on trunk after
|
|
the commit of MAPREDUCE-1385. (Amareshwari Sriramadasu via vinodkv)
|
|
|
|
MAPREDUCE-1520. Fix TestMiniMRLocalFS failure caused by regression in
|
|
getting user working dir. (Amareshwari Sriramadasu via cdouglas)
|
|
|
|
MAPREDUCE-1512. RAID uses HarFileSystem directly instead of
|
|
FileSystem.get (Rodrigo Schmidt via dhruba)
|
|
|
|
MAPREDUCE-1435. Fix symlink handling in task work directory when
|
|
cleaning up, essentially to avoid following links.
|
|
(Ravi Gummadi via yhemanth)
|
|
|
|
MAPREDUCE-1518. RaidNode does not run the deletion check on the
|
|
directory that stores the parity files. (Rodrigo Schmidt via dhruba)
|
|
|
|
MAPREDUCE-1573. TestStreamingAsDifferentUser fails if run as tt_user.
|
|
(Ravi Gummadi via vinodkv)
|
|
|
|
MAPREDUCE-927. Cleanup of task-logs should happen in TaskTracker instead
|
|
of the Child. (Amareshwari Sriramadasu via vinodkv)
|
|
|
|
MAPREDUCE-1578. Decouple HadoopArchives vesrion from HarFileSystem version.
|
|
(Rodrigo Schmidt via szetszwo)
|
|
|
|
MAPREDUCE-1422. Fix cleanup of localized job directory to work if files
|
|
with non-deletable permissions are created within it.
|
|
(Amar Kamat via yhemanth)
|
|
|
|
MAPREDUCE-1306. Randomize the arrival of heartbeat responses in Mumak.
|
|
(Tamas Sarlos via cdouglas)
|
|
|
|
MAPREDUCE-1579. archive: check and possibly replace the space charater
|
|
in source paths. (szetszwo)
|
|
|
|
MAPREDUCE-1536. DataDrivenDBInputFormat does not split date columns correctly.
|
|
(Aaron Kimball via enis)
|
|
|
|
MAPREDUCE-890. After HADOOP-4491, the user who started mapred system is
|
|
not able to run job. (Ravi Gummadi via vinodkv)
|
|
|
|
MAPREDUCE-1615. Fix compilation of TestSubmitJob. (cdouglas)
|
|
|
|
MAPREDUCE-1508. Protect against NPE in TestMultipleLevelCaching. (Aaron
|
|
Kimball via cdouglas)
|
|
|
|
MAPREDUCE-1497. Suppress spurious findbugs warning about IndexCache
|
|
synchronization. (Amareshwari Sriramadasu via cdouglas)
|
|
|
|
MAPREDUCE-1420. Fix TestTTResourceReporting failure. (Scott Chen via
|
|
cdouglas)
|
|
|
|
MAPREDUCE-1480. Correctly initialize child RecordReaders in
|
|
CombineFileInputFormat. (Aaron Kimball via cdouglas)
|
|
|
|
MAPREDUCE-1348. Fix block forensics packaging. (Tom White via cdouglas)
|
|
|
|
MAPREDUCE-1628. HarFileSystem shows incorrect replication numbers and
|
|
permissions. (szetszwo via mahadev)
|
|
|
|
MAPREDUCE-1602. Fix the error message for the case that src does not
|
|
exist. (szetszwo)
|
|
|
|
MAPREDUCE-1585. Create Hadoop Archives version 2 with filenames
|
|
URL-encoded (rodrigo via mahadev)
|
|
|
|
MAPREDUCE-1523. Sometimes rumen trace generator fails to extract the job
|
|
finish time. (dick king via mahadev)
|
|
|
|
MAPREDUCE-1635. ResourceEstimator does not work after MAPREDUCE-842.
|
|
(Amareshwari Sriramadasu via vinodkv)
|
|
|
|
MAPREDUCE-889. binary communication formats added to Streaming by
|
|
HADOOP-1722 should be documented. (Klaas Bosteels via tomwhite)
|
|
|
|
MAPREDUCE-1031. ant tar target doens't seem to compile tests in contrib
|
|
projects. (Aaron Kimball via tomwhite)
|
|
|
|
MAPREDUCE-1692. Removed unused testcase TestStreamedMerge.
|
|
(Sreekanth Ramakrishnan and Amareshwari Sriramadasu via yhemanth)
|
|
|
|
MAPREDUCE-1062. Fix ReliabilityTest to work with retired jobs. (Sreekanth
|
|
Ramakrishnan via cdouglas)
|
|
|
|
MAPREDUCE-1409. IOExceptions thrown from FIleOutputCommitter::abortTask
|
|
should cause the task to fail. (Amareshwari Sriramadasu via cdouglas)
|
|
|
|
MAPREDUCE-1695. Include capacity scheduler in findbugs and javadoc-dev
|
|
targets and also fix existing warnings. (Hong Tang via yhemanth)
|
|
|
|
MAPREDUCE-1494. Ensure TestJobDirCleanup verifies the correct paths.
|
|
(Amareshwari Sriramadasu via cdouglas)
|
|
|
|
MAPREDUCE-1622. Include missing slf4j dependencies. (cdouglas)
|
|
|
|
MAPREDUCE-1515. Accept java5.home from build.properties, not only from the
|
|
command line when building forrest docs. (Al Thompson via cdouglas)
|
|
|
|
MAPREDUCE-1618. Add missing javadoc to JobStatus::*JobAcls. (Amareshwari
|
|
Sriramadasu via cdouglas)
|
|
|
|
MAPREDUCE-1219. Remove job level metrics from jobtracker metrics to ease
|
|
undue load on jobtracker. (Sreekanth Ramakrishnan via sharad)
|
|
|
|
MAPREDUCE-1604. Add Forrest documentation for Job ACLs.
|
|
(Amareshwari Sriramadasu via yhemanth)
|
|
|
|
MAPREDUCE-1705. Archiving and Purging of HDFS parity files should
|
|
handle globbed policies accurately. (Rodrigo Schmidt via dhruba)
|
|
|
|
MAPREDUCE-1612. job conf file is not accessible from job history web page.
|
|
(Ravi Gummadi and Sreekanth Ramakrishnan via vinodkv)
|
|
|
|
MAPREDUCE-1397. NullPointerException observed during task failures.
|
|
(Amareshwari Sriramadasu via vinodkv)
|
|
|
|
MAPREDUCE-1728. Oracle timezone strings do not match Java.
|
|
(Aaron Kimball via tomwhite)
|
|
|
|
MAPREDUCE-1609. TaskTracker.localizeJob should not set permissions on
|
|
job log directory recursively. (Amareshwari Sriramadasu via vinodkv)
|
|
|
|
MAPREDUCE-1657. After task logs directory is deleted, tasklog servlet
|
|
displays wrong error message about job ACLs. (Ravi Gummadi via vinodkv)
|
|
|
|
MAPREDUCE-1727. TestJobACLs fails after HADOOP-6686. (Ravi Gummadi via vinodkv)
|
|
|
|
MAPREDUCE-1611. Refresh nodes and refresh queues doesnt work with service
|
|
authorization enabled. (Amar Kamat via vinodkv)
|
|
|
|
MAPREDUCE-1276. Correct flaws in the shuffle related to connection setup
|
|
and failure attribution. (Amareshwari Sriramadasu via cdouglas)
|
|
|
|
MAPREDUCE-1372. ConcurrentModificationException in JobInProgress.
|
|
(Dick King and Amareshwari Sriramadasu via tomwhite)
|
|
|
|
MAPREDUCE-118. Fix Job.getJobID(). (Amareshwari Sriramadasu via sharad)
|
|
|
|
MAPREDUCE-913. TaskRunner crashes with NPE resulting in held up slots,
|
|
UNINITIALIZED tasks and hung TaskTracker. (Amareshwari Sriramadasu and
|
|
Sreekanth Ramakrishnan via vinodkv)
|
|
|
|
MAPREDUCE-1725. Fix MapReduce API incompatibilities between 0.20 and 0.21.
|
|
(tomwhite)
|
|
|
|
MAPREDUCE-1606. TestJobACLs may timeout as there are no slots for launching
|
|
JOB_CLEANUP task. (Ravi Gummadi via vinodkv)
|
|
|
|
MAPREDUCE-1765. Correct streaming documentation for StreamXmlRecordReader.
|
|
(Corinne Chandel via amareshwari)
|
|
|
|
MAPREDUCE-1880. Fix BigDecimal.divide(..) in the pi example. (szetszwo)
|
|
|
|
MAPREDUCE-1885. Revert FileSystem create method that takes CreateFlags
|
|
(MapReduce part of HADOOP-6826). (Ravi Gummadi via tomwhite)
|
|
|
|
MAPREDUCE-1870. Harmonize MapReduce JAR library versions with Common and
|
|
HDFS. (tomwhite)
|
|
|
|
MAPREDUCE-1791. Remote cluster control functionality needs JavaDocs
|
|
improvement (Konstantin Boudnik)
|
|
|
|
MAPREDUCE-1942. 'compile-fault-inject' should never be called directly.
|
|
(Konstantin Boudnik)
|
|
|
|
MAPREDUCE-1876. Fixes TaskAttemptStartedEvent to correctly log event type
|
|
for all task types. (Amar Kamat via amareshwari)
|
|
|
|
MAPREDUCE-1926. MapReduce distribution is missing build-utils.xml.
|
|
(tomwhite)
|
|
|
|
MAPREDUCE-2012. Some contrib tests fail in branch 0.21 and trunk.
|
|
(Amareshwari Sriramadasu via tomwhite)
|
|
|
|
MAPREDUCE-1980. Fixes TaskAttemptUnsuccessfulCompletionEvent and
|
|
TaskAttemptFinishedEvent to correctly log event type for all task types.
|
|
(Amar Kamat via amareshwari)
|
|
|
|
MAPREDUCE-1856. Extract a subset of tests for smoke (DOA) validation (cos)
|
|
|