big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
12,508 Commits 2 Branches 2 Tags 264 MiB
7f393a6f61
Commit Graph

2259 Commits

Author SHA1 Message Date
Tsuyoshi Ozawa
96677bef00 YARN-3573. MiniMRYarnCluster constructor that starts the timeline server using a boolean should be marked deprecated. Contributed by Brahma Reddy Battula. 2015-10-27 12:35:01 +09:00
Junping Du
eb6379ca25 MAPREDUCE-6508. TestNetworkedJob fails consistently due to delegation token changes on RM. Contributed by Akira AJISAKA 2015-10-23 10:05:46 -07:00
Jason Lowe
ee9b80acf8 MAPREDUCE-6518. Set SO_KEEPALIVE on shuffle connections. Contributed by Chang Li 2015-10-21 21:44:43 +00:00
Jason Lowe
cb26cd4bee MAPREDUCE-6489. Fail fast rogue tasks that write too much to local disk. Contributed by Maysam Yabandeh 2015-10-21 14:01:23 +00:00
Robert Kanter
0c4af0f998 MAPREDUCE-6495. Docs for archive-logs tool (rkanter) 2015-10-20 17:34:34 -07:00
Sangjin Lee
58590fef49 YARN-2556. Tool to measure the performance of the timeline server (Chang Li via sjlee) 2015-10-16 16:53:20 -07:00
Haohui Mai
da8441d0fe HDFS-9167. Update pom.xml in other modules to depend on hdfs-client instead of hdfs. Contributed by Mingliang Liu. 2015-10-13 15:24:24 -07:00
Karthik Kambatla
4aa9b3e75c MAPREDUCE-6302. Incorrect headroom can lead to a deadlock between map and reduce allocations. (kasha) 2015-10-09 07:37:39 -07:00
Jason Lowe
a0bca2b5ad YARN-261. Ability to fail AM attempts. Contributed by Andrey Klochkov and Rohith Sharma K S 2015-10-09 14:17:38 +00:00
Akira Ajisaka
584cf3bb35 MAPREDUCE-6479. Add missing mapred job command options in mapreduce document. Contributed by nijel. 2015-10-09 10:54:12 +09:00
Robert Kanter
1bca1bb517 MAPREDUCE-6503. archive-logs tool should use HADOOP_PREFIX instead of HADOOP_HOME (rkanter) 2015-10-06 16:49:50 -07:00
Xuan
8f08532bde YARN-1897. CLI and core support for signal container functionality. Contributed by Ming Ma 2015-10-02 18:50:47 -07:00
rohithsharmaks
439f43ad3d MAPREDUCE-6485. Create a new task attempt with failed map task priority if in-progress attempts are unassigned. (Xianyin Xin via rohithsharmaks) 2015-10-02 20:34:34 +05:30
Akira Ajisaka
195793c6f3 MAPREDUCE-6497. Fix wrong value of JOB_FINISHED event in JobHistoryEventHandler. Contributed by Shinichi Yamashita. 2015-10-01 10:22:45 +02:00
Robert Kanter
5db371f52f MAPREDUCE-6494. Permission issue when running archive-logs tool as different users (rkanter) 2015-09-30 17:33:53 -07:00
Jason Lowe
ab11085b81 MAPREDUCE-6492. AsyncDispatcher exit with NPE on TaskAttemptImpl#sendJHStartEventForAssignedFailTask. Contributed by Bibin A Chundatt 2015-09-28 22:12:03 +00:00
Harsh J
66dad854c0 MAPREDUCE-6471. Document distcp incremental copy. Contributed by Neelesh Srinivas Salian. 2015-09-28 13:12:43 +05:30
Robert Kanter
d3c49e7662 MAPREDUCE-6480. archive-logs tool may miss applications (rkanter) 2015-09-25 15:02:42 -07:00
Zhihai Xu
97a08807ec MAPREDUCE-6484. Yarn Client uses local address instead of RM address as token renewer in a secure cluster when RM HA is enabled. Contributed by Zhihai Xu 2015-09-24 08:12:07 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
1f707ecffd Release process for 2.6.1: Set the release date for 2.6.1 2015-09-23 15:10:50 -07:00
Jian He
89cab1ba5f YARN-1651. CapacityScheduler side changes to support container resize. Contributed by Wangda Tan 2015-09-23 13:29:38 -07:00
Jian He
83a18add10 YARN-1449. AM-NM protocol changes to support container resizing. Contributed by Meng Ding & Wangda Tan) 2015-09-23 13:29:36 -07:00
Jian He
dfe2cb849f YARN-3866. AM-RM protocol changes to support container resizing. Contributed by Meng Ding 2015-09-23 13:29:36 -07:00
Harsh J
cc2b473990 MAPREDUCE-5045. UtilTest#isCygwin method appears to be unused. Contributed by Neelesh Srinivas Salian. 2015-09-22 21:38:06 +05:30
Tsuyoshi Ozawa
dfd807afab HADOOP-12428. Fix inconsistency between log-level guards and statements. Contributed by Jagadesh Kiran N and Jackie Chang. 2015-09-22 12:54:29 +09:00
Zhihai Xu
e3ace31e2b MAPREDUCE-6460. TestRMContainerAllocator.testAttemptNotFoundCausesRMCommunicatorException fails. Contributed by Zhihai Xu. 2015-09-19 00:10:06 -07:00
Haohui Mai
602335dfe6 MAPREDUCE-6483. Replace deprecated method NameNode.getUri() with DFSUtilClient.getNNUri() in TestMRCredentials. Contributed by Mingliang Liu. 2015-09-18 13:36:09 -07:00
Jason Lowe
f97dc034fe Update CHANGES.txt to reflect commit of MR-5982 to 2.7.2 2015-09-18 19:24:10 +00:00
Wangda Tan
372ad270a0 MAPREDUCE-6478. Add an option to skip cleanupJob stage or ignore cleanup failure during commitJob. (Junping Du via wangda) 2015-09-18 10:10:32 -07:00
Jason Lowe
288c8854a6 Update CHANGES.txt to reflect LineRecordReader fixes committed to 2.7.2 2015-09-18 15:14:18 +00:00
Jason Lowe
ee4ee6af6a MAPREDUCE-5982. Task attempts that fail from the ASSIGNED state can disappear. Contributed by Chang Li 2015-09-17 21:37:39 +00:00
Jason Lowe
3f82f582e5 MAPREDUCE-5002. AM could potentially allocate a reduce container to a map attempt. Contributed by Chang Li 2015-09-17 18:17:29 +00:00
Jason Lowe
58d1a02b8d MAPREDUCE-6481. LineRecordReader may give incomplete record and wrong position/key information for uncompressed input sometimes. Contributed by Zhihai Xu 2015-09-17 14:30:18 +00:00
Haohui Mai
941001bfa6 MAPREDUCE-6477. Replace usage of deprecated NameNode.DEFAULT_PORT in TestFileSystem. Contributed by Mingliang Liu. 2015-09-16 13:06:26 -07:00
Jason Lowe
62943b8e3a MAPREDUCE-6472. MapReduce AM should have java.io.tmpdir=./tmp to be consistent with tasks. Contributed by Naganarasimha G R 2015-09-15 19:43:20 +00:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
fbbb7ff1ed Updating all CHANGES.txt files to move entires from future releases into 2.6.1 
section given the large number of backports to 2.6.1.
 2015-09-10 13:45:57 -07:00
Jason Lowe
8e615588d5 MAPREDUCE-6474. ShuffleHandler can possibly exhaust nodemanager file descriptors. Contributed by Kuhu Shukla 2015-09-10 16:00:17 +00:00
Karthik Kambatla
119cc75e7e MAPREDUCE-6415. Create a tool to combine aggregated logs into HAR files. (Robert Kanter via kasha) 2015-09-09 17:45:19 -07:00
Tsuyoshi Ozawa
9b685773ec MAPREDUCE-6442. Stack trace is missing when error occurs in client protocol provider's constructor Contributed by Chang Li. 2015-09-05 11:21:03 +09:00
Andrew Wang
e1feaf6db0 HADOOP-12358. Add -safely flag to rm to prompt when deleting many files. Contributed by Xiaoyu Yao. 2015-09-04 13:42:55 -07:00
Akira Ajisaka
00804e2457 Revert "MAPREDUCE-6363. [NNBench] Lease mismatch error when running with multiple mappers. Contributed by Brahma Reddy Battula." 
This reverts commit 75a2560e51.
 2015-09-02 12:22:16 +09:00
Zhihai Xu
cbb249534a MAPREDUCE-6452. NPE when intermediate encrypt enabled for LocalRunner. Contributed by Zhihai Xu 2015-08-28 12:13:23 -07:00
Andrew Wang
50fa383704 Revert "MAPREDUCE-6455. Unable to use surefire > 2.18. (Charlie Helin via kasha)" 
This reverts commit 61bf9cae6f.
 2015-08-27 10:15:19 -07:00
Jian He
57c7ae1aff YARN-4014. Support user cli interface in for Application Priority. Contributed by Rohith Sharma K S 2015-08-24 20:36:44 -07:00
Karthik Kambatla
61bf9cae6f MAPREDUCE-6455. Unable to use surefire > 2.18. (Charlie Helin via kasha) 2015-08-22 00:39:04 -07:00
Akira Ajisaka
2ba90c93d7 MAPREDUCE-6357. MultipleOutputs.write() API should document that output committing is not utilized when input path is absolute. Contributed by Dustin Cote. 2015-08-21 10:41:54 +09:00
Vinod Kumar Vavilapalli
33474939a0 Creating 2.6.2 entries in CHANGES.txt files. 2015-08-20 17:21:06 -07:00
Karthik Kambatla
8dfec7a197 MAPREDUCE-6439. AM may fail instead of retrying if RM shuts down during the allocate call. (Anubhav Dhoot via kasha) 2015-08-15 00:52:11 -07:00
Karthik Kambatla
27d24f96ab MAPREDUCE-5817. Mappers get rescheduled on node transition even after all reducers are completed. (Sangjin Lee via kasha) 2015-08-14 12:29:50 -07:00
Allen Wittenauer
fb1be0b310 MAPREDUCE-6257. Document encrypted spills (Bibin A Chundatt via aw) 2015-08-06 10:13:51 -07:00
Junping Du
e73a928a63 MAPREDUCE-6443. Add JvmPauseMonitor to JobHistoryServer. Contributed by Robert Kanter. 
(cherry picked from commit c8469357bad481ca8e341050553a5ae6d98bf8e5)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2015-08-06 07:03:46 -07:00
Colin Patrick Mccabe
21d10ccc6e HADOOP-7824. NativeIO.java flags and identifiers must be set correctly for each platform, not hardcoded to their Linux values (Martin Walsh via Colin P. McCabe) 2015-07-31 15:01:34 -07:00
Allen Wittenauer
666cafca8d HADOOP-12249. pull argument parsing into a function (aw) 2015-07-31 14:32:21 -07:00
Jason Lowe
d0e0ba8010 MAPREDUCE-6394. Speed up Task processing loop in HsTasksBlock#render(). Contributed by Ray Chiang 2015-07-31 18:17:54 +00:00
Zhihai Xu
93d50b7824 MAPREDUCE-6433. launchTime may be negative. Contributed by Zhihai Xu 2015-07-30 23:07:31 -07:00
Allen Wittenauer
cf74772064 MAPREDUCE-5801. Uber mode's log message is missing a vcore reason (Steven Wong via aw) 2015-07-21 10:58:52 -07:00
Allen Wittenauer
ee36f4f9b8 HADOOP-10979. Auto-entries in hadoop_usage (aw) 2015-07-16 16:58:11 -07:00
Chris Douglas
f4ccdb11dc MAPREDUCE-6427. Fix typo in JobHistoryEventHandler. Contributed by Ray Chiang 2015-07-14 14:55:43 -07:00
Akira Ajisaka
19295b36d9 YARN-3381. Fix typo InvalidStateTransitonException. Contributed by Brahma Reddy Battula. 2015-07-13 17:52:13 +09:00
Akira Ajisaka
e04faf8e46 MAPREDUCE-6421. Fix findbugs warning in RMContainerAllocator.reduceNodeLabelExpression. Contributed by Brahma Reddy Battula. 2015-07-13 14:29:14 +09:00
Devaraj K
fffb15bb43 MAPREDUCE-6426. TestShuffleHandler#testGetMapOutputInfo is failing. 
Contributed by zhihai xu.
 2015-07-09 15:00:26 +05:30
Chris Douglas
7e2fe8c9f2 MAPREDUCE-6038. A boolean may be set error in the Word Count v2.0 in MapReduce Tutorial. Contributed by Tsuyoshi Ozawa 2015-07-07 10:27:04 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
bf89ddb9b8 Release process for 2.7.1: Set the release date for 2.7.1. 2015-07-06 16:39:59 -07:00
Devaraj K
bff67dfe2f MAPREDUCE-6425. ShuffleHandler passes wrong "base" parameter to 
getMapOutputInfo if mapId is not in the cache. Contributed by zhihai xu.
 2015-07-06 13:46:37 +05:30
Jason Lowe
eac1d18943 MAPREDUCE-6418. MRApp should not shutdown LogManager during shutdown. Contributed by Chang Li 2015-07-01 18:01:22 +00:00
Jason Lowe
826715622e MAPREDUCE-6420. Interrupted Exception in LocalContainerLauncher should be logged in warn/info level. Contributed by Chang Li 2015-07-01 17:50:02 +00:00
Jason Lowe
2ac87df578 MAPREDUCE-6376. Add avro binary support for jhist files. Contributed by Ray Chiang 2015-07-01 16:00:03 +00:00
Robert Kanter
3a72bfd082 MAPREDUCE-6121. JobResourceUpdater#compareFs() doesn't handle HA namespaces (rchiang via rkanter) 2015-06-30 16:49:27 -07:00
Colin Patrick Mccabe
9c63825f10 MAPREDUCE-6407. Migrate MAPREDUCE native build to new CMake framework (Alan Burlison via Colin P. McCabe) 2015-06-30 16:17:06 -07:00
Jason Lowe
b6ba56457c MAPREDUCE-6384. Add the last reporting reducer info for too many fetch failure diagnostics. Contributed by Chang Li 2015-06-30 21:22:30 +00:00
Vinod Kumar Vavilapalli
aad6a7d5db Adding release 2.7.2 to CHANGES.txt. 2015-06-28 16:29:12 -07:00
Jason Lowe
aa5b15b03b MAPREDUCE-6413. TestLocalJobSubmission is failing with unknown host. Contributed by zhihai xu 2015-06-25 19:50:07 +00:00
Jason Lowe
72d08a0e41 MAPREDUCE-6400. Multiple shuffle transfer fails because input is closed too early. Contributed by Brahma Reddy Battula, Akira AJISAKA, and Gera Shegalov. 2015-06-24 15:29:11 +00:00
Vinod Kumar Vavilapalli
d481684c7c MAPREDUCE-6410. Fixed MapReduce JobHistory server to use the right (login) UGI to refresh log and cleaner settings. Contributed by Varun Saxena. 2015-06-23 11:39:55 -07:00
Devaraj K
41ae7768eb MAPREDUCE-6406. Update 
FileOutputCommitter.FILEOUTPUTCOMMITTER_ALGORITHM_VERSION_DEFAULT to match
mapred-default.xml. Contributed by Ray Chiang.
 2015-06-23 10:23:44 +05:30
Jason Lowe
077250d8d7 MAPREDUCE-5948. org.apache.hadoop.mapred.LineRecordReader does not handle multibyte record delimiters well. Contributed by Vinayakumar B, Rushabh Shah, and Akira AJISAKA 2015-06-22 21:59:20 +00:00
Gera Shegalov
11ac848207 MAPREDUCE-6408. Queue name and user name should be printed on the job page. (Siqi Li via gera) 2015-06-22 14:40:32 -07:00
Akira Ajisaka
445b13240c MAPREDUCE-6403. Fix typo in the usage of NNBench. Contributed by Jagadesh Kiran N. 2015-06-22 15:32:39 +09:00
Gera Shegalov
055cd5a9a1 MAPREDUCE-6305. AM/Task log page should be able to link back to the job. (Siqi Li via gera) 2015-06-20 12:38:36 -07:00
Gera Shegalov
63d40d5286 MAPREDUCE-6316. Task Attempt List entries should link to the task overview. (Siqi Li via gera) 2015-06-20 11:38:21 -07:00
Devaraj K
311a4179cc MAPREDUCE-6405. NullPointerException in App Attempts page. Contributed by 
Siqi Li and Gera Shegalov.
 2015-06-20 10:35:04 +05:30
Gera Shegalov
10107243be MAPREDUCE-6395. Improve the commit failure messages in MRAppMaster recovery. (Brahma Reddy Battula via gera) 2015-06-19 02:22:59 -07:00
Devaraj K
1babe50a2c MAPREDUCE-6373. The logger reports total input paths but it is referring 
to input files. Contributed by Bibin A Chundatt.
 2015-06-18 11:42:22 +05:30
Akira Ajisaka
85cc644f92 MAPREDUCE-6396. TestPipeApplication fails by NullPointerException. Contributed by Brahma Reddy Battula. 2015-06-15 15:28:31 -07:00
Akira Ajisaka
75a2560e51 MAPREDUCE-6363. [NNBench] Lease mismatch error when running with multiple mappers. Contributed by Brahma Reddy Battula. 2015-06-15 14:52:26 -07:00
Devaraj K
c7729efee8 MAPREDUCE-6389. Fix BaileyBorweinPlouffe CLI usage message. Contributed by 
Brahma Reddy Battula.
 2015-06-10 21:05:10 +05:30
Devaraj K
6785661e55 MAPREDUCE-6350. JobHistory doesn't support fully-functional search. 
Contributed by Siqi Li.
 2015-06-10 12:40:29 +05:30
cnauroth
025a3a8be0 MAPREDUCE-6392. Document mapred class path options. Contributed by Brahma Reddy Battula. 2015-06-08 16:22:03 -07:00
Tsuyoshi Ozawa
6afe20a7a4 MAPREDUCE-6388. Remove deprecation warnings from JobHistoryServer classes. Contributed by Ray Chiang. 2015-06-08 15:06:10 -07:00
Jason Lowe
b3ffa87003 MAPREDUCE-6354. ShuffleHandler should be able to log shuffle connections. Contributed by Chang Li 2015-06-05 22:38:31 +00:00
Karthik Kambatla
6786daab33 MAPREDUCE-6387. Serialize the recently added Task#encryptedSpillKey field at the end. (Arun Suresh via kasha) 2015-06-05 09:14:06 -07:00
Devaraj K
09b8241e6e MAPREDUCE-6383. Pi job (QuasiMonteCarlo) should not try to read the 
results file if its job fails. Contributed by Harsh J.
 2015-06-05 21:22:47 +05:30
Devaraj K
790a861766 MAPREDUCE-6377. JHS sorting on state column not working in webUi. 
Contributed by zhihai xu.
 2015-06-05 15:50:16 +05:30
Akira Ajisaka
dbed757cba HADOOP-12058. Fix dead links to DistCp and Hadoop Archives pages. Contributed by Kazuho Fujii. 2015-06-05 01:45:34 +09:00
Gera Shegalov
e8302071ff MAPREDUCE-6382. Don't escape HTML links in Diagnostics in JHS job overview. (Siqi Li via gera) 2015-06-03 23:46:06 -07:00
Robert Kanter
cc70df98e7 MAPREDUCE-5965. Hadoop streaming throws error if list of input files is high. Error is: "error=7, Argument list too long at if number of input file is high" (wilfreds via rkanter) 2015-06-03 18:41:45 -07:00
Gera Shegalov
d90c13e2da MAPREDUCE-6174. Combine common stream code into parent class for InMemoryMapOutput and OnDiskMapOutput. (Eric Payne via gera) 2015-06-03 16:50:26 -07:00
Zhijie Shen
bc85959edd Revert "YARN-1462. Made RM write application tags to timeline server and exposed them to users via generic history web UI and REST API. Contributed by Xuan Gong." 
This reverts commit 4a9ec1a824.
 2015-06-03 14:15:56 -07:00
Jason Lowe
107da29ff9 MAPREDUCE-6374. Distributed Cache File visibility should check permission of full path. Contributed by Chang Li 2015-06-03 20:19:27 +00:00
Zhijie Shen
4a9ec1a824 YARN-1462. Made RM write application tags to timeline server and exposed them to users via generic history web UI and REST API. Contributed by Xuan Gong. 2015-05-30 21:05:36 -07:00