big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
13,938 Commits 2 Branches 2 Tags 264 MiB
8ef546c1ee
Commit Graph

1590 Commits

Author SHA1 Message Date
Akira Ajisaka
4ee4e5ca2b MAPREDUCE-6526. Remove usage of metrics v1 from hadoop-mapreduce. (aajisaka) 2016-05-03 10:46:11 +09:00
Jing Zhao
63e5412f1a HDFS-9427. HDFS should not default to ephemeral ports. Contributed by Xiaobing Zhou. 2016-04-22 15:14:40 -07:00
Harsh J
7da5847cf1 MAPREDUCE-2398. MRBench: setting the baseDir parameter has no effect. Contributed by Michael Noll and Wilfred Spiegelenburg. 2016-04-21 17:53:51 +05:30
Jian He
1e48eefe58 MAPREDUCE-6680. JHS UserLogDir scan algorithm sometime could skip directory with update in CloudFS (Azure FileSystem, S3, etc. Contributed by Junping Du 2016-04-20 19:02:10 -07:00
Xuan
e6c0742012 YARN-4965. Distributed shell AM failed due to ClientHandlerException thrown by jersey. Contributed by Junping Du 2016-04-16 19:39:18 -07:00
Eric Payne
cc8b83a8e8 MAPREDUCE-6649. getFailureInfo not returning any failure info. Contributed by Eric Badger 2016-04-16 22:07:27 +00:00
Wangda Tan
8b2880c0b6 MAPREDUCE-6513. MR job got hanged forever when one NM unstable for some time. (Varun Saxena via wangda) 2016-04-14 11:00:53 -07:00
Eric Payne
1fec06e037 MAPREDUCE-6633. AM should retry map attempts if the reduce task encounters commpression related errors. Contributed by Rushabh Shah 2016-04-09 16:51:57 +00:00
Robert Kanter
3be1ab485f MAPREDUCE-6647. MR usage counters use the resources requested instead of the resources allocated (haibochen via rkanter) 2016-04-06 17:15:43 -07:00
Junping Du
de96d7c88a MAPREDUCE-6670. TestJobListCache#testEviction sometimes fails on Windows with timeout. Contributed by Gergely Novák. 2016-04-06 08:32:35 -07:00
Allen Wittenauer
0a74610d1c HADOOP-11393. Revert HADOOP_PREFIX, go back to HADOOP_HOME (aw) 2016-03-31 07:51:05 -07:00
Akira Ajisaka
690d8a368d MAPREDUCE-6663. [NNBench] Refactor nnbench as a Tool implementation. Contributed by Brahma Reddy Battula. 2016-03-30 11:42:54 +09:00
Akira Ajisaka
0050fa5f1c MAPREDUCE-6543. Migrate MR client test cases part 2. Contributed by Dustin Cote. 2016-03-29 18:17:52 +09:00
Rohith Sharma K S
0ef8bbfd87 MAPREDUCE-6662. Clear ASF Warnings on test data files. Contributed by Vinayakumar B 2016-03-29 10:32:21 +08:00
Junping Du
90fcb16c0f MAPREDUCE-6655. Fix a typo (STRICT_IE6) in Encrypted Shuffle. Contributed by Wei-Chiu Chuang. 2016-03-28 05:30:42 -07:00
Akira Ajisaka
115be193df Revert "MAPREDUCE-6543. Migrate MR client test cases part 2. Contributed by Dustin Cote." 
This reverts commit 2c268cc936.
 2016-03-28 16:05:35 +09:00
Jason Lowe
3f622a143c MAPREDUCE-6535. TaskID default constructor results in NPE on toString(). Contributed by Daniel Templeton 2016-03-25 20:04:32 +00:00
Akira Ajisaka
2c268cc936 MAPREDUCE-6543. Migrate MR client test cases part 2. Contributed by Dustin Cote. 2016-03-25 15:17:27 +09:00
Akira Ajisaka
368c77376a MAPREDUCE-6656. [NNBench] OP_DELETE operation isn't working after MAPREDUCE-6363. Contributed by J.Andreina. 2016-03-23 23:22:43 +09:00
Akira Ajisaka
0bfe5a0167 MAPREDUCE-6580. Test failure: TestMRJobsWithProfiler. Contributed by Eric Badger. 2016-03-23 13:18:06 +09:00
Li Lu
57ead18a85 MAPREDUCE-6110. JobHistoryServer CLI throws NullPointerException with job ids that do not exist. (Kai Sasaki via gtcarrera9) 2016-03-22 14:27:32 -07:00
Karthik Kambatla
7fae4c68e6 YARN-4732. *ProcessTree classes have too many whitespace issues 2016-03-20 10:47:38 -07:00
Wangda Tan
6529c87551 MAPREDUCE-6579. JobStatus#getFailureInfo should not output diagnostic information when the job is running. (Akira AJISAKA via wangda) 2016-03-15 17:28:40 -07:00
Akira Ajisaka
247a790609 MAPREDUCE-6520. Migrate MR Client test cases part 1. Contributed by Dustin Cote. 2016-03-11 22:51:20 +09:00
Akira Ajisaka
6876b9f9f9 Revert "MAPREDUCE-6520. Migrate MR Client test cases part 1." 
This reverts commit ef68b44195.
 2016-03-11 22:50:57 +09:00
Akira Ajisaka
ef68b44195 MAPREDUCE-6520. Migrate MR Client test cases part 1. 2016-03-11 22:20:55 +09:00
Varun Vasudev
e51a8c1056 YARN-4737. Add CSRF filter support in YARN. Contributed by Jonathan Maron. 2016-03-07 15:26:44 +05:30
Harsh J
4f9fe3ac03 MAPREDUCE-6648. Add yarn.app.mapreduce.am.log.level to mapred-default.xml (harsh) 2016-03-07 13:09:57 +05:30
Masatake Iwasaki
cbd31328a6 HADOOP-12470. In-page TOC of documentation should be automatically generated by doxia macro (iwasakims) 2016-03-04 14:11:36 +09:00
Robert Kanter
ff0ee84d77 MAPREDUCE-4785. TestMRApp occasionally fails (haibochen via rkanter) 2016-03-03 16:38:07 -08:00
Robert Kanter
0f72da7e28 MAPREDUCE-6622. Add capability to set JHS job cache to a task-based limit (rchiang via rkanter) 2016-02-26 17:57:17 -08:00
Robert Kanter
c4d4df8de0 YARN-4701. When task logs are not available, port 8041 is referenced instead of port 8042 (haibochen via rkanter) 2016-02-25 12:48:02 -08:00
Robert Kanter
4343a4cf77 MAPREDUCE-6640. mapred job -history command should be able to take Job ID (rkanter) 2016-02-23 17:37:49 -08:00
Varun Vasudev
c6f2d761d5 MAPREDUCE-6635. Unsafe long to int conversion in UncompressedSplitLineReader and IndexOutOfBoundsException. Contributed by Junping Du. 2016-02-23 13:05:18 +05:30
Tsuyoshi Ozawa
b68901d7dd YARN-4708. Missing default mapper type in TimelineServer performance test tool usage. Contributed by Kai Sasaki. 2016-02-22 07:54:53 +09:00
Robert Kanter
6eae4337d1 MAPREDUCE-6613. Change mapreduce.jobhistory.jhist.format default from json to binary (rchiang via rkanter) 2016-02-19 17:15:54 -08:00
Andrew Wang
37577852ba MAPREDUCE-6637. Testcase Failure : TestFileInputFormat.testSplitLocationInfo. Contributed by Brahma Reddy Battula. 2016-02-19 16:33:17 -08:00
Robert Kanter
8eee59ce6b MAPREDUCE-6627. Add machine-readable output to mapred job -history command (rkanter) 2016-02-18 17:58:26 -08:00
Robert Kanter
a365a3941c HADOOP-12817. Enable TLS v1.1 and 1.2 (rkanter) 2016-02-18 11:09:50 -08:00
Varun Vasudev
2440671a11 MAPREDUCE-6634. Log uncaught exceptions/errors in various thread pools in mapreduce. Contributed by Sidharta Seethana. 2016-02-18 14:18:36 +05:30
Tsuyoshi Ozawa
453e7e0d10 MAPREDUCE-6341. addendum patch to fix typos. Contributed by John Michael Luy. 2016-02-17 14:17:32 +09:00
Akira Ajisaka
5e565ce7a7 Revert "HDFS-9686. Remove useless boxing/unboxing code. Contributed by Kousuke Saruta." 
This reverts commit fe124da5ff.
 2016-02-10 23:43:33 +09:00
Akira Ajisaka
a0b1f10a30 MAPREDUCE-6626. Reuse ObjectMapper instance in MapReduce. Contributed by Lin Yiqun. 2016-02-10 03:05:07 +09:00
Akira Ajisaka
fe124da5ff HDFS-9686. Remove useless boxing/unboxing code. Contributed by Kousuke Saruta. 2016-02-06 19:31:01 +09:00
Robert Kanter
ccbba4a4de MAPREDUCE-6620. Jobs that did not start are shown as starting in 1969 in the JHS web UI (haibochen via rkanter) 2016-02-02 15:19:35 -08:00
Jason Lowe
43e669b22d MAPREDUCE-6621. Memory Leak in JobClient#submitJobInternal(). Contributed by Xuan Gong 2016-02-02 19:30:45 +00:00
Wangda Tan
9875325d5c YARN-4340. Add list API to reservation system. (Sean Po via wangda) 2016-02-02 10:17:33 +08:00
Jason Lowe
59a212b6e1 MAPREDUCE-6618. YarnClientProtocolProvider leaking the YarnClient thread. Contributed by Xuan Gong 2016-02-01 16:05:06 +00:00
Akira Ajisaka
df99ea8a92 MAPREDUCE-6616. Fail to create jobhistory file if there are some multibyte characters in the job name. Contributed by Kousuke Saruta. 2016-01-29 16:20:29 +09:00
Robert Kanter
dca0dc8ac2 MAPREDUCE-6431. JobClient should be an AutoClosable (haibochen via rkanter) 2016-01-27 17:11:07 -08:00
Jian He
d0d7c22168 MAPREDUCE-6610. JobHistoryEventHandler should not swallow timeline response. Contributed by Li Lu 2016-01-25 23:01:03 -08:00
Akira Ajisaka
f5c8c857be MAPREDUCE-6605. Fix typos mapreduce.map.skip.proc.count.autoincr and mapreduce.reduce.skip.proc.count.autoincr in mapred-default.xml. Contributed by Kai Sasaki. 2016-01-22 18:35:43 +09:00
Karthik Kambatla
d40859fab1 YARN-4526. Make SystemClock singleton so AppSchedulingInfo could use it. (kasha) 2016-01-18 10:58:14 +01:00
Colin Patrick Mccabe
5af5f9575a HADOOP-12714. Fix hadoop-mapreduce-client-nativetask unit test which fails because it is not able to open the "glibc bug spill" file. (cmccabe) 2016-01-15 13:53:05 -08:00
Jason Lowe
9fbd579ab5 MAPREDUCE-6554. MRAppMaster servicestart failing with NPE in MRAppMaster#parsePreviousJobHistory. Contributed by Bibin A Chundatt 2016-01-15 16:52:53 +00:00
Colin Patrick Mccabe
b1ed28fa77 HADOOP-8887. Use a Maven plugin to build the native code using CMake (cmccabe) 2016-01-14 11:02:34 -08:00
Akira Ajisaka
5cc44d18aa MAPREDUCE-6601. Fix typo in Job#setUseNewAPI. Contributed by Kai Sasaki. 2016-01-15 00:44:24 +09:00
Akira Ajisaka
7b0964f354 MAPREDUCE-6363. [NNBench] Lease mismatch error when running with multiple mappers. Contributed by Vlad Sharanhovich and Bibin A Chundatt. 2016-01-14 10:40:22 +09:00
Jason Lowe
f657b54281 MAPREDUCE-6473. Job submission can take a long time during Cluster initialization. Contributed by Kuhu Shukla 2016-01-12 23:56:23 +00:00
Wangda Tan
9e792da014 YARN-4582. Label-related invalid resource request exception should be able to properly handled by application. (Bibin A Chundatt via wangda) 2016-01-12 12:53:31 +08:00
Junping Du
8c1adeaa26 MAPREDUCE-6068. Illegal progress value warnings in map tasks. Contributed by Binglin Chang. 2016-01-11 06:12:19 -08:00
Sangjin Lee
f6f16118d3 MAPREDUCE-6577. MR AM unable to load native library without MR_AM_ADMIN_USER_ENV set (sjlee) 2016-01-05 15:22:50 -08:00
Akira Ajisaka
7dafee11d8 MAPREDUCE-6593. TestJobHistoryEventHandler.testTimelineEventHandling fails on trunk because of NPE. Contributed by Naganarasimha G R. 2016-01-04 16:30:55 +09:00
Akira Ajisaka
2a59d01503 MAPREDUCE-6589. TestTaskLog outputs a log under directory other than target/test-dir. (aajisaka) 2015-12-30 01:33:47 +09:00
Akira Ajisaka
bee542817d MAPREDUCE-6584. Remove trailing whitespaces from mapred-default.xml. (aajisaka) 2015-12-30 01:03:12 +09:00
Wangda Tan
32b3dc11eb MAPREDUCE-6574. MR AM should print host of failed tasks. (Mohammad Shahid Khan via wangda) 2015-12-28 11:03:23 -08:00
Rohith Sharma K S
a308e86621 MAPREDUCE-6419. JobHistoryServer doesn't sort properly based on Job ID when Job id's exceed 9999. (Mohammad Shahid Khan via rohithsharmaks) 2015-12-24 11:55:42 +05:30
Akira Ajisaka
7995a6ea4d MAPREDUCE-6583. Clarify confusing sentence in MapReduce tutorial document. Contributed by Kai Sasaki. 2015-12-21 00:16:14 +09:00
Jian He
85c2466048 YARN-4164. Changed updateApplicationPriority API to return the updated application priority. Contributed by Rohith Sharma K S 2015-12-18 14:13:48 -08:00
Xuan
d4e766de93 MAPREDUCE-6566. Add retry support to mapreduce CLI tool. Contributed by Varun Vasudev 2015-12-16 15:26:03 -08:00
Vinod Kumar Vavilapalli
60fe6d53d9 Revert "MAPREDUCE-6566. Add retry support to mapreduce CLI tool. Contributed by Varun Vasudev" 
This reverts commit fc470840a0.
 2015-12-16 15:11:38 -08:00
Zhihai Xu
5b7078d069 MAPREDUCE-6436. JobHistory cache issue. Contributed by Kai Sasaki 2015-12-15 00:58:23 -08:00
Xuan
fc470840a0 MAPREDUCE-6566. Add retry support to mapreduce CLI tool. Contributed by Varun Vasudev 2015-12-07 14:15:14 -08:00
Steve Loughran
65f395226b HADOOP-12321. Make JvmPauseMonitor an AbstractService. (Sunil G via Stevel) [includes HDFS-8947 MAPREDUCE-6462 and YARN-4072] 2015-12-06 17:43:35 +00:00
Jian He
9f77ccad73 YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999). Contributed by Mohammad Shahid Khan and Varun Saxena 2015-12-03 12:48:50 -08:00
cnauroth
c37c3f41b3 Revert "HADOOP-11505. Various native parts use bswap incorrectly and unportably (Alan Burlison via aw)" 
This reverts commit 6725e7f1be.
 2015-11-28 17:21:11 -08:00
Allen Wittenauer
6725e7f1be HADOOP-11505. Various native parts use bswap incorrectly and unportably (Alan Burlison via aw) 2015-11-27 18:25:45 -08:00
Robert Kanter
7fd00b3db4 MAPREDUCE-6549. multibyte delimiters with LineRecordReader cause duplicate records (wilfreds via rkanter) 2015-11-25 17:03:38 -08:00
Junping Du
15d577bfbb Tests in mapreduce-client-app are writing outside of target. Contributed by Akira AJISAKA. 2015-11-25 09:15:26 -08:00
Tsuyoshi Ozawa
3e85542a7a MAPREDUCE-6555. TestMRAppMaster fails on trunk. (Junping Du via ozawa) 2015-11-26 01:02:27 +09:00
Akira Ajisaka
5ec44fc35c MAPREDUCE-6553. Replace '\u2b05' with '<-' in rendering job configuration. Contributed by Gabor Liptak. 2015-11-25 16:30:23 +09:00
Jason Lowe
f634505d48 MAPREDUCE-5870. Support for passing Job priority through Application Submission Context in Mapreduce Side. Contributed by Sunil G 2015-11-24 22:07:26 +00:00
Jason Lowe
cab3c7c889 MAPREDUCE-5883. "Total megabyte-seconds" in job counters is slightly misleading. Contributed by Nathan Roberts 2015-11-24 22:01:03 +00:00
Jian He
8676a118a1 YARN-4349. Support CallerContext in YARN. Contributed by Wangda Tan 2015-11-23 17:19:48 -08:00
Haohui Mai
4bff073b4d Revert "HADOOP-12572. Update Hadoop's lz4 to r131. Contributed by Kevin Bowling." 
This reverts commit 4c061e607e.
 2015-11-23 15:00:56 -08:00
Haohui Mai
000e12f6fa HDFS-9153. Pretty-format the output for DFSIO. Contributed by Kai Zheng. 2015-11-22 16:18:07 -08:00
Haohui Mai
4c061e607e HADOOP-12572. Update Hadoop's lz4 to r131. Contributed by Kevin Bowling. 2015-11-22 15:18:24 -08:00
Akira Ajisaka
a7f776ffdf MAPREDUCE-6499. Add elapsed time for retired job in JobHistoryServer WebUI. Contributed by Lin Yiqun. 2015-11-17 15:16:47 +09:00
Jian He
fcd7888029 Revert "YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999) Contributed by Mohammad Shahid Khan" 
This reverts commit 8fbea531d7.

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-11-16 20:18:44 -08:00
Jian He
6502d59e73 MAPREDUCE-5485. Allow repeating job commit by extending OutputCommitter API. Contributed by Junping Du 2015-11-16 17:06:41 -08:00
Sangjin Lee
9ad708a0df MAPREDUCE-6540. TestMRTimelineEventHandling fails (sjlee) 2015-11-11 15:37:24 -08:00
Jason Lowe
df68eac825 MAPREDUCE-6533. testDetermineCacheVisibilities of TestClientDistributedCacheManager is broken. Contributed by Chang Li 2015-11-11 17:26:16 +00:00
Jian He
8fbea531d7 YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999) Contributed by Mohammad Shahid Khan 2015-11-09 10:43:45 -08:00
Tsuyoshi Ozawa
668e897538 MAPREDUCE-5763. Warn message about httpshuffle in NM logs. Contributed by Akira AJISAKA. 2015-11-09 15:25:31 +09:00
Akira Ajisaka
04d97f8abb MAPREDUCE-6525. Fix test failure of TestMiniMRClientCluster.testRestart. Contributed by Masatake Iwasaki. 2015-11-03 01:50:07 +09:00
Jason Lowe
6344b6a769 MAPREDUCE-6528. Memory leak for HistoryFileManager.getJobSummary(). Contributed by Junping Du 2015-10-30 15:31:38 +00:00
Tsuyoshi Ozawa
ce31b22739 YARN-4320. TestJobHistoryEventHandler fails as AHS in MiniYarnCluster no longer binds to default port 8188. Contributed by Varun Saxena. 2015-10-30 18:00:20 +09:00
Xuan
7412ff48ee YARN-4313. Race condition in MiniMRYarnCluster when getting history 
server address. Contributed by Jian He
 2015-10-29 17:36:36 -07:00
Jason Lowe
cf953b6258 MAPREDUCE-6515. Update Application priority in AM side from AM-RM heartbeat. Contributed by Sunil G 2015-10-29 18:05:01 +00:00
Allen Wittenauer
68ce93c32e MAPREDUCE-6435. MapReduce client assumes the world is x86 (Alan Burlison via aw) 2015-10-27 12:29:41 -07:00