big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
3,078 Commits 2 Branches 2 Tags 264 MiB
28ca281cd6
Commit Graph

3078 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Siddharth Seth
5fdfa2b4bd MAPREDUCE-3822. Changed FS counter computation to use all occurences of the same FS scheme, instead of randomly using one. (Contributed by Mahadev Konar) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1241721 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-08 00:56:28 +00:00
Arun Murthy
47c5d51bd8 HADOOP-7841. Added to CHANGES.txt 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1241716 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-08 00:41:14 +00:00
Arun Murthy
23e4adabfa HADOOP-7813. Added to CHANGES.txt 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1241714 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-08 00:36:26 +00:00
Arun Murthy
04f685ea76 MAPREDUCE-3827. Changed Counters to use ConcurrentSkipListMap for performance. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1241711 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-08 00:16:54 +00:00
Siddharth Seth
ccc2807d86 MAPREDUCE-3823. Ensure counters are calculated only once after a job finishes. (Contributed by Vinod Kumar Vavilapalli) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1241692 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-07 23:07:03 +00:00
Arun Murthy
d8e9f84fb1 Fixed jira id in commit for MAPREDUCE-3826. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1241688 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-07 23:04:29 +00:00
Arun Murthy
7dfe20d5a8 MAPREDUCE-3823. Fixed a bug in RM web-ui which broke sorting. Contributed by Jonathan Eagles. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1241685 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-07 22:59:58 +00:00
Arun Murthy
59b88655bc MAPREDUCE-3833. Fixed a bug in reinitiaziling of queues. Contributed by Jason Lowe. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1241659 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-07 22:08:55 +00:00
Vinod Kumar Vavilapalli
90bc439b66 MAPREDUCE-3815. Fixed MR AM to always use hostnames and never IPs when requesting containers so that scheduler can give off data local containers correctly. Contributed by Siddarth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1241654 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-07 21:58:54 +00:00
Siddharth Seth
fb498926be MAPREDUCE-3436. JobHistory webapp address should use the host configured in the jobhistory address. (Contributed by Ahmed Radwan) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1241620 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-07 20:55:10 +00:00
Jitendra Nath Pandey
acacde55e6 HDFS-2901. Improvements for SBN web UI - not show under-replicated/missing blocks. Contributed by Brandon Li. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1241568 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-07 19:29:39 +00:00
Suresh Srinivas
28eadb7cd7 HDFS-2899. Service protocol changes in DatanodeProtocol to add multiple storages. Contributed by Suresh Srinivas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1241519 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-07 16:59:48 +00:00
Mahadev Konar
aaace5e84e MAPREDUCE-3709. TestDistributedShell is failing. (Hitesh Shah via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1241325 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-07 01:53:54 +00:00
Arun Murthy
6e376a39a0 MAPREDUCE-3817. Fixed bin/mapred to allow running of distcp and archive jobs. Contributed by Arpit Gupta. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1241323 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-07 01:49:50 +00:00
Arun Murthy
35f12b9556 MAPREDUCE-3697. Support binary compatibility for Counters after MAPREDUCE-901. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1241319 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-07 01:41:28 +00:00
Todd Lipcon
7f0a99f6e6 HDFS-2794. Active NN may purge edit log files before standby NN has a chance to read them. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1241317 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-07 01:39:14 +00:00
Aaron Myers
2d1406e9e7 HADOOP-8027. Visiting /jmx on the daemon web interfaces may print unnecessary error in logs. Contributed by Aaron T. Myers. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1241303 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-07 01:09:34 +00:00
Mahadev Konar
59d1f07d6c MAPREDUCE-3794. Support mapred.Task.Counter and mapred.JobInProgress.Counter enums for compatibility (Tom White via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1241299 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-07 00:53:36 +00:00
Konstantin Shvachko
7e4bd89590 HDFS-2886. CreateEditLogs should generate a realistic edit log. Contributed by Konstantin Shvachko. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1241296 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-07 00:44:27 +00:00
Arun Murthy
8dbf374f16 MAPREDUCE-3811. Made jobclient-to-AM retries configurable. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1241285 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-07 00:20:36 +00:00
Arun Murthy
75d2d6185e MAPREDUCE-3809. Ensure that there is no needless sleep in Task at the end of the task. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1241282 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-07 00:13:40 +00:00
Vinod Kumar Vavilapalli
08479378c2 MAPREDUCE-3818. Fixed broken compilation in TestSubmitJob after the patch for HDFS-2895. Contributed by Suresh Srinivas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1241253 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-06 23:16:37 +00:00
Arun Murthy
4f6839f23d MAPREDUCE-3354. Changed scripts so that jobhistory server is started by bin/mapred instead of bin/yarn. Contributed by Jonathan Eagles. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1241250 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-06 23:11:26 +00:00
Robert Joseph Evans
bdf837a64a MAPREDUCE-3804. yarn webapp interface vulnerable to cross scripting attacks (Dave Thompson via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1241225 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-06 22:34:28 +00:00
Vinod Kumar Vavilapalli
9f2b77aee4 MAPREDUCE-3808. Fixed an NPE in FileOutputCommitter for jobs with maps but no reduces. Contributed by Robert Joseph Evans. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1241217 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-06 22:16:26 +00:00
Arun Murthy
c6282df3e5 MAPREDUCE-3813. Added a cache for resolved racks. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1241209 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-06 22:06:58 +00:00
Arun Murthy
28a2eb9d72 MAPREDUCE-3810. Performance tweaks - reduced logging in AM and defined hascode/equals for ResourceRequest & Priority. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1241205 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-06 22:01:22 +00:00
Robert Joseph Evans
2c91c9d716 HADOOP-8017. Configure hadoop-main pom to get rid of M2E plugin execution not covered (Eric Charles via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1241190 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-06 21:33:27 +00:00
Aaron Myers
db7c013877 HDFS-2733. Document HA configuration and CLI. Contributed by Aaron T. Myers. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1241183 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-06 21:18:11 +00:00
Vinod Kumar Vavilapalli
3f02d8dd5c MAPREDUCE-3814. Fixed MRV1 compilation. (Arun C Murthy via vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1241177 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-06 21:04:30 +00:00
Suresh Srinivas
2a9e430ff9 HDS-2895. Remove Writable wire protocol types and translators to complete transition to protocol buffers. Contributed by Suresh Srinivas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1241007 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-06 13:33:35 +00:00
Eli Collins
b837bbb7d5 HDFS-2893. The start/stop scripts don't start/stop the 2NN when using the default configuration. Contributed by Eli Collins 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1240928 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-06 08:52:50 +00:00
Eli Collins
296b6c0063 HDFS-2894. HA: automatically determine the nameservice Id if only one nameservice is configured. Contributed by Eli Collins 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1240917 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-06 08:25:52 +00:00
Eli Collins
2e4cf977ae HDFS-2752. HA: exit if multiple shared dirs are configured. Contributed by Eli Collins 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1240916 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-06 08:24:11 +00:00
Eli Collins
c17b4f8eef HDFS-2819. Document new HA-related configs in hdfs-default.xml. Contributed by Eli Collins 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1240914 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-06 08:21:06 +00:00
Harsh J
da7b0d90c6 HDFS-2868. Expose xceiver counts via the DataNode MXBean. (harsh) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1240897 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-06 05:25:12 +00:00
Arun Murthy
c7007a8a35 MAPREDUCE-3747. Initialize queue metrics upfront and added start/finish time to RM Web-UI. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1240886 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-06 03:38:44 +00:00
Suresh Srinivas
f88574acde HDFS-2880. Protobuf chagnes in DatanodeProtocol to add multiple storages. Contributed by Suresh Srinivas. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1240653 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-05 01:39:30 +00:00
Mahadev Konar
23e336a0ed MAPREDUCE-3765. FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb setting (Hitesh Shah via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1240634 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-04 22:44:43 +00:00
Mahadev Konar
f3c35222e6 MAPREDUCE-3775. Change MiniYarnCluster to escape special chars in testname. (Hitesh Shah via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1240603 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-04 20:20:00 +00:00
Eli Collins
a660c5dd2c Add missing file from previous commit. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1240601 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-04 20:13:48 +00:00
Eli Collins
7b913180be HDFS-2808. HA: haadmin should use namenode ids. Contributed by Eli Collins 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1240600 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-04 20:12:51 +00:00
Mahadev Konar
c7d64769c1 MAPREDUCE-3759. ClassCastException thrown in -list-active-trackers when there are a few unhealthy nodes (vinodkv via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1240598 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-04 20:03:37 +00:00
Mahadev Konar
f2280f612c MAPREDUCE-3795. "job -status" command line output is malformed. (vinodkv via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1240593 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-04 19:32:10 +00:00
Mahadev Konar
caed1cfaf9 MAPREDUCE-3723. TestAMWebServicesJobs & TestHSWebServicesJobs incorrectly asserting tests (Bhallamudi Venkata Siva Kamesh via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1240590 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-04 18:55:03 +00:00
Mahadev Konar
2c32cd7008 MAPREDUCE-3791. can't build site in hadoop-yarn-server-common. (mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1240587 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-04 18:41:50 +00:00
Jitendra Nath Pandey
205f0470f4 HDFS-2785. Update webhdfs and httpfs for host-based token support. Contributed by Robert Joseph Evans. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1240460 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-04 06:57:40 +00:00
Aaron Myers
ec6961b39c HDFS-2792. Make fsck work. Contributed by Aaron T. Myers. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1240449 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-04 03:10:39 +00:00
Aaron Myers
3cc71933e9 HDFS-2890. DFSUtil#getSuffixIDs should skip unset configurations. Contributed by Aaron T. Myers. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1240447 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-04 03:00:17 +00:00
Todd Lipcon
11df1c2561 HDFS-2874. Edit log should log to shared dirs before local dirs. Contributed by Todd Lipcon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-1623@1240445 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-04 02:52:53 +00:00