big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

HADOOP-12866. add a subcommand for gridmix (Kai Sasaki via aw)

Browse Source
This commit is contained in:
Allen Wittenauer 2016-05-06 14:07:02 -07:00
parent 2835f146b5
commit 1c5bbf6499
3 changed files with 13 additions and 15 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
hadoop-common-project/hadoop-common/src/main/bin/hadoop
View File

@ -36,6 +36,7 @@ function hadoop_usage
hadoop_add_subcommand "dtutil" "operations related to delegation tokens"
hadoop_add_subcommand "envvars" "display computed Hadoop environment variables"
hadoop_add_subcommand "fs" "run a generic filesystem user client"
hadoop_add_subcommand "gridmix" "submit a mix of synthetic job, modeling a profiled from production load"
hadoop_add_subcommand "jar <jar>" "run a jar file. NOTE: please use \"yarn jar\" to launch YARN applications, not this command."
hadoop_add_subcommand "jnipath" "prints the java.library.path"
hadoop_add_subcommand "kerbname" "show auth_to_local principal conversion"
@ -158,6 +159,11 @@ case ${COMMAND} in
fs)
CLASS=org.apache.hadoop.fs.FsShell
;;
gridmix)
CLASS=org.apache.hadoop.mapred.gridmix.Gridmix
hadoop_add_to_classpath_tools hadoop-rumen
hadoop_add_to_classpath_tools hadoop-gridmix
;;
jar)
if [[ -n "${YARN_OPTS}" ]] || [[ -n "${YARN_CLIENT_OPTS}" ]]; then
hadoop_error "WARNING: Use \"yarn jar\" to launch YARN applications."

4
hadoop-common-project/hadoop-common/src/site/markdown/CommandsManual.md
View File

@ -157,6 +157,10 @@ For every subcommand that connects to a service, convenience flags are provided
This command is documented in the [File System Shell Guide](./FileSystemShell.html). It is a synonym for `hdfs dfs` when HDFS is in use.
### `gridmix`
Gridmix is a benchmark tool for Hadoop cluster. More information can be found in the [Gridmix Guide](../../hadoop-gridmix/GridMix.html).
### `jar`
Usage: `hadoop jar <jar> [mainClass] args...`

18
hadoop-tools/hadoop-gridmix/src/site/markdown/GridMix.md.vm
View File

@ -75,16 +75,16 @@ Jobs submitted by GridMix have names of the form
Usage
-----
Basic command-line usage without configuration parameters:
Gridmix is provided as hadoop subcommand. Basic command-line usage without configuration parameters:
```
java org.apache.hadoop.mapred.gridmix.Gridmix [-generate <size>] [-users <users-list>] <iopath> <trace>
$ hadoop gridmix [-generate <size>] [-users <users-list>] <iopath> <trace>
```
Basic command-line usage with configuration parameters:
```
java org.apache.hadoop.mapred.gridmix.Gridmix \
$ hadoop gridmix \
-Dgridmix.client.submit.threads=10 -Dgridmix.output.directory=foo \
[-generate <size>] [-users <users-list>] <iopath> <trace>
```
@ -125,18 +125,6 @@ uncompressed. Use "-" as the value of this parameter if you
want to pass an *uncompressed* trace via the standard
input-stream of GridMix.
GridMix expects certain library *JARs* to be present in the *CLASSPATH*.
One simple way to run GridMix is to use `hadoop jar` command to run it.
You also need to add the JAR of Rumen to classpath for both of client and tasks
as example shown below.
```
HADOOP_CLASSPATH=$HADOOP_HOME/share/hadoop/tools/lib/hadoop-rumen-2.5.1.jar \
$HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/share/hadoop/tools/lib/hadoop-gridmix-2.5.1.jar \
-libjars $HADOOP_HOME/share/hadoop/tools/lib/hadoop-rumen-2.5.1.jar \
[-generate <size>] [-users <users-list>] <iopath> <trace>
```
The supported configuration parameters are explained in the
following sections.