HADOOP-12866. add a subcommand for gridmix (Kai Sasaki via aw)

This commit is contained in:
Allen Wittenauer 2016-05-06 14:07:02 -07:00
parent 2835f146b5
commit 1c5bbf6499
3 changed files with 13 additions and 15 deletions

View File

@ -36,6 +36,7 @@ function hadoop_usage
hadoop_add_subcommand "dtutil" "operations related to delegation tokens" hadoop_add_subcommand "dtutil" "operations related to delegation tokens"
hadoop_add_subcommand "envvars" "display computed Hadoop environment variables" hadoop_add_subcommand "envvars" "display computed Hadoop environment variables"
hadoop_add_subcommand "fs" "run a generic filesystem user client" hadoop_add_subcommand "fs" "run a generic filesystem user client"
hadoop_add_subcommand "gridmix" "submit a mix of synthetic job, modeling a profiled from production load"
hadoop_add_subcommand "jar <jar>" "run a jar file. NOTE: please use \"yarn jar\" to launch YARN applications, not this command." hadoop_add_subcommand "jar <jar>" "run a jar file. NOTE: please use \"yarn jar\" to launch YARN applications, not this command."
hadoop_add_subcommand "jnipath" "prints the java.library.path" hadoop_add_subcommand "jnipath" "prints the java.library.path"
hadoop_add_subcommand "kerbname" "show auth_to_local principal conversion" hadoop_add_subcommand "kerbname" "show auth_to_local principal conversion"
@ -158,6 +159,11 @@ case ${COMMAND} in
fs) fs)
CLASS=org.apache.hadoop.fs.FsShell CLASS=org.apache.hadoop.fs.FsShell
;; ;;
gridmix)
CLASS=org.apache.hadoop.mapred.gridmix.Gridmix
hadoop_add_to_classpath_tools hadoop-rumen
hadoop_add_to_classpath_tools hadoop-gridmix
;;
jar) jar)
if [[ -n "${YARN_OPTS}" ]] || [[ -n "${YARN_CLIENT_OPTS}" ]]; then if [[ -n "${YARN_OPTS}" ]] || [[ -n "${YARN_CLIENT_OPTS}" ]]; then
hadoop_error "WARNING: Use \"yarn jar\" to launch YARN applications." hadoop_error "WARNING: Use \"yarn jar\" to launch YARN applications."

View File

@ -157,6 +157,10 @@ For every subcommand that connects to a service, convenience flags are provided
This command is documented in the [File System Shell Guide](./FileSystemShell.html). It is a synonym for `hdfs dfs` when HDFS is in use. This command is documented in the [File System Shell Guide](./FileSystemShell.html). It is a synonym for `hdfs dfs` when HDFS is in use.
### `gridmix`
Gridmix is a benchmark tool for Hadoop cluster. More information can be found in the [Gridmix Guide](../../hadoop-gridmix/GridMix.html).
### `jar` ### `jar`
Usage: `hadoop jar <jar> [mainClass] args...` Usage: `hadoop jar <jar> [mainClass] args...`

View File

@ -75,16 +75,16 @@ Jobs submitted by GridMix have names of the form
Usage Usage
----- -----
Basic command-line usage without configuration parameters: Gridmix is provided as hadoop subcommand. Basic command-line usage without configuration parameters:
``` ```
java org.apache.hadoop.mapred.gridmix.Gridmix [-generate <size>] [-users <users-list>] <iopath> <trace> $ hadoop gridmix [-generate <size>] [-users <users-list>] <iopath> <trace>
``` ```
Basic command-line usage with configuration parameters: Basic command-line usage with configuration parameters:
``` ```
java org.apache.hadoop.mapred.gridmix.Gridmix \ $ hadoop gridmix \
-Dgridmix.client.submit.threads=10 -Dgridmix.output.directory=foo \ -Dgridmix.client.submit.threads=10 -Dgridmix.output.directory=foo \
[-generate <size>] [-users <users-list>] <iopath> <trace> [-generate <size>] [-users <users-list>] <iopath> <trace>
``` ```
@ -125,18 +125,6 @@ uncompressed. Use "-" as the value of this parameter if you
want to pass an *uncompressed* trace via the standard want to pass an *uncompressed* trace via the standard
input-stream of GridMix. input-stream of GridMix.
GridMix expects certain library *JARs* to be present in the *CLASSPATH*.
One simple way to run GridMix is to use `hadoop jar` command to run it.
You also need to add the JAR of Rumen to classpath for both of client and tasks
as example shown below.
```
HADOOP_CLASSPATH=$HADOOP_HOME/share/hadoop/tools/lib/hadoop-rumen-2.5.1.jar \
$HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/share/hadoop/tools/lib/hadoop-gridmix-2.5.1.jar \
-libjars $HADOOP_HOME/share/hadoop/tools/lib/hadoop-rumen-2.5.1.jar \
[-generate <size>] [-users <users-list>] <iopath> <trace>
```
The supported configuration parameters are explained in the The supported configuration parameters are explained in the
following sections. following sections.