big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

HADOOP-15168. Add kdiag tool to hadoop command. Contributed by Bharat Viswanadham.

Browse Source
This commit is contained in:
Hanisha Koneru 2018-02-02 12:51:27 -08:00
parent d4e13a4647
commit 51cb6c5380
4 changed files with 28 additions and 21 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
hadoop-common-project/hadoop-common/src/main/bin/hadoop
View File

@ -45,6 +45,7 @@ function hadoop_usage
hadoop_add_subcommand "key" client "manage keys via the KeyProvider"
hadoop_add_subcommand "trace" client "view and modify Hadoop tracing settings"
hadoop_add_subcommand "version" client "print the version"
hadoop_add_subcommand "kdiag" client "Diagnose Kerberos Problems"
hadoop_generate_usage "${HADOOP_SHELL_EXECNAME}" true
}
@ -148,6 +149,9 @@ function hadoopcmd_case
kerbname)
HADOOP_CLASSNAME=org.apache.hadoop.security.HadoopKerberosName
;;
kdiag)
HADOOP_CLASSNAME=org.apache.hadoop.security.KDiag
;;
key)
HADOOP_CLASSNAME=org.apache.hadoop.crypto.key.KeyShell
;;

7
hadoop-common-project/hadoop-common/src/main/bin/hadoop.cmd
View File

@ -149,7 +149,7 @@ call :updatepath %HADOOP_BIN_PATH%
exit /b
)
set corecommands=fs version jar checknative conftest distch distcp daemonlog archive classpath credential kerbname key trace
set corecommands=fs version jar checknative conftest distch distcp daemonlog archive classpath credential kerbname key trace kdiag
for %%i in ( %corecommands% ) do (
if %hadoop-command% == %%i set corecommand=true
)
@ -231,6 +231,10 @@ call :updatepath %HADOOP_BIN_PATH%
set CLASS=org.apache.hadoop.security.HadoopKerberosName
goto :eof
:kdiag
set CLASS=org.apache.hadoop.security.KDiag
goto :eof
:key
set CLASS=org.apache.hadoop.crypto.key.KeyShell
goto :eof
@ -307,6 +311,7 @@ call :updatepath %HADOOP_BIN_PATH%
@echo credential interact with credential providers
@echo jnipath prints the java.library.path
@echo kerbname show auth_to_local principal conversion
@echo kdiag diagnose kerberos problems
@echo key manage keys via the KeyProvider
@echo trace view and modify Hadoop tracing settings
@echo daemonlog get/set the log level for each daemon

6
hadoop-common-project/hadoop-common/src/site/markdown/CommandsManual.md
View File

@ -187,6 +187,12 @@ user name.
Example: `hadoop kerbname user@EXAMPLE.COM`
### `kdiag`
Usage: `hadoop kdiag`
Diagnose Kerberos Problems
### `key`
Usage: `hadoop key <subcommand> [options]`

32
hadoop-common-project/hadoop-common/src/site/markdown/SecureMode.md
View File

@ -435,17 +435,12 @@ or a specific principal in a named keytab.
The output of the command can be used for local diagnostics, or forwarded to
whoever supports the cluster.
The `KDiag` command has its own entry point; it is currently not hooked up
to the end-user CLI.
It is invoked simply by passing its full classname to one of the `bin/hadoop`,
`bin/hdfs` or `bin/yarn` commands. Accordingly, it will display the kerberos client
state of the command used to invoke it.
The `KDiag` command has its own entry point; It is invoked by passing `kdiag` to
`bin/hadoop` command. Accordingly, it will display the kerberos client state
of the command used to invoke it.
```
hadoop org.apache.hadoop.security.KDiag
hdfs org.apache.hadoop.security.KDiag
yarn org.apache.hadoop.security.KDiag
hadoop kdiag
```
The command returns a status code of 0 for a successful diagnostics run.
@ -525,7 +520,7 @@ some basic Kerberos preconditions.
#### `--out outfile`: Write output to file.
```
hadoop org.apache.hadoop.security.KDiag --out out.txt
hadoop kdiag --out out.txt
```
Much of the diagnostics information comes from the JRE (to `stderr`) and
@ -534,7 +529,7 @@ To get all the output, it is best to redirect both these output streams
to the same file, and omit the `--out` option.
```
hadoop org.apache.hadoop.security.KDiag --keytab zk.service.keytab --principal zookeeper/devix.example.org@REALM > out.txt 2>&1
hadoop kdiag --keytab zk.service.keytab --principal zookeeper/devix.example.org@REALM > out.txt 2>&1
```
Even there, the output of the two streams, emitted across multiple threads, can
@ -543,15 +538,12 @@ name in the Log4j output to distinguish background threads from the main thread
helps at the hadoop level, but doesn't assist in JVM-level logging.
#### `--resource <resource>` : XML configuration resource to load.
When using the `hdfs` and `yarn` commands, it is often useful to force
load the `hdfs-site.xml` and `yarn-site.xml` resource files, to pick up any Kerberos-related
configuration options therein.
The `core-default` and `core-site` XML resources are always loaded.
To load XML configuration files, this option can be used. As by default, the
`core-default` and `core-site` XML resources are only loaded. This will help,
when additional configuration files has any Kerberos related configurations.
```
hdfs org.apache.hadoop.security.KDiag --resource hbase-default.xml --resource hbase-site.xml
yarn org.apache.hadoop.security.KDiag --resource yarn-default.xml --resource yarn-site.xml
hadoop kdiag --resource hbase-default.xml --resource hbase-site.xml
```
For extra logging during the operation, set the logging and `HADOOP_JAAS_DEBUG`
@ -580,9 +572,9 @@ nor `"/"` characters.
### Example
```
hdfs org.apache.hadoop.security.KDiag \
hadoop kdiag \
--nofail \
--resource hbase-default.xml --resource hbase-site.xml \
--resource hdfs-site.xml --resource yarn-site.xml \
--keylen 1024 \
--keytab zk.service.keytab --principal zookeeper/devix.example.org@REALM
```