big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

HDFS-7195. Update user doc of secure mode about Datanodes don't require root or jsvc. Contributed by Chris Nauroth.

Browse Source
This commit is contained in:
cnauroth 2014-10-09 11:51:12 -07:00
parent 180afa2f86
commit 9097183983
3 changed files with 58 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

12
hadoop-common-project/hadoop-common/src/main/conf/hadoop-env.sh
View File

@ -206,7 +206,10 @@ esac
# custom functions. See hadoop-functions.sh for more information.
#
# The jsvc implementation to use. Jsvc is required to run secure datanodes.
# The jsvc implementation to use. Jsvc is required to run secure datanodes
# that bind to privileged ports to provide authentication of data transfer
# protocol. Jsvc is not required if SASL is configured for authentication of
# data transfer protocol using non-privileged ports.
# export JSVC_HOME=/usr/bin
#
@ -263,8 +266,11 @@ esac
# This is the default:
# export HADOOP_DATANODE_OPTS="-Dhadoop.security.logger=ERROR,RFAS"
# On secure datanodes, user to run the datanode as after dropping privileges
# This **MUST** be uncommented to enable secure HDFS!
# On secure datanodes, user to run the datanode as after dropping privileges.
# This **MUST** be uncommented to enable secure HDFS if using privileged ports
# to provide authentication of data transfer protocol. This **MUST NOT** be
# defined if SASL is configured for authentication of data transfer protocol
# using non-privileged ports.
# export HADOOP_SECURE_DN_USER=hdfs
# Supplemental options for secure datanodes

47
hadoop-common-project/hadoop-common/src/site/apt/SecureMode.apt.vm
View File

@ -244,7 +244,7 @@ KVNO Timestamp Principal
10.222.0.0-15 and 10.113.221.221
can impersonate any user belonging to any group.
----
----
<property>
<name>hadoop.proxyuser.oozie.hosts</name>
<value>10.222.0.0/16,10.113.221.221</value>
@ -273,6 +273,33 @@ KVNO Timestamp Principal
You must specify <<<HADOOP_SECURE_DN_USER>>> and <<<JSVC_HOME>>>
as environment variables on start up (in hadoop-env.sh).
As of version 2.6.0, SASL can be used to authenticate the data transfer
protocol. In this configuration, it is no longer required for secured clusters
to start the DataNode as root using jsvc and bind to privileged ports. To
enable SASL on data transfer protocol, set <<<dfs.data.transfer.protection>>>
in hdfs-site.xml, set a non-privileged port for <<<dfs.datanode.address>>>, set
<<<dfs.http.policy>>> to <HTTPS_ONLY> and make sure the
<<<HADOOP_SECURE_DN_USER>>> environment variable is not defined. Note that it
is not possible to use SASL on data transfer protocol if
<<<dfs.datanode.address>>> is set to a privileged port. This is required for
backwards-compatibility reasons.
In order to migrate an existing cluster that used root authentication to start
using SASL instead, first ensure that version 2.6.0 or later has been deployed
to all cluster nodes as well as any external applications that need to connect
to the cluster. Only versions 2.6.0 and later of the HDFS client can connect
to a DataNode that uses SASL for authentication of data transfer protocol, so
it is vital that all callers have the correct version before migrating. After
version 2.6.0 or later has been deployed everywhere, update configuration of
any external applications to enable SASL. If an HDFS client is enabled for
SASL, then it can connect successfully to a DataNode running with either root
authentication or SASL authentication. Changing configuration for all clients
guarantees that subsequent configuration changes on DataNodes will not disrupt
the applications. Finally, each individual DataNode can be migrated by
changing its configuration and restarting. It is acceptable to have a mix of
some DataNodes running with root authentication and some DataNodes running with
SASL authentication temporarily during this migration period, because an HDFS
client enabled for SASL can connect to both.
* Data confidentiality
@ -391,6 +418,10 @@ Configuration for <<<conf/core-site.xml>>>
| <<<dfs.http.policy>>> | <HTTP_ONLY> or <HTTPS_ONLY> or <HTTP_AND_HTTPS> | |
| | | HTTPS_ONLY turns off http access. This option takes precedence over |
| | | the deprecated configuration dfs.https.enable and hadoop.ssl.enabled. |
| | | If using SASL to authenticate data transfer protocol instead of |
| | | running DataNode as root and using privileged ports, then this property |
| | | must be set to <HTTPS_ONLY> to guarantee authentication of HTTP servers. |
| | | (See <<<dfs.data.transfer.protection>>>.) |
*-------------------------+-------------------------+------------------------+
| <<<dfs.namenode.https-address>>> | <nn_host_fqdn:50470> | |
*-------------------------+-------------------------+------------------------+
@ -440,6 +471,9 @@ Configuration for <<<conf/hdfs-site.xml>>>
| | | Secure DataNode must use privileged port |
| | | in order to assure that the server was started securely. |
| | | This means that the server must be started via jsvc. |
| | | Alternatively, this must be set to a non-privileged port if using SASL |
| | | to authenticate data transfer protocol. |
| | | (See <<<dfs.data.transfer.protection>>>.) |
*-------------------------+-------------------------+------------------------+
| <<<dfs.datanode.http.address>>> | <0.0.0.0:1006> | |
| | | Secure DataNode must use privileged port |
@ -457,6 +491,17 @@ Configuration for <<<conf/hdfs-site.xml>>>
| <<<dfs.encrypt.data.transfer>>> | <false> | |
| | | set to <<<true>>> when using data encryption |
*-------------------------+-------------------------+------------------------+
| <<<dfs.data.transfer.protection>>> | | |
| | | <authentication> : authentication only \
| | | <integrity> : integrity check in addition to authentication \
| | | <privacy> : data encryption in addition to integrity |
| | | This property is unspecified by default. Setting this property enables |
| | | SASL for authentication of data transfer protocol. If this is enabled, |
| | | then <<<dfs.datanode.address>>> must use a non-privileged port, |
| | | <<<dfs.http.policy>>> must be set to <HTTPS_ONLY> and the |
| | | <<<HADOOP_SECURE_DN_USER>>> environment variable must be undefined when |
| | | starting the DataNode process. |
*-------------------------+-------------------------+------------------------+
Configuration for <<<conf/hdfs-site.xml>>>

3
hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
View File

@ -662,6 +662,9 @@ Release 2.6.0 - UNRELEASED
HDFS-7217. Better batching of IBRs. (kihwal)
HDFS-7195. Update user doc of secure mode about Datanodes don't require root
or jsvc. (cnauroth)
OPTIMIZATIONS
HDFS-6690. Deduplicate xattr names in memory. (wang)