big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

YARN-9229. Document docker registry deployment with NFS Gateway. Contributed by Eric Yang.

Browse Source
This commit is contained in:
Suma Shivaprasad 2019-02-11 14:54:17 -08:00
parent 6c999fe4b0
commit 1ce2e91c4b
1 changed files with 116 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

118
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/DockerContainers.md
View File

@ -654,8 +654,8 @@ will fail and the container will be killed on NodeManager restart.
proc /proc proc nosuid,nodev,noexec,hidepid=2,gid=yarn 0 0
```
Connecting to a Secure Docker Repository
----------------------------------------
Connecting to a Docker Trusted Registry
--------------------------------------
The Docker client command will draw its configuration from the default location,
which is $HOME/.docker/config.json on the NodeManager host. The Docker
@ -683,6 +683,120 @@ host into the secure repo using the Docker login command:
Note that this approach means that all users will have access to the secure
repo.
Hadoop integrates with Docker Trusted Registry via YARN service API. Docker registry can store Docker images on HDFS, S3 or external storage using CSI driver.
### Docker Registry on HDFS
NFS Gateway provides capability to mount HDFS as NFS mount point. Docker Registry can configure to write to HDFS mount point using standard file system API.
In hdfs-site.xml, configure NFS configuration:
```
<property>
<name>nfs.exports.allowed.hosts</name>
<value>* rw</value>
</property>
<property>
<name>nfs.file.dump.dir</name>
<value>/tmp/.hdfs-nfs</value>
</property>
<property>
<name>nfs.kerberos.principal</name>
<value>nfs/_HOST@EXAMPLE.COM</value>
</property>
<property>
<name>nfs.keytab.file</name>
<value>/etc/security/keytabs/nfs.service.keytab</value>
</property>
```
Run NFS Gateway on all datanodes as hdfs user using:
```
$ $HADOOP_HOME/bin/hdfs --daemon start nfs3
```
On each datanode, nfs mount point is exposed to /hdfs, using:
```
# mount -t nfs -o vers=3,proto=tcp,nolock,noacl,sync $DN_IP:/ /hdfs
```
Where DN_IP is the IP address of the datanode.
Container-executor.cfg is configured to allow trusted Docker images from library.
```
[docker]
docker.privileged-containers.enabled=true
docker.trusted.registries=library,registry.docker-registry.registry.example.com:5000
docker.allowed.rw-mounts=/tmp,/usr/local/hadoop/logs,/hdfs
```
Docker Registry can be started using YARN service:
registry.json
```
{
"name": "docker-registry",
"version": "1.0",
"kerberos_principal" : {
"principal_name" : "registry/_HOST@EXAMPLE.COM",
"keytab" : "file:///etc/security/keytabs/registry.service.keytab"
},
"components" :
[
{
"name": "registry",
"number_of_containers": 1,
"artifact": {
"id": "registry:latest",
"type": "DOCKER"
},
"resource": {
"cpus": 1,
"memory": "256"
},
"run_privileged_container": true,
"configuration": {
"env": {
"YARN_CONTAINER_RUNTIME_DOCKER_RUN_OVERRIDE_DISABLE":"true",
"YARN_CONTAINER_RUNTIME_DOCKER_MOUNTS":"/hdfs/apps/docker/registry:/var/lib/registry"
},
"properties": {
"docker.network": "host"
}
}
}
]
}
```
YARN service configures docker mounts from /hdfs/apps/docker/registry to /var/lib/registry inside docker container.
```
yarn app -launch docker-registry /tmp/registry.json
```
Docker trusted registry is deployed in YARN framework, and the URL to access the registry following Hadoop Registry DNS format:
```
registry.docker-registry.$USER.$DOMAIN:5000
```
When docker-registry application reaches STABLE state in YARN, user can push or pull docker images to Docker Trusted Registry by prefix image name with registry.docker-registry.registry.example.com:5000/.
### Docker Registry on S3
Docker Registry provides its own S3 driver and YAML configuration. YARN service configuration can generate YAML template, and enable direct Docker Registry to S3 storage. This option is the top choice for deploying Docker Trusted Registry on AWS.
### Docker Registry with CSI Driver
CSI driver enables third party storage system to expose as posix mount point in the container. This allows Docker Trusted Registry to write docker images to an external storage.
Example: MapReduce
------------------