big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

YARN-8376. Separate white list for docker.trusted.registries and docker.privileged-container.registries. Contributed by Eric Yang

Browse Source
This commit is contained in:
Eric Badger 2019-03-14 19:39:00 +00:00
parent d60673c470
commit 688b177fc6
3 changed files with 24 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/utils/docker-util.c
View File

@ -115,7 +115,15 @@ int check_trusted_image(const struct configuration *command_config, const struct
int ret = 0;
int no_registry_prefix_in_image_name = 0;
char *image_name = get_configuration_value("image", DOCKER_COMMAND_FILE_SECTION, command_config);
char **privileged_registry = get_configuration_values_delimiter("docker.trusted.registries", CONTAINER_EXECUTOR_CFG_DOCKER_SECTION, conf, ",");
char *privileged = NULL;
char **privileged_registry = NULL;
privileged = get_configuration_value("privileged", DOCKER_COMMAND_FILE_SECTION, command_config);
if (privileged != NULL && strcasecmp(privileged, "true") == 0 ) {
privileged_registry = get_configuration_values_delimiter("docker.privileged-containers.registries", CONTAINER_EXECUTOR_CFG_DOCKER_SECTION, conf, ",");
}
if (privileged_registry == NULL) {
privileged_registry = get_configuration_values_delimiter("docker.trusted.registries", CONTAINER_EXECUTOR_CFG_DOCKER_SECTION, conf, ",");
}
char *registry_ptr = NULL;
if (image_name == NULL) {
ret = INVALID_DOCKER_IMAGE_NAME;
@ -158,6 +166,7 @@ int check_trusted_image(const struct configuration *command_config, const struct
}
free_and_exit:
free(privileged);
free(image_name);
free_values(privileged_registry);
return ret;

4
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/utils/test_docker_util.cc
View File

@ -782,7 +782,9 @@ namespace ContainerExecutor {
struct configuration container_cfg, cmd_cfg;
struct args buff = ARGS_INITIAL_VALUE;
int ret = 0;
std::string container_executor_cfg_contents[] = {"[docker]\n docker.privileged-containers.enabled=1\n docker.trusted.registries=hadoop",
std::string container_executor_cfg_contents[] = {"[docker]\n docker.privileged-containers.enabled=1\n"
" docker.trusted.registries=library\n"
" docker.privileged-containers.registries=hadoop",
"[docker]\n docker.privileged-containers.enabled=true\n docker.trusted.registries=hadoop",
"[docker]\n docker.privileged-containers.enabled=True\n docker.trusted.registries=hadoop",
"[docker]\n docker.privileged-containers.enabled=0",

11
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/DockerContainers.md
View File

@ -280,6 +280,7 @@ are allowed. It contains the following properties:
| `docker.allowed.volume-drivers` | Comma separated list of volume drivers which are allowed to be used. By default, no volume drivers are allowed. |
| `docker.host-pid-namespace.enabled` | Set to "true" or "false" to enable or disable using the host's PID namespace. Default value is "false". |
| `docker.privileged-containers.enabled` | Set to "true" or "false" to enable or disable launching privileged containers. Default value is "false". |
| `docker.privileged-containers.registries` | Comma separated list of privileged docker registries for running privileged docker containers. By default, no registries are defined. |
| `docker.trusted.registries` | Comma separated list of trusted docker registries for running trusted privileged docker containers. By default, no registries are defined. |
| `docker.inspect.max.retries` | Integer value to check docker container readiness. Each inspection is set with 3 seconds delay. Default value of 10 will wait 30 seconds for docker container to become ready before marked as container failed. |
| `docker.no-new-privileges.enabled` | Enable/disable the no-new-privileges flag for docker run. Set to "true" to enable, disabled by default. |
@ -306,6 +307,7 @@ yarn.nodemanager.linux-container-executor.group=yarn
[docker]
module.enabled=true
docker.privileged-containers.enabled=true
docker.privileged-containers.registries=local
docker.trusted.registries=centos
docker.allowed.capabilities=SYS_CHROOT,MKNOD,SETFCAP,SETPCAP,FSETID,CHOWN,AUDIT_WRITE,SETGID,NET_RAW,FOWNER,SETUID,DAC_OVERRIDE,KILL,NET_BIND_SERVICE
docker.allowed.networks=bridge,host,none
@ -650,6 +652,15 @@ When docker images have been certified by developers and testers to be trustwort
docker.trusted.registries=library
```
Fine grained access control can also be defined using `docker.privileged-containers.registries` to allow only a subset of Docker images to run as privileged containers. If `docker.privileged-containers.registries` is not defined, YARN will fall back to use `docker.trusted.registries` as access control for privileged Docker images. Fine grained access control example:
```
[docker]
docker.privileged-containers.enabled=true
docker.privileged-containers.registries=local/centos:latest
docker.trusted.registries=library
```
In development environment, local images can be tagged with a repository name prefix to enable trust. The recommendation of choosing a repository name is using a local hostname and port number to prevent accidentially pulling docker images from Docker Hub or use reserved Docker Hub keyword: "local". Docker run will look for docker images on Docker Hub, if the image does not exist locally. Using a local hostname and port in image name can prevent accidental pulling of canonical images from docker hub. Example of tagging image with localhost:5000 as trusted registry:
```