big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
8b23814415
hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine
History
Wangda Tan 8b23814415 YARN-8960. [Submarine] Can't get submarine service status using the command of "yarn app -status" under security environment. (Zac Zhou via wangda) 
Change-Id: I21b1addc9c32817650ea744a8f2e6b5602f2f4d4
2018-11-19 08:55:30 -08:00
..
installation YARN-8870. [Submarine] Add submarine installation scripts. (Xun Liu via wangda) 2018-10-16 13:51:02 -07:00
src YARN-8960. [Submarine] Can't get submarine service status using the command of "yarn app -status" under security environment. (Zac Zhou via wangda) 2018-11-19 08:55:30 -08:00
pom.xml Changed version in trunk to 3.3.0-SNAPSHOT. 2018-10-02 22:41:41 +05:30
README.md YARN-8800. Updated documentation of Submarine with latest examples. Contributed by Wangda Tan. 2018-09-29 00:01:04 +05:30

README.md

Overview

              _                              _
             | |                            (_)
  ___  _   _ | |__   _ __ ___    __ _  _ __  _  _ __    ___
 / __|| | | || '_ \ | '_ ` _ \  / _` || '__|| || '_ \  / _ \
 \__ \| |_| || |_) || | | | | || (_| || |   | || | | ||  __/
 |___/ \__,_||_.__/ |_| |_| |_| \__,_||_|   |_||_| |_| \___|

                             ?
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~|^"~~~~~~~~~~~~~~~~~~~~~~~~~o~~~~~~~~~~~
        o                   |                  o      __o
         o                  |                 o     |X__>
       ___o                 |                __o
     (X___>--             __|__            |X__>     o
                         |     \                   __o
                         |      \                |X__>
  _______________________|_______\________________
 <                                                \____________   _
  \                                                            \ (_)
   \    O       O       O                                       >=)
    \__________________________________________________________/ (_)

Submarine is a project which allows infra engineer / data scientist to run unmodified Tensorflow programs on YARN.

Goals of Submarine:

  • It allows jobs easy access data/models in HDFS and other storages.
  • Can launch services to serve Tensorflow/MXNet models.
  • Support run distributed Tensorflow jobs with simple configs.
  • Support run user-specified Docker images.
  • Support specify GPU and other resources.
  • Support launch tensorboard for training jobs if user specified.
  • Support customized DNS name for roles (like tensorboard.$user.$domain:6006)

Please jump to QuickStart guide to quickly understand how to use this framework.

Please jump to Examples to try other examples like running Distributed Tensorflow Training for CIFAR 10.

If you're a developer, please find Developer guide for more details.