Welcome to GoDocker project

GoDocker is a batch scheduler for computing tasks, based on Docker. It acts as a frontend above several Docker scheduler/management tools (Docker Swarm, Kubernetes, Apache Mesos) for end users in a multi user system (ldap, ...).

With GoDocker, end user can submit some jobs (basically a script) to execute on a remote node, accoding to its requirements (cpu, mem, ports, gpu, ..). The software will mount required and authorized directories (home directory, some shared resources) in the container, directories being limited and controlled by GoDocker.

It also manages the scheduling of the tasks according to user/projects priorities and quotas.

At last, user can connect to the container for interactive sessions via SSH.

All components are scalable and can be set in HA or distributed mode, and all components can be monitored (Prometheus metrics, Consul or etcd checks, ...).

Most of the features are available via plugins, meaning that new systems or features can be added to extend the software.

GoDocker will give the power of Docker images (reproductibility, easy tool usage, ...) to users not having Docker priviledges on your system, it will control and limit the resources a user can use.

From a end-user perspective, user jobs will be run in containers meaning they are isolated from other jobs (cpu and mem reservation). He can also execute his job in a container specific to his tasks with all the software he needs, and using different Linux operating systems (Debian, CentOS, ...).

Look at feature for more details.

Releases

https://godocker.atlassian.net/projects/GOD?selectedItem=com.atlassian.jira.jira-projects-plugin:release-page&status=released

Features

Setup

If not using Mesos, and not willing to install mesos python libraries, it is possible to delete plugins/mesos.* files.

Docker swarm

Swarm must listen on a tcp port, no specific requirement

2 plugins are available: swarm for the standalone swarm manager, and dockerswarm for the docker integrated swarm mode.

Kubernetes

No specific requirement

Apache Mesos

Mesos is the favorite framework for GoDocker as it provides deeper control on tasks and provides more feature.

Depending on Mesos release, one may need to install python eggs on scheduler node. Recent releases of Mesos install them with their packages, but some older releases provide them separately. From go-docker 1.3.2 (included), HTTP API of mesos is used and software does not need anymore mesos python libs installation.

Increase executor timeout for image pulls:

echo '5mins' > /etc/mesos-slave/executor_registration_timeout

typical slave config for GoDocker (/etc/mesos-slave directory):

[mesos-slave]# ls
attributes containerizers executor_registration_timeout isolation
[mesos-slave] cat attributes
hostname:192.168.1.37   <= IP of the slave node, optional from go-docker >= 1.3.2
[mesos-slave] cat containerizers 
docker,mesos
[mesos-slave] cat executor_registration_timeout 
5mins
[mesos-slave] cat isolation
filesystem/linux,docker/runtime,cgroups/cpu,cgroups/mem,network/cni

filesystem/linux,docker/runtime isolation params are needed for Mesos unified containerizer.

cgroups/cpu,cgroups/mem,network/cni are needed for CNI support with Docker images.

From go-docker 1.3.2, hostname in attributes is not needed anymore as long as slave is declared with a routable hostname in the network. If not the case, then hostname attribute still need to bne declared.

For CNI support, one need additional files network_cni_config_dir and network_cni_plugins_dir (see Mesos configuration/setup guide)

For unified containerize, an other file is needed, image_providers, with content docker

More info on Mesos: http://mesos.apache.org/documentation/latest/container-image/, http://mesos.apache.org/documentation/latest/mesos-containerizer/

Installation

see README.md and INSTALL.md

cAdvisor must be deployed on all nodes for monitoring (global cpu/ram/io usage).

docker run --volume=/:/rootfs:ro --volume=/var/run:/var/run:rw --volume=/sys:/sys:ro --volume=/var/lib/docker/:/var/lib/docker:ro --publish=8080:8080 --detach=true --name=cadvisor google/cadvisor:latest

Or one can use bubble-chamber project (https://github.com/osallou/sysdig-analyser) in place of cadvisor.

Bubble-chamber records containers usage with details of what occurs in the container (cpu/ram/io per process within the container), based on sysdig. With bubble-chamber, user and/or admin can view, during and after job completion, what are the precise resource usage per process, which files were accessed (read/write but also system files access). Stats are recorded for a defined amount of time, with details per minutes/hour/day.