Managing

Configuration

Configuration is done in go-d.ini . This file is used by all GoDocker components.

You can use the go-d.ini.example as a basis example.

File uses the YAML syntax.

Many parameters can be overriden at runtime with environment variables (see README.md for the complete list). Those are prefixed with GODOCKER, for example one can set GODOCKER_REDIS_HOST to set the Redis database host, based on a dynamic dns host name in the Docker or cloud setup.

Requirements

A shared directory (nfs, ...) need to be accessible on all slave nodes, as well as on all nodes where GoDocker components are installed (scheduler, watcher, etc.).

Processes

go-d-scheduler, go-d-watcher, etc. have start/stop methods. Use stop when possible and avoid killing the process as cleanup will not be done and may lead in mesos to restart with a different framework Id.

Execute process with no argument to get help.

Stopping a process will not impact running jobs but will either prevent scheduling new jobs or updating job status.

Users

If using local system users, users need to be manually created via scripts in seed directory.

If using LDAP, users can connect using their credentials, and are automatically created in GoDocker at first login.

Software will get user uids, gids from web server hosts using Linux commands. Email, home directory etc.. will be extracted from LDAP.

When a job is executed, if no root access is required, job will use user id and group ids, allowing native access rights to the volume files and directories.

Files created in job directory will be owned by the user, allowing user direct access from a server with shared directory mount.

It is possible to modify default quotas and priority per user in the administration tab of the web server.

Users also have a user data directory (with quotas) to read/write files. He can access it via the FTP server. This directory can be used by guest users to upload some data to analyse before submitting a job.

Guest users

If guest support is configured, external users can connect using Google,etc. credentials.

Guest users have to be activated after a first login by administrator to submit jobs.

When a guest submit a job, he will run jobs with configured user id/gids but will not have access to his home directory, he will only have access to the job directory or user data directory.

Volumes

The list of allowed volumes in container are configured in go-d.ini. This is a static list of volume name/path/acl. Selected volumes will be mounted in container when requested.

home volume is a specific volume that will get user home directory.

If some users need a specific volume access (directory shared by a team for example), it is possible to create a project (see Projects). Project can have a list of directories to mount in container.

Those directories will be automatically mounted and accessible only to the users declared in the project.

Projects

Projects are a way to group users will common quotas and priority. If no project is selected at job creation, default project will be used.

Allowed users have to been manually added in project. If a user is in a project, project will be proposed in web interface at job creation.

Project can also contain a list of volumes to be mounted in container when project is selected.

Only administrator has access to projects.

Project owner is only informative, as a contact point for this project. Project owner is not by default a member of the project. If needed, he must be added to the member list.

Maintenance

Administrator can switch maintenance on/off.

When maintenance is on, users have read access to their job, but cannot submit new ones.

Running jobs continue to live during this time.

When a Mesos, Swarm, ... node is put under maintenance (via Mesos, ... operations), GoDocker will stop scheduling jobs to those nodes but will not kill running jobs.

Cleanup

go-d-archive daemon service manage the archiving of the jobs. When a job is archived, it is kept in database but all files in the job directory are deleted (and user quota updated).

User can archive a job via the Web UI/API or the admin can cron the archive of the old job via the go-d-clean.py script.

go-d-clean.py, with no arguments, clean all old jobs. With arguments (job identifiers), specified jobs will be archived, whatever their age.