What is Singularity?
Singularity is a Linux containerisation tool suitable for HPC environments. It uses its own container format and also has features to import Docker containers.
What is Docker?
Docker is a platform that employs features of the Linux kernel to run software in a container. The software housed in a docker container is not just a single program but an entire OS distribution, or at least enough of the OS to enable the program to work.
Docker can be thought of as somewhat like a software distribution mechanism like yum or apt. It also can be thought of as an expanded version of a chroot jail, or a reduced version of a virtual machine.
Read more about Docker on the official web site.
Docker vs Singularity
Here are some important differences between Docker and Singularity.
- Docker and Singularity have their own container formats
- Docker containers may be imported to run via Singularity. A full example is below.
- Docker containers usually run as root. However as a Melbourne Bioinformatics (formerly VLSCI) user you have a regular user account which does not have root privileges.
- Singularity allows for containers that run as a regular user. When importing a Docker container, Singularity removes any elements which can only run as root. The resulting containers are therefore able to run as a regular user.
Importing a Docker image
If you are working on a Docker container, you may use Singularity to convert this container to the Singularity format. Once done you may upload your Singularity container to a Melbourne Bioinformatics cluster such as snowy or barcoo, and run jobs using your container. The container is kept in your project's own storage area. This is different from Shifter which imports containers via Docker hub.
Here is an example. Note that most of these steps are performed on your local system, not a Melbourne Bioinformatics cluster!
Melbourne Bioinformatics filesystems
If you need to use the Melbourne Bioinformatics filesystems,
vlsci inside your container, you will need to make sure folders exist for those filesystems to be mounted on.
Start your container (in this example we will use
ubuntu:latest) and create those three directories.
$ sudo docker run -it ubuntu:latest bash root@119565e80448:/# cd / root@119565e80448:/# mkdir hsm scratch vlsci
Exporting your Docker image
Find the name of your Docker image using
$ sudo docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS NAMES 119565e80448 ubuntu:latest "bash" 2 minutes ago Up 2 minutes boring_nobel
In this example the name is
boring_nobel. Export this image to a tarball
$ sudo docker export boring_nobel > ubuntu.tar
Preparing a Singularity image
You will need to have Singularity installed on your local computer in order to prepare your image. This is because the 'create' and 'import' operations of Singularity require root, which you do not have on Melbourne Bioinformatics systems.
Create an empty singularity image, and then import the exported docker image into it:
$ sudo singularity create ubuntu.img Creating a sparse image with a maximum size of 1024MiB... Using given image size of 1024 Formatting image (/sbin/mkfs.ext3) Done. Image can be found at: ubuntu.img $ sudo singularity import ubuntu.img ubuntu.tar
Uploading your Singularity image
You should upload your new singularity image to a Melbourne Bioinformatics cluster using your usual method. For example, to use
scp to upload to
snowy you might do:
$ scp ubuntu.img firstname.lastname@example.org:
Using your Singularity image
From here onwards, the steps are performed on a Melbourne Bioinformatics cluster. We previously transferred
ubuntu.img to snowy.
If you ssh into snowy and load the
singularity module, you can then run commands inside your container.
$ ssh email@example.com $ module load singularity $ singularity exec ubuntu.img /bin/bash $ cat /etc/issue Ubuntu 16.04.1 LTS \n \l
If you created the three directories for Melbourne Bioinformatics filesystems you should find they are mounted inside your container.
$ mount | grep gpfs /dev/projects on /vlsci type gpfs (rw,relatime) /dev/hsm on /hsm type gpfs (rw,relatime) /dev/scratch on /scratch type gpfs (rw,relatime)
WARNING If your container mounts the /vlsci, /hsm or /scratch directories, software inside the container may be able to destroy any data on these filesystems for which you have write permissions. You should proceed with caution.
Using Singularity containers inside a SLURM job
It's straightforward to run your jobs inside a Singularity container.
In your sbatch script, ensure you load the singularity module. You can run any job inside the container by prefacing it with
singularity exec <your image name>.
Here is a trivial example which we will save as
#!/bin/bash #SBATCH --job-name=singularity_test #SBATCH -p main #SBATCH --ntasks=1 module load singularity # where are we running? hostname singularity exec ubuntu.img hostname # which OS is this? cat /etc/issue singularity exec ubuntu.img cat /etc/issue
If you run this with sbatch:
$ sbatch singularity_test.slurm Submitted batch job 915487
Once the job completes, let's have a look at the result.
$ cat slurm-915487.out snowy006 snowy006 Red Hat Enterprise Linux Server release 6.8 (Santiago) Kernel \r on an \m Ubuntu 16.04.1 LTS \n \l
Lines 1 and 3 of the output were run directly on the OS of the compute node which handled your job. Lines 2 and 4 were run on the same compute node but inside the Singularity container.
If you have any questions about Singularity, the Melbourne Bioinformatics team is always able to help.