Open Science Grid¶

Preparing a Docker container for use at Open Science Grid¶

For a variety of reasons, Open Science Grid (OSG) will not take Singularity containers; rather, they prefer Docker containers registered at Docker Hub.

This is a short note of ARC staff's first experience preparing one for OSG for a group of graduate students in SEAS.

First thing was to register at Docker Hub so there is a username and password that can be used. This example will use Bennet's personal Docker Hub name throughout.

Install Docker on a machine on which you have root. I used an older Dell PowerEdge workstation running Ubuntu 18.04 to create the image.

The following were preparatory steps

$ sudo apt-get install apt-transport-https ca-certificates curl \
    gnupg-agent software-properties-common
$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
# Verify fingerprint
$ sudo apt-key fingerprint 0EBFCD88

Add this line to the end of /etc/apt/sources.list

deb [arch=amd64] https://download.docker.com/linux/ubuntu bionic stable

Install Docker, and run two basic tests

$ sudo apt-get install docker-ce docker-ce-cli containerd.io
$ apt-cache madison docker-ce
$ sudo docker run hello-world

Create something small to test that we can create an image and run it to get a prompt. Create a Dockerfile containing (the timezone is magic, but you need it)

FROM ubuntu:18.04
ENV LANG="C"
ENV R_LIBS_SITE="/usr/local/Rlibs"
ENV TZ=America/Detroit

RUN mkdir -p /usr/local/Rlibs
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
RUN apt-get update
RUN apt-get -y install software-properties-common

Now build it with

$ sudo docker build -t ubuntu/test -f Dockerfile $PWD

and run it with this, where the -it gets you an interactive terminal, the --rm instructs Docker to remove the image at the conclusion of the run, and finally the shell (or other program) you want to run.

$ sudo docker run -it --rm ubuntu/test /bin/bash

Now, you finish the Dockerfile, build the image, and run it interactively to test that the program(s) you installed inside really work.

$ sudo docker build -t seas/lidar -f Dockerfile $PWD
$ sudo docker run -it --rm seas/lidar /bin/bash
docker# R
>   [etc.]
>q()

Next, you use docker to log into https://hub.docker.com with

$ sudo docker login --username=justbennet

Get the image ID from

$ sudo docker images

and create a tag to use at Docker Hub,

$ sudo docker tag 091239e28f00 justbennet/seas:lidar

and finally, push it to Docker Hub,

$ sudo docker push justbennet/seas:lidar

You can now use

$ sudo singularity build lidar.sif docker://justbennet/seas:lidar

to build the Singularity image and verify that it still works.