Initial commit

README.md

@@ -1,2 +1,10 @@
 # singularity-test-containers
-Dockerfiles / Definition files used for containers pulled from Singularity tests
+
+Dockerfiles & Definition files used for containers pulled from the Singularity
+test suite.
+
+ - `docker-aufs-sanity`
+ - `docker-singularity-userperms`
+ - `docker-singularity-linkwh`
+
+

docker-aufs-sanity/Dockerfile

@@ -0,0 +1,20 @@
+FROM alpine
+MAINTAINER David Trudgian
+
+# Create 3 directories, eache containing file1
+RUN mkdir -p /test/whiteout-dir /test/whiteout-file /test/normal-dir
+RUN touch /test/whiteout-dir/file1 /test/whiteout-file/file1 /test/normal-dir/file1
+
+# Remove opaque-dir, re-create it
+RUN rm /test/whiteout-dir/file1
+RUN rmdir /test/whiteout-dir
+RUN mkdir /test/whiteout-dir
+
+# Put a file 2 in each dir
+RUN touch /test/whiteout-dir/file2 /test/whiteout-file/file2 /test/normal-dir/file2
+
+# Remove file1 from the whiteout dir
+RUN rm /test/whiteout-file/file1
+
+CMD find /test
+

docker-aufs-sanity/README.md

@@ -0,0 +1,75 @@
+# docker-aufs-sanity
+
+[![Docker Pulls](https://img.shields.io/docker/pulls/dctrud/docker-aufs-sanity.svg)](https://hub.docker.com/r/dctrud/docker-aufs-sanity/)
+
+This repo contains a Dockerfile defining the container `dctrud/docker-aufs-sanity` on [docker hub](https://hub.docker.com/r/dctrud/docker-aufs-sanity/).
+
+This is a small docker container that can be used to test container software that extracts docker multi-layer images (e.g. Singularity), to ensure that they handle aufs whiteout correctly.
+
+If your extraction of docker layers handles whiteout correctly then you should see the following when you run the container:
+
+```bash
+$ docker run dctrud/docker-aufs-sanity
+/test
+/test/normal-dir
+/test/normal-dir/file1
+/test/normal-dir/file2
+/test/whiteout-dir
+/test/whiteout-dir/file2
+/test/whiteout-file
+/test/whiteout-file/file2
+```
+
+If your aufs extraction is incorrect you will likely see an additional `file1` inside the directories `whiteout-dir/`, `whiteout-file/`, or both. You will also likely see the whiteout marker files (beginning `.wh.`) E.g. with Singularity v2.4:
+
+```bash
+$ singularity run docker://dctrud/docker-aufs-sanity
+/test
+/test/normal-dir
+/test/normal-dir/file1
+/test/normal-dir/file2
+/test/whiteout-dir
+/test/whiteout-dir/file1
+/test/whiteout-dir/.wh.file1
+/test/whiteout-dir/file2
+/test/whiteout-file
+/test/whiteout-file/file1
+/test/whiteout-file/file2
+/test/whiteout-file/.wh.file1
+/test/.wh.whiteout-dir
+```
+
+
+## Explanation
+
+Each Dockerfile `RUN` step creates a separate container filesystem layer, with content that builds upon previous layers. When you `rm` a file inside a `RUN` step it must be removed from your container - but the `RUN` cannot modify the previous immutable layers. Docker uses the aufs whiteout standard to record when you `rm` a file that was created in a previous layer. If you create a file `file1` in layer 1, and remove it in layer 2, then the tar for layer 2 will contain a hidden marker file `.wh.file1` which tells us we should remove `file1` as we extract layer 2.
+
+### About this test
+
+Here we create 3 directories to test directory, file, and no whiteout.
+
+**Directory Whiteout**
+
+Directory `/test/whiteout-dir/` is created, and has a single file, `file1` added inside it.
+
+We then remove `file1` and `/test/whiteout-dir/` itself. At this point there is no `/test/whiteout-dir/` visible in our container. Docker will mark this by creating a whiteout file `/test/.wh.whiteout-dir` in the layer tar, which instructs the extracting tool to remove the directory during extraction, just as we did in our Dockerfile RUN steps.
+
+Finally we recreate `/test/whiteout-dir/` and put a new file `file2` into it.
+
+If we extract our container layers correctly we should see `/test/whiteout-dir/` containing a single file named `file2`.
+
+
+**File Whiteout**
+
+Directory `/test/whiteout-file/` is created, and has a single file, `file1` added inside it. In a later RUN step we add a new file `file2` into the directory.
+
+Finally, we remove `/test/whiteout-file/file1` in a RUN step. Docker will mark this by creating a whiteout file `/test/whiteout-file/.wh.file1` in the layer tar, which instructs the extracting tool to remove `file1` during extraction, just as we did in our Dockerfile RUN steps.
+
+If we extract our container layers correctly we should see `/test/whiteout-file/` containing a single file named `file2`.
+
+**No Whiteout**
+
+Directory `/test/normal-dir/` has two files, `file1` and `file2`, added to it with no removals made in later RUN steps.
+
+If we extract our container layers correctly we should see `/test/normal-dir/` containing both `file1` and `file2`.
+

docker-singuarity-userperms/Dockerfile

@@ -0,0 +1,14 @@
+FROM alpine
+MAINTAINER David Trudgian
+
+# Create a directory, and put a file in it
+RUN mkdir /testdir && touch /testdir/testfile
+
+# Take away owner perms
+RUN chmod 555 /testdir
+
+# Remove the test file - should dissappear without issue
+RUN rm /testdir/testfile
+
+CMD find /testdir
+

docker-singuarity-userperms/README.md

@@ -0,0 +1,47 @@
+# docker-singularity-userperms
+
+This is a test container for Singularity issue #977.
+
+If a docker container holds a file or directory without owner write perms, and singularity build is run as a user (non-root), we must force owner write perms at extraction. If we don't do this, the file or dir in question cannot be modified at later layers, and build may fail.
+
+
+**Expected Behaviour, fixed in PR #1295**
+
+Container builds OK, and on run there is nothing listed inside /testdir.
+
+```
+$ singularity pull docker://dctrud/docker-singularity-userperms
+WARNING: pull for Docker Hub is not guaranteed to produce the
+WARNING: same image on repeated pull. Use Singularity Registry
+WARNING: (shub://) to pull exactly equivalent images.
+Docker image path: index.docker.io/dctrud/docker-singularity-userperms:latest
+Cache folder set to /home/dave/.singularity/docker
+Importing: base Singularity environment
+WARNING: Building container as an unprivileged user. If you run this container as root
+WARNING: it may be missing some functionality.
+Building Singularity FS image...
+Building Singularity SIF container image...
+Singularity container built: ./docker-singularity-userperms.simg
+Cleaning up...
+Done. Container is at: ./docker-singularity-userperms.simg
+
+$ singularity run docker-singularity-userperms.simg 
+/testdir
+```
+
+**Prior Behaviour**
+
+```
+$ singularity pull docker://dctrud/docker-singularity-userperms
+WARNING: pull for Docker Hub is not guaranteed to produce the
+WARNING: same image on repeated pull. Use Singularity Registry
+WARNING: (shub://) to pull exactly equivalent images.
+Docker image path: index.docker.io/dctrud/docker-singularity-userperms:latest
+Cache folder set to /home/dave/.singularity/docker
+Importing: base Singularity environment
+ERROR  : Error applying whiteout marker from docker layer.
+ABORT  : Retval = 255
+Cleaning up...
+rm: cannot remove '/tmp/.singularity-build.HkwFPi/testdir/testfile': Permission denied
+ERROR: pulling container failed!
+```

docker-singularity-linkwh/Dockerfile

@@ -0,0 +1,4 @@
+FROM centos:7
+RUN rm /etc/localtime
+RUN cp /usr/share/zoneinfo/America/Los_Angeles /etc/localtime
+

docker-singularity-linkwh/README.md

@@ -0,0 +1,8 @@
+# docker-singularity-userperms
+
+This is a test container for Singularity issue #1592 #1576.
+
+See `e2e/docker/docker.go`
+
+The image should build cleanly. If it does not, then whiteout of symbolic links
+is not working correctly.