Creating sandbox with non-existent parent directory fails

[mem]

Version of Singularity:

What version of Singularity are you using? Run:

$ singularity version
3.4.1

Expected behavior

$ ./builddir/singularity build --sandbox $PWD/issue-4517.s/image docker://centos:7

should work

Actual behavior

What actually happend? Why was it incorrect?

$ ./builddir/singularity build --sandbox $PWD/issue-4517.s/image docker://centos:7
INFO:    Starting build...
Getting image source signatures
Skipping fetch of repeat blob sha256:d8d02d45731499028db01b6fa35475f91d230628b4e25fab8e3c015594dc3261
Copying config sha256:acab94af64effb1f7481666a37788e7a59465e723f0b0fe0a0f458f3f4856638
 1.05 KiB / 1.05 KiB [======================================================] 0s
Writing manifest to image destination
Storing signatures
2019/09/25 20:08:25  info unpack layer: sha256:d8d02d45731499028db01b6fa35475f91d230628b4e25fab8e3c015594dc3261
2019/09/25 20:08:26  warn rootless{usr/bin/ping} ignoring (usually) harmless EPERM on setxattr "security.capability"
2019/09/25 20:08:27  warn rootless{usr/sbin/arping} ignoring (usually) harmless EPERM on setxattr "security.capability"
2019/09/25 20:08:27  warn rootless{usr/sbin/clockdiff} ignoring (usually) harmless EPERM on setxattr "security.capability"
INFO:    Creating sandbox directory...
FATAL:   While performing build: sandbox assemble failed: exit status 1: mv: cannot move '/tmp/sbuild-830025760/fs' to '/home/mem/devel/sylabs/singularity/src/github.com/sylabs/singularity/issue-4517.s/image': No such file or directory

Note how this is failing with a "no such file or directory" message.

If the directory $PWD/issue-4517.s is created before running singularity, the process will fail as in issue #4517, which is a different failure.

This one seems to be caused by a non-existent parent directory.

[mem]

Current master does work:

$ rm -rf $PWD/issue-4517.s

$ ./builddir/singularity build --sandbox $PWD/issue-4517.s/image docker://centos:7
INFO:    Starting build...
Getting image source signatures
Skipping fetch of repeat blob sha256:d8d02d45731499028db01b6fa35475f91d230628b4e25fab8e3c015594dc3261
Copying config sha256:acab94af64effb1f7481666a37788e7a59465e723f0b0fe0a0f458f3f4856638
 1.05 KiB / 1.05 KiB [======================================================] 0s
Writing manifest to image destination
Storing signatures
2019/09/25 20:11:54  info unpack layer: sha256:d8d02d45731499028db01b6fa35475f91d230628b4e25fab8e3c015594dc3261
2019/09/25 20:11:54  warn rootless{usr/bin/ping} ignoring (usually) harmless EPERM on setxattr "security.capability"
2019/09/25 20:11:55  warn rootless{usr/sbin/arping} ignoring (usually) harmless EPERM on setxattr "security.capability"
2019/09/25 20:11:55  warn rootless{usr/sbin/clockdiff} ignoring (usually) harmless EPERM on setxattr "security.capability"
INFO:    Creating sandbox directory...
INFO:    Build complete: /home/mem/devel/sylabs/singularity/src/github.com/sylabs/singularity/issue-4517.s/image

this seems to be specific to 3.4.1 (and before?)

[DrDaveD]

So which PR in master fixed this and should be backported to release-3.4?

[dtrudg]

Fixed in the 3.5 series within PRs that changed the sandbox mv vs in-place behaviour