"mv: can't preserve ownership of ..." when running rebar3 compile

[ddeboer]

Environment

$ rebar3 report "compile"

Rebar3 report
 version 3.3.1
 generated at 2016-09-12T09:06:46+00:00
=================
Please submit this along with your issue at https://github.com/erlang/rebar3/issues (and feel free to edit out private information, if any)
-----------------
Task: compile
Entered as:
  compile
-----------------
Operating System: x86_64-alpine-linux-musl
ERTS: Erlang/OTP 18 [erts-7.3.1] [source] [64-bit] [smp:2:2] [async-threads:0] [kernel-poll:false]
Root Directory: /usr/lib/erlang
Library directory: /usr/lib/erlang/lib
-----------------
Loaded Applications:
bbmustache: 1.0.4
certifi: 0.4.0
cf: 0.2.1
compiler: 6.0.3
crypto: 3.6.3
cth_readable: 1.2.3
erlware_commons: 0.21.0
eunit: 2.2.13
eunit_formatters: 0.3.1
getopt: 0.8.2
inets: 6.2.2
kernel: 4.2
providers: 1.6.0
public_key: 1.1.1
relx: 3.21.0
sasl: 2.7
snmp: 5.2.2
ssl_verify_fun: 1.1.1
stdlib: 2.8
syntax_tools: 1.7
tools: 2.8.3

-----------------
Escript path: /opt/zotonic/rebar3
Providers:
  app_discovery as clean clean compile compile compile cover ct deps dialyzer do edoc escriptize eunit help install install_deps list lock new path pkgs release relup report shell state tar tree unlock update upgrade upgrade upgrade version xref

Current behaviour

Rebar clones the dependencies in /tmp and then moves them to their destination path. However, mv tries to preserve permissions, which doesn’t always work when moving the files to a different filesystem than /tmp is on. My use case: a Docker container running on Docker for Mac, which relies on the fuse.osxfs filesystem.

$ DEBUG=1 rebar3 compile

===> Moving checkout "/tmp/.tmp_dir714388033350" to "/opt/zotonic/_build/default/lib/gen_smtp"
===> sh info:
    cwd: "/opt/zotonic"
    cmd: mv /tmp/.tmp_dir714388033350 /opt/zotonic/_build/default/lib/gen_smtp

===>    opts: [{use_stdout,false},abort_on_error]

===> Port Cmd: mv /tmp/.tmp_dir714388033350 /opt/zotonic/_build/default/lib/gen_smtp
Port Opts: [exit_status,{line,16384},use_stdio,stderr_to_stdout,hide,eof]

===> rebar_fetch exception error {badmatch,
                                         {ok,
                                          "mv: can't preserve ownership of '/opt/zotonic/_build/default/lib/gen_smtp/.git/objects/pack/pack-6bd4ff9f801691ecabe6ef19c7dc5ccb9fcf1e79.idx': Permission denied\nmv: can't preserve ownership of '/opt/zotonic/_build/default/lib/gen_smtp/.git/objects/pack/pack-6bd4ff9f801691ecabe6ef19c7dc5ccb9fcf1e79.pack': Permission denied\n"}} [{rebar_file_utils,

Expected behaviour

Installing the dependencies should work even if file permissions can’t be guaranteed. Some possible solutions:

1. cp && rm the files, as copy will not try to preserve file permissions. This may be slower, however, when moving within the same filesystem.

2. The problematic files, that are owned444, are only in the .git directory:

   bash-4.3# ls -alt
   total 1800
   drwxr-xr-x    2 root     root          4096 Sep 12 09:13 .
   -r--r--r--    1 root     root        105036 Sep 12 09:13 pack-0b545f8d9e8a31ab894676fa0d209069afdd04d2.idx
   -r--r--r--    1 root     root       1725727 Sep 12 09:13 pack-0b545f8d9e8a31ab894676fa0d209069afdd04d2.pack
   drwxr-xr-x    4 root     root          4096 Sep 12 09:13 ..
   

   So another solution is to rm .git before the file is moved. Or does Rebar rely on the .git directory in the dependency directories?

[ferd]

We do rely on the .git directory to validate versions, so that isn't a good idea. Would removing file permissions risk breaking some files (say pre-written escripts) which require executable rights? In such a case, I don't think dropping permissions would be acceptable.

[ddeboer]

I understand.

A third, and better, solution then is to keep using mv but see whether we can ignore the error message that mv returns. After all, the files are moved successfully, and only lose their permissions when transferring between (some) filesystems.

[ddeboer]

Alternatively, clone the files directly in the destination directory (instead of first in /tmp and then moving them). Is there a reason for cloning in /tmp first?

[ferd]

Yeah, this allows to avoid clearing up the directory if it exists first, I believe, which is faster.

It would be a good idea to possibly just parse the return value and promote it to a warning and see it as a success however, rather than a fatal error in this case. I like that solution personally.

[ddeboer]

Agreed on the solution.

I’ll try to make some time and a PR. 😉

[ferd]

Thanks! Let us know if you need any help and we'll be glad to provide it!