multiple_app_files error point to the same file

[kpy3]

Hello,

rebar3 crashes on upgrade/tree command with error

===> Uncaught error: multiple_app_files ["/Users/yes/Projects/q/_build/default/lib/riak_kv/src/riak_kv.app.src",                                                "/Users/yes/Projects/q/_build/default/lib/riak_kv/src/riak_kv.app.src"]

Rebar3 report
 version 3.4.3
 generated at 2017-08-30T10:28:46+00:00
=================
Please submit this along with your issue at https://github.com/erlang/rebar3/issues (and feel free to edit out private information, if any)
-----------------
Task: rebar3
Entered as:
  rebar3 upgrade
-----------------
Operating System: x86_64-apple-darwin16.6.0
ERTS: Erlang/OTP 17 [erts-6.3.1] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:0] [hipe] [kernel-poll:false] [dtrace]
Root Directory: /Users/yes/.kerl/install/17.4.1
Library directory: /Users/yes/.kerl/install/17.4.1/lib
-----------------
Loaded Applications:
bbmustache: 1.3.0
certifi: 2.0.0
cf: 0.2.2
common_test: 1.9
compiler: 5.0.3
crypto: 3.4.2
cth_readable: 1.3.0
dialyzer: 2.7.3
edoc: 0.7.16
erlware_commons: 1.0.0
eunit: 2.2.9
eunit_formatters: 0.3.1
getopt: 0.8.2
inets: 5.10.5
kernel: 3.1
providers: 1.6.0
public_key: 0.22.1
relx: 3.23.1
sasl: 2.4.1
snmp: 5.1.1
ssl_verify_fun: 1.1.2
stdlib: 2.3
syntax_tools: 1.6.17
tools: 2.7.1

-----------------
Escript path: undefined
Providers:
  app_discovery as clean clean compile compile compile config cover ct cut deps dialyzer do docs edoc escriptize eunit get-deps help info install install_deps key list lock new owner path pkgs publish release relup report search shell state tar tree unlock update upgrade upgrade upgrade user version xref 

Here is debug output

===> Expanded command sequence to be run: [{default,app_discovery},
                                                  {default,install_deps},
                                                  {default,lock},
                                                  {default,upgrade}]
===> Evaluating config script "/Users/yes/Projects/q/_build/default/lib/decimal/rebar.config.script"
===> Evaluating config script "/Users/yes/Projects/q/_build/default/lib/decimal/rebar.config.script"
===> Evaluating config script "/Users/yes/Projects/q/_build/default/lib/erocksdb/rebar.config.script"
===> Evaluating config script "/Users/yes/Projects/q/_build/default/lib/erocksdb/rebar.config.script"
===> Evaluating config script "/Users/yes/Projects/q/_build/default/lib/gproc/rebar.config.script"
===> Evaluating config script "/Users/yes/Projects/q/_build/default/lib/gproc/rebar.config.script"
===> Evaluating config script "/Users/yes/Projects/q/_build/default/lib/jiffy/rebar.config.script"
===> Evaluating config script "/Users/yes/Projects/q/_build/default/lib/jiffy/rebar.config.script"
===> Evaluating config script "/Users/yes/Projects/q/_build/default/lib/lager/rebar.config.script"
===> Evaluating config script "/Users/yes/Projects/q/_build/default/lib/lager/rebar.config.script"
===> Not adding provider hex user from module rebar3_hex_user because it already exists from module rebar3_hex_user
===> Not adding provider hex cut from module rebar3_hex_cut because it already exists from module rebar3_hex_cut
===> Not adding provider hex config from module rebar3_hex_config because it already exists from module rebar3_hex_config
===> Not adding provider hex key from module rebar3_hex_key because it already exists from module rebar3_hex_key
===> Not adding provider hex info from module rebar3_hex_info because it already exists from module rebar3_hex_info
===> Not adding provider hex owner from module rebar3_hex_owner because it already exists from module rebar3_hex_owner
===> Not adding provider hex docs from module rebar3_hex_docs because it already exists from module rebar3_hex_docs
===> Not adding provider hex search from module rebar3_hex_search because it already exists from module rebar3_hex_search
===> Not adding provider hex publish from module rebar3_hex_pkg because it already exists from module rebar3_hex_pkg
===> Not adding provider asn compile from module provider_asn1_compile because it already exists from module provider_asn1_compile
===> Not adding provider asn clean from module provider_asn1_clean because it already exists from module provider_asn1_clean
===> Verifying dependencies...
===> sh info:
	cwd: "/Users/yes/Projects/q"
	cmd: git rev-parse --short=7 -q HEAD

===> 	opts: [{cd,"/Users/yes/Projects/q/_build/default/lib/backoff"}]

===> Port Cmd: git rev-parse --short=7 -q HEAD
Port Opts: [{cd,"/Users/yes/Projects/q/_build/default/lib/backoff"},
            exit_status,
            {line,16384},
            use_stdio,stderr_to_stdout,hide,eof]

===> Comparing git ref d91c51e with d91c51e
===> sh info:
	cwd: "/Users/yes/Projects/q"
	cmd: git rev-parse --short=7 -q HEAD

===> 	opts: [{cd,"/Users/yes/Projects/q/_build/default/lib/bcmon"}]

===> Port Cmd: git rev-parse --short=7 -q HEAD
Port Opts: [{cd,"/Users/yes/Projects/q/_build/default/lib/bcmon"},
            exit_status,
            {line,16384},
            use_stdio,stderr_to_stdout,hide,eof]

===> Comparing git ref 2d11095 with 2d11095
===> Evaluating config script "/Users/yes/Projects/q/_build/default/lib/lager/rebar.config.script"
===> Evaluating config script "/Users/yes/Projects/q/_build/default/lib/lager/rebar.config.script"
===> sh info:
	cwd: "/Users/yes/Projects/q"
	cmd: git rev-parse --short=7 -q HEAD

===> 	opts: [{cd,"/Users/yes/Projects/q/_build/default/lib/cowboy"}]

===> Port Cmd: git rev-parse --short=7 -q HEAD
Port Opts: [{cd,"/Users/yes/Projects/q/_build/default/lib/cowboy"},
            exit_status,
            {line,16384},
            use_stdio,stderr_to_stdout,hide,eof]

===> Comparing git ref d08c2ab with d08c2ab
===> sh info:
	cwd: "/Users/yes/Projects/q"
	cmd: git rev-parse --short=7 -q HEAD

===> 	opts: [{cd,"/Users/yes/Projects/q/_build/default/lib/cowboy_param_extractor"}]

===> Port Cmd: git rev-parse --short=7 -q HEAD
Port Opts: [{cd,"/Users/yes/Projects/q/_build/default/lib/cowboy_param_extractor"},
            exit_status,
            {line,16384},
            use_stdio,stderr_to_stdout,hide,eof]

===> Comparing git ref 0b57efc with 0b57efc
===> sh info:
	cwd: "/Users/yes/Projects/q"
	cmd: git rev-parse --short=7 -q HEAD

===> 	opts: [{cd,"/Users/yes/Projects/q/_build/default/lib/decimal"}]

===> Port Cmd: git rev-parse --short=7 -q HEAD
Port Opts: [{cd,"/Users/yes/Projects/q/_build/default/lib/decimal"},
            exit_status,
            {line,16384},
            use_stdio,stderr_to_stdout,hide,eof]

===> Comparing git ref 70a7e65 with 70a7e65
===> Evaluating config script "/Users/yes/Projects/q/_build/default/lib/decimal/rebar.config.script"
===> sh info:
	cwd: "/Users/yes/Projects/q"
	cmd: git rev-parse --short=7 -q HEAD

===> 	opts: [{cd,"/Users/yes/Projects/q/_build/default/lib/edown"}]

===> Port Cmd: git rev-parse --short=7 -q HEAD
Port Opts: [{cd,"/Users/yes/Projects/q/_build/default/lib/edown"},
            exit_status,
            {line,16384},
            use_stdio,stderr_to_stdout,hide,eof]

===> Comparing git ref 4d18dc4 with 4d18dc4
===> sh info:
	cwd: "/Users/yes/Projects/q"
	cmd: git rev-parse --short=7 -q HEAD

===> 	opts: [{cd,"/Users/yes/Projects/q/_build/default/lib/eep"}]

===> Port Cmd: git rev-parse --short=7 -q HEAD
Port Opts: [{cd,"/Users/yes/Projects/q/_build/default/lib/eep"},
            exit_status,
            {line,16384},
            use_stdio,stderr_to_stdout,hide,eof]

===> Comparing git ref 8f6e5e3 with 8f6e5e3
===> sh info:
	cwd: "/Users/yes/Projects/q"
	cmd: git rev-parse --short=7 -q HEAD

===> 	opts: [{cd,"/Users/yes/Projects/q/_build/default/lib/erlang_js"}]

===> Port Cmd: git rev-parse --short=7 -q HEAD
Port Opts: [{cd,"/Users/yes/Projects/q/_build/default/lib/erlang_js"},
            exit_status,
            {line,16384},
            use_stdio,stderr_to_stdout,hide,eof]

===> Comparing git ref fcf8476 with fcf8476
===> sh info:
	cwd: "/Users/yes/Projects/q"
	cmd: git rev-parse --short=7 -q HEAD

===> 	opts: [{cd,"/Users/yes/Projects/q/_build/default/lib/erocksdb"}]

===> Port Cmd: git rev-parse --short=7 -q HEAD
Port Opts: [{cd,"/Users/yes/Projects/q/_build/default/lib/erocksdb"},
            exit_status,
            {line,16384},
            use_stdio,stderr_to_stdout,hide,eof]

===> Comparing git ref e6dfa13 with e6dfa13
===> Evaluating config script "/Users/yes/Projects/q/_build/default/lib/erocksdb/rebar.config.script"
===> sh info:
	cwd: "/Users/yes/Projects/q"
	cmd: git rev-parse --short=7 -q HEAD

===> 	opts: [{cd,"/Users/yes/Projects/q/_build/default/lib/folsomite"}]

===> Port Cmd: git rev-parse --short=7 -q HEAD
Port Opts: [{cd,"/Users/yes/Projects/q/_build/default/lib/folsomite"},
            exit_status,
            {line,16384},
            use_stdio,stderr_to_stdout,hide,eof]

===> Comparing git ref 0d94fe3 with 0d94fe3
===> Evaluating config script "/Users/yes/Projects/q/_build/default/lib/folsom/rebar.config.script"
===> Evaluating config script "/Users/yes/Projects/q/_build/default/lib/folsom/rebar.config.script"
===> sh info:
	cwd: "/Users/yes/Projects/q"
	cmd: git rev-parse --short=7 -q HEAD

===> 	opts: [{cd,"/Users/yes/Projects/q/_build/default/lib/gen_leader"}]

===> Port Cmd: git rev-parse --short=7 -q HEAD
Port Opts: [{cd,"/Users/yes/Projects/q/_build/default/lib/gen_leader"},
            exit_status,
            {line,16384},
            use_stdio,stderr_to_stdout,hide,eof]

===> Comparing git ref a119cb9 with a119cb9
===> sh info:
	cwd: "/Users/yes/Projects/q"
	cmd: git rev-parse --short=7 -q HEAD

===> 	opts: [{cd,"/Users/yes/Projects/q/_build/default/lib/gh_streamer"}]

===> Port Cmd: git rev-parse --short=7 -q HEAD
Port Opts: [{cd,"/Users/yes/Projects/q/_build/default/lib/gh_streamer"},
            exit_status,
            {line,16384},
            use_stdio,stderr_to_stdout,hide,eof]

===> Comparing git ref 28cd622 with 28cd622
===> Evaluating config script "/Users/yes/Projects/q/_build/default/lib/gproc/rebar.config.script"
===> Evaluating config script "/Users/yes/Projects/q/_build/default/lib/gproc/rebar.config.script"
===> Evaluating config script "/Users/yes/Projects/q/_build/default/lib/jiffy/rebar.config.script"
===> Evaluating config script "/Users/yes/Projects/q/_build/default/lib/jiffy/rebar.config.script"
===> Evaluating config script "/Users/yes/Projects/q/_build/default/lib/lager/rebar.config.script"
===> Evaluating config script "/Users/yes/Projects/q/_build/default/lib/lager/rebar.config.script"
===> sh info:
	cwd: "/Users/yes/Projects/q"
	cmd: git rev-parse --short=7 -q HEAD

===> 	opts: [{cd,"/Users/yes/Projects/q/_build/default/lib/gh_utils"}]

===> Port Cmd: git rev-parse --short=7 -q HEAD
Port Opts: [{cd,"/Users/yes/Projects/q/_build/default/lib/gh_utils"},
            exit_status,
            {line,16384},
            use_stdio,stderr_to_stdout,hide,eof]

===> Comparing git ref 0d79b15 with 0d79b15
===> Evaluating config script "/Users/yes/Projects/q/_build/default/lib/jiffy/rebar.config.script"
===> Evaluating config script "/Users/yes/Projects/q/_build/default/lib/jiffy/rebar.config.script"
===> Evaluating config script "/Users/yes/Projects/q/_build/default/lib/lager/rebar.config.script"
===> Evaluating config script "/Users/yes/Projects/q/_build/default/lib/lager/rebar.config.script"
===> sh info:
	cwd: "/Users/yes/Projects/q"
	cmd: git rev-parse --short=7 -q HEAD

===> 	opts: [{cd,"/Users/yes/Projects/q/_build/default/lib/gproc"}]

===> Port Cmd: git rev-parse --short=7 -q HEAD
Port Opts: [{cd,"/Users/yes/Projects/q/_build/default/lib/gproc"},
            exit_status,
            {line,16384},
            use_stdio,stderr_to_stdout,hide,eof]

===> Comparing git ref 1d16f5e with 1d16f5e
===> Evaluating config script "/Users/yes/Projects/q/_build/default/lib/gproc/rebar.config.script"
===> sh info:
	cwd: "/Users/yes/Projects/q"
	cmd: git rev-parse --short=7 -q HEAD

===> 	opts: [{cd,"/Users/yes/Projects/q/_build/default/lib/intervals"}]

===> Port Cmd: git rev-parse --short=7 -q HEAD
Port Opts: [{cd,"/Users/yes/Projects/q/_build/default/lib/intervals"},
            exit_status,
            {line,16384},
            use_stdio,stderr_to_stdout,hide,eof]

===> Comparing git ref 343bd8f with 343bd8f
===> sh info:
	cwd: "/Users/yes/Projects/q"
	cmd: git rev-parse --short=7 -q HEAD

===> 	opts: [{cd,"/Users/yes/Projects/q/_build/default/lib/jiffy"}]

===> Port Cmd: git rev-parse --short=7 -q HEAD
Port Opts: [{cd,"/Users/yes/Projects/q/_build/default/lib/jiffy"},
            exit_status,
            {line,16384},
            use_stdio,stderr_to_stdout,hide,eof]

===> Comparing git ref 06aaf44 with 06aaf44
===> Evaluating config script "/Users/yes/Projects/q/_build/default/lib/jiffy/rebar.config.script"
===> sh info:
	cwd: "/Users/yes/Projects/q"
	cmd: git rev-parse --short=7 -q HEAD

===> 	opts: [{cd,"/Users/yes/Projects/q/_build/default/lib/jsx"}]

===> Port Cmd: git rev-parse --short=7 -q HEAD
Port Opts: [{cd,"/Users/yes/Projects/q/_build/default/lib/jsx"},
            exit_status,
            {line,16384},
            use_stdio,stderr_to_stdout,hide,eof]

===> Comparing git ref 352d6b2 with 352d6b2
===> sh info:
	cwd: "/Users/yes/Projects/q"
	cmd: git rev-parse --short=7 -q HEAD

===> 	opts: [{cd,"/Users/yes/Projects/q/_build/default/lib/lager"}]

===> Port Cmd: git rev-parse --short=7 -q HEAD
Port Opts: [{cd,"/Users/yes/Projects/q/_build/default/lib/lager"},
            exit_status,
            {line,16384},
            use_stdio,stderr_to_stdout,hide,eof]

===> Comparing git ref c3970a9 with c3970a9
===> Evaluating config script "/Users/yes/Projects/q/_build/default/lib/lager/rebar.config.script"
===> sh info:
	cwd: "/Users/yes/Projects/q"
	cmd: git rev-parse --short=7 -q HEAD

===> 	opts: [{cd,"/Users/yes/Projects/q/_build/default/lib/lhttpc"}]

===> Port Cmd: git rev-parse --short=7 -q HEAD
Port Opts: [{cd,"/Users/yes/Projects/q/_build/default/lib/lhttpc"},
            exit_status,
            {line,16384},
            use_stdio,stderr_to_stdout,hide,eof]

===> Comparing git ref 5ce99d8 with 5ce99d8
===> sh info:
	cwd: "/Users/yes/Projects/q"
	cmd: git rev-parse --short=7 -q HEAD

===> 	opts: [{cd,"/Users/yes/Projects/q/_build/default/lib/mimetypes"}]

===> Port Cmd: git rev-parse --short=7 -q HEAD
Port Opts: [{cd,"/Users/yes/Projects/q/_build/default/lib/mimetypes"},
            exit_status,
            {line,16384},
            use_stdio,stderr_to_stdout,hide,eof]

===> Comparing git ref 0800544 with 0800544
===> sh info:
	cwd: "/Users/yes/Projects/q"
	cmd: git rev-parse --short=7 -q HEAD

===> 	opts: [{cd,"/Users/yes/Projects/q/_build/default/lib/poolboy"}]

===> Port Cmd: git rev-parse --short=7 -q HEAD
Port Opts: [{cd,"/Users/yes/Projects/q/_build/default/lib/poolboy"},
            exit_status,
            {line,16384},
            use_stdio,stderr_to_stdout,hide,eof]

===> Comparing git ref 8bb45fb with 8bb45fb
===> sh info:
	cwd: "/Users/yes/Projects/q"
	cmd: git rev-parse --short=7 -q HEAD

===> 	opts: [{cd,"/Users/yes/Projects/q/_build/default/lib/recon"}]

===> Port Cmd: git rev-parse --short=7 -q HEAD
Port Opts: [{cd,"/Users/yes/Projects/q/_build/default/lib/recon"},
            exit_status,
            {line,16384},
            use_stdio,stderr_to_stdout,hide,eof]

===> Comparing git ref fcc1a7d with fcc1a7d
===> sh info:
	cwd: "/Users/yes/Projects/q"
	cmd: git rev-parse --short=7 -q HEAD

===> 	opts: [{cd,"/Users/yes/Projects/q/_build/default/lib/riak_core"}]

===> Port Cmd: git rev-parse --short=7 -q HEAD
Port Opts: [{cd,"/Users/yes/Projects/q/_build/default/lib/riak_core"},
            exit_status,
            {line,16384},
            use_stdio,stderr_to_stdout,hide,eof]

===> Comparing git ref 8d65a47 with 8d65a47
===> Evaluating config script "/Users/yes/Projects/q/_build/default/lib/lager/rebar.config.script"
===> Evaluating config script "/Users/yes/Projects/q/_build/default/lib/lager/rebar.config.script"
===> Evaluating config script "/Users/yes/Projects/q/_build/default/lib/eleveldb/rebar.config.script"
===> Evaluating config script "/Users/yes/Projects/q/_build/default/lib/eleveldb/rebar.config.script"
===> Evaluating config script "/Users/yes/Projects/q/_build/default/lib/exometer_core/rebar.config.script"
===> Evaluating config script "/Users/yes/Projects/q/_build/default/lib/exometer_core/rebar.config.script"
===> Uncaught error in rebar_core. Run with DEBUG=1 to stacktrace or consult rebar3.crashdump
===> Uncaught error: multiple_app_files ["/Users/yes/Projects/q/_build/default/lib/riak_kv/src/riak_kv.app.src",
                                                "/Users/yes/Projects/q/_build/default/lib/riak_kv/src/riak_kv.app.src"]
===> When submitting a bug report, please include the output of `rebar3 report "your command"`

[ferd]

Do you have a link to the exact project source and/or the rebar.config file to make it easier to reproduce?

[kpy3]

I've extracted part of rebar.config from our project, you can find it here with additional info.

[ferd]

Reproduced after ensuring cuttlefish does not warn on export_all. Weird thing.

[ferd]

Ah got it. The problem stems from the fact that riak_kv states an src_dirs value as ["./priv/tracers", "./src"] and then adds one in erl_opts as {src_dirs, ["src", "priv/tracers"]}. Those are treated as distinct paths and path deduplication does not work on them, yielding multiple .app.src files that are identical once the path is expanded.

Should probably just deduplicate the source paths at the root in rebar_dir if they have the same prefix like that, and the rest would be automatically fixed.

[ferd]

Patch at #1624

[ferd]

Do note that after this patch you'll still have at least one failure in the sext app since it declares a single {src_dir, ["example"]} path, which needs to be changed to {src_dir, ["example","src"]}.

[kpy3]

Noted, thanks!