How to compile in *_SUITE_data directories?

[sirihansen]

We have some apps that need to compile some extra test code specific for certain suites. The source for this is placed in the suite's data dir, and it is built by a common_test hook which is part of the 'ts' stuff under common_test/test_server - i.e. specific for running OTP tests. The code can be erlang, c, java..., and the build result is typically also placed in the data dir so there won't be any conflict between test suites. Some times the file structure for the outcome is also important, e.g. when needing a whole OTP application for testing upgrade or other OTP application mechanisms.

Now I'm looking into how rebar3 can be used for building and testing of split out OTP applications. I find that rebar3 builds erlang code in my data dirs and places the result flat in the test directory. Is there a way to avoid this?

I also wonder if there is a "better" and/or more "rebar3 typical" way to build the code in the data dirs? A plugin to hook into the compilation under the test profile? Any other idea?

[talentdeficit]

the rebar compiler recurses into subdirs by default but there's an (internal) option to disable that. it could pretty easily be used to stop ct from compiling subdir code. would that address this issue?

[sirihansen]

Great! I guess it would solve the problem of compiled data dir code ending up in the test directory. Which option is it?

[talentdeficit]

it's not currently exposed to user code, it's just for internal calls to the compiler. i'll try to make a pr today to expose it. the option is here and will need to be set here and here via an option in rebar.config

[psyeugenic]

@talentdeficit did you do a PR for this? I could try to implement it otherwise. I need to learn and navigate the common test provider anyhow.

[sirihansen]

Finally had a look at the 'recursive' option, and it would probably solve the problem in most cases. However, to be "always correct" we would need to set it to false only for the test directory, since some of our applications have deep src directories. Any thoughts on connecting compile options to directories?

[sirihansen]

If I was to add a 'recursive' option to the config, should it be

{erl_opts,[{recursive,boolean()}]}

or a new top level opt, e.g.

{compile_opts,[{recursive,boolean()}]}

And if using erl_opts, then should the same apply to mib_opts, xrl_opts, yrl_opts?

No matter what, I think that ct_compile_opts and eunit_compile_opts must have the same?

[talentdeficit]

i'm not sure how feasible it currently is to compile only specific directories recursively. as an all-or-nothing option it should probably be something like {erlc_compiler, [{recursive, true}]}. erl_opts is for passing options directly to compile:file

alternately, if we want to pursue recursive compilation for subsets of directories, it could be attached as an argument to src_dirs and extra_src_dirs. something like {src_dirs, ["foo"]} and {src_dirs, [{"bar", [recursive]}]} could resolve to [{"foo", []}, {"bar", [recursive]}]. we already support multiple src_dirs declarations in a config

ct_compile_opts and eunit_compile_opts should probably be deprecated. they date to a time before we had profiles which are a much better solution to the problem

[sirihansen]

Thanks! I'll look at src_dirs and extra_src_dirs to see what it would require to set recursiveness pr directory.

[ferd]

I think I can close this since #1382 was merged in. Let me know if it's not the case and we'll reopen it.