llvmPackages_10: cross-compilation support

[thefloweringash]

Motivation for this change

Cross compilation of clang 10. Ultimately to make bootstrap tools for platforms that use clang in their stdenv.

Tested building on x86_64-linux with

nix-build \
  --keep-going \
  --argstr crossSystem aarch64-linux \
  -A llvmPackages_10.llvm \
  -A llvmPackages_10.lld \
  -A llvmPackages_10.compiler-rt \
  -A llvmPackages_10.libcxx \
  -A llvmPackages_10.libcxxabi \
  -A llvmPackages_10.clang-unwrapped \
  -A pkgsCross.wasi32.hello

Note that presently I can't test the following due to a build cycle in gcc:

cycle detected in the references of '/nix/store/dkc8jicyd1pibg3hfgasbnvl6schw9da-gcc-debug-9.3.0-aarch64-unknown-linux-gnu-lib' from '/nix/store/nv8nb53m5ypywdgjh6b38pxi0klncgls-gcc-debug-9.3.0-aarch64-unknown-linux-gnu'

nix-build \
  --argstr crossSystem aarch64-linux \
  -A llvmPackages_10.clang \
  -A llvmPackages_10.libcxxClang

Note that while it's possible to build a wrapper, the wrapper uses the build platform's shell so I was unable to test it.

Some interesting points:

• clang-tblgen doesn't seem to have an install rule within clang, which suggests it's never meant to be installed. To export it without altering the interface I added a tablegen output.
• llvm-config is installed by llvm to work a bit like pkg-config and help locate components of llvm. Installing the native version to a different output didn't work, since it assumes it's installed alongside llvm. I made up the name llvm-config-native to avoid a conflict, but I'm hoping there's a nicer way to do this. The two versions here are:
  • the installed version, which runs on the host platform, installed by cmake as llvm-config. This tool is probably never used when building with nixpkgs.
  • the "NATIVE" version, which runs on the build platform, installed in postInstall as llvm-config-native. This is required for the lld and clang builds.
• I don't see why lld would be required to build clang so I removed it. The build didn't fail, but I don't have a strong argument as for why it should or should not be included.

Things done

• Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
• Built on platform(s)
  • NixOS
  • macOS
  • other Linux distributions
• Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
• Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
• Tested execution of all binary files (usually in ./result/bin/)
• Determined the impact on package closure size (by running nix path-info -S before and after)
• Ensured that relevant documentation is up to date
• Fits CONTRIBUTING.md.

[SuperSandro2000]

@thefloweringash Can you fix the merge conflict?

[matthewbauer]

useLLVM support still works:

$ nix build github:thefloweringash/nixpkgs?ref=clang-10-cross-compile#pkgsCross.wasi32.hello

[thefloweringash]

Rebased to fix the merge conflict, and updated to include current state of testing.

[Ericson2314]

N.B. I started fixing conflicts and then got carried away; I'll try to open the patch to staging soon.

[thefloweringash]

Going to close this in favor of #111487. There's a similar version of this in #105026 for llvm 11 for making bootstrap tools.

[Ericson2314]

@thefloweringash well I just edited that one' descriptions to point out I didn't really end up fixing all the cross things :)

[thefloweringash]

@thefloweringash well I just edited that one' descriptions to point out I didn't really end up fixing all the cross things :)

Testing the one use case I'm focused on right now (cross compiling unwrapped llvmPackages_11), it looks like it works as long as I remove lld from clang-unwrapped's buildInputs.

[Ericson2314]

@thefloweringash do you know why it's deemed needed at all?

[Ericson2314]

Also would you like to review #111487 ? We have quite little time before the 21.05 staging freeze.

[Ericson2314]

Hmm it was db29857. Fishy... I'll leave a comment there.