Support runtime bootstrapping

[tmds]

The way source-build bootstraps .NET requires a working version of .NET.

That means:

• The build host needs to be a platform that already supports .NET.
• And, to target an unsupported platform a cross-compilation toolchain is needed from the build host for that platform.

In practice: when .NET gets built for s390x for RHEL, we start with a bootstrap from x64 Ubuntu because Ubuntu provides better support for cross-compilation. Even with that support, we need to take special care the cross-compilation target is compatible with RHEL. For example: it can't target a version of glibc that is too new.

This requires building a very specific environment.
That environment can then be re-used for bootstrapping new major versions of .NET for the target platform.

The alternative to this is to support creating a bootstrap SDK directly on the target platform by combining: pre-compiled managed bits, with native assets that are built directly on the platform.
This type of build isn't supported. And maybe there are reasons why this can't even work.

This issue is to have some awareness of the steps not covered by source-build for bootstrapping unsupported platforms, and source-build's bootstrap limitations.

cc @omajid @uweigand @jkotas @janvorli

[janvorli]

Actually, in the 1.x / 2.x / 3.x era, the bootstrapping script that was being used that I have created was working exactly the way suggested as an alternative above. It was running on the target platform. It used a working SDK from a bit-compatible platform as an input, extracted the commit hashes of the native bits, checked out the related repos at those commits and compiled the native parts. Then it copied the native bits over the ones in the SDK and also uncrossgened the managed assemblies, as the R2R code would not be compatible with the target architecture.
The old script is here:
https://github.com/dotnet/source-build/blob/release/3.1/scripts/bootstrap/buildbootstrapcli.sh

[janvorli]

I've forgotten to mention that the script also updated the RID graph in the Microsoft.NETCore.App.deps.json

[tmds]

I don't think the approach was actively abandoned, but it went 'missing'.

The script was removed in #2001. The documentation is still around: https://github.com/dotnet/source-build/blob/main/Documentation/boostrap-new-os.md.

The EngineeringPlanLinuxDistroRepos focuses on 'toolset bootstrapping' instead of 'runtime bootstrapping'

Toolset bootstrapping is different than runtime bootstapping provided in scripts/bootstrap/buildbootstrapcli.sh. This script is only needed when starting with a new distro on which .NET Core has never built. For the distros being discussed in this document, there are already existing .NET Core runtime and SDK builds available. For that reason, runtime bootstrapping is outside the scope of this document.

If source-build would still support the runtime bootstrapping, that's what we'd be using for s390x instead of using the Ubuntu cross-build. (note: for s390x we need to bootstrap for mono.)

[janvorli]

If I remember it correctly, one of the reasons for removing the script was that the source-build folks had another plan for bootstrapping (maybe the cross build you've mentioned is the one) and the fact that the script would need to be moved to the new repo structure.

[jkotas]

The old bootstrapping script also assumed that no managed tools are needed to build the unmanaged runtime parts. It goes against our desire to write the custom build tools in C#. Are the unmanaged runtime parts buildable without a working .NET runtime? There is a good chance we have picked up a managed tool dependency somewhere.

[tmds]

It's a feature that .NET can be bootstrapped using only a native toolchain.
If we're not maintaining it, it will stop working.

The way we're now doing a runtime bootstrap by cross-compiling pushes the problem to the source-build user. He/she needs to create a cross-compilation environment from a supported platform to the unsupported platform.

[jkotas]

The way we're now doing a runtime bootstrap by cross-compiling pushes the problem to the source-build user.

Yes. Is it that bad? The alternative discussed here has a cost too that are paid elsewhere. Would the overall benefits of switching the source build to this alternative offset the overall costs?

[tmds]

For a package maintainer, it's preferable that package dependencies can be met by the distro without much effort.
That's not the case now. We're using another distro's cross-compiler. And, we need to be picky about the version to be sure the result works on RHEL.

In some cases a cross-compiler won't even be readily available. Suppose we'd add Alpine support today. afaik cross-compilers that ship with Ubuntu target the same version of glibc as the build platform. You can't use them for a musl based distro.

The cost is now for the package maintainer which needs to setup a cross-compilation environment. Each distro needs to address that on its own.
Runtime bootstrap provides a common solution.

Since it worked previously it is probably doable to make it work again.

We manage to get everything built today, so this isn't urgent.

If you also think it's important and within source-build scope, maybe we can target .NET 8?

[tmds]

@jkotas @janvorli I think we should maintain this feature. Can we include this for .NET 8?

[jkotas]

I do think it makes sense to maintain this hacky bootstrapping script as something that is guaranteed to work. The design is fragile - how do you know that CoreLib built for a different platform/architecture is going to work on your architecture? The implementation requires joining assets from multiple different build machines. It is something that we explicitly want to avoid in the source build design for .NET 8.

I think it is ok to keep the script up to date as one-off solution for people who are doing the initial port to a new platform, to allow them to do the initial bootstrap with some massaging of the process.

[wfurt]

Even if not perfect, it certainly helped with FreeBSD support back in a day. (We end up going with cross-compilation as easier to maintain)

[tmds]

I'm going to close this issue thanks to the .NET 9 vmr supporting cross-targetting.
This solves the bootstrapping issue for an important set of cases.

I don't think we still need an issue for "general bootstrap" support. Users can create new issues when they run into specific issues while bootstrapping.