Add bootstrap tools for aarch64-darwin and add to supportedSystems #123557

thefloweringash · 2021-05-18T17:52:43Z

Motivation for this change

Fully enable aarch64-darwin, follow up to #105026.

Bootstrap tools come from #105026 (comment). The hashes that nix uses for executable files is not the same as the sha256 of their contents, so the executable tools do not match the comment. I applied the regular TOFU approach.

Things done

grahamc

~~I don't understand why my hashes don't match what you put here, do you know whats up?~~

~~Maybe we could use the hashes I provided here? #105026 (comment)~~

pkgs/stdenv/darwin/default.nix

thefloweringash · 2021-05-18T18:03:40Z

I don't understand why my hashes don't match what you put here, do you know whats up?

Maybe we could use the hashes I provided here? #105026 (comment)

IIUC when nix hashes an executable file it uses a nested file format to encode the executable metadata, which affects the hash. It should be the difference between nix-hash $file and nix-hash --flat $file. The matching hash is non-executable, so it’s a flat hash.

grahamc · 2021-05-18T18:07:42Z

[grahamc@hyperchicken:~]$ (for f in bootstrap-tools.cpio.bz2  bzip2  cpio  mkdir  sh; do url="https://tarballs.nixos.org/stdenv-darwin/aarch64/20acd4c4f14040485f40e55c0a76c186aa8ca4f3/$f"; nix-prefetch-url "$url"; done; )
path is '/nix/store/cn4c1gcqd84v5ybx07kkspprd8w5nshx-bootstrap-tools.cpio.bz2'
1v2332k33akm6mrm4bj749rxnnmc2pkbgcslmd0bbkf76bz2ildy
path is '/nix/store/9xi1jm60gbi5wb2n3bnbimrjh8nka763-bzip2'
1c7a2civpwviwr917xrn0dxgch59c7ajs5va2b3qfga0ll70zcca
path is '/nix/store/as2w0c2g5rn7d9rmnqb4vnlx9drhbz3q-cpio'
1pay9iaj6332nb5ikw36ch0ds2c94s2jxfgxg22aaaqld0yaajwv
path is '/nix/store/v8gi3b5960xz6wh1qfqn6j7rvahzc2qp-mkdir'
0n3644mwpnqmc90ds1x8q0rhkasp1zcdlk3bm84749p0smrd654k
path is '/nix/store/1bxpipz2w1qqa1w442fprcjs17prm0hj-sh'
132qg34xrl56isis2qx9q779znig16flg1yisfapqg6y4kc71fxl

[grahamc@hyperchicken:~]$ for f in bootstrap-tools.cpio.bz2  bzip2  cpio  mkdir  sh; do printf "%s: %s\n" "$f" "$(nix-hash --flat --base32 --type sha256 /nix/store/71mly5hl2qxmm1fmp1jwn2qwbdbwxpvs-stdenv-bootstrap-tools-aarch64-apple-darwin/on-server/$f)"; done
bootstrap-tools.cpio.bz2: 1v2332k33akm6mrm4bj749rxnnmc2pkbgcslmd0bbkf76bz2ildy
bzip2: 1c7a2civpwviwr917xrn0dxgch59c7ajs5va2b3qfga0ll70zcca
cpio: 1pay9iaj6332nb5ikw36ch0ds2c94s2jxfgxg22aaaqld0yaajwv
mkdir: 0n3644mwpnqmc90ds1x8q0rhkasp1zcdlk3bm84749p0smrd654k
sh: 132qg34xrl56isis2qx9q779znig16flg1yisfapqg6y4kc71fxl

[grahamc@hyperchicken:~]$ for f in bootstrap-tools.cpio.bz2  bzip2  cpio  mkdir  sh; do printf "%s: %s\n" "$f" "$(nix-hash --base32 --type sha256 /nix/store/71mly5hl2qxmm1fmp1jwn2qwbdbwxpvs-stdenv-bootstrap-tools-aarch64-apple-darwin/on-server/$f)"; done
bootstrap-tools.cpio.bz2: 1y2jkykx13r7f3a0w9kjnp62rp4yp3zddcq9c9xp86qz0lgn2qi7
bzip2: 1khs8s5klf76plhlvlc1ma838r8pc1qigk9f5bdycwgbn0nx240q
cpio: 17pxq61yjjvyd738fy9f392hc9cfzkl612sdr9rxr3v0dgvm8y09
mkdir: 1m9nk90paazl93v43myv2ay68c1arz39pqr7lk5ddbgb177hgg8a
sh: 17m3xrlbl99j3vm7rzz3ghb47094dyddrbvs2a6jalczvmx7spnj

grahamc

Yep, my mistake. 👍

prusnak · 2021-05-18T18:09:53Z

This change is not required? (saw it somewhere on the IRC)

--- a/pkgs/top-level/release.nix
+++ b/pkgs/top-level/release.nix
@@ -180,7 +180,7 @@ let
           # Cross compiled bootstrap tools
           aarch64-darwin =
             let
-              bootstrap = import ../stdenv/darwin/make-bootstrap-tools.nix { system = "x86_64-darwin"; crossSystem = "aarch64-darwin"; };
+              bootstrap = import ../stdenv/darwin/make-bootstrap-tools.nix { system = "aarch64-darwin"; };
             in {
               # Distribution only for now
               inherit (bootstrap) dist;

grahamc · 2021-05-18T18:13:15Z

This change is not required?

I'd rather keep it cross-built so that in the future we can easily re-bootstrap it without using a known good aarch64-darwin machine.

thefloweringash · 2021-05-18T18:13:16Z

This change is not required?

--- a/pkgs/top-level/release.nix
+++ b/pkgs/top-level/release.nix
@@ -180,7 +180,7 @@ let
           # Cross compiled bootstrap tools
           aarch64-darwin =
             let
-              bootstrap = import ../stdenv/darwin/make-bootstrap-tools.nix { system = "x86_64-darwin"; crossSystem = "aarch64-darwin"; };
+              bootstrap = import ../stdenv/darwin/make-bootstrap-tools.nix { system = "aarch64-darwin"; };
             in {
               # Distribution only for now
               inherit (bootstrap) dist;

The bootstrap tools can be built either cross or native. Currently the cross compiled tools have had more testing, so I’m inclined to keep them for now. I’ve also seen reports that natively rebuilt tools behave slightly differently (shows up as a dylib not found error when building expat), so until I’ve investigated that I think we’re best to stick with the tools that we know to work.

andir · 2021-05-18T18:13:44Z

This change is not required? (saw it somewhere on the IRC)

--- a/pkgs/top-level/release.nix
+++ b/pkgs/top-level/release.nix
@@ -180,7 +180,7 @@ let
           # Cross compiled bootstrap tools
           aarch64-darwin =
             let
-              bootstrap = import ../stdenv/darwin/make-bootstrap-tools.nix { system = "x86_64-darwin"; crossSystem = "aarch64-darwin"; };
+              bootstrap = import ../stdenv/darwin/make-bootstrap-tools.nix { system = "aarch64-darwin"; };
             in {
               # Distribution only for now
               inherit (bootstrap) dist;

Maybe we should leave the cross bootstrap job for now and add a native boostrap job as well? At least until we are happy?

prusnak · 2021-05-18T18:14:21Z

I thought so, thanks for explanation @grahamc @thefloweringash

vcunat · 2021-05-18T18:18:30Z

The test attribute from bootstrap would be nice (ran natively on aarch64-darwin). But not necessarily in this PR.

vcunat

I checked the downloads

$ nix build -f. --argstr system aarch64-darwin stdenv.bootstrapTools
[2 built, 3 copied (0.4 MiB), 75.5 MiB DL]
error: a 'aarch64-darwin' with features {} is required to build '/nix/store/7kcayxwk8khycxw1agmcyfm9vpsqpw4s-bootstrap-tools.drv' ...

and that plain hashes of the resulting paths match what hydra built, by manual sha256sum.

jonringer · 2021-05-18T19:04:34Z

I think this should be merged as soon as possible, to allow for the aarch64-darwin builders to find as many regressions as possible

andir · 2021-05-18T19:05:05Z

pkgs/top-level/release.nix

@@ -11,7 +11,7 @@
 { nixpkgs ? { outPath = (import ../../lib).cleanSource ../..; revCount = 1234; shortRev = "abcdef"; revision = "0000000000000000000000000000000000000000"; }
 , officialRelease ? false
  # The platforms for which we build Nixpkgs.
-, supportedSystems ? [ "x86_64-linux" "x86_64-darwin" "aarch64-linux" ]
+, supportedSystems ? [ "x86_64-linux" "x86_64-darwin" "aarch64-linux" "aarch64-darwin" ]


Is this right? will this not block channels?

it will only block for failing constituents

Nevermind. We have an explicit list of jobs + arch here since a while..

It will, about as much as x86_64-darwin, but I'd hope it works out OK.

IIUC, this will just add the aarch64-darwin packages to the jobset. Channel blocking is done by adding certain jobs. We should probably have a follow-up PR which at least has the fundamental packages listed so we don't ship completely broken aarch64-darwin updates. But that can be done in another PR.

In particular note that NixOS channels won't be affected, just as they're not affected by x86_64-darwin.

A more interesting question (for later than this PR) is what the nixpkgs-21.05-darwin channel should include...

A more interesting question (for later than this PR) is what the nixpkgs-21.05-darwin channel should include...

probably both x86_64 and aarch64

grahamc · 2021-05-18T19:11:27Z

What tier of support are we shooting for here? https://github.com/NixOS/rfcs/blob/master/rfcs/0046-platform-support-tiers.md

jonringer · 2021-05-18T19:14:49Z

What tier of support are we shooting for here? https://github.com/NixOS/rfcs/blob/master/rfcs/0046-platform-support-tiers.md

I would say that this will eventually replace x86_64-darwin, so it should be on similar footing. Which seems to be tier 1.

vcunat · 2021-05-18T19:15:49Z

I see it in Tier 2 in that non-normative appendix. And I failed to find a current normative table :-)

grahamc · 2021-05-18T19:15:50Z

If we're going for tier-3 I think we should remove it from supportedSystems. If we're going for tier-2 we'll need to get builders for it for ofborg. x86-64-darwin is currently listed at tier-2, but I think practically considered to be tier-1.

prusnak · 2021-05-18T19:19:04Z

Tier 2 sounds good. Especially because it will eventually replace x86_64-darwin like jonringer suggests.

grahamc · 2021-05-18T19:25:03Z

I think it is too early to add it to supportedSystems today. I think instead we should have it be its own jobset and potentially its own channel until it has some time to simmer, and get full support in to ofborg. My concerns here are:

I'd like to get some day to day experience with it blocking a channel that isn't the primary channel, before we go making a major commitment to the tier.
ensure we have sufficient hardware for it in hydra
extend our ofborg support to also evaluate aarch64-darwin deeply (this might require some tuning on ofborg)
if possible, acquire hardware to build aarch64-darwin in ofborg as well

vcunat · 2021-05-18T19:42:20Z

OK, so for this moment I just pushed the bootstrap hashes to staging-next as 5d22db3 (with typo in commit message) and I modified the staging-next jobset to include aarch64-darwin.

jonringer · 2021-05-18T20:06:43Z

I think it is too early to add it to supportedSystems today. I think instead we should have it be its own jobset and potentially its own channel until it has some time to simmer, and get full support in to ofborg.

Agreed. My main concern is if we have "Apple M1 Support" in the release notes, I would just like for that platform to be usable. Higher tier support can come later when other pieces are in place.

But I would like to mention that x86_64-darwin was in a limbo state for a long time due to ofborg's lack of build capacity and stagnating contributions for that platform. Also the very real concern about people not having hardware to support a platform (especially for apple, which is expensive, and other issues).

Also, shoutout to @stephank for doing 100+ darwin fixes for ZHF 🥳

OK, so for this moment I just pushed the bootstrap hashes to staging-next as 5d22db3 (with typo in commit message) and I modified the staging-next jobset to include aarch64-darwin.

Perfect, I guess that's what I was wanting to hear. Just concerned about this being usable by release time in 10 days. Get those jobs simmering :)

grahamc · 2021-05-18T20:08:41Z

Agreed. My main concern is if we have "Apple M1 Support" in the release notes, I would just like for that platform to be usable. Higher tier support can come later when other pieces are in place.

👍

But I would like to mention that x86_64-darwin was in a limbo state for a long time due to ofborg's lack of build capacity and stagnating contributions for that platform. Also the very real concern about people not having hardware to support a platform (especially for apple, which is expensive, and other issues).

Yes, that was no good. I should have asked much sooner for the foundation to fund it. I've also started the process to request an additional x86_64-darwin machine and 2x aarch64-darwin machines for ofborg.

Also, shoutout to @stephank for doing 100+ darwin fixes for ZHF partying_face

🥳 🎉 yes, thank you!

domenkozar · 2021-05-21T08:48:55Z

Agreed. My main concern is if we have "Apple M1 Support" in the release notes, I would just like for that platform to be usable. Higher tier support can come later when other pieces are in place.

But I would like to mention that x86_64-darwin was in a limbo state for a long time due to ofborg's lack of build capacity and stagnating contributions for that platform. Also the very real concern about people not having hardware to support a platform (especially for apple, which is expensive, and other issues).

Yes, that was no good. I should have asked much sooner for the foundation to fund it. I've also started the process to request an additional x86_64-darwin machine and 2x aarch64-darwin machines for ofborg.

Also, shoutout to @stephank for doing 100+ darwin fixes for ZHF partying_face

yes, thank you!

Could we reassign 2 aarch64-darwin machines from hydra to ofborg? If that makes it quicker than buying new hardware and wiring it up.

domenkozar · 2021-05-29T18:34:17Z

I've opened #124887 and this PR is no longer needed.

thefloweringash added 2 commits May 19, 2021 02:00

release.nix: add aarch64-darwin as a supportedSystem

cbaf2ee

stdenv/darwin: add bootstrap tools for aarch64-darwin

20a9447

github-actions bot added the 6.topic: stdenv Standard environment label May 18, 2021

ofborg bot added the 6.topic: darwin Running or building packages on Darwin label May 18, 2021

grahamc requested changes May 18, 2021

View reviewed changes

pkgs/stdenv/darwin/default.nix Show resolved Hide resolved

pkgs/stdenv/darwin/default.nix Show resolved Hide resolved

pkgs/stdenv/darwin/default.nix Show resolved Hide resolved

pkgs/stdenv/darwin/default.nix Show resolved Hide resolved

pkgs/stdenv/darwin/default.nix Show resolved Hide resolved

ofborg bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 1-10 10.rebuild-linux: 1 labels May 18, 2021

grahamc approved these changes May 18, 2021

View reviewed changes

prusnak approved these changes May 18, 2021

View reviewed changes

vcunat approved these changes May 18, 2021

View reviewed changes

jonringer approved these changes May 18, 2021

View reviewed changes

andir reviewed May 18, 2021

View reviewed changes

domenkozar closed this May 29, 2021

grahamc mentioned this pull request May 31, 2021

Revert "release.nix: add aarch64-darwin as a supportedSystem" #125076

Merged

thefloweringash deleted the aarch64-darwin-bootstrap-tools branch July 10, 2021 02:17

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add bootstrap tools for aarch64-darwin and add to supportedSystems #123557

Add bootstrap tools for aarch64-darwin and add to supportedSystems #123557

thefloweringash commented May 18, 2021

grahamc left a comment •

edited

Loading

thefloweringash commented May 18, 2021

grahamc commented May 18, 2021

grahamc left a comment

prusnak commented May 18, 2021 •

edited

Loading

grahamc commented May 18, 2021

thefloweringash commented May 18, 2021

andir commented May 18, 2021

prusnak commented May 18, 2021

vcunat commented May 18, 2021

vcunat left a comment

jonringer commented May 18, 2021

andir May 18, 2021

jonringer May 18, 2021

andir May 18, 2021

vcunat May 18, 2021

jonringer May 18, 2021

vcunat May 18, 2021

vcunat May 18, 2021

jonringer May 18, 2021

grahamc commented May 18, 2021

jonringer commented May 18, 2021

vcunat commented May 18, 2021

grahamc commented May 18, 2021

prusnak commented May 18, 2021 •

edited

Loading

grahamc commented May 18, 2021 •

edited

Loading

vcunat commented May 18, 2021

jonringer commented May 18, 2021

grahamc commented May 18, 2021 •

edited

Loading

domenkozar commented May 21, 2021 •

edited

Loading

domenkozar commented May 29, 2021

Add bootstrap tools for aarch64-darwin and add to supportedSystems #123557

Add bootstrap tools for aarch64-darwin and add to supportedSystems #123557

Conversation

thefloweringash commented May 18, 2021

Motivation for this change

Things done

grahamc left a comment • edited Loading

Choose a reason for hiding this comment

thefloweringash commented May 18, 2021

grahamc commented May 18, 2021

grahamc left a comment

Choose a reason for hiding this comment

prusnak commented May 18, 2021 • edited Loading

grahamc commented May 18, 2021

thefloweringash commented May 18, 2021

andir commented May 18, 2021

prusnak commented May 18, 2021

vcunat commented May 18, 2021

vcunat left a comment

Choose a reason for hiding this comment

jonringer commented May 18, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

grahamc commented May 18, 2021

jonringer commented May 18, 2021

vcunat commented May 18, 2021

grahamc commented May 18, 2021

prusnak commented May 18, 2021 • edited Loading

grahamc commented May 18, 2021 • edited Loading

vcunat commented May 18, 2021

jonringer commented May 18, 2021

grahamc commented May 18, 2021 • edited Loading

domenkozar commented May 21, 2021 • edited Loading

domenkozar commented May 29, 2021

grahamc left a comment •

edited

Loading

prusnak commented May 18, 2021 •

edited

Loading

prusnak commented May 18, 2021 •

edited

Loading

grahamc commented May 18, 2021 •

edited

Loading

grahamc commented May 18, 2021 •

edited

Loading

domenkozar commented May 21, 2021 •

edited

Loading