fix: platforms, architectures and dpkg_status after #115

[jjmaestro]

After landing #115 we need platforms to be non-dev. This would have been caught in the e2e test but since #115 didn't make changes to the e2e test, it went unnoticed.

To prevent this, I've also copied the examples/debian_snapshot to the e2e test. IMHO this helps catch bugs that affect third-party repos using rules_distroless as a dependency, such as the platforms dependency.

I've also verified that we do need the select in the architecture, as mentioned in my code review. Otherwise, the Config in the manifest is always set to arm64 regardless of the architecture that we are building.

I've also fixed the unmapped Debian architectures (e.g. "all"). Basically, _ARCHITECTURE_MAP[arch] or arch fails when a Debian arch is not in the map. It should be _ARCHITECTURE_MAP.get(arch, arch) to default to arch when it's not in the map. To exercise this and ensure it always works I've:

• removed "arm64" since it's a valid CPU platform already (and without this fix it would have already failed in all the examples and the e2e test).
• added armhf mapping to armv7e-mf
• changed Debian's ppc64el mapping to the exact matching platforms CPU (ppc64le)
• added a test to exercise arch all (which is both an arch that's not in the map while being "a special Debian arch" and also a valid platforms CPU constraint).
• Moved architecture doc links next to _ARCHITECTURE_MAP and added a bunch more links to the Debian wikis.

Finally, I've fixed _PACKAGES, :packages and :dpkg_status to be different per-architecture, because there are cases where the packages differ between architectures. E.g. currently in the examples/ubuntu_snapshot, coreutils depends on libssl3 on amd64 but there's no such dependency on arm64. Thus, the list of packages and dpkg_status has to be different.

[thesayyn]

I am little at unease about the symlinks, this will most likely break BCR, because of

rules_distroless/.gitattributes

Line 10 in 7b342e2

examples export-ignore

and how BCR uses release archives (created via git archive with examples folder excluded) to run presubmit.

Can we just improve e2e so it catches this case?

[jjmaestro]

Ah, I didn't know about BCR! Sure, I'll copy over the test and will make a note in it to keep it in sync.

[jjmaestro]

@thesayyn Added another fix I caught while adding one of the tests that I wrote for #100 :) Let me know if you like the PR and let the workflows run when you can (I've manually tested the PR so it should hopefully run green). Thanks!

[jjmaestro]

@thesayyn Added yet-another fix, the "new" dpk_status from #115 would fail if there were different packages installed per-architecture. This is actually the case in at least one of the examples in the repo. See b0cc2da for more details.

[thesayyn]

This is great. Thanks for the fixes!

There are small changes to be made but overall good. (sorry for the conflicts.)

[thesayyn]

would fail if there were different packages installed per-architecture

Yup, this is a great catch and paves the way for per architecture debian repositories and packages #56

[jjmaestro]

Oh wow, I just fixed the PR and I was checking "Compare" and the rebase still shows!!? even when it's rebased over main! 😭 I was expecting that GH would separate rebasing over main VS e.g. other branches and would NOT include those as "changes"... at least provide a way to see "all changes" and "changes VS last state of the PR".

@thesayyn this, again, will make it difficult to review, sorry 😞 It'll be easier if you check the final state of the commits, but basically, I've done as you requested in your comments.

[jjmaestro]

OK, I've produced such a diff myself with:

$ git co -b fix-pr-115-OLD-rebased b0cc2da
$ git rebase main

$ git rev-parse --short fix-pr-115-OLD-rebased
0f4a221

$ git rev-parse --short fix-pr-115            
7711ba0

git diff fix-pr-115-OLD-rebased fix-pr-115 -- . ':(exclude)*.lock.json'

But GH doesn't let me attach it as a file to the comment!?? srsly... it seems to only allow PDF, images, movies, etc.

Anyway, here you go, in case it makes the review easier:

EDIT: I've pushed the old rebased branch so that it's easier to compare via GH with a manual compare link: Compare 0f4a221...7711ba0

[thesayyn]

You have some buildifier errors.

[thesayyn]

https://github.com/GoogleContainerTools/rules_distroless/actions/runs/12284130637/job/34279199329?pr=120#step:3:40

[jjmaestro]

@thesayyn I've created #135 from main that fixes some buildifier stuff that got through the cracks (somehow didn't run pre-commit run I think?). I'll rebase on that one and see if there's anything left here as well.

[jjmaestro]

OK, this is with the rebase, now fixing the broken buildifiers left here...

[thesayyn]

@thesayyn I've created #135 from main that fixes some buildifier stuff that got through the cracks (somehow didn't run pre-commit run I think?). I'll rebase on that one and see if there's anything left here as well.

It's just Readme files over there, which is not related to your changes, you have some unused loads.

[jjmaestro]

OK, it should be green now! If you also push #135 for main, then everything should be green 👌 I'm still a bit baffled because I thought I had pre-commit installed and running... maybe I deactivated it at some point and didn't realize... anyway, it's now set so it should all be fine from here!

[thesayyn]

This made macos unhappy.

[thesayyn]

https://github.com/GoogleContainerTools/rules_distroless/actions/runs/12284400987/job/34281214039

[jjmaestro]

@thesayyn :S will look into it now!

[jjmaestro]

@thesayyn OK, so it's a bit weird, at least to me. I've managed to run all tests in //examples manually, e..g going one by one through bazel test //examples/flatten/... and so on, except //examples/debian_snapshot:test and //examples/ubuntu_snapshot:test because they are SKIPPED because they are guarded by target_compatible_with.

Even so, if I try to run them, I get, as expected

% bazel test //examples/debian_snapshot:test --cache_test_results=no
WARNING: Ignoring JAVA_HOME, because it must point to a JDK, not a JRE.
Target //examples/debian_snapshot:test failed to build
Use --verbose_failures to see the command lines of failed build steps.
ERROR: Analysis of target '//examples/debian_snapshot:test' failed; build aborted: Target //examples/debian_snapshot:test is incompatible and cannot be built, but was explicitly requested.
Dependency chain:
    //examples/debian_snapshot:test (efbef2)   <-- target platform (@@platforms//host:host) didn't satisfy constraint @@platforms//os:linux
INFO: Elapsed time: 0.119s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
ERROR: Build did NOT complete successfully
ERROR: No test targets were found, yet testing was requested

However! If I run bazel test //examples/... then it breaks!!

% bazel test //examples/... --cache_test_results=no
WARNING: Ignoring JAVA_HOME, because it must point to a JDK, not a JRE.
ERROR: /private/var/tmp/_bazel_jjmaestro/be9c074a2171a3cb6c673d10113bd570/external/_main~apt~noble/BUILD.bazel:196:12: configurable attribute "controls" in @@_main~apt~noble//:dpkg_status doesn't match this configuration. Would a default condition help?

Conditions checked:
 @@_main~apt~noble//:linux_amd64
 @@_main~apt~noble//:linux_arm64

To see a condition's definition, run: bazel query --output=build <condition label>.

This instance of @@_main~apt~noble//:dpkg_status has configuration identifier 69bb7f0. To inspect its configuration, run: bazel config 69bb7f0.

For more help, see https://bazel.build/docs/configurable-attributes#faq-select-choose-condition.

Use --verbose_failures to see the command lines of failed build steps.
ERROR: Analysis of target '//examples/ubuntu_snapshot:_noble_index_json' failed; build aborted: Analysis failed
INFO: Elapsed time: 0.178s, Critical Path: 0.03s
INFO: 11 processes: 11 internal.
ERROR: Build did NOT complete successfully
//examples/cacerts:test_cacerts                                       NO STATUS
//examples/debian_snapshot:test                                         SKIPPED
//examples/flatten:test_flatten                                       NO STATUS
//examples/flatten:test_flatten_dedup_listing                         NO STATUS
//examples/flatten:test_flatten_dedup_mtree                           NO STATUS
//examples/group:test_group                                           NO STATUS
//examples/group:test_group_content                                   NO STATUS
//examples/home:test_home                                             NO STATUS
//examples/locale:test_bookworm                                       NO STATUS
//examples/locale:test_bullseye                                       NO STATUS
//examples/os_release:test_os_release                                 NO STATUS
//examples/os_release:test_os_release_alternative_path                NO STATUS
//examples/os_release:test_os_release_content                         NO STATUS
//examples/passwd:test_passwd                                         NO STATUS
//examples/passwd:test_passwd_content                                 NO STATUS
//examples/statusd:test_statusd                                       NO STATUS

Executed 0 out of 16 tests: 16 were skipped.

So, somehow, something is running on a previous phase / stage? :-?

Anyway, I'll add a default step to the selects but not sure if that will be enough to fix it.

[jjmaestro]

OK, got it fixed in #136!