Msgpack 1.0 compatibility

[jrbourbeau]

In msgpack 1.0 the default value for strict_map_key was changed from False to True (see https://github.com/msgpack/msgpack-python#major-breaking-changes-in-msgpack-10). This causes failures for us as we use map keys which are not bytes or str. This PR updates our use of msgpack.loads to include strict_map_key=False

Closes #3491

[mrocklin]

Thanks for handling this @jrbourbeau

[jrbourbeau]

CI has passed against both msgpack 1.0 and a pre-1.0 release to check for backwards compatibility

I've also set 0.6.0 as the minimum supported msgpack version (released in Nov 2018) because that's when the strict_map_key option was added. This seems reasonable to me, but if there's reason to think this 0.6.0 restriction is too aggressive we can add some additional logic surrounding the msgpack version to support older versions too

[TomAugspurger]

Requiring>=0.6 sounds fine.

[jrbourbeau]

Great, I just set the minimum version for our CI environments and will merge this PR when the current set of CI builds pass

[jrbourbeau]

The test failure seems unrelated and I've raised a separate issue for it (xref #3498)

[Carreau]

May I suggest a new build/build number on conda-forge that update the metadata info to pin msgpack to <1.0.0 ? I've read recently that there should be a simpler way to do that via repodata patch, but I haven't used that before.

[mrocklin]

cc @jakirkham

…

On Tue, Feb 18, 2020 at 3:55 PM Matthias Bussonnier < ***@***.***> wrote: May I suggest a new build/build number on conda-forge that update the metadata info to pin msgpack to <1.0.0 ? I've read recently <https://twitter.com/ocefpaf/status/1227292336662597632> that there should be a simpler way to do that via repodata patch, but I haven't used that before. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#3494?email_source=notifications&email_token=AACKZTFJKDI6UCAAOR2NCM3RDRYPVA5CNFSM4KXHETCKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMFZ34I#issuecomment-587963889>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACKZTF6PXQW6TXFKD4JRQ3RDRYPVANCNFSM4KXHETCA> .

[jakirkham]

A PR to the conda-forge feedstock to do that would be welcome 🙂

[Carreau]

A PR to the conda-forge feedstock to do that would be welcome 🙂

Do you know how to patch the repodata, or do I just bump the build number ?

[Carreau]

See conda-forge/distributed-feedstock#114

[jakirkham]

Thanks for doing that. Merged! 😄

[jakirkham]

Do you know how to patch the repodata, or do I just bump the build number ?

Ah sorry missed this bit.

The patches live here. Though bumping the build number works also and may be a bit more straightforward.

[jrbourbeau]

Thanks for updating the conda-forge recipe @Carreau!

[Carreau]

Jakirkham, I'm more thinking that all previous version of distributed (or at least a few of those) should get msgpack<1;
Bumping the build only affect latest one. It's something that I think will come up more and more often to update dependencies compat after the releases. Hence why I was thinking about repodata patch.

[jakirkham]

Yeah hot-fixing would still be appropriate Matthias 🙂