Pip fails on different compatible ~= version specifiers

[robertschweizer]

pip version

21.0.1

Python version

3.6.8

OS

Windows 10

Additional information

$ python -VV
Python 3.6.8 (tags/v3.6.8:3c6b436a57, Dec 24 2018, 00:16:47) [MSC v.1916 64 bit (AMD64)]

Description

When using multiple ~= version specifiers that are compatible, pip fails with an error.

Expected behavior

pip should install one of the versions that satisfy both requirements.

How to Reproduce

1. Make sure you have the latest numpy

$ pip freeze | grep numpy
numpy==1.19.5

2. Install two overlapping version ranges that would require a downgrade:

pip install numpy~=1.18 numpy~=1.18.0

Output

$ pip install numpy~=1.18 numpy~=1.18.0
Looking in indexes: https://nexus-pypi-ci:****@nexus.devops.definiens.com/repository/pypi-all/simple
Requirement already satisfied: numpy~=1.18 in c:\users\kftf964\appdata\local\programs\python\python36\lib\site-packages (1.19.5)
ERROR: Exception:
Traceback (most recent call last):
  File "c:\users\kftf964\appdata\local\programs\python\python36\lib\site-packages\pip\_internal\cli\base_command.py", line 189, in _main
    status = self.run(options, args)
  File "c:\users\kftf964\appdata\local\programs\python\python36\lib\site-packages\pip\_internal\cli\req_command.py", line 178, in wrapper
    return func(self, options, args)
  File "c:\users\kftf964\appdata\local\programs\python\python36\lib\site-packages\pip\_internal\commands\install.py", line 317, in run
    reqs, check_supported_wheels=not options.target_dir
  File "c:\users\kftf964\appdata\local\programs\python\python36\lib\site-packages\pip\_internal\resolution\resolvelib\resolver.py", line 122, in resolve
    requirements, max_rounds=try_to_avoid_resolution_too_deep,
  File "c:\users\kftf964\appdata\local\programs\python\python36\lib\site-packages\pip\_vendor\resolvelib\resolvers.py", line 453, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "c:\users\kftf964\appdata\local\programs\python\python36\lib\site-packages\pip\_vendor\resolvelib\resolvers.py", line 347, in resolve
    failure_causes = self._attempt_to_pin_criterion(name, criterion)
  File "c:\users\kftf964\appdata\local\programs\python\python36\lib\site-packages\pip\_vendor\resolvelib\resolvers.py", line 221, in _attempt_to_pin_criterion
    raise InconsistentCandidate(candidate, criterion)
pip._vendor.resolvelib.resolvers.InconsistentCandidate: Provided candidate AlreadyInstalledCandidate(numpy 1.19.5 (c:\users\kftf964\appdata\local\programs\python\python36\lib\site-packages)) does not satisfy SpecifierRequirement('numpy~=1.18'), SpecifierRequirement('numpy~=1.18.0')

Code of Conduct

• I agree to follow the PSF Code of Conduct

We found this problem in a pretty complex internal dependency tree. It's reproducible with a simple numpy install though.

I'd be very happy about a fix! For now, we work around this by explicitly stating version ranges, e.g. >=4.1,<5.

[uranusjr]

The error is an integrity check in the resolver to make sure the versions it gets make sense. The resolver first requests a list of candidates to use with find_matches, and check the list’s integrity with is_satisfied_by. Since 1.19.5 is not compatible with ~=1.18.0 (equivalent to >=1.18.0,==1.18.*), it should not be returned by find_matches in the first place. So you should start here

pip/src/pip/_internal/resolution/resolvelib/provider.py

Lines 133 to 161 in bbf8466

	def find_matches(self, requirements):
	# type: (Sequence[Requirement]) -> Iterable[Candidate]
	if not requirements:
	return []
	name = requirements[0].project_name
	def _eligible_for_upgrade(name):
	# type: (str) -> bool
	"""Are upgrades allowed for this project?
	This checks the upgrade strategy, and whether the project was one
	that the user specified in the command line, in order to decide
	whether we should upgrade if there's a newer version available.
	(Note that we don't need access to the `--upgrade` flag, because
	an upgrade strategy of "to-satisfy-only" means that `--upgrade`
	was not specified).
	"""
	if self._upgrade_strategy == "eager":
	return True
	elif self._upgrade_strategy == "only-if-needed":
	return (name in self._user_requested)
	return False
	return self._factory.find_candidates(
	requirements,
	constraint=self._constraints.get(name, Constraint.empty()),
	prefers_installed=(not _eligible_for_upgrade(name)),
	)

and try to find out where the provided specifiers are not correctly applied to the returning candidate list.

[teije01]

We encountered the same problem today. Thanks for the clear issue description

[uranusjr]

This seems like a bug in packaging. Reported in pypa/packaging#421.

[kasium]

pypa/packaging#421 was fixed. Can this be fixed in pip as well? What is missing?

[pradyunsg]

That needs a new packaging release, which is blocked on dropping legacy version.

[yuvalmarciano]

Is there any update here?

[astrojuanlu]

packaging 22.0 was released in December 2022. Latest stable version is 23.1. Currently, pip vendors packaging 21.3:

pip/src/pip/_vendor/packaging/__about__.py

Lines 16 to 20 in 5e8e88e

	__title__ = "packaging"
	__summary__ = "Core utilities for Python packages"
	__uri__ = "https://github.com/pypa/packaging"
	__version__ = "21.3"

Would a PR upgrading packaging be welcome?

[pfmoore]

Would a PR upgrading packaging be welcome?

Thanks for the offer, but we need to work out how we handle the deprecation of non-PEP 440 versions. This is being worked on.