Skip to content

test_dependency_versions.py silently skips checking Pillow and pyyaml due to a case-sensitivity bug #635

Description

@yakew7

Where: tests/test_dependency_versions.py's test_requirements_lock_versions_satisfy_pyproject_minimums.

The gap: the test builds pyproject_versions and locked_versions dicts keyed by the dependency name exactly as regex-captured from each file, then does a case-sensitive dict lookup:

for dependency, minimum in pyproject_versions.items():
    locked = locked_versions.get(dependency)
    dependency = normalize_dependency(dependency)   # only affects the later error message
    if locked is None:
        continue
    assert version_tuple(locked) >= version_tuple(minimum), ...

normalize_dependency() exists specifically to reconcile naming differences between the two files, but it's applied to the variable used in the failure message only - never to the dict key used for the lookup itself. pyproject.toml declares Pillow>=11.1 and pyyaml>=6.0; requirements-lock.txt (a real pip freeze) pins them as pillow==11.1.0 and PyYAML==6.0.3 - different casing on each side.

Repro:

import re
from pathlib import Path
pyproject_text = Path('pyproject.toml').read_text()
lock_text = Path('requirements-lock.txt').read_text()
pyproject_versions = dict(re.findall(r'([A-Za-z0-9_.-]+)\s*>=\s*([0-9]+(?:\.[0-9]+)*)', pyproject_text))
locked_versions = dict(re.findall(r'^([A-Za-z0-9_.-]+)==([0-9]+(?:\.[0-9]+)*)', lock_text, re.MULTILINE))
print(locked_versions.get('Pillow'))   # None - case mismatch
print(locked_versions.get('pyyaml'))   # None - case mismatch

Since locked is None for both, the loop's if locked is None: continue fires and the version assertion never runs for these two real dependencies. Simulating a deliberately-broken lock (pillow==9.0.0, below the declared >=11.1 minimum) still produces zero failures - this test would stay green even if Pillow's locked version were downgraded below what pyproject.toml requires.

Fix direction: normalize both dicts' keys with normalize_dependency() before building them (or at least before the lookup), matching how tests/test_declared_dependencies.py already normalizes both sides correctly.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workinghelp wantedExtra attention is needed

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions