uv-inspired: cross-platform lockfile, hashes, sync, tree, add/remove#8
Open
voidreamer wants to merge 8 commits into
Open
uv-inspired: cross-platform lockfile, hashes, sync, tree, add/remove#8voidreamer wants to merge 8 commits into
voidreamer wants to merge 8 commits into
Conversation
Track every (requester, constraint) pair encountered during depth-first resolution. When a later request is incompatible with an already-chosen version, emit a diagnostic that names the chosen version, lists every package that asked for the name, and marks the incompatible row. The "package not found" and "no matching version" messages now include the requester (parent package id, "<request>" for top-level, or "<lockfile>" for stale pins) so deep dependency failures are traceable at a glance. Adds Display impls for VersionConstraint and PackageRequest so the diagnostics round-trip the original constraint syntax. The resolver still does not backtrack -- the first chosen version wins, and conflicts are surfaced rather than worked around -- but every error now identifies both sides of the conflict.
Each pin now stores a SHA-256 of the package definition file alongside
its version. When the resolver loads a lockfile it compares each
recorded hash against the file currently on disk and emits a warning
if they differ -- the case where someone edits a shared package on a
studio filesystem after a project locked, which used to be silent.
Pin gains a custom Deserialize that accepts both the new map form
python: { version: "3.11", content_hash: "..." }
and the legacy string form
python: "3.11"
so existing anvil.lock files keep working without re-locking.
Adds the sha2 crate, a content_hash() helper on Package, and a
source_path field that the loader populates so the hash can be
recomputed without re-walking the package paths.
Variant application moved from package load time to the resolver: the package cache now stores raw definitions (variants intact, requires/env not yet merged), and the resolver merges the variant for the chosen platform when materialising each resolved package. This lets `anvil lock --all-platforms` resolve the same request set as if running on linux, macos, and windows in turn from a single scan. The lockfile schema gains two fields, both with serde defaults so older files still parse: - platforms: which platforms were resolved at lock time - platform_pins: per-platform overlay applied on top of common `pins` Pins shared by every locked platform (same name + version + content hash) live in `pins`; pins that differ per platform live under `platform_pins[<platform>]`. When the resolver loads a lockfile it overlays the entry for the current running platform via the new Lockfile::effective_pins helper, so a single anvil.lock is correct on any locked target. The package scan cache salt is bumped so caches written by older anvil binaries (which stored variant-merged Packages) get re-built rather than confusing the new resolve-time variant logic.
--locked: re-resolve the locked request set fresh and diff against the pins on disk before running any command. Any drift -- different version, different content hash, missing or extra package -- aborts with a per-line breakdown. Fails fast when there is no lockfile. Useful for CI to catch a forgotten `anvil lock` re-run. --frozen: refuses to use anything not already pinned. Every name the resolver touches (top-level requests, transitive requires) must appear in anvil.lock; otherwise the command fails with the offending package and its requester. Skipping a fresh resolve was already the default whenever a lockfile is present, so this is purely a guard against unintended drift on render farms or shared workstations. The two flags are mutually exclusive (clap conflicts_with) and apply globally to every command that constructs a Resolver. Lockfile gains a Lockfile::diff_pins helper that the --locked path uses to format the failure message; the same helper will be reused by `anvil sync`.
Walks every effective pin for the current platform and, for each: - confirms the pinned name+version exists in the package paths - compares content hashes (if recorded) and reports drift - validates each command-alias target resolves to an executable Prints one line per pin (ok / warn / fail) plus a summary, and exits non-zero only when one or more pins fail outright (missing package). Hash drift and broken command targets are warnings -- the lockfile is intact, but the operator should know. Useful before farm jobs that must not fail mid-run on a missing alias, or as a smoke test after a workstation rsyncs a new package set.
Resolves the requested package set and walks the requires graph, printing a Unicode-box ASCII tree with one root per top-level request. Repeated nodes (diamond dependencies, shared transitive deps) are marked `name-version (*)` after their first appearance, which keeps the output compact and terminates cycles.
Repeatable flag. Loads the existing lockfile, drops only the named pins, then resolves -- so the upgraded names get the highest matching version while every other pin stays exactly where it was. Matches the mid-show case where you want python to bump but every other dep must not move. Errors out clearly if there is no anvil.lock to start from. Without the flag, `anvil lock` keeps its existing always-fresh behaviour.
Both commands mutate anvil.lock's `requests` array and re-resolve so
the lockfile reflects the new project package set. This lets users
manage the project's package list without editing YAML or repeating
themselves on the command line.
anvil add maya-2024 -- create lockfile if missing,
append maya-2024
anvil add maya-2025 -- replace any existing request for
'maya' with the new constraint
anvil remove arnold -- drop every request whose package
name is 'arnold' (constraints OK)
Both refuse to write an empty lockfile (unusual state, probably a
mistake), and `anvil remove` refuses to run without an existing
anvil.lock to mutate.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Eight uv-shaped improvements to the lockfile and resolver, ordered by user-visible impact.
Resolver / lockfile (tier 1)
(requester, constraint)pair is recorded, and a mismatch produces a multi-line diagnostic naming the chosen version, every package that asked for the name, and the incompatible row. "Package not found" / "no matching version" gain the requester chain.package_pathsfilesystems; legacy string-form pins (python: "3.11") keep parsing via untagged-enum compat.anvil lock --all-platformsresolves once per linux/macos/windows from a single cache (variant application moved from load time to resolve time). Common pins live inpins; per-platform diffs land inplatform_pins; reader overlays the entry for the running platform.CLI ergonomics (tier 2)
--locked(re-resolve and diff vs disk; CI gate) and--frozen(refuse anything not pinned; render farms) as global flags.anvil sync— per-pin verification (existence, hash drift, command-target reachability) with green/red counts.anvil tree— box-drawing dependency graph;(*)marks repeats so diamond deps don't print twice.anvil lock --upgrade-package <name>— surgical bumps. Drops only the named pin(s); everything else stays exactly where it was.anvil add/anvil remove— mutate the lockfile's request set and re-resolve so the project's package list is managed without hand-editing YAML.Test plan
lockrecordscontent_hash:; legacypins: { python: "3.11" }still parses; modifying a YAML after lock fires a drift warning.--all-platformswritesplatform_pins:overlay; single-platform lock omits it.--lockedpasses when fresh, fails on stale lock with a readable diff, fails when no lock exists.--frozenuses the lock verbatim, fails when an unpinned name is requested, fails when no lock exists.anvil syncexits 0 on a healthy lockfile, non-zero when a pin's package is missing, warns (exit 0) on hash drift.anvil treerenders ├── / └── connectors and(*)for diamond deps.--upgrade-package pythonre-resolves only python and keeps every other pin.anvil add/anvil removecreate/append/replace/drop requests and re-lock; both refuse to write an empty lockfile.Notes
schema=v2|...) so caches written by older binaries (which pre-merged variants intoPackage) get rebuilt automatically.