Skip to content

Verify registry checksums, including MD5 and SHA-1 - #3

Merged
andrew merged 1 commit into
git-pkgs:mainfrom
simonchrz:registry-checksums
Oct 14, 2026
Merged

andrew merged 1 commit into
git-pkgs:mainfrom
simonchrz:registry-checksums

Conversation

@simonchrz

Copy link
Copy Markdown
Contributor

Closes #1.

Registries publish checksums that aren't SRI. Some are algorithm-prefixed hex (Cargo, Pub, RubyGems and others give sha256-<hex>). Some use SHA-1: npm's dist.shasum fallback and Packagist's dist.shasum. Conda falls back to MD5, and Vagrant gives the algorithm and hex separately. This module couldn't parse or verify any of those, so the proxy treated a valid checksum as unusable.

Changes

Algorithms (algorithm.go)

  • Adds MD5 and SHA1 after the existing constants, so the existing values don't change.
  • NewReader calculates both. ParseHex accepts all five algorithms and checks the 16-byte and 20-byte lengths.
  • Strength is now an explicit ranking (byStrength): MD5 < SHA-1 < SHA-256 < SHA-384 < SHA-512. Before, strongestAlgorithm compared the constants' numeric values, which no longer follow strength.

Registry checksums (digest.go, reader.go)

  • ParseChecksum parses one algorithm-prefixed hexadecimal checksum, <alg>-<hex> or <alg>:<hex>. It is case-insensitive and ignores surrounding whitespace.
  • Digest.Checksum returns the canonical <alg>-<hex> form.
  • Result.VerifyDigest checks a completed stream against one exact digest in any algorithm. It needs a complete stream (ErrIncomplete otherwise) and needs the reader to have calculated that algorithm.

SRI stays strict

  • ParseSRI rejects MD5 and SHA-1, even as valid base64 of the right length.
  • Digest.SRI returns "" for them, and FormatSRI leaves them out.
  • Result.Verify keeps the SRI strongest-algorithm rule. It refuses an SRI list that contains a weak digest, even one built by hand.

Docs

  • The package doc and README describe MD5 and SHA-1 as compatibility checks for checksums publishers already supply, never to be chosen for new integrity metadata. The README gains a "Verify a registry checksum" section.

Julia's git-tree-sha1 is left out, as the issue says: it identifies a tree, not the downloaded tarball.

Tests

checksum_test.go:

  • Hex parsing: MD5 and SHA-1 hex parsing with canonical output, and digest-length errors.
  • ParseChecksum: all five algorithms, both separators, upper case and whitespace, plus malformed values (no prefix, wrong length, base64, unknown algorithm, two entries).
  • Weak digests stay out of SRI: they are rejected by ParseSRI (alone and in a list), have no SRI encoding, and are refused by Result.Verify.
  • Reader and VerifyDigest: the reader calculates MD5 and SHA-1. VerifyDigest passes on a match, fails on a mismatch, needs a complete stream, and needs the calculated algorithm (or the zero Digest).
  • Strength ranking: each algorithm ranks below the next one in the list.

The existing multi-hash SRI tests pass unchanged. A new FuzzParseChecksum round-trips through Checksum.

I checked that the tests catch regressions by breaking each rule in turn: weak names in ParseSRI, SRI encoding for weak digests, Verify accepting them, VerifyDigest always matching or skipping the completeness check, and ranking by constant order. Every one fails a test.

go test -race ./..., 20 s of FuzzParseChecksum and 10 s of FuzzSRIRoundTrip, make lint's golangci-lint linters (v2.13.1) and govulncheck all pass.

This unblocks git-pkgs/proxy#473's follow-up: npm's dist.shasum fallback and Packagist's dist.shasum check.

Registries publish checksums that are not SRI: algorithm-prefixed hex
such as Cargo's sha256-<hex>, npm's and Packagist's SHA-1 shasum, Conda's
MD5. Callers could not parse or verify them, so a valid checksum was
treated as unusable.

- Add verification-only MD5 and SHA1 algorithms, calculated by NewReader
  and accepted by ParseHex with their digest lengths checked
- Add ParseChecksum for a single algorithm-prefixed hexadecimal checksum,
  Digest.Checksum for its canonical form, and Result.VerifyDigest to check
  a stream against one exact digest
- Keep MD5 and SHA-1 out of SRI: ParseSRI rejects them, Digest.SRI is empty
  for them, FormatSRI leaves them out, and Result.Verify refuses them
- Rank algorithms by an explicit strength order rather than the order of
  the constants, which the new algorithms no longer follow
- Document MD5 and SHA-1 as compatibility checks only

Closes git-pkgs#1
@andrew
andrew merged commit b1e7c9d into git-pkgs:main Oct 14, 2026
4 checks passed
simonchrz added a commit to simonchrz/proxy that referenced this pull request Oct 14, 2026
v0.2.0 adds verification-only SHA-1 and MD5, algorithm-prefixed checksum
parsing and exact-digest verification (git-pkgs/integrity#3).
simonchrz added a commit to simonchrz/proxy that referenced this pull request Oct 14, 2026
v0.2.0 adds verification-only SHA-1 and MD5, algorithm-prefixed checksum
parsing and exact-digest verification (git-pkgs/integrity#3).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Parse and verify registry-native checksums alongside SRI

2 participants