Skip to content

test: matrix scenarios - Homebrew consumer path, N-1 to N upgrade, state preservation #2562

Open

Description

@codeforester

Goal

Automate the Homebrew consumer path and upgrade rehearsal, including preservation of per-user Base state.

Background

Homebrew is a single /opt/homebrew owned by the admin account, so the brew matrix accounts can run the formula but cannot install or upgrade it; the admin drives install and upgrade and each brew user verifies the consumer experience. The upgrade rehearsal is manual today.

Evidence that this path needs CI-independent coverage: the 1.10.3 fix (CHANGELOG) was found by a Homebrew readback after the 1.10.2 publication, when a packaged install nested inside an unrelated Git checkout reported the checkout's version.

Next-release planning review of 2026-10-14/15 against main at 7faf6dfb. Tracking: #2555.

Prerequisites: #2559, #2560.

Scope

  • Admin-side step: install the released formula, then upgrade to the candidate formula or tap branch.
  • Per-user steps for both brew cells: pre-upgrade state capture (version, projects list, config checksum, venv present), upgrade, post-upgrade comparison.
  • Consumer smoke: base-demo setup, check, doctor, test, demo; package-aware basectl test base.
  • Regression scenario for packaged version identity when the Cellar path sits inside a Git checkout.

Acceptance Criteria

  • Both brew cells pass N-1 to N on the latest two releases.
  • State captured before upgrade is preserved after it, with differences reported by name.
  • The shared-prefix limitation and who runs which step are documented.
  • The version-identity regression fails on a build that reintroduces the 1.10.3 bug (recorded).

Validation

<matrix runner> --cells brew-bash,brew-zsh --scenario brew-upgrade --from v1.10.4 --to <candidate>

Attach the result JSON to the PR.

Non-Goals

  • Do not create a second Homebrew prefix.
  • Do not change the release process.

Project Fields

  • Status: Backlog
  • Priority: P1
  • Area: Packaging
  • Initiative: Contract Hardening
  • Size: M
  • Milestone: v1.12.0
  • Category: ci

Agent Assignment

Assignee: codeforester. Blocked on #2559 and #2560. Needs the admin account for brew mutations.

Activity

  1. added this to the v1.12.0 milestone on Oct 15, 2026
  2. self-assigned this
    on Oct 15, 2026
  3. added
    ciContinuous integration, tests, automation, or release workflows
    on Oct 15, 2026
  4. codeforester commented on Oct 19, 2026

    @codeforester
    CollaboratorAuthor

    Follow-up from #2596/#2608: the core stale-managed-runtime contract is now merged in #2608. #2562 should consume that contract for the remaining full N-1 to N Homebrew consumer scenario: seed the previous-version environment, upgrade, compare preserved state, verify the exact installed base-cli, run basectl check base --verify-project-runtime, and prove basectl test base reaches the declared test command.

    No separate issue is needed; this scope already belongs to #2562. #2596 is being closed as the completed core defect, while #2562 remains the follow-up matrix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

ciContinuous integration, tests, automation, or release workflows

Type

No type

Projects

  • Status
    Backlog

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions