Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
116 changes: 116 additions & 0 deletions .github/workflows/copy-audit-evidence.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
name: Copy audit UI evidence

on:
push:
branches: [test/copy-audit-ui-evidence]
paths:
- '.github/workflows/copy-audit-evidence.yml'
- 'scripts/ui-evidence/**'
workflow_dispatch:

permissions:
contents: read

jobs:
capture:
name: ${{ matrix.scenario }} before and after
runs-on: ubuntu-latest
timeout-minutes: 25
strategy:
fail-fast: false
matrix:
include:
- scenario: auth
after: 94d238c4d8475439a0f569d05590d270f6ac69d9
- scenario: project-feedback
after: 2f500ace5aa4de98d9422899f18baa2a16f62e5b
- scenario: editor
after: b3c1a666b5f9cdcec94be7480ca978e7fa5397a2
- scenario: onboarding
after: 5318d2e84137dc57e70e747c02cf0d898031df57
env:
NEXT_TELEMETRY_DISABLED: '1'
PUPPETEER_SKIP_DOWNLOAD: 'true'
EVIDENCE_SCENARIO: ${{ matrix.scenario }}
AFTER_SHA: ${{ matrix.after }}
BEFORE_DIR: ${{ github.workspace }}/before
AFTER_DIR: ${{ github.workspace }}/after
EVIDENCE_OUTPUT: ${{ github.workspace }}/artifacts/${{ matrix.scenario }}

steps:
- name: Initialize retained diagnostics
run: |
mkdir -p "$EVIDENCE_OUTPUT"
printf 'scenario=%s\nafter=%s\nharness=%s\n' "$EVIDENCE_SCENARIO" "$AFTER_SHA" "$GITHUB_SHA" > "$EVIDENCE_OUTPUT/setup.txt"

- name: Checkout capture infrastructure
uses: actions/checkout@v4
with:
path: harness

- name: Checkout exact audited baseline
uses: actions/checkout@v4
with:
ref: dd33611e3debb0170b801f42370cda55d6c5caa7
path: before

- name: Checkout exact feature revision
uses: actions/checkout@v4
with:
ref: ${{ matrix.after }}
path: after

- uses: pnpm/action-setup@v4
with:
package_json_file: before/package.json

- uses: actions/setup-node@v4
with:
node-version: '22'
cache: pnpm
cache-dependency-path: before/pnpm-lock.yaml

- name: Install locked app dependencies without unrelated native postinstalls
working-directory: before
run: pnpm install --frozen-lockfile --ignore-scripts

- name: Install independent feature dependencies
working-directory: after
run: pnpm install --frozen-lockfile --ignore-scripts

- name: Install locked evidence tooling
working-directory: harness/scripts/ui-evidence
run: npm ci --ignore-scripts

- name: Install the pinned Chromium browser
id: browser-setup
working-directory: harness/scripts/ui-evidence
run: |
npx playwright install --with-deps chromium
echo "CHROME_PATH=$(node -e "console.log(require('playwright').chromium.executablePath())")" >> "$GITHUB_ENV"

- name: Build the exact feature frontend revision
id: feature-build
if: matrix.scenario == 'project-feedback' || matrix.scenario == 'editor'
working-directory: after/frontend
run: |
set -o pipefail
set +e
pnpm build 2>&1 | tee "$EVIDENCE_OUTPUT/${EVIDENCE_SCENARIO}-after-build.log"
status=$?
set -e
printf '%s\n' "$status" > "$EVIDENCE_OUTPUT/${EVIDENCE_SCENARIO}-build-exit-code.txt"
exit "$status"

- name: Capture real before and after components with local fixture responses
if: ${{ !cancelled() && steps.browser-setup.outcome == 'success' }}
run: node harness/scripts/ui-evidence/run.mjs

- name: Retain screenshots, source SHAs, fixture observations and logs
if: always()
uses: actions/upload-artifact@v4
with:
name: copy-audit-${{ matrix.scenario }}-${{ github.run_id }}
path: artifacts/${{ matrix.scenario }}/
if-no-files-found: error
retention-days: 7
218 changes: 218 additions & 0 deletions .github/workflows/copy-audit-integration.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,218 @@
name: Copy audit feature integration

on:
push:
branches: [test/copy-audit-ui-evidence]
paths:
- '.github/workflows/copy-audit-integration.yml'
workflow_dispatch:

permissions:
contents: read

jobs:
validate:
name: ${{ matrix.scenario }} with CI repair
runs-on: ubuntu-latest
timeout-minutes: 45
strategy:
fail-fast: false
matrix:
include:
- scenario: onboarding
feature_sha: 5318d2e84137dc57e70e747c02cf0d898031df57
jest_config: jest.onboarding-copy.config.js
- scenario: project-feedback
feature_sha: 2f500ace5aa4de98d9422899f18baa2a16f62e5b
jest_config: jest.project-feedback.config.js
- scenario: auth
feature_sha: 94d238c4d8475439a0f569d05590d270f6ac69d9
jest_config: jest.auth-copy.config.js
- scenario: editor
feature_sha: b3c1a666b5f9cdcec94be7480ca978e7fa5397a2
jest_config: jest.editor-feedback.config.js
defaults:
run:
shell: bash
working-directory: integration
env:
CI: 'true'
NEXT_TELEMETRY_DISABLED: '1'
PUPPETEER_SKIP_DOWNLOAD: 'true'
CI_SHA: f5ea2bfdf2037ace6ad50f6c3c7feaa9df0693d9
FEATURE_SHA: ${{ matrix.feature_sha }}
FRONTEND_JEST_CONFIG: ${{ matrix.jest_config }}
ARTIFACT_DIR: ${{ github.workspace }}/ci-integration-artifacts/${{ matrix.scenario }}

steps:
- name: Initialize retained diagnostics
working-directory: ${{ github.workspace }}
run: |
mkdir -p "$ARTIFACT_DIR"
printf 'feature_sha=%s\nci_sha=%s\nworkflow_sha=%s\nrun_id=%s\nrun_attempt=%s\n' \
"$FEATURE_SHA" "$CI_SHA" "$GITHUB_SHA" "$GITHUB_RUN_ID" "$GITHUB_RUN_ATTEMPT" \
> "$ARTIFACT_DIR/requested-revisions.txt"

- name: Checkout the exact CI repair with full history
id: checkout
uses: actions/checkout@v4
with:
repository: CodeFox-Repo/codefox
ref: ${{ env.CI_SHA }}
path: integration
fetch-depth: 0
persist-credentials: false

- name: Merge the exact feature locally without creating a commit
id: merge
run: |
set -euo pipefail
exec > >(tee "$ARTIFACT_DIR/merge.log") 2>&1
test "$(git rev-parse HEAD)" = "$CI_SHA"
git fetch --no-tags origin "$FEATURE_SHA"
test "$(git rev-parse FETCH_HEAD)" = "$FEATURE_SHA"
merge_base=$(git merge-base "$CI_SHA" "$FEATURE_SHA")
git -c user.name='Copy audit integration' \
-c user.email='copy-audit-integration@users.noreply.github.com' \
-c core.hooksPath=/dev/null \
merge --no-commit --no-ff "$FEATURE_SHA"
# HEAD must remain the CI repair; the staged tree is the integration.
test "$(git rev-parse HEAD)" = "$CI_SHA"
integration_tree=$(git write-tree)
git cat-file -e "$integration_tree:frontend/$FRONTEND_JEST_CONFIG"
printf 'INTEGRATION_TREE=%s\n' "$integration_tree" >> "$GITHUB_ENV"
printf '{\n "feature_sha": "%s",\n "ci_sha": "%s",\n "merge_base": "%s",\n "integration_tree": "%s",\n "workflow_sha": "%s"\n}\n' \
"$FEATURE_SHA" "$CI_SHA" "$merge_base" "$integration_tree" "$GITHUB_SHA" \
> "$ARTIFACT_DIR/provenance.json"
git status --short > "$ARTIFACT_DIR/merged-status.txt"
git diff --stat "$merge_base" "$integration_tree" > "$ARTIFACT_DIR/merged-diff-stat.txt"
# Include eligible changes from both the feature and the CI repair.
git diff --name-only --diff-filter=ACMR -z "$merge_base" "$integration_tree" -- \
'*.js' '*.jsx' '*.ts' '*.tsx' > "$ARTIFACT_DIR/changed-js-ts.nul"
cat "$ARTIFACT_DIR/provenance.json"

- name: Set up Node.js 22
uses: actions/setup-node@v4
with:
node-version: '22'

- name: Set up the declared pnpm version
uses: pnpm/action-setup@v4
with:
version: 9.1.0
package_json_file: integration/package.json
run_install: false

- name: Install independently from the lockfile
id: install
run: |
set -euo pipefail
exec > >(tee "$ARTIFACT_DIR/install.log") 2>&1
# No restored node_modules, shared checkout links, or dependency cache.
test ! -e node_modules
test ! -e frontend/node_modules
test ! -e backend/node_modules
node --version
pnpm --version
test "$(pnpm --version)" = '9.1.0'
pnpm install --frozen-lockfile --store-dir "$RUNNER_TEMP/copy-audit-pnpm-store"

# A failed validation stays blocking, while unrelated checks still run.
- name: Check frontend TypeScript
id: frontend_types
if: ${{ !cancelled() && steps.merge.outcome == 'success' && steps.install.outcome == 'success' }}
run: |
pnpm exec tsc --project frontend/tsconfig.json --noEmit 2>&1 | tee "$ARTIFACT_DIR/frontend-types.log"

- name: Check backend TypeScript
id: backend_types
if: ${{ !cancelled() && steps.merge.outcome == 'success' && steps.install.outcome == 'success' }}
run: |
pnpm exec tsc --project backend/tsconfig.json --noEmit 2>&1 | tee "$ARTIFACT_DIR/backend-types.log"

- name: Build frontend
id: frontend_build
if: ${{ !cancelled() && steps.merge.outcome == 'success' && steps.install.outcome == 'success' }}
working-directory: integration/frontend
run: |
pnpm build 2>&1 | tee "$ARTIFACT_DIR/frontend-build.log"

- name: Build backend
id: backend_build
if: ${{ !cancelled() && steps.merge.outcome == 'success' && steps.install.outcome == 'success' }}
working-directory: integration/backend
run: |
pnpm build 2>&1 | tee "$ARTIFACT_DIR/backend-build.log"

- name: Run all source guards
id: source_guards
if: ${{ !cancelled() && steps.merge.outcome == 'success' && steps.install.outcome == 'success' }}
run: |
pnpm check 2>&1 | tee "$ARTIFACT_DIR/source-guards.log"

- name: Run the matching focused frontend suite
id: frontend_tests
if: ${{ !cancelled() && steps.merge.outcome == 'success' && steps.install.outcome == 'success' }}
working-directory: integration/frontend
run: |
pnpm exec jest --config "$FRONTEND_JEST_CONFIG" --runInBand 2>&1 | tee "$ARTIFACT_DIR/frontend-jest.log"

- name: Run the full backend Jest suite
id: backend_tests
if: ${{ !cancelled() && steps.merge.outcome == 'success' && steps.install.outcome == 'success' }}
working-directory: integration/backend
run: |
pnpm exec jest --runInBand 2>&1 | tee "$ARTIFACT_DIR/backend-jest.log"

- name: Lint changed eligible JavaScript and TypeScript without fixes
id: lint
if: ${{ !cancelled() && steps.merge.outcome == 'success' && steps.install.outcome == 'success' }}
run: |
set -euo pipefail
pnpm exec node <<'NODE' 2>&1 | tee "$ARTIFACT_DIR/lint-selection.log"
const fs = require('node:fs');
const path = require('node:path');
const { ESLint } = require('eslint');
(async () => {
const output = process.env.ARTIFACT_DIR;
const candidates = fs.readFileSync(path.join(output, 'changed-js-ts.nul'), 'utf8').split('\0').filter(Boolean);
const eslint = new ESLint({ resolvePluginsRelativeTo: path.resolve('backend') });
const eligible = [];
const ignored = [];
for (const file of candidates) {
if (await eslint.isPathIgnored(file)) {
ignored.push(file);
console.log(`Ignored by existing ESLint configuration: ${file}`);
} else {
eligible.push(file);
console.log(`Linting: ${file}`);
}
}
fs.writeFileSync(path.join(output, 'lint-files.nul'), eligible.map(file => `${file}\0`).join(''));
fs.writeFileSync(path.join(output, 'lint-files.json'), JSON.stringify({ eligible, ignored }, null, 2) + '\n');
})().catch(error => { console.error(error); process.exitCode = 1; });
NODE
mapfile -d '' files < "$ARTIFACT_DIR/lint-files.nul"
if (( ${#files[@]} )); then
pnpm exec eslint --resolve-plugins-relative-to backend "${files[@]}" 2>&1 | tee "$ARTIFACT_DIR/eslint.log"
else
echo 'No changed files are eligible under the existing ESLint configuration.' | tee "$ARTIFACT_DIR/eslint.log"
fi

- name: Retain validation outcomes
if: always()
working-directory: ${{ github.workspace }}
env:
STEP_RESULTS: ${{ toJSON(steps) }}
run: |
mkdir -p "$ARTIFACT_DIR"
printf '%s\n' "$STEP_RESULTS" > "$ARTIFACT_DIR/step-outcomes.json"

- name: Upload integration provenance and logs
if: always()
uses: actions/upload-artifact@v4
with:
name: copy-audit-integration-${{ matrix.scenario }}-${{ github.run_id }}-${{ github.run_attempt }}
path: ${{ env.ARTIFACT_DIR }}/
if-no-files-found: error
retention-days: 7
36 changes: 36 additions & 0 deletions scripts/ui-evidence/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# Copy-audit UI evidence (CI only)

This infrastructure is deliberately separate from the feature changes. The
workflow runs on pushes to `test/copy-audit-ui-evidence`, so it can collect
review evidence before any workflow is merged into the default branch.

Each matrix job checks out the exact audited baseline and one exact published
feature commit, installs their locked dependencies independently, then starts
actual Next/React components with deterministic local fixture responses. Only
pnpm's download cache is shared. Never symlink dependencies or build output
between source trees: Next's output tracing can follow those links during
cleanup.

The runner verifies both source SHAs and writes them together with its own
harness SHA and the fixture limitations into `run.json`. Images visibly label
their fixture scope and source commit. Application copy is never replaced by
the capture scripts. The editor harness adds a disposable route importing the
real components and a minimal layout retaining the real font definitions; that override is recorded explicitly.

All browser network requests outside the local fixture endpoints are blocked
(or, for the decorative GitHub star counter, answered with a fixed local
response). All identities, strings and tokens are synthetic. Auth/onboarding
assert zero submitted mutations. Project actions and deployment use local
mock outcomes, not live services. These captures do not validate production
accounts, emails, paid generation, persistence, deployment or real clipboard
permission handling.

The workflow uses read-only repository permissions and no secrets. It pins the
browser tooling, fails on assertions, retains raw PNGs/text observations and
logs for seven days, and uploads artifacts even if a scenario fails. Failure
captures and metadata help diagnose incomplete runs. Inspect the pixels before
calling the result visually verified.

The harnesses have syntax/YAML validation locally; browser execution is owned
by the isolated repository CI jobs. Do not use these scripts as an alternate
route around a denied local browser connection.
13 changes: 13 additions & 0 deletions scripts/ui-evidence/auth/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Auth copy UI evidence

The capture script starts the actual Next frontends from two checkouts, with synthetic GraphQL responses and all external browser requests blocked. It captures signup requirements and unavailable password-reset email before and after. It never submits registration, login, email, payment, or generation mutations.

Prerequisites: Node 20+, repository dependencies installed from pnpm-lock.yaml with pnpm 9.1.0; Playwright plus Chromium available to this script. Make the original base commit dd33611e3debb0170b801f42370cda55d6c5caa7 and the auth fix commit available as separate checkouts. Install each checkout independently; never symlink node_modules between source trees.

Run:

AUTH_COPY_BEFORE_DIR=/absolute/base AUTH_COPY_AFTER_DIR=/absolute/fix AUTH_COPY_OUTPUT_DIR=/absolute/artifacts node capture.mjs

Ports: browser fixtures 3204 (before) and 3104 (after); internal Next servers 3206 and 3106. Fixture responses are local only. Both contexts block requests to any other origin. Entered strings are synthetic example.test addresses and a dummy password, with no submissions.

Output: four PNGs plus metadata.json recording exact checkout SHAs, fixture state, viewport, and that no mutations were submitted. These are fixture-state screenshots, not production sign-in or email-delivery validation.
Loading
Loading