Skip to content

test: harden fixtures and streamline CI - #83

Open
gbaudrit wants to merge 5 commits into
mainfrom
codex/test-suite-review
Open

test: harden fixtures and streamline CI#83
gbaudrit wants to merge 5 commits into
mainfrom
codex/test-suite-review

Conversation

@gbaudrit

@gbaudrit gbaudrit commented Aug 31, 2026

Copy link
Copy Markdown
Owner

Summary

This PR follows a complete review of the repository test suites and CI execution path.

  • isolates every Testing web host in a single data directory and deletes owned SQLite/data-protection files on shutdown;
  • treats a missing, empty, or whitespace-only Data:TestingDirectory as unconfigured and creates an owned temporary directory;
  • preserves the existing non-Testing storage fallbacks under <ContentRoot>/.agentstration, even when Data:Directory is customized;
  • lets fixtures provide Data:TestingDirectory when they own cleanup, including startup-failure scenarios;
  • enables MSTest class-level parallelism for the Management suite while preserving the existing DoNotParallelize guard around environment-variable tests;
  • clears only the SQLite pools owned by each fixture or test host, preventing parallel fixtures from disposing one another's live connections;
  • removes the obsolete build.yml workflow, whose platform, AEP, and container jobs were fully duplicated by ci.yml;
  • normalizes the mixed line endings in McpToolCatalogTests.cs in a dedicated commit.

Audit findings

  • The repository has 11 platform test projects plus the separate AEP test project.
  • The Management project was the critical path: 129 tests took about 3m39 serially.
  • Web test hosts created identity, control-plane, work-plane, flow-plane, runtime, scheduler, and data-protection files in unrelated temporary locations. More than 123,000 agentstration-* entries had accumulated in %TEMP% on the review machine. Existing files were deliberately not deleted by this PR.
  • A declarative-bootstrap startup-failure test could not reach the hosted cleanup service; its data is now owned and removed by the test fixture itself.
  • Full validation exposed that process-wide SqliteConnection.ClearAllPools() calls were unsafe after enabling class-level parallelism. Cleanup is now scoped to the database files owned by each fixture.
  • .github/workflows/build.yml repeated the complete platform test, AEP test, and container build already covered (with newer actions, formatting, timeouts, and Windows lifecycle coverage) by .github/workflows/ci.yml.
  • No coverage threshold is currently enforced. This PR does not introduce a speculative threshold or new coverage dependency.
  • Two live-provider tests remain intentionally optional and inconclusive unless LocalAI/llama.cpp endpoints are configured.

Compatibility

  • Data:Directory continues to configure identity, scheduler, data-protection, and related general data paths outside Testing as before.
  • The control, work, flow, and runtime plane fallbacks remain <ContentRoot>/.agentstration/*.db outside Testing, unless their explicit Data:*Path setting is provided.
  • The unified per-host directory applies only to the Testing environment; this PR does not migrate or relocate production data.

Before / after

  • Management suite: ~3m39 -> 2m00-2m32 across repeated runs.
  • Complete platform suite: ~4m31 -> ~2m33.
  • Owned test directories during the final complete run: 201 before / 201 after (no growth).
  • CI no longer runs a second copy of the platform, AEP, and container jobs.

Validation

  • dotnet format Agentstration.slnx --no-restore --verify-no-changes — passed
  • dotnet build Agentstration.slnx --configuration Release --no-restore — 0 warnings, 0 errors
  • dotnet test Agentstration.slnx --configuration Release --no-build — 539 total, 537 passed, 2 optional skipped, 0 failed
  • Management suite repeated under class-level parallelism — 129/129 passed on every run, including two focused repetitions plus the final complete suite
  • Testing-directory lifecycle coverage includes missing, empty, and whitespace-only configuration
  • Final complete run temporary-directory count — 201 before, 201 after
  • dotnet build aep/Aep.slnx --configuration Release --no-restore — 0 warnings, 0 errors
  • dotnet test aep/Aep.slnx --configuration Release --no-build — 9/9 passed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant