Skip to content

feat: add deterministic shared triage foundation - #4

Merged
alexdancer merged 7 commits into
mainfrom
fm/huddle-quick-demo-shared-foundation-a1
Jul 29, 2026
Merged

feat: add deterministic shared triage foundation#4
alexdancer merged 7 commits into
mainfrom
fm/huddle-quick-demo-shared-foundation-a1

Conversation

@alexdancer

Copy link
Copy Markdown
Owner

Intent

Land only the bounded shared executable foundation that lets the later Evidence Desk and Operations streams work in parallel: framework-free @huddle/application ports and DTOs; per-fired-rule signal/evidence identity and exact additional evidence; server-only GuideAccess composition; ordered DB/migration seams; PostgreSQL-driver isolation; truthful synthetic-only legacy/import policies; half-open America/Chicago boundaries; separate response semantics from timing profiles; centralized fingerprinted synthetic-demo severity; and the captain-approved portfolio regression/no-accuracy-claim specification. Do not build UI, import/refresh pipelines, model runtime, deployment, reset, README packaging, or the full simulator/eval. Preserve deterministic, grounded, synthetic-only boundaries.

What Changed

  • Add framework-free application ports and DTOs for board, evidence, compilation, and import flows, plus server-only guide access and ordered PostgreSQL seams.
  • Harden deterministic signal compilation with per-rule identities, canonical fingerprints, exact supporting evidence, reconciled confidence and ranking, and half-open Chicago calendar boundaries.
  • Separate response semantics from timing profiles, contain legacy data as synthetic-only and non-displayable, and align specifications with the bounded regression and no-accuracy-claim policy.

Risk Assessment

✅ Low: The follow-up consistently closes the composite activity-identity and civil-date gaps without expanding scope, and the full source review found no remaining material issue.

Testing

After base-to-target intent inspection, the focused automated tests and runtime evidence probe passed, directly demonstrating the shared foundation’s deterministic, grounded, synthetic-only boundaries. No screenshot was captured because this change intentionally contains no UI; the reviewer-visible artifact is the actual developer-facing runtime JSON.

Evidence: Shared foundation runtime evidence

Runtime JSON demonstrates 21 acceptance assertions, including per-rule evidence identity, DST-safe half-open windows, synthetic-only policies, timing-profile separation, server/driver boundaries, migration ordering, and portfolio no-accuracy-claim policy.

{
  "scenario": "shared executable foundation runtime probe",
  "applicationPolicy": {
    "canonicalIdenticalDuplicates": "collapse",
    "divergentSameIdentity": "reject-identity-group-atomically",
    "divergentSessionAggregate": "reject-session-group-atomically",
    "references": "fixed-seeded-synthetic-only",
    "rejectedRawRetention": "none"
  },
  "packageBoundaries": {
    "applicationRuntimeDependencies": [
      "@huddle/core"
    ],
    "databasePackageOwnsPostgresDriver": true,
    "nonDatabasePackagesWithPostgresDriver": [],
    "guideAccessCompositionIsServerOnly": true,
    "guideAccessRequiresVerifiedAuthBeforeDatabaseScope": true,
    "guideAccessReturnsSyntheticOnlyCapability": true
  },
  "chicagoHalfOpenBoundary": {
    "timezone": "America/Chicago",
    "start": "2026-03-02T06:00:00.000Z",
    "end": "2026-03-09T05:00:00.000Z",
    "elapsedHoursAcrossSpringDst": 167,
    "exactStartIncluded": true,
    "oneMillisecondBeforeEndIncluded": true,
    "exactEndIncluded": false
  },
  "timingPolicySeparation": {
    "seededItemId": "wp:TEKS.4.5A-01",
    "responseItemType": "multiple_choice",
    "timingProfile": "word_problem",
    "boundMs": 300000,
    "overBoundElapsedMsAfterIngest": null,
    "engagedMsRetained": 299000,
    "resultingTimingQuality": "engaged"
  },
  "opaqueActivityIdentity": {
    "sha256": "101b92ecec6f9a2f72e8e802d7e7ee0492db2f270146c007125c0f369accd801",
    "stableForSameCompositeIdentity": true,
    "exposesSourceComponent": false,
    "exposesEventComponent": false
  },
  "firedSignals": [
    {
      "ruleId": "guessing.skill",
      "signalId": 1,
      "signalIdentity": "11111111-1111-1111-1111-111111111111:TEKS.4.3E:guessing.skill:2026-03-02T06:00:00.000Z:2026-03-09T05:00:00.000Z:99355166dba09562221fbdb1784f70c0787fbf2d6bc0a880932f0441e264825a",
      "behaviorFingerprint": "99355166dba09562221fbdb1784f70c0787fbf2d6bc0a880932f0441e264825a",
      "evidenceFingerprint": "72662a747896079284e40b3939da40646217570ac48490dfa1bc207cabe48e4a",
      "rawConfidence": 0.9,
      "finalConfidence": 0.63,
      "conflictMultiplier": 0.7,
      "exactActivityIds": [
        "101b92ecec6f9a2f72e8e802d7e7ee0492db2f270146c007125c0f369accd801"
      ],
      "additionalEvidenceLinks": [
        {
          "signalId": 2,
          "cause": "disengagement",
          "severity": 0.7,
          "finalConfidence": 0.5599999999999999,
          "ruleId": "engagement.cross",
          "scope": {
            "kind": "cross-skill"
          }
        }
      ],
      "conflicts": [
        {
          "family": "engagement",
          "suggestedCause": "disengagement",
          "ruleId": "engagement.cross"
        }
      ],
      "evidenceFingerprintStableAfterDatabaseRekey": true
    },
    {
      "ruleId": "engagement.cross",
      "signalId": 2,
      "signalIdentity": "11111111-1111-1111-1111-111111111111:cross-skill:engagement.cross:2026-03-02T06:00:00.000Z:2026-03-09T05:00:00.000Z:99355166dba09562221fbdb1784f70c0787fbf2d6bc0a880932f0441e264825a",
      "behaviorFingerprint": "99355166dba09562221fbdb1784f70c0787fbf2d6bc0a880932f0441e264825a",
      "evidenceFingerprint": "ed507ea9a14213141d11c031f7401be543dcc43e3fdb1c97e5407ce3e75b303a",
      "rawConfidence": 0.8,
      "finalConfidence": 0.5599999999999999,
      "conflictMultiplier": 0.7,
      "exactActivityIds": [
        "101b92ecec6f9a2f72e8e802d7e7ee0492db2f270146c007125c0f369accd801"
      ],
      "additionalEvidenceLinks": [
        {
          "signalId": 1,
          "cause": "guessing",
          "severity": 0.8,
          "finalConfidence": 0.63,
          "ruleId": "guessing.skill",
          "scope": {
            "kind": "skill",
            "skill": {
              "code": "TEKS.4.3E",
              "name": "Add and subtract fractions with equal denominators"
            }
          }
        }
      ],
      "conflicts": [
        {
          "family": "answer-choice",
          "suggestedCause": "guessing",
          "ruleId": "guessing.skill"
        }
      ],
      "evidenceFingerprintStableAfterDatabaseRekey": true
    }
  ],
  "migrationSeams": {
    "orderedReservations": [
      {
        "name": "008_import_receipt_session.sql",
        "offset": 94
      },
      {
        "name": "009_board_run_head.sql",
        "offset": 177
      },
      {
        "name": "010_evidence_acknowledgment.sql",
        "offset": 275
      }
    ],
    "strictlyOrdered": true,
    "legacyRowsDefaultNonDisplayable": true,
    "legacyRowsForcedNonDisplayable": true,
    "masteryExcludesExactAnchor": true
  },
  "portfolioPolicySpecification": {
    "captainApprovedNoAccuracyClaim": true,
    "completeAccuracyBundleRequiredBeforePilot": true,
    "accuracyRegressionApproverNotInvented": true
  }
}

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

🔧 **Review** - 7 issues found → auto-fixed (3) ✅
  • 🚨 db/migrations/007_shared_foundation.sql:17 - The required “truthful synthetic-only legacy/import policies” containment remains bypassable. Existing rows are marked legacy, but the new column defaults to false; the unchanged legacy nightly writer omits this column, so its newly inserted provenance-free entries pass legacy_non_displayable IS FALSE and become visible. Default legacy rows to non-displayable and let only the future run-backed compiler explicitly publish displayable entries.
  • 🚨 packages/signal-engine/src/windows.ts:67 - The required “half-open America/Chicago boundaries” do not hold end-to-end. boardWindows returns Chicago midnight, but scripts/nightly.ts still uses UTC end-of-day as the mastery and engine end anchor; rules also query mastery at ctx.now instead of window.end. For a 2026-03-09 board this can include Monday daytime activity belonging to the next period. Use one canonical windowEnd/asOf for every decision read and reserve now for provenance.
  • 🚨 packages/signal-engine/src/engine.ts:220 - The required “per-fired-rule signal/evidence identity and exact additional evidence” is incomplete because conflicts are calculated inside each skill or cross-skill evaluation before the complete student signal set exists. A skill-scoped cause and an overlapping cross-skill cause therefore retain conflictMultiplier: 1 and omit each other from conflicts; additionally, the locally recomputed additionalCauses.finalConfidence omits the linked signal's conflict multiplier. Reconcile all student signals and final confidences before assembling bundles, fingerprints, and ranking disclosures.
  • 🚨 packages/signal-engine/src/engine.ts:244 - Each signal's behavior fingerprint hashes only the active unit subset passed to evaluateContext. Per-skill and cross-skill signals consequently receive different fingerprints, and changing a cross-skill rule leaves every per-skill fingerprint unchanged even though that rule can change board dominance. Compute one fingerprint from the complete rule set at runEngine and pass it into every evaluation.
  • 🚨 packages/signal-engine/src/engine.ts:315 - evidenceFingerprint is assigned the raw, non-canonical JSON bundle rather than a lowercase SHA-256 fingerprint. This cannot satisfy the planned 64-hex persistence constraint or provide stable finding identity. Hash canonical evidence at the shared identity boundary, excluding persistence-only surrogate IDs as specified by the data model.
  • 🚨 db/migrations/007_shared_foundation.sql:11 - The required separation of response semantics from timing profiles is lost during migration: every existing item receives standard_multiple_choice, including the four seeded wp: items whose canonical timing profile is word_problem. Later ingest will apply the 180-second bound instead of 300 seconds. Backfill timing profiles from the canonical seeded mapping before treating the column as authoritative.
  • 🚨 packages/signal-engine/src/confidence.ts:27 - The new confidence breakdown does not implement the frozen rule-requirement matrix. It derives timing attenuation solely from attempt quality, so timing-free answer-choice or engagement rules are reduced to 0.45 on none data instead of remaining at 1.0; it also compounds a 0.75 conflict penalty by family count although the contract specifies one 0.70 penalty. Because final confidence is now a ranking tiebreaker, this changes board order. Calculate multipliers from the rule's declared timing requirement and the one-time conflict policy.

🔧 Fix: Fix deterministic signal boundaries and legacy containment
5 errors still open:

  • 🚨 packages/signal-engine/src/engine.ts:224 - The required “per-fired-rule signal/evidence identity” still records the config-derived version for every signal instead of the fired contract’s record.rule.version. Changing a custom rule from v1 to v2 changes the behavior fingerprint while its signal and evidence continue reporting the same local rule version. Persist the fired rule’s version and reserve the behavior fingerprint for whole-policy identity.
  • 🚨 packages/signal-engine/src/evidence.ts:38 - The durable identity/exact-evidence fix still uses database surrogate IDs as semantic keys before hashing. Null IDs collapse to 0, which can include unrelated attempts and create false cross-skill conflicts; tied timestamps are ordered by ID before ordinals and derived values are computed, so rekeying can still change the fingerprint after IDs are stripped. Carry a stable source-event identity through rule evidence, overlap checks, and bundle ordering.
  • 🚨 packages/signal-engine/src/evidence.ts:66 - The required “separate response semantics from timing profiles” is not preserved end-to-end. The engine drops itemType and timingProfile, evidence hardcodes every attempt as multiple_choice, and winsorize silently defaults an omitted profile to standard_multiple_choice; numeric/short-text evidence is therefore false and omitted word-problem profiles receive the 180-second bound. Carry resolved item metadata through the shared context and require the timing profile at winsorization.
  • 🚨 packages/signal-engine/src/engine.ts:485 - The required half-open America/Chicago invariant still fails for attendance. Date-only spans are interpreted in the process timezone and extended by a fixed 24 hours; against Chicago windows, a July 21–23 absence parsed at UTC midnight overlaps 67 rather than 72 hours, and DST creates another mismatch. Normalize inclusive attendance dates to Chicago [start, end+1) civil boundaries and prorate using civil days.
  • 🚨 specs/001-huddle-triage-board/spec.md:753 - The changed captain-approved portfolio gate conflicts with contracts/eval-harness.md, which still says external synthetic-demo promotion before the complete eval bundle is unresolved. This leaves later streams with contradictory release criteria. Align that contract with the bounded grounding/fallback-regression and no-accuracy-claim policy while retaining the full eval requirement for pilots or accuracy-backed claims.

🔧 Fix: Align evidence identity, metadata, calendar, and portfolio gates
2 errors still open:

  • 🚨 packages/signal-engine/src/contract.ts:48 - The approved “per-fired-rule signal/evidence identity and exact additional evidence” correction still keys evidence only by sourceEventId, while both the current schema and target ingest contract define event identity using source as well. For one student with source-a/event-1 and source-b/event-1, a rule citing either ID makes bundle selection and cross-skill overlap match both attempts. Create an opaque composite activity identity at the persistence/compiler boundary and use it throughout without exposing the raw source.
  • 🚨 packages/signal-engine/src/windows.ts:54 - The approved “half-open America/Chicago boundaries” fix remains process-timezone dependent. PostgreSQL date values are loaded as process-local Date objects, but chicagoDateOnlyBoundary extracts their UTC date with toISOString(); in an east-of-UTC process, a stored 2026-03-08 becomes 2026-03-07 before the Chicago boundary is calculated. Preserve absence dates as YYYY-MM-DD civil strings from the database/domain boundary and convert those directly.

🔧 Fix: Harden activity identity and civil-date boundaries
✅ Re-checked - no issues remain.

✅ **Test** - passed

✅ No issues found.

  • Inspected git diff --name-status 5d25aba567c4fd66d5aa6dd5856984e9b2946969..61253dbf7f83586ef5a89876423b8eb9d8ca5b89 against the authoritative intent, plan, and constitution.
  • Ran the targeted 17-file Vitest command covering application contracts, GuideAccess, DB seams and isolation, activity identity, ingest timing, seed policy, narration requests, evidence identity, Chicago boundaries, confidence, ranking, and determinism.
  • Ran npm exec tsx -- /var/folders/nt/rdk7cjs538l8zphln24q2k900000gn/T/no-mistakes-evidence/01KYQDPZ9FGPFA49HHJ69KP6EC/shared-foundation-probe.ts.
  • Verified all 21 assertions in shared-foundation-runtime.json, recorded SHA-256 a12dba7e3a9a9045056d307e74ed5f6fb1687ca0bebf4d34c36a13269ba0004b, and confirmed git status --short remained clean.
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

@alexdancer
alexdancer merged commit d192492 into main Jul 29, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant