Expose scoped source scan discovery limits

docs/DOGFOOD.md

@@ -16,7 +16,7 @@ The harness creates a fresh `mkdtemp` directory under `os.tmpdir()` with mode `0
 ## What It Does
 
 1. Checks `git rev-parse HEAD` and `git status --porcelain=v1` before and after scanning. If either changes mid-scan, it fails.
-2. Scans the repo with `scanSourceGraph` using bounded limits: 64 files, depth 8, 1 MiB total, 256 KiB per file, 128 records max.
+2. Scans the repo with `scanSourceGraph` using bounded limits: 64 files, depth 8, 1 MiB total, 256 KiB per file, 128 records max. The receipt and stdout summary report `scanBounds.maxFilesHit` and `scanBounds.maxDepthHit` as scoped discovery-uncertainty flags. Both false does not prove complete coverage: ignored directories, non-source extensions, and read/size exclusions remain outside these flags.
 3. Spawns the CLI via `StdioClientTransport` with the absolute path `packages/context-tools/bin/encrypted-context.mjs`.
 4. Connects an MCP `Client` named `z1p-dogfood` version `0.1.0`.
 5. Verifies required tools exist, then creates a context, appends scan records, lists, retrieves with `maxBytes: 8192` / `maxRecords: 8`, checks `bytesUsed` exactly, tests invalid `maxBytes: 1` rejection, rechecks retrieval, closes, reconnects, and verifies `context_read` head and record count match.
@@ -26,7 +26,8 @@ The harness creates a fresh `mkdtemp` directory under `os.tmpdir()` with mode `0
 ## Caveats
 
 - **Timeout scope.** The MCP phase has a 120-second watchdog with a further two seconds allowed for transport cleanup. The initial scan and Git checks are outside that watchdog. Cancellation semantics are not qualified by this check.
-- **Bounded navigation, not whole-repo coverage.** The scanner reads at most 64 files. Results may omit relevant code. Always read full source before editing.
+- **Bounded navigation, not whole-repo coverage.** The scanner reads at most 64 files. Results may omit relevant code. `scanBounds` reports only whether the file cap or depth cap was reached during discovery; it is not a coverage proof. Always read full source before editing.
+- **`recordsOmitted` is candidate omissions.** `recordsOmitted` keeps its legacy numeric formula (`filesScanned + symbolsFound - records.length`) and counts constructed candidate records not retained under `maxRecords`. It is not an unknown unvisited-file count.
 - **Freshness.** Every run creates a new collection. Re-run after any source edit or before any new task. The one-shot command always makes a fresh collection.
 - **Dirty status equality does not prove unchanged file contents.** If the repo is dirty, the harness cannot verify content stability.
 - **MCP client is not desktop acceptance.** This harness exercises the MCP protocol and persistence. It does not measure real user acceptance or savings.
@@ -111,3 +112,28 @@ enlarge the signed v1 format or automatically sign repository source.
 The [first paired diagnostic trial](PAIRED-TRIAL.md) now records the scanner
 truncation diagnosis, actual fixture checks and worker usage including repairs.
 It is a host-assisted context-selection experiment, not end-to-end billing proof.
+
+## Scan-bound reporting acceptance — 21 September 2026
+
+The follow-up implementation adds `scanBounds` without changing signed records,
+discovery quotas or `filesSkipped` semantics. Six regression tests cover capped
+and below-cap scans, exact-cap uncertainty, depth pruning, ignored directories
+and repeatability. All 106 tests, independent package checks and both benchmark
+gates passed. A real harness run confirmed identical flags in stdout and the
+saved receipt, plus invalid-request rejection and restart persistence.
+
+Ollama Flash (`deepseek-v4.1-flash:cloud`, thinking off) produced the accepted
+patch in one dispatched request: 13,528 input and 2,619 output tokens, 16,147
+total, with no repair. A preceding busy receipt was rejected locally before
+dispatch while another job owned the endpoint; its usage fields are unknown.
+This was a real coding task using Z1P navigation, not a paired savings test.
+Host selection/review usage and billing cost remain unknown. Local worker
+receipts: `/private/tmp/z1p-scan-flags.mMGZHK/`.
+
+A bounded Claude CLI acceptance attempt requested Haiku, disabled built-in
+tools and approved only repository refresh/search for that invocation. Claude
+returned HTTP 429 before any tool call: weekly limit reached, reset reported
+as 22 September at noon Europe/London. The CLI reported zero tokens and cost.
+No retry or fallback was attempted, and saved permissions were not changed.
+Actual Claude tool-use acceptance remains blocked; connection health alone
+does not establish it.

packages/context-tools/src/source-scan.test.ts

@@ -227,4 +227,84 @@ describe('bounded TypeScript and JavaScript source graph scan', () => {
     expect(fileRecord.text).not.toContain('localOne')
     expect(fileRecord.text).not.toContain('localTwo')
   })
+
+  it('reports maxFilesHit at cap and preserves bounded discovery semantics', async () => {
+    const root = await fixture()
+    await source(root, 'a.ts', 'export function a() {}')
+    await source(root, 'b.ts', 'export function b() {}')
+    await source(root, 'c.ts', 'export function c() {}')
+    const first = await scanSourceGraph(root, { observedAt: 123, maxFiles: 1, maxRecords: 128 })
+    expect(first).toMatchObject({
+      filesScanned: 1,
+      filesSkipped: 0,
+      symbolsFound: 1,
+      scanBounds: { maxFilesHit: true, maxDepthHit: false },
+    })
+    expect(first.records).toHaveLength(2)
+    expect(Math.max(0, first.filesScanned + first.symbolsFound - first.records.length)).toBe(0)
+
+    const second = await scanSourceGraph(root, { observedAt: 123, maxFiles: 64, maxRecords: 1 })
+    expect(second).toMatchObject({
+      filesScanned: 3,
+      filesSkipped: 0,
+      symbolsFound: 3,
+      scanBounds: { maxFilesHit: false, maxDepthHit: false },
+    })
+    expect(second.records).toHaveLength(1)
+    expect(second.filesScanned + second.symbolsFound - second.records.length).toBe(5)
+  })
+
+  it('reports maxFilesHit at an exact cap with no additional eligible files', async () => {
+    const root = await fixture()
+    await source(root, 'only.ts', 'export function only() {}')
+    const result = await scanSourceGraph(root, { observedAt: 123, maxFiles: 1 })
+    expect(result.scanBounds.maxFilesHit).toBe(true)
+    expect(result.scanBounds.maxDepthHit).toBe(false)
+    expect(result.filesScanned).toBe(1)
+  })
+
+  it('leaves both scan bounds false for a below-cap flat scan', async () => {
+    const root = await fixture()
+    await source(root, 'one.ts', 'export function one() {}')
+    await source(root, 'two.ts', 'export function two() {}')
+    const result = await scanSourceGraph(root, { observedAt: 123, maxFiles: 64, maxDepth: 8 })
+    expect(result.scanBounds).toEqual({ maxFilesHit: false, maxDepthHit: false })
+  })
+
+  it('reports maxDepthHit only when an otherwise traversable directory is not descended', async () => {
+    const root = await fixture()
+    await source(root, 'root.ts', 'export function rootFn() {}')
+    await source(root, 'nested/deep.ts', 'export function deep() {}')
+    const shallow = await scanSourceGraph(root, { observedAt: 123, maxDepth: 0 })
+    expect(shallow.filesScanned).toBe(1)
+    expect(shallow.scanBounds).toEqual({ maxFilesHit: false, maxDepthHit: true })
+    expect(shallow.records.map(record => record.source)).toContain('repo://root.ts')
+    expect(shallow.records.every(record => !record.source.includes('nested'))).toBe(true)
+
+    const deep = await scanSourceGraph(root, { observedAt: 123, maxDepth: 4 })
+    expect(deep.filesScanned).toBe(2)
+    expect(deep.scanBounds).toEqual({ maxFilesHit: false, maxDepthHit: false })
+    expect(deep.records.map(record => record.source)).toContain('repo://nested/deep.ts')
+  })
+
+  it('does not set maxDepthHit for ignored directories that are never inspected', async () => {
+    const root = await fixture()
+    await source(root, 'root.ts', 'export function rootFn() {}')
+    await source(root, '.hidden/no.ts', 'export function hidden() {}')
+    await source(root, 'node_modules/no.ts', 'export function dependency() {}')
+    const result = await scanSourceGraph(root, { observedAt: 123, maxDepth: 0 })
+    expect(result.filesScanned).toBe(1)
+    expect(result.scanBounds).toEqual({ maxFilesHit: false, maxDepthHit: false })
+    expect(result.records.every(record => !record.source.includes('hidden') && !record.source.includes('node_modules'))).toBe(true)
+  })
+
+  it('keeps records unchanged across repeated scans with the same observedAt and flags', async () => {
+    const root = await fixture()
+    await source(root, 'a.ts', 'export function a() {}')
+    await source(root, 'nested/b.ts', 'export function b() {}')
+    const first = await scanSourceGraph(root, { observedAt: 123, maxFiles: 1 })
+    const second = await scanSourceGraph(root, { observedAt: 123, maxFiles: 1 })
+    expect(first.records).toEqual(second.records)
+    expect(first.scanBounds).toEqual(second.scanBounds)
+  })
 })

packages/context-tools/src/source-scan.ts

@@ -23,6 +23,19 @@ export interface SourceGraphScan {
   symbolsFound: number
   importsFound: number
   callsFound: number
+  /** Scoped uncertainty about discovery. Both flags false does not prove
+   * complete coverage: ignored directories, non-source extensions, and
+   * read/byte/size exclusions remain outside this signal. */
+  scanBounds: {
+    /** True when the eligible file discovery count reached the configured
+     * maxFiles cap. This is not proof that additional eligible files exist;
+     * the cap may have been reached exactly at the last eligible file. */
+    maxFilesHit: boolean
+    /** True when an otherwise traversable directory was not descended
+     * because the configured maxDepth was reached. Ignored/hidden
+     * directories that are never inspected do not set this flag. */
+    maxDepthHit: boolean
+  }
 }
 
 type SymbolKind = 'function' | 'class' | 'method' | 'interface' | 'type' | 'enum' | 'variable'
@@ -98,6 +111,8 @@ export async function scanSourceGraph(root: string, options: SourceGraphScanOpti
 
   const discovered: string[] = []
   let filesSkipped = 0
+  let maxFilesHit = false
+  let maxDepthHit = false
   async function visit(directory: string, depth: number): Promise<void> {
     if (discovered.length >= maxFiles) return
     const entries = []
@@ -108,15 +123,21 @@ export async function scanSourceGraph(root: string, options: SourceGraphScanOpti
       const path = join(directory, entry.name)
       const info = await lstat(path).catch(() => undefined)
       if (!info || info.isSymbolicLink()) { if (info?.isSymbolicLink()) filesSkipped++; continue }
-      if (info.isDirectory()) { if (depth < maxDepth) await visit(path, depth + 1); if (discovered.length >= maxFiles) return; continue }
+      if (info.isDirectory()) {
+        if (depth < maxDepth) await visit(path, depth + 1)
+        else maxDepthHit = true
+        if (discovered.length >= maxFiles) return
+        continue
+      }
       if (!info.isFile() || !extensions.has(extname(entry.name))) continue
       if (!validRecordSource(`repo://${normal(canonicalRoot, path)}`)) { filesSkipped++; continue }
       if (info.size > maxFileBytes) { filesSkipped++; continue }
       discovered.push(path)
-      if (discovered.length >= maxFiles) return
+      if (discovered.length >= maxFiles) { maxFilesHit = true; return }
     }
   }
   await visit(canonicalRoot, 0)
+  if (discovered.length >= maxFiles) maxFilesHit = true
   discovered.sort((a, b) => normal(canonicalRoot, a).localeCompare(normal(canonicalRoot, b)))
   const selected: { path: string; text: string; bytes: number }[] = []
   let bytesRead = 0
@@ -263,5 +284,6 @@ export async function scanSourceGraph(root: string, options: SourceGraphScanOpti
   })
   return { root: canonicalRoot, records, filesScanned: parsed.length, filesSkipped, bytesRead,
     symbolsFound: parsed.reduce((sum, file) => sum + file.symbols.length, 0),
-    importsFound: parsed.reduce((sum, file) => sum + file.imports.length, 0), callsFound }
+    importsFound: parsed.reduce((sum, file) => sum + file.imports.length, 0), callsFound,
+    scanBounds: { maxFilesHit, maxDepthHit } }
 }

scripts/dogfood.mjs

@@ -194,6 +194,10 @@ async function main() {
         symbolsFound: scan.symbolsFound,
         importsFound: scan.importsFound,
         callsFound: scan.callsFound,
+        scanBounds: {
+          maxFilesHit: scan.scanBounds.maxFilesHit,
+          maxDepthHit: scan.scanBounds.maxDepthHit,
+        },
         recordsRetained: scan.records.length,
         recordsOmitted: omitted,
       },
@@ -204,6 +208,8 @@ async function main() {
       },
       warnings: [
         'Bounded navigation, not whole-repository coverage; excluded files are not counted as omissions.',
+        'recordsOmitted counts constructed candidate records not retained under maxRecords; it is not an unknown unvisited-file count.',
+        'scanBounds are scoped discovery-uncertainty flags; both false does not prove complete repository coverage.',
         'Git status equality does not prove unchanged dirty contents.',
         'Retained local keys/cache; retrieval receipt is plaintext. Read actual source before edits.',
       ],
@@ -222,6 +228,10 @@ async function main() {
       dirty: before.dirty ? 'yes' : 'no',
       filesScanned: scan.filesScanned,
       filesSkipped: scan.filesSkipped,
+      scanBounds: {
+        maxFilesHit: scan.scanBounds.maxFilesHit,
+        maxDepthHit: scan.scanBounds.maxDepthHit,
+      },
       recordsRetained: scan.records.length,
       recordsOmitted: omitted,
       collection,