External ingest: harden finalize against 412 phase mismatch

[bhavyaus]

Fixes the "GitHub Copilot: Build Codebase Semantic Index" failure reported in #320915, where, for non-GitHub repositories, document upload succeeds but POST /external/code/ingest/finalize returns HTTP 412 with request sent for phase which was not the current phase.

Root cause

The external-ingest protocol is phase-gated server-side: create checkpoint → reconcile coded symbols → upload documents → finalize. The server only advances to the finalize phase once it has received every expected document. The client previously:

• called finalize exactly once with no reconciliation, and
• swallowed per-document upload failures (logging them but counting them as uploaded), so a stranded server surfaced later as a confusing finalize 412 rather than the real cause.

Fixes #320915

[Copilot]

Pull request overview

This PR hardens the Copilot external-ingest indexing flow against server-side phase gating, specifically addressing cases where POST /external/code/ingest/finalize returns HTTP 412 (“phase mismatch”) after document upload appears to have succeeded. It improves correctness and diagnosability by reconciling missing documents and surfacing real upload failures earlier, and it adds protocol-level tests using a mock GitHub fetcher.

Changes:

• Add a bounded finalize retry loop that re-polls /external/code/ingest/batch, re-uploads still-missing documents, backs off briefly, and retries finalize.
• Stop counting failed document uploads as successful; instead fail the pass with a descriptive error (including status and requestId when available), and abort deterministically on 404 (“ingest gone”).
• Add an ExternalIngestClient test suite that exercises the real client against a mock IGithubApiFetcherService for retries, re-upload behavior, retry exhaustion, and error surfacing.

Show a summary per file

File	Description
extensions/copilot/src/platform/workspaceChunkSearch/node/codeSearch/externalIngestClient.ts	Implements finalize 412 reconciliation/retry and makes document-upload failures fail the ingestion pass deterministically.
extensions/copilot/src/platform/workspaceChunkSearch/test/node/externalIngest.spec.ts	Adds end-to-end-ish protocol tests for finalize retries and upload failure behavior using a recording mock fetcher.

Copilot's findings

Comments suppressed due to low confidence (1)

extensions/copilot/src/platform/workspaceChunkSearch/node/codeSearch/externalIngestClient.ts:473

• The upload task throws when the server requests a docSha that isn't present in mappings, but that rejection is later swallowed by the outer Promise.all(uploading) error handler, letting the pass continue and potentially turning a client-side mapping mismatch into a confusing finalize 412/retry flow. Treat this as a recorded failure so the pass fails deterministically with a descriptive error.

							const fileEntry = mappings.get(requestedDocSha);
							if (!fileEntry) {
								throw new Error(`No mapping for docSha: ${requestedDocSha}`);
							}

• Files reviewed: 2/2 changed files
• Comments generated: 1

[Copilot]

Copilot's findings

• Files reviewed: 2/2 changed files
• Comments generated: 1