NavigateEntry.mainRequestId is never populated, so the snapshot origin guard rejects legitimate redirects

[platonai]

Summary

NavigateEntry.mainRequestId is never populated for real navigations, so InteractiveBrowserEmulator.captureNavigationSnapshot rejects perfectly good captures whose committed URL differs from the fetch URL by a redirect.

The guard's own contract says a redirect committed by this navigation is fine, because it checks the network record:

// browser4-core/browser4-protocol/.../InteractiveBrowserEmulator.kt
val entry = driver.navigateEntry
if (entry.userTypedUrl.isBlank() || !urlsReferToSameDocument(entry.userTypedUrl, taskUrl)) { ...throw... }
if (entry.mainRequestId.isNotBlank()) { return snapshot.html }   // <-- never taken
...throw WebDriverException("... no main-document request was issued for this navigation")

In practice the second if is never true, so every redirect lands in the failure branch.

Reproduction

Fetch a product URL that the site answers with a 302 that only changes variant/tracking parameters:

• fetch https://www.amazon.com/TOZO-Cancelling-Bluetooth-Waterproof-Headphones/dp/B0DGKWQMSM/ref=zg_bs_g_172541_d_sccl_8/145-...?psc=1
• committed https://www.amazon.com/TOZO-Cancelling-Bluetooth-Waterproof-Headphones/dp/B0DGKWQMSM/ref=zg_bs_g_172541_d_sccl_8/145-...?th=1

Log (crawl/swarm fetch, hundreds of occurrences):

WARN a.p.p.p.b.e.i.InteractiveBrowserEmulator - Tab origin mismatch: refusing to capture
'...?th=1' for fetch '...?psc=1' — the snapshot still shows an earlier fetch's document and
no main-document request was issued for this navigation
INFO a.p.p.p.b.d.LoadingWebDriverPool - Closing driver that doesn't work unexpectedly #37: RETIRED

Effect: the task fails (expectation failed), a working driver is retired, and the page is retried until it gives up — for a document that was in fact the requested page.

The same happens with the site's own entry-page hrefs, so it is not a hand-built-URL problem.

Questions for the driver/network layer

1. Who is supposed to set NavigateEntry.mainRequestId — the Network/Fetch domain tracker on the main-frame request of the navigation this fetch created?
2. Why is it blank here: is the request tracker not enabled/attached for these drivers, is the redirect committed from the disk cache without a Network.requestWillBeSent for the navigation, or is the entry replaced when the redirect commits?
3. mainFrameReceived / networkRequestCount on the same entry did not provide the evidence either (they are exposed but the guard does not use them) — are they reliable enough to be the "this navigation really happened" signal?

Suggested resolution

Either populate mainRequestId for redirect-committed navigations, or expose a reliable "this navigation issued a main-document request" signal that the guard can trust. When that exists, the guard can accept a redirect whose committed URL is a different document (not only a query-parameter variation), which is what the original design intended.

Workaround applied in Browser4

UrlDocumentMatcher.referToSamePageIgnoringQuery (new, with unit tests) accepts a committed URL that differs from the fetch URL only in the query string (same scheme/host/port/path). That unblocks the Amazon case above and keeps the guard's protection for genuinely different documents, but it does not explain or fix the missing request id.

Related Browser4 commits: d924559079 (guard relaxation + UrlDocumentMatcher).