[Bug]: bidi reports transfer size as responseBodySize, so request().sizes() over-reports the body by the header size

[ayaangazali]

Version

1.64.0-next, d1ead3ecc. Chrome 141 over --project=bidi-chrome-page.

Steps to reproduce

Serve a body of a known length and read sizes():

server.setRoute('/fixed', (req, res) => {
  const plain = 'x'.repeat(2000);
  res.writeHead(200, { 'content-type': 'text/plain', 'content-length': String(plain.length) });
  res.end(plain);
});
const resp = await page.goto(server.PREFIX + '/fixed');
console.log(await resp.request().sizes());

Expected

responseBodySize of 2000, as chromium over CDP reports:

chromium (CDP)  {"requestBodySize":0,"requestHeadersSize":674,"responseBodySize":2000,"responseHeadersSize":151}

Actual

bidi-chrome     {"requestBodySize":0,"requestHeadersSize":343,"responseBodySize":2151,"responseHeadersSize":132}

2151 for a 2000 byte body. That number is exactly CDP's body plus CDP's header size (2000 + 151), so what is coming back is the whole transfer, headers included, in the body field. Same overshoot on the other shapes I tried: a gzipped body whose encoded length is 35 reports 208, and a chunked response whose wire body is 15 reports 172.

Cause

packages/playwright-core/src/server/bidi/bidiNetworkManager.ts:168:

response.setTransferSize(params.response.bodySize);
response.setEncodedBodySize(params.response.bodySize);

One protocol field feeds two different metrics. The codebase treats them as distinct, and network.ts:719 spells the relationship out in the fallback path:

transferSize = responseHeadersSize + encodedBodySize;

so setting both from one value cannot be right for both. CDP keeps them apart:

response.setTransferSize(event.encodedDataLength);
response.responseHeadersSize().then(size => response.setEncodedBodySize(event.encodedDataLength - size));

WebKit only sets setEncodedBodySize(event.metrics?.responseBodyBytesReceived) and leaves transfer size to the fallback.

If BiDi's network.ResponseData carries headersSize alongside bodySize, subtracting it the way CDP does would line the two up. I did not check what Chrome actually populates there, so that part is a suggestion rather than a diagnosis.

Scope

Only the bidi backend, which I realise is still experimental and tracked under #32577, so this is a low-priority one. Filing it discretely because the value is plainly wrong rather than merely missing, and the cause is two lines.

I am a freshman in college doing my best to contribute usefully, so if bidi gaps are better collected on #32577 than filed one by one, tell me and I will move it there.

[ashrafiucse]

Confirmed on Chrome 152 (stable) with current main, and I instrumented bidiNetworkManager to log the raw network.ResponseCompleted params. The full picture is messier than the two lines suggest — the two engines populate these fields with opposite semantics:

response	engine	bodySize	headersSize
fixed, 2000-byte body	Chrome	2151	132
fixed, 2000-byte body	Firefox	2000	151
gzip, 29 bytes on the wire	Chrome	202	154
gzip, 29 bytes on the wire	Firefox	29	173
chunked, 15-byte body	Chrome	182	138
chunked, 15-byte body	Firefox	15	157

Decoding it:

Chrome treats bodySize as the full transfer size — status line + headers + wire body, identical to CDP's encodedDataLength. Check: 2151 = "HTTP/1.1 200 OK\r\n"(17) + headers(132) + CRLF(2) + 2000; the gzip one is 202 = 173 + 29 and the chunked one is 182 = 157 + 25 (25 = the chunk framing). But its headersSize counts the header lines only — no status line, no final CRLF (132, not 151).

Firefox is crossed: bodySize is the actual body (2000 / the 29 gzip bytes / 15 chunk-data bytes), and headersSize is the full raw header block including status line and terminator (151 / 173 / 157 — these match CDP's responseHeadersSize exactly).

So each of the two lines is correct on exactly one engine:

• setTransferSize(bodySize) — right on Chrome, wrong on Firefox (should be headersSize + bodySize)
• setEncodedBodySize(bodySize) — right on Firefox, wrong on Chrome

And the suggested subtraction bodySize - headersSize doesn't quite land either: on Chrome it gives 2151 − 132 = 2019 (still 19 over — status line + CRLF), and on Firefox it would regress the currently-correct 2000 down to 1849. Any fix has to branch on the engine. For Chrome the remaining 19 bytes are reconstructable from ${protocol} ${status} ${statusText}\r\n, though the empty-reason-phrase edge case is worth a look. Worth noting the reported responseHeadersSize has the same cross-engine skew (132 vs 151 vs CDP's 151), so aligning that at the same time would make the three backends agree.

(Meta-question: this is fine as its own issue IMO — the values are wrong, not merely missing. The gaps list on #32577 reads more like missing-API tracking.)

Nice diagnosis by the way — "body plus CDP's header size" is exactly what was happening, and pointing at the two lines was spot on.

[ayaangazali]

This is much better than what I had, thanks for actually instrumenting it. My suggested subtraction was wrong and I should not have offered it as even a maybe.

I ran the other half independently and your two columns hold up. Same probe, same build d1ead3ecc, 2000 byte body with content-length set:

firefox over bidi   responseBodySize=2000  responseHeadersSize=151
chrome  over bidi   responseBodySize=2151  responseHeadersSize=132
chromium over CDP   responseBodySize=2000  responseHeadersSize=151

So Firefox over BiDi is already correct on both fields and agrees with CDP exactly, while Chrome over BiDi is wrong on both. That matches your reading that each of the two lines is right on exactly one engine, and it means the only user-visible breakage through sizes() today is Chrome over BiDi. Good catch on responseHeadersSize too, I had recorded the 132 but not registered that it was the same skew rather than a separate quirk.

Your point that bodySize - headersSize regresses Firefox from a correct 2000 down to 1849 is the part that kills my suggestion outright, since it would break the engine that currently works. Branching on the engine does look unavoidable given the field semantics are inverted.

Please go ahead with it, you are well ahead of me here and I have nothing to add beyond the confirmation above. Happy to re-run anything on either bidi transport if a second set of numbers is useful at any point.