last_modified of HEAD and GET are not always in sync

[tthverst]

In the text below you can see the first entry which shows the headers of the HEAD response where the last_modified header is
Wed, 15 Jul 2020 20:08:32 GMT.
The second entry is the if-modified-since header for the GET request.
The third entry shows the headers of the GET response where the last_modified header is Wed, 15 Jul 2020 20:07:46 GMT
The url is https://a.4cdn.org/tv/catalog.json.

Why are these two last_modified headers different?

---

1: {connection: keep-alive, last-modified: Wed, 15 Jul 2020 20:08:32 GMT, cache-control: public, must-revalidate, proxy-revalidate, cf-request-id: 03f5b0e4fe0000faa0c4955200000001, date: Wed, 15 Jul 2020 20:08:33 GMT, access-control-allow-origin: http://boards.4chan.org, vary: Accept-Encoding,Origin, Accept-Encoding, content-encoding: gzip, cf-cache-status: EXPIRED, expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct", access-control-max-age: 1728000, content-type: application/json, server: cloudflare, access-control-allow-headers: If-Modified-Since, accept-ranges: bytes, access-control-allow-methods: GET, OPTIONS, content-length: 76148, cf-ray: 5b361db4cdd3faa0-AMS, etag: "5f0f6240-12974", expires: -1}
2: {if-modified-since: Wed, 15 Jul 2020 20:07:46 GMT}
3: {connection: keep-alive, cache-control: public, must-revalidate, proxy-revalidate, last-modified: Wed, 15 Jul 2020 20:07:46 GMT, cf-request-id: 03f5b0e8d20000fa38d4806200000001, date: Wed, 15 Jul 2020 20:08:33 GMT, access-control-allow-origin: http://boards.4chan.org, vary: Accept-Encoding,Origin, Accept-Encoding, age: 35, cf-cache-status: UPDATING, expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct", access-control-max-age: 1728000, server: cloudflare, access-control-allow-headers: If-Modified-Since, access-control-allow-methods: GET, OPTIONS, cf-ray: 5b361dbae89bfa38-AMS, etag: "5f0f6212-12885", expires: -1}

[desuwa]

Works fine here.

How exactly are you doing your requests? Make sure you send the same Origin header every time.

[tthverst]

Thank you for your quick response.
The weird thing is, is that it does not happen every request.

I'm making an app with flutter. http is a dart package.
url is https://a.4cdn.org/tv/catalog.json.
previous_last_modified is Wed, 15 Jul 2020 20:07:46 GMT.

final head = await http.head(url);
final last_modified = head.headers[HttpHeaders.lastModifiedHeader]; // Wed, 15 Jul 2020 20:08:32 GMT
final response = await http.get(url, headers: { HttpHeaders.ifModifiedSinceHeader: previous_last_modified });
final response_last_modified = response.headers[HttpHeaders.lastModifiedHeader] // Wed, 15 Jul 2020 20:07:46 GMT

I'll try to explicitly state the origin header. I'll also check if previous_last_modified is always the same as response_last_modified
or if it was just a coincidence this time.

---

EDIT: Explicitly stating the origin header did not change anything. Also previous_last_modified was always the same as response_last_modified.

[tthverst]

I will also make the requests via Postman to make sure it is not a Flutter problem.

---

EDIT: I've found that when the HEAD and GET are received in the same second this problem occurs (See also the date headers in my first comment). I could not reproduce this in Postman because I'm not fast enough.
If I add a delay of 1 second between the requests the problem does not occur. But this is obviously not viable.