Skip to content

Make session.cookies the only cookie store and follow redirects in Python - #160

Merged
FlorianREGAZ merged 4 commits into
mainfrom
fix/cookies-and-redirects
Oct 10, 2026
Merged

FlorianREGAZ merged 4 commits into
mainfrom
fix/cookies-and-redirects

Conversation

@FlorianREGAZ

Copy link
Copy Markdown
Owner

#25: session.cookies.clear() doesn't clear cookies

The Go session kept a cookie jar of its own, keyed by session id, so a cookie removed from session.cookies (by clear() or del) was still sent. Every request now sends withoutCookieJar: true, which makes session.cookies the only place cookies are kept.

Without a jar, upstream adds every requestCookies entry to the request without checking it against the URL (cffi_dist/main.go L235-243: SetCookies only runs when a jar exists, otherwise each cookie goes through req.AddCookie). So cookies are no longer sent as requestCookies. http.cookiejar (add_cookie_header via the existing MockRequest) picks the ones for the request URL by domain, path, secure flag and expiry, and they are sent as a Cookie header.

Separate commit: extract_cookies_to_jar now extracts straight into the session jar, so when a server expires a cookie it is removed. Before, it was dropped from the per-response jar only. That removes the xfail on test_server_can_delete_a_session_cookie.

Fixes #25

#120: cookies set during redirects are lost

followRedirects is now always false. With allow_redirects=True, execute_request follows redirects itself, the way requests does:

  • 301/302/303/307/308 with a Location are followed, at most 30 times, then TooManyRedirects (a TLSClientException) is raised
  • a relative Location is resolved with urljoin, and the fragment carries over unless the Location has its own
  • 303 turns anything but HEAD into GET, and 301/302 turn POST into GET. When the method changes, the body and its Content-Type/Content-Length/Transfer-Encoding headers are dropped. 307/308 keep the method and body
  • Authorization, and a Cookie header passed in, are dropped when the redirect changes host, port or scheme (an http to https upgrade on default ports keeps them, like requests' should_strip_auth)
  • each hop's Set-Cookie goes into the session jar, and the next hop gets the jar's cookies for its URL
  • the final response has response.history (the earlier responses, oldest first) and response.url set to the final URL

AsyncSession merges response.cookies into the shared jar on the event loop thread. So the final response.cookies now holds the cookies of every hop, and AsyncSession keeps them without any change to how it touches shared state.

Fixes #120

Behaviour changes

  • Cookies removed from session.cookies are no longer sent, and a cookie the server expires is removed from the jar.
  • Cookies are sent as a Cookie header built in Python, not as requestCookies. A Cookie header passed with the request is still sent, with the jar's cookies after it, but it is not sent on to another origin during redirects.
  • Cookie values are sent exactly as stored. The native library used to strip " from them.
  • response.cookies on a redirected response holds the cookies of every hop. requests only puts the final response's cookies there; each response in history still has its own.
  • response.history is new, and so is tls_client.exceptions.TooManyRedirects.
  • The redirect rules above (method rewriting, header stripping, the limit of 30) now come from this library instead of Go's net/http.
  • allow_redirects still defaults to False. requests defaults to True for everything except HEAD; it might make sense to switch the default in a future major version, but this PR leaves it alone.

Testing

  • Unit tests with the fake library: a scripted redirect chain with Set-Cookie on each hop. They check the URL, method, body, headers and cookies of every hop, that followRedirects is false and withoutCookieJar true, relative/fragment resolution, the 301/302/303/307/308 method and body rules, credential stripping across origins, the 30-redirect limit, and domain/path/secure/expiry matching of sent cookies. There are also AsyncSession tests: redirect-hop cookies land in the shared jar, and cleared cookies aren't sent.
  • Integration tests against the local httpbin: /cookies/set → /cookies → cookies.clear() → /cookies is empty, del of one cookie, /cookies/delete, /redirect/3, /relative-redirect/3, /absolute-redirect/3, /redirect-to, POST through /redirect-to?status_code=301|302|303|307|308, /redirect/31 raising, Authorization and cookies not following a redirect from 127.0.0.1 to localhost, and /cookies/set with allow_redirects for both Session and AsyncSession.
  • A local http.server endpoint sets a cookie on every hop of a chain (httpbin can't do that), including a hop that expires one, for both Session and AsyncSession.
  • uv run ruff check, uv run ruff format --check, uv run mypy and uv run pytest (422 passed, 6 xfailed) pass.

🤖 Generated with Claude Code

@FlorianREGAZ
FlorianREGAZ force-pushed the fix/cookies-and-redirects branch from abe27e0 to bdf9bb3 Compare October 10, 2026 11:49
FlorianREGAZ and others added 4 commits October 10, 2026 13:52
Set-Cookie headers were parsed into a fresh jar and merged into the session jar, so a cookie the server expired was
dropped from the fresh jar but stayed in the session jar. Extract into the session jar directly instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The library kept a cookie jar per session of its own, so cookies removed from session.cookies, by clear() or del, were
still sent. Requests now run with withoutCookieJar, and the cookies for the request URL are picked from the Python jar
by domain, path, secure flag and expiry and sent as a Cookie header. With withoutCookieJar the library adds every
requestCookies entry to the request without matching it to the URL, which is why they aren't sent that way anymore.

A Cookie header passed with the request is still sent, with the jar's cookies after it. Cookie values are sent as
stored; the library used to strip double quotes from them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With followRedirects the library followed redirects itself, and Python only saw the Set-Cookie headers of the last
response, so cookies set on the way were lost. followRedirects is now always false, and with allow_redirects
execute_request follows 301, 302, 303, 307 and 308 the way requests does:

- every hop's cookies go into the session jar and the next hop is sent the jar's cookies for its URL
- a relative Location is resolved against the current URL and the fragment carries over
- 303 turns anything but HEAD into a GET, 301 and 302 turn a POST into one; the body and its Content-Type,
  Content-Length and Transfer-Encoding headers are dropped when the method changes, 307 and 308 resend them
- Authorization and a Cookie header passed in aren't sent to another host, port or scheme
- after 30 redirects TooManyRedirects, a TLSClientException, is raised

The final response gets response.history with the earlier responses, and response.cookies holds the cookies of every
hop, so AsyncSession, which merges response.cookies into the shared jar, keeps them too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Python 3.10 orders them differently from later versions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@FlorianREGAZ
FlorianREGAZ force-pushed the fix/cookies-and-redirects branch from bdf9bb3 to 406de78 Compare October 10, 2026 11:53
@FlorianREGAZ
FlorianREGAZ merged commit 45199e8 into main Oct 10, 2026
34 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

cookies not extracted in autoRedirect requests !!! "session.cookies.clear()" does not work properly

1 participant