Skip to content

fix(government_us): send the FedInvest session cookie and CSRF token with the price download - #7858

Open
artcashin wants to merge 1 commit into
openbq-org:developfrom
artcashin:fix/treasury-prices-session
Open

artcashin wants to merge 1 commit into
openbq-org:developfrom
artcashin:fix/treasury-prices-session

Conversation

@artcashin

Copy link
Copy Markdown
Contributor

What

treasury_prices fails for every request: TreasuryDirect answers the FedInvest CSV download with 403.

TreasuryDirect rebuilt FedInvest. The download (securityPriceDetail) is now refused unless the request carries the session cookie and the _csrf token of the price form (selectSecurityPriceDate). The cookies are set for the www host. The page is still public; no account is involved.

Change

UsTreasuryPricesFetcher.extract_data reads the form, takes the token, and posts the download on the same session. It uses the existing get_requests_session and make_request helpers; nothing new is added to the utilities.

Tests

  • test_treasury_models.py: the payload test now checks the form request, the token in the payload, and that both requests share one session; a new test covers a form with no token. The HTTP-error and encoding tests go through the same two-step fake.
  • test_us_treasury_fetchers.py::test_treasury_prices_fetcher: cassette re-recorded for the two requests, with the cookie and token values replaced by placeholders.
  • government_us suite: 3,261 pass. ruff check and ruff format --check clean on the touched files.
  • Live on 2026-11-18: 462 rows for 2026-11-17.

Not changed

transform_data reads the CSV with header=0, and the file has no header row, so the first security of the day is dropped. That is the existing behaviour and is left for a separate change.

🤖 Generated with Claude Code

…th the price download

TreasuryDirect rebuilt FedInvest. The CSV download now answers 403 unless
the request carries the session cookie and the _csrf token of the price
form, so every treasury_prices request failed. Read the form first and
post the download on the same session.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant