Name the day and the reason when an async Binance download fails - #48
Merged
Conversation
The async daily path returned a bare exception and logged `f"Failed to download:
{result}"`. `str(exc)` is empty for an exception raised with no arguments, so the
line rendered with no subject and no reason at all. Run 33988501549 of the book
repository's reader-install job emitted 569 of them, which is what turned a
five-symbol Binance outage into a whole seven-dataset job failing with nothing to
read. The premium-index monthly path had the same shape at debug level.
`fetch_one` now carries the date out with the failure, and both sites log
structured fields through `_failure_reason`, which falls back to the exception's
type when it carries no message:
Failed to download date=2024-01-02 reason=TimeoutError symbol=BTCUSDT
Four tests, including one that drives the async daily path with an argument-less
`TimeoutError` and asserts the old subject-less line is not what comes out.
`uv run pytest tests/ -q`: 3,623 passed, 280 deselected. ruff and ty clean.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
An async Binance download failure said nothing about itself.
The async daily path returned a bare exception from
fetch_oneand loggedf"Failed to download: {result}".str(exc)is empty for an exception raised with no arguments, so the line rendered with no subject and no reason:That is from run 33988501549 of the book repository's
Reader installjob. Five of nineteen Binance perpetual symbols were unavailable upstream; six of the seven free datasets downloaded fine. Reading only the log, it was indistinguishable from every source failing, and diagnosing it meant going to the raw job-log REST endpoint and counting[OK]/[FAIL]lines in the summary block. The premium-index monthly path had the same shape at debug level (Monthly fetch failed for (2021, 1):).fetch_onenow carries the date out alongside the failure, and both sites log structured fields through a shared_failure_reason, which falls back to the exception's type when it carries no message of its own:Four tests in
tests/test_binance_public_edge_cases.py: the three_failure_reasonbranches (argument-less, with a message, whitespace-only), and one that drives_fetch_daily_data_asyncwith an argument-lessTimeoutErrorand asserts both that the date, symbol and reason are present and that the old subject-lessFailed to download:is not.Verification
uv run pytest tests/ -q: 3,623 passed, 280 deselected.ruff check,ruff format --checkandty checkclean.🤖 Generated with Claude Code
https://claude.ai/code/session_01WfX5GbAHc5EVwdUnRSoUPT