Skip to content

gh-153364: Limit frame, coroutine, and task-waiter chain walks#153365

Open
maurycy wants to merge 14 commits into
python:mainfrom
maurycy:tachyon-parse_async_frame_chain-limit
Open

gh-153364: Limit frame, coroutine, and task-waiter chain walks#153365
maurycy wants to merge 14 commits into
python:mainfrom
maurycy:tachyon-parse_async_frame_chain-limit

Conversation

@maurycy

@maurycy maurycy commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

The PR hardens and cleans up chain walks a bit.

It introduces a const (MAX_FRAME_CHAIN_DEPTH), uses it in the existing process_frame_chain(), which is called by get_stack_trace(), as well as adds it to parse_coro_chain(), used in both sync and async pathways, and to parse_async_frame_chain().

Also, it adds MAX_TASK_WAITER_CHAIN_DEPTH for the purpose of task-waiter limits in get_async_stack_trace().

The obvious context here is avoiding infinite loops. Truth be told, I think that in some places it unexpectedly works (ie: doesn't hang) because incomplete / torn-reads result in an exception. :-)

@@ -325,10 +327,19 @@ int
parse_coro_chain(

@maurycy maurycy Jul 8, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it actually need to be recursive?

To put aside recursive v. iterative; CS101 strikes: if you ask me, I was thinking about a visited set here... not applicable to this fix but cycle detected would be such a nice touch. :-)

@maurycy maurycy Jul 9, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realized we used Floyd's algo at least once:

PyObject *slow_o = o; /* Floyd's cycle detection algo */

I, for one, like Brent's :-)

@maurycy maurycy Jul 9, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar recursive problem here:

process_task_and_waiters(

@maurycy maurycy Jul 10, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wondering:

0:04:10 load avg: 5.21 mem: 175.8 MiB [282/505/1] test_external_inspection worker non-zero exit code (Exit code 3221225725 (STATUS_STACK_OVERFLOW))

0:04:10 load avg: 5.21 mem: 175.8 MiB [282/505/1] test_external_inspection worker non-zero exit code (Exit code 3221225725 (STATUS_STACK_OVERFLOW))

On one hand, it hints at iterative approach being the right now. On the other hand, I've just lowered the limit: 353f815 and, eventually, c9f1ebd

#define MAX_STACK_CHUNK_SIZE (16 * 1024 * 1024) /* 16 MB max for stack chunks */
#define MAX_LONG_DIGITS 64 /* Allows values up to ~2^1920 */
#define MAX_SET_TABLE_SIZE (1 << 20) /* 1 million entries max for set iteration */
#define MAX_FRAME_CHAIN_DEPTH (1024 + 512) /* Iteration bound for frame chain walks */

@maurycy maurycy Jul 8, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not the scope but maybe MAX_THREADS, MAX_STACK_CHUNKS, MAX_LINETABLE_ENTRIES or MAX_ITERATIONS are worth keeping here too

@maurycy maurycy changed the title gh-153364: Add frame limit in get_async_stack_trace() and parse_coro_chain() gh-153364: Limit frame, coroutine, and task-waiter chain walks Jul 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant