gh-153364: Limit frame, coroutine, and task-waiter chain walks#153365
gh-153364: Limit frame, coroutine, and task-waiter chain walks#153365maurycy wants to merge 14 commits into
Conversation
| @@ -325,10 +327,19 @@ int | |||
| parse_coro_chain( | |||
There was a problem hiding this comment.
Does it actually need to be recursive?
To put aside recursive v. iterative; CS101 strikes: if you ask me, I was thinking about a visited set here... not applicable to this fix but cycle detected would be such a nice touch. :-)
There was a problem hiding this comment.
Similar recursive problem here:
cpython/Modules/_remote_debugging/asyncio.c
Line 662 in a159c68
There was a problem hiding this comment.
Wondering:
0:04:10 load avg: 5.21 mem: 175.8 MiB [282/505/1] test_external_inspection worker non-zero exit code (Exit code 3221225725 (STATUS_STACK_OVERFLOW))
0:04:10 load avg: 5.21 mem: 175.8 MiB [282/505/1] test_external_inspection worker non-zero exit code (Exit code 3221225725 (STATUS_STACK_OVERFLOW))
On one hand, it hints at iterative approach being the right now. On the other hand, I've just lowered the limit: 353f815 and, eventually, c9f1ebd
| #define MAX_STACK_CHUNK_SIZE (16 * 1024 * 1024) /* 16 MB max for stack chunks */ | ||
| #define MAX_LONG_DIGITS 64 /* Allows values up to ~2^1920 */ | ||
| #define MAX_SET_TABLE_SIZE (1 << 20) /* 1 million entries max for set iteration */ | ||
| #define MAX_FRAME_CHAIN_DEPTH (1024 + 512) /* Iteration bound for frame chain walks */ |
There was a problem hiding this comment.
Not the scope but maybe MAX_THREADS, MAX_STACK_CHUNKS, MAX_LINETABLE_ENTRIES or MAX_ITERATIONS are worth keeping here too
get_async_stack_trace() and parse_coro_chain()
The PR hardens and cleans up chain walks a bit.
It introduces a const (
MAX_FRAME_CHAIN_DEPTH), uses it in the existingprocess_frame_chain(), which is called byget_stack_trace(), as well as adds it toparse_coro_chain(), used in both sync and async pathways, and toparse_async_frame_chain().Also, it adds
MAX_TASK_WAITER_CHAIN_DEPTHfor the purpose of task-waiter limits inget_async_stack_trace().The obvious context here is avoiding infinite loops. Truth be told, I think that in some places it unexpectedly works (ie: doesn't hang) because incomplete / torn-reads result in an exception. :-)
_remote_debugging: No frame limit inget_async_stack_trace()#153364