From 439a76ab9bd1e66cd0ece59ecf8c72ef846fae0b Mon Sep 17 00:00:00 2001 From: Jiayu Liu Date: Wed, 8 Apr 2026 12:20:30 +0800 Subject: [PATCH] docs: fix misleading timeout_ms description in README The README described timeout_ms as "the maximum time to wait for more requests before processing the batch", implying an early wake-up on a full batch. In reality, it is a fixed sleep per loop iteration when the batch is not full. Updated to match the accurate docstrings. Co-Authored-By: Claude Opus 4.6 (1M context) --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 7154973..e586980 100755 --- a/README.md +++ b/README.md @@ -73,7 +73,7 @@ Below is a basic example of how to use the Batched API to process text data in b For more advanced usage, such as customizing batch size and timeout dynamically, the Batched API provides decorators that allow fine-grained control over the batching process. - **Batch Size**: You can specify the max. number of requests to group together in a single batch. -- **Timeout**: The maximum time to wait for more requests before processing the batch. +- **Timeout**: The time in milliseconds to sleep between batch generation attempts when the batch is not full. - **Small Batch Threshold**: The threshold to give more priority to smaller batches. - **Pad Token**: The token to use for padding when batching tensors, only for `@inference.dynamically` and `@aio.inference.dynamically`.