You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Feature Request]: Provide asynchronous file relations and Arrow analytics APIs #30066
Checked relevant Python SDK issues; this report concerns the SDK client path.
Environment
Repository main 3e50e1b32d8da61bbbf70c89989fb08b671809de, Python 3.12, SQLAlchemy 1.4/2.x. Initial findings were reproduced offline with synthetic inputs; real-service validation will be linked from the implementation PR.
Priority: P1. Related: #30049 and draft implementation #30054 (synchronous scope).
Problem and requirements
AsyncClient has no equivalent of the proposed synchronous analytics interface. Applications should not need blocking file/result operations on the event loop.
Provide async context-managed file relations, async Arrow batch iteration, DataFrame/Arrow upload and import with equivalent type/precision/commit-outcome contracts. Honor cancellation/deadlines and pool backpressure. Preserve optional dependencies and explicit shared-path limitations. No Flight or zero-copy promise is implied.
Design considerations
Use SQLAlchemy async streaming/aiomysql unbuffered retrieval; bound Arrow conversion and staging work and move blocking I/O off the event loop where appropriate. Retain transaction settings through COMMIT, distinguish unknown outcomes and define cancellation cleanup. Do not merely wrap complete synchronous result materialization in a thread.
Acceptance tests
Compare complete values and schemas with synchronous APIs; measure event-loop heartbeat during large transfers; cancel during staging/LOAD/read/COMMIT, exhaust a small pool, inject network stalls and verify outcome/resource ownership. Run dependency, same-host dual-CN and permissions matrices with no unexpected skips.
Implementation tracking
Design/implementation follow-up remains open. This requirement is not claimed as implemented by the current bug-fix/performance PRs. Related synchronous analytics draft: #30054.
Async connection pool configuration
Expose pool_size, max_overflow, pool_timeout and pool_recycle rather than retaining fixed testing defaults. Verify bounded backpressure and cancellation with a one-connection pool; preserve documented default behavior.
Activity