[griessel-jc]

Summary

Large query results can be processed or exported without collecting every row in memory.

• Add lazy row streams and bounded row batches to both clients, preserving parameter binding, value conversion, polling, retries, and query instrumentation.
• Add CSV and JSONL exports to caller-owned text streams, including metadata-derived CSV headers for empty results and exact Decimal text.
• Reject malformed streaming data and duplicate export column names, stop fetching when consumption stops, and document partial-output behavior.

Docs: updated

Supported behavior

Snowflake returns completed query results in individually retrievable partitions.
The client consumes those partitions on demand; memory scales with the largest
decoded server partition plus the application batch or serialized row.
This does not provide rows before statement execution completes or impose a byte limit.

Sources: Snowflake response handling
and SQL API reference.

Backwards compatibility

Additive public APIs: query_stream, query_batches, export_csv, and export_jsonl,
with synchronous and asynchronous equivalents. Existing query, execute, and
QueryHandle.result behavior and return types remain unchanged. No new runtime dependencies.
Streaming data validation applies only to the new APIs.

Review

The full implementation at 8f36a2d has been checked. The subsequent test-only
commit d6fdc63 strengthens immediate inner-iterator cleanup assertions for the
public stream wrappers; production code is unchanged. Negative controls confirm
that those assertions fail when cleanup is removed, on both clients.

Exempt from re-review: test-only at d6fdc63, tests/test_streaming.py.

Changes

 AGENTS.md                       |   9 +-
 README.md                       |  25 +++-
 docs/getting-started.md         |  12 ++
 docs/streaming.md               | 149 +++++++++++++++++++++
 snowflake_sql_api/aclient.py    | 114 +++++++++++++++-
 snowflake_sql_api/client.py     | 102 ++++++++++++++-
 snowflake_sql_api/exports.py    |  71 ++++++++++
 snowflake_sql_api/pagination.py |  82 +++++++++++-
 snowflake_sql_api/types.py      |  19 +--
 tests/conftest.py               |   6 +
 tests/smoke/test_smoke.py       |  37 ++++++
 tests/streaming_support.py      |  73 +++++++++++
 tests/test_exports.py           | 221 +++++++++++++++++++++++++++++++
 tests/test_regression.py        |   4 +-
 tests/test_streaming.py         | 279 ++++++++++++++++++++++++++++++++++++++++
 15 files changed, 1170 insertions(+), 33 deletions(-)

Test plan

• Full suite: 278 passed, 9 live tests skipped; 96% coverage.

• Repository hooks: lint, formatting, types, and secret detection pass.

• Wheel and source distribution build successfully.

• Negative controls for eager fetching, query instrumentation, and immediate stream cleanup.

• Sync/async coverage for lazy fetching, cross-partition batches, gzip and bare-array responses, retries, empty results, value conversion, and failure cleanup.

• Live Snowflake smoke tests: credentials are not configured in this environment.

• CI: awaiting maintainer approval for the fork workflow run.

• No visual effect.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

• Configuration used: defaults
• Review profile: CHILL
• Plan: Advanced
• Run ID: 28d65381-66f1-44a7-b360-5c5079946e88

📥 Commits

Reviewing files that changed from the base of the PR and between af00371 and d6fdc63.

📒 Files selected for processing: 15

• AGENTS.md
• README.md
• docs/getting-started.md
• docs/streaming.md
• snowflake_sql_api/aclient.py
• snowflake_sql_api/client.py
• snowflake_sql_api/exports.py
• snowflake_sql_api/pagination.py
• snowflake_sql_api/types.py
• tests/conftest.py
• tests/smoke/test_smoke.py
• tests/streaming_support.py
• tests/test_exports.py
• tests/test_regression.py
• tests/test_streaming.py

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

📝 Walkthrough

The clients now support lazy row streaming, bounded batches, and streamed CSV and JSONL exports. The change adds row decoding and partition iterators, serialization utilities, synchronous and asynchronous client APIs, documentation, and tests.

Changes: Streaming and Exports

Layer / File(s)	Summary
Row decoding and partition iteration snowflake_sql_api/types.py, snowflake_sql_api/pagination.py, tests/test_streaming.py	Adds single-row conversion and synchronous and asynchronous iterators that validate and decode rows as partitions are consumed. Tests cover conversion, partition handling, malformed data, and cleanup.
Client streaming and batching snowflake_sql_api/client.py, snowflake_sql_api/aclient.py, tests/streaming_support.py, tests/test_streaming.py, tests/smoke/test_smoke.py, tests/conftest.py, README.md, docs/getting-started.md, docs/streaming.md, AGENTS.md, tests/test_regression.py	Both clients now prepare results for eager collection or lazy iteration, and expose row-stream and batch APIs. Tests and documentation cover consumption, parameter binding, hooks, polling, and resource cleanup.
CSV and JSONL exports snowflake_sql_api/exports.py, snowflake_sql_api/client.py, snowflake_sql_api/aclient.py, tests/test_exports.py, docs/streaming.md, AGENTS.md	Adds CSV and JSONL serialization and client export methods that write to caller-owned sinks and return row counts. Tests and documentation cover value encoding, validation, failures, and partial output.

Priority: ➖ Normal

Estimated code review effort: 4 (Complex) | ~45 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant Caller
  participant SnowflakeClient
  participant iter_result_rows
  participant Transport
  participant CSVWriter
  participant TextSink
  Caller->>SnowflakeClient: Call export_csv
  SnowflakeClient->>iter_result_rows: Start streamed result
  iter_result_rows->>Transport: Fetch partition as needed
  Transport-->>iter_result_rows: Return partition response
  iter_result_rows-->>CSVWriter: Provide decoded row
  CSVWriter->>TextSink: Write header and serialized rows
  SnowflakeClient-->>Caller: Return number of rows written

Loading

Suggested reviewers: hampsterx

Merge Risk: ⚪ Minimal · up to d6fdc

The reviewed streaming and export changes have no identified merge-blocking issue; normal checks remain appropriate.

Pre-merge checks: 4 · 1

Failed checks: 1 warning

Check name	Status	Explanation	Resolution
Docstring Coverage		Docstring coverage is 27.16% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 81 functions across 11 files. (4 skipped:…	Write docstrings for the functions missing them to satisfy the coverage threshold.

Passed checks: 4 passed

Check name	Status	Explanation
Description Check		Check skipped - CodeRabbit’s high-level summary is enabled.
Title check		The title clearly and concisely describes the primary changes: streaming query APIs and CSV/JSONL exports.
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.

Full details: Docstring Coverage

Explanation

Docstring coverage is 27.16% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 81 functions across 11 files. (4 skipped: 4 unsupported.)

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

• Fix all pre-merge checks with AI

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}