Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions .out-of-scope/format-conversion.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Conversion to Other Formats

This package converts between Python values and TOON, and nothing else. It does not parse or write CSV, YAML, or XML, does not auto-detect input formats, and does not batch-convert files or bundle them into archives.

## Why this is out of scope

The spec scopes TOON as a translation layer for the JSON data model: "produce data as JSON in code, encode to TOON for downstream consumption, and decode back to JSON if needed" (Introduction, Purpose and Scope). Python values are the hub. Any other format reaches TOON through a parser that already turns it into dicts and lists:

```python
import csv
import json

encode(json.load(f))
encode({"rows": list(csv.DictReader(f))})

rows = decode(text)["rows"]
writer = csv.DictWriter(out, fieldnames=rows[0].keys())
writer.writeheader()
writer.writerows(rows)
```

Shipping our own CSV or YAML reader would duplicate the standard library and mature third-party parsers, and every edge case of those formats would turn into a bug report here. As Johann put it on the CSV pull request: "this is completely out of scope for the TOON Python package. There are numerous battle-tested CSV parsers available" ([#39](https://github.com/toon-format/toon-python/pull/39#issuecomment-3539108955)).

The `toon` command follows the same line: it converts between JSON and TOON, because JSON is the data model TOON carries (§2).

## Prior requests

- [#25](https://github.com/toon-format/toon-python/issues/25) / [#35](https://github.com/toon-format/toon-python/pull/35): "batch processing" – multi-format conversion between JSON, YAML, XML, CSV, and TOON with auto-detection
- [#38](https://github.com/toon-format/toon-python/discussions/38): "TOON to CSV python lib" – merging `tooncsv` into this package
- [#39](https://github.com/toon-format/toon-python/pull/39): "Add CSV Parsing & Writing Support to toon_format"
31 changes: 31 additions & 0 deletions .out-of-scope/framework-integrations.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# Framework Integrations

This package does not ship integrations for LLM or data frameworks – no LangChain serializers or output parsers, no agent-framework adapters, no pandas API, and no MCP server. The core library encodes Python values to TOON and decodes TOON back to Python values. Everything else builds on those two calls.

## Why this is out of scope

Every one of these integrations is a thin layer over `encode()` and `decode()`, but each one ties the package to someone else's release cycle. An integration for LangChain, pandas, or an MCP server means another optional dependency to pin, another API that breaks on its own schedule, and another test matrix – for code that is a single call on the user's side:

```python
# pandas
encode({"customers": df[["Customer", "Customer question"]].to_dict("records")})

# Pydantic models, dataclasses, agent-framework payloads
encode(model.model_dump())
encode(dataclasses.asdict(obj))
```

The DataFrame example already emits a tabular array (§9.3), `customers[3]{Customer,"Customer question"}:` with one row per record, so a dedicated pandas API wouldn't produce anything better.

Johann closed the LangChain integration with the same reasoning: "a LangChain integration is out of scope for the core library, which should stay focused on encoding and decoding. This would be a great fit as a separate package (e.g. `toon-langchain`)" ([#45](https://github.com/toon-format/toon-python/pull/45#issuecomment-4073109814)). The MCP server got the same answer: "toon-python should stay focused on encoding/decoding. If there's appetite for this, it would be better as a separate repository" ([#31](https://github.com/toon-format/toon-python/pull/31#issuecomment-4073055880)).

Integrations are welcome as separate packages, or on the framework's side.

The optional `pydantic` extra ([#46](https://github.com/toon-format/toon-python/pull/46)) exists and is not part of this decision. Host-type normalization through a `JSONEncoder.default`-style hook is allowed by the spec (§3) and is a regular feature request, not an integration.

## Prior requests

- [#31](https://github.com/toon-format/toon-python/pull/31): "Add Model Context Protocol (MCP) Server for TOON Format"
- [#44](https://github.com/toon-format/toon-python/discussions/44): "Agent framework integration" (LangChain, Pydantic AI)
- [#45](https://github.com/toon-format/toon-python/pull/45): "feat(langchain): add ToonSerializer and ToonOutputParser"
- [#56](https://github.com/toon-format/toon-python/issues/56): "An interface with Pandas"
28 changes: 28 additions & 0 deletions .out-of-scope/json-text-helpers.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# JSON Text Helpers

`encode()` takes Python values and `decode()` returns Python values, the same way `json.dumps` and `json.loads` work. The package does not add variants that read or write JSON text: no `json_indent` option on `decode()`, no `encode_json(text)`, no `loads` alias for `json.loads`.

## Why this is out of scope

The `json` module already does the JSON half, and each helper would only wrap one call to it:

```python
import json

encode(json.loads(text)) # JSON text → TOON
json.dumps(decode(toon_text), indent=2) # TOON → pretty-printed JSON
```

A `json_indent` option on `decode()` also breaks the return type: with it set, `decode()` returns a `str` instead of the decoded value, so every typed caller has to narrow a `JsonValue | str` union – for output that `json.dumps` produces without it.

The public API also stays close to the reference implementation, where `encode()` and `decode()` work on in-memory values only. @smortezah raised this on the `encode_json` pull request: "To maintain alignment with the original TypeScript implementation of toon-format, I suggest we avoid adding excessive functions" ([#57](https://github.com/toon-format/toon-python/pull/57)).

The `toon` command is where JSON text belongs: it reads and writes JSON files and indents its JSON output on decode.

JSON `null` needs no helper either: `json.loads` returns `None`, which `encode()` emits as `null` (§2).

## Prior requests

- [#10](https://github.com/toon-format/toon-python/issues/10): "JSON indentation option in decode method"
- [#37](https://github.com/toon-format/toon-python/pull/37): "feat: Add JSON indentation option to decode() method"
- [#57](https://github.com/toon-format/toon-python/pull/57): "feat: add encode_json and loads helpers for better JSON null support"
15 changes: 15 additions & 0 deletions .out-of-scope/native-backend.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Native Backend

This package is pure Python. It does not bind the Rust implementation through PyO3 or ship any other compiled backend, optional or not.

## Why this is out of scope

A pure Python package installs from a single wheel on every interpreter and platform – CPython, PyPy, GraalPy, free-threaded builds – without a compiler toolchain. A native backend trades that for a wheel matrix per Python version, OS, and architecture, plus a fallback path for everything the matrix misses. @Justar96 laid out the same reasoning when the question first came up: "We're prioritizing pure python for maximal portability (no wheels/toolchains) interpreter coverage PyPy/GraalPy" ([#9](https://github.com/toon-format/toon-python/discussions/9#discussioncomment-14871615)).

Sharing one implementation across languages doesn't buy correctness either. The ports validate against the same language-agnostic test suite (Appendix C), so the Python and Rust decoders agree because both pass it, not because they share code. A binding would also tie Python releases to the Rust crate's API and release schedule.

Anyone who needs a Rust-backed binding can publish it as a separate package.

## Prior requests

- [#9](https://github.com/toon-format/toon-python/discussions/9): "Using the rust library instead of a pure python implementation"
Loading