Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,10 @@ Heavily inspired by Rust's [Iterator trait](https://doc.rust-lang.org/std/iter/t
pip install itrx
```

> **Using an AI coding agent?** `itrx` ships an installable [agent skill](#agent-skill) — run
> `uv run itrx-skill --install` and your agent gets a built-in reference for writing `Itr` chains
> correctly, without needing this whole README in context.

## Quick Start & Core Functionality

### Chaining
Expand Down Expand Up @@ -105,6 +109,27 @@ When working with `Itr`, keep these points in mind:
* **No Rewinding:** It's not possible to rewind an `Itr` to an earlier state. You can "preview" the next value using the `peek()` method, and conditionally consume it with `next_if()`.
* **Infinite Iterators:** Be cautious with open-ended iterators (e.g., those from `itertools.count()` or custom generators). Eager evaluation methods (like `collect()`, `count()`, `reduce()`) will attempt to consume the entire sequence, potentially leading to infinite loops or out-of-memory errors if applied to an infinite source.

## Agent skill

`itrx` ships a `SKILL.md` for AI coding agents (e.g. Claude Code) covering the `Itr` API, the
lazy/eager split described above, and the common pitfalls (single-pass iterators, infinite
sources, `groupby` vs `chunk_by`), so an agent doesn't need this whole README in context to write
correct `Itr` chains.

Install it into a project as a symlink to the version installed in the current environment:

```sh
uv run itrx-skill --install [PATH] # default PATH: .agents
uv run itrx-skill --remove [PATH] # default PATH: .agents
```

(Or, without `uv`, activate the virtualenv `itrx` is installed in and drop the `uv run` prefix —
`itrx-skill` is a normal console-script entry point, so it's only on `PATH` while that environment
is active.)

This creates (or removes) `PATH/skills/itrx`, symlinked to the skill bundled inside the installed
`itrx` package, so it always matches the version in use.

## API Reference

For a complete list of `Itr` methods and their detailed descriptions, please refer to the [API documentation](./doc/apidoc.md).
Expand Down
3 changes: 3 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,9 @@ classifiers = [
"Homepage" = "https://github.com/virgesmith/itrx"
"Bug Tracker" = "https://github.com/virgesmith/itrx/issues"

[project.scripts]
itrx-skill = "itrx.skill_cli:main"

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
Expand Down
6 changes: 6 additions & 0 deletions relnotes.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
## Unreleased

### New features

- Installable **agent skill**: the package now bundles a `SKILL.md` reference for AI coding agents, plus an `itrx-skill` console script to symlink it into a project (`itrx-skill --install [PATH]` / `--remove [PATH]`, default `PATH=.agents`, creating `PATH/skills/itrx`). The symlink points at the skill inside the installed `itrx`, so it always matches the version in use. See the "Agent skill" section of the README.

## 0.4.0

### Breaking changes
Expand Down
168 changes: 168 additions & 0 deletions src/itrx/skill/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,168 @@
---
name: itrx
description: >
Use when writing, editing, or debugging code that uses itrx's `Itr` — a chainable, lazy
iterator adaptor for Python inspired by Rust's Iterator trait. Covers the lazy/eager split,
the full method set by category, single-pass and infinite-iterator pitfalls, and how
`collect` materialises results. Triggers: "itrx", "Itr", chainable/fluent iterator, a
Rust-style `.map().filter().collect()` chain in Python, replacing nested `itertools` calls.
---

# Developing with itrx

`itrx` wraps any Python iterable, iterator, or generator in an `Itr[T]` that exposes a fluent,
chainable, mostly-lazy API modelled on Rust's [`Iterator`](https://doc.rust-lang.org/std/iter/trait.Iterator.html)
trait. It is zero-dependency: in most cases each method is thin syntactic sugar over `itertools`
or a builtin, so the runtime cost over hand-written `itertools` is negligible.

```py
from itrx import Itr

Itr(range(100)).rev().step_by(4).skip(10).map(lambda x: x * x).filter(lambda x: x % 10 == 9).for_each(print)
```

`Itr` implements the iterator protocol itself (`__iter__`/`__next__`), so an `Itr` can be passed
anywhere an iterator is accepted — `for x in itr`, `list(itr)`, `sum(itr)`, or as the `other`
argument to `zip`/`chain` on another `Itr`.

This skill is a quick reference, not the full API. For a method's exact signature, parameters,
raises and doctest, consult the generated per-method reference:

<https://github.com/virgesmith/itrx/blob/main/doc/apidoc.md>

Read it whenever you need a detail this file doesn't state — it is generated from the source
docstrings and covers every public method. Note it tracks `main`, so it may describe methods
newer than the installed version; `Itr.<method>.__doc__` (or `help(Itr.<method>)`) is the
authoritative check for what the *installed* version actually has. In a checkout of the itrx repo
itself the same file is at `doc/apidoc.md`, and the narrative docs are in `README.md`.

## When reaching for `Itr` is worth it

The reason to use it is **readability**, not speed — it will not make a pipeline faster than the
equivalent `itertools` code, because it *is* that code underneath. Reach for it when:

- **A pipeline reads inside-out as builtins.** `filter(lambda .., map(lambda .., islice(islice(reversed(x), None, None, 4), 10, None)))`
has to be read from the innermost call outwards; the `Itr` chain reads left to right in the
order the data actually flows.
- **You want laziness by default without writing generators.** Every intermediate step returns a
new `Itr` and pulls items only on demand, so an infinite source stays workable right up to the
terminal call.
- **The operation exists in Rust's `Iterator` but not as a Python builtin** — `fold`, `inspect`,
`partition`, `position`, `intersperse`, `interleave`, `dedup`, `chunk_by`, `unzip`, `map_while`,
`next_chunk`, `step_by`, `rolling`.

Conversely, it is **not** worth it for a single `map`/`filter` (a comprehension is clearer), for
code already dominated by numpy/pandas vectorised calls, or where the data is a materialised
`list` you are indexing into rather than streaming.

## Constructing

`Itr(it)` accepts any `Iterable[T]` and immediately calls `iter()` on it. Wrapping a **sequence**
(list, tuple, str, range) is safe to do repeatedly — each `Itr` gets its own fresh iterator — but
wrapping a **generator or iterator** hands over ownership: consuming the `Itr` consumes it.

## Lazy vs eager — the single most important distinction

**Lazy** (return a new `Itr` and consume nothing until pulled, so they are safe on infinite
sources — with the exception of `product`, which materialises `other` up front like
`itertools.product` does):

`accumulate`, `batched`, `chain`, `chunk_by`, `copy`, `cycle`, `dedup`, `enumerate`, `filter`,
`flat_map`, `flatten`, `inspect`, `interleave`, `intersperse`, `map`, `map_dict`, `map_while`,
`pairwise`, `partition`, `product`, `repeat`, `rolling`, `skip`, `skip_while`, `step_by`, `take`,
`take_while`, `tee`, `unzip`, `zip`, `zip_longest`

**Eager** (consume the iterator, return a concrete value; **never** on an infinite source):

- Collection: `collect`, `last`, `next`, `next_chunk`, `next_if`, `nth`, `position`, `peek`
- Aggregation: `all`, `any`, `consume`, `count`, `find`, `fold`, `for_each`, `max`, `min`, `prod`,
`reduce`, `sum`
- Whole-input reordering: `groupby`, `sorted_by`, `value_counts`, `rev`

Note that some of these consume only as far as they need to: `next`, `next_chunk`, `nth`,
`next_if`, `peek`, `find`, `position`, `any` and `all` short-circuit, so they *are* safe on an
infinite source. `collect`, `count`, `last`, `consume`, `fold`, `reduce`, `sum`, `prod`, `max`,
`min`, `for_each`, `rev`, `groupby`, `sorted_by` and `value_counts` are not.

## Outputs

`collect(container=tuple)` materialises into `tuple` (default), `list`, `set`, or `dict`:

```py
>>> from itrx import Itr
>>> Itr(("apple", "banana", "carrot")).groupby(len).collect(dict)
{5: ('apple',), 6: ('banana', 'carrot')}
```

- `collect(dict)` requires the iterator to yield 2-tuples — pair up with `zip`, `enumerate`,
`groupby`, `chunk_by`, `value_counts` or a `map` that returns pairs first.
- `collect(set)` is the "unique" operation (and drops order).
- `for_each(func)` is the side-effecting terminal; `consume()` exhausts for side effects alone
(pair it with `inspect`); `fold(init, func)` is the general terminal reduction.

## Method notes and gotchas

- **Single-pass, no rewinding.** Like any Python iterator, an `Itr` is consumed once. To use a
sequence twice, use `copy()` (splits at the current position into an independent `Itr`),
`tee(n)`, or re-wrap the original source. There is no `reset`.
- **`peek()` does not advance**, and returns the same value on repeated calls; it raises
`StopIteration` when exhausted. `next_if(predicate)` consumes the next item **only** if it
satisfies the predicate, otherwise leaves it in place and returns `None` (also `None` when
exhausted) — the correct tool for conditional lookahead parsing.
- **`tee(n)` and `copy()` share a buffer.** After `tee`, do not consume the original `Itr` — use
the returned ones. If one tee lags far behind another the buffer grows to hold the gap, so
memory can blow up on a large or infinite source. `tee(n)` raises `ValueError` for `n < 1`.
- **`nth(n)` is 0-based** (like Rust's `Iterator::nth`): `nth(0)` is the first item. It raises
`ValueError` for `n < 0` and `StopIteration` if the iterator is shorter than `n + 1`.
- **`next()` and `last()` raise `StopIteration`** on an empty iterator, they do not return `None`.
`find(predicate)` *does* return `None` when nothing matches.
- **`groupby(key)` and `sorted_by(key)` sort the whole input up front**, so they reorder output,
require mutually-orderable keys, and must not touch an infinite source. Use the lazy
**`chunk_by(key)`** to group *consecutive* runs without sorting (the `itertools.groupby`
semantics) — it preserves order and works on infinite iterators.
- **`value_counts()` is eager** and yields `(value, count)` most-common-first with ties by first
appearance (pandas semantics, not sorted by key). Items must be hashable. Collect it into a
`dict` for a lookup table.
- **`dedup()` removes only *adjacent* duplicates**, keeping the first of each run. It compares by
equality (items need not be hashable) and stays lazy — it is not "unique". For global
uniqueness use `collect(set)`, accepting the loss of order.
- **`rev()` materialises the entire remaining sequence** into memory before yielding — unavoidable,
but never call it on an unbounded source.
- **`repeat(n)` tees the iterator `n` times**, so it buffers the whole sequence for large `n`; it
always returns a new `Itr` and leaves the original exhausted. `cycle()` repeats indefinitely.
- **`flatten()` removes exactly one level** of nesting and every item must itself be iterable
(note that `str` is, which flattens strings into characters). `flat_map(f)` is
`map(f).flatten()`.
- **`zip(other)` stops at the shorter input** (no `strict=True` mode); `zip_longest(other,
fillvalue=...)` runs to the longer, padding. `interleave(other)` alternates and then yields the
tail of whichever is longer.
- **`starmap(f)` unpacks each item as `*args`** — use it after `zip`, `enumerate`, `pairwise` or
`product` instead of `map(lambda t: f(*t))`.
- **`map_dict(mapping)`** looks each item up in a `dict` (a `defaultdict` works); a missing key
raises `KeyError`.
- **`batched(n)` yields non-overlapping tuples** (the last possibly short); **`rolling(n)`** yields
overlapping windows of exactly `n` (a generalisation of `pairwise`); **`step_by(n)`** keeps every
n-th item starting with the first.
- **`partition(predicate)`** returns `(matching, non_matching)` as two lazy `Itr`s built on a
`copy()`, so the predicate runs twice per item and the same buffering caveat as `tee` applies.
- **`product(other)` materialises `other`** (as `itertools.product` does), so the *other* iterable
must be finite even though the chain stays lazy in `self`.
- **`inspect(func)` is the lazy debugging hook** — it calls `func` on each item and passes it
through unchanged, so you can drop it mid-chain without altering results.

## Typing

`Itr[T]` is generic and every method is fully annotated, so type checkers track the element type
through a chain: `Itr(range(3)).map(str)` is `Itr[str]`, `.enumerate()` is `Itr[tuple[int, str]]`,
`.collect(list)` is `list[str]`. The package ships a `py.typed` marker. When a chain's inferred
type is wrong, the fix is usually to annotate the lambda or the source, not to cast the `Itr`.

## Working on itrx itself (not just using it)

If you are editing this library's own source (`src/itrx/itr.py`) rather than using it in a
downstream project, the repo's `AGENTS.md` governs the workflow. In short: no new runtime
dependencies (it is zero-dependency by design), new non-terminal methods must return an `Itr` and
stay lazy, every public method needs a doctest, the full gate suite is
`uv run ruff check && uv run ruff format --check && uv run ty check src && uv run pytest`, and
coverage is enforced at 100%. Regenerate `doc/apidoc.md` with
`uv run python src/scripts/introspect.py` whenever the public API changes.
100 changes: 100 additions & 0 deletions src/itrx/skill_cli.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
import argparse
import os
from pathlib import Path

_SKILL_NAME = "itrx"
_DEFAULT_PATH = ".agents"


def _source_dir() -> Path:
"""The bundled skill directory, shipped inside this package."""
return Path(__file__).parent / "skill"


def _target_path(root: str) -> Path:
return Path(root) / "skills" / _SKILL_NAME


def _is_ours(target: Path, source: Path) -> bool:
if not target.is_symlink():
return False
try:
return target.resolve() == source.resolve()
except OSError:
return False


def _link_target(source: Path, target: Path) -> str:
"""A relative link target, or an absolute one if there is no relative path (different windows drives)."""
try:
return os.path.relpath(source.resolve(), target.parent.resolve())
except ValueError:
return str(source.resolve())


def _install(root: str) -> int:
source = _source_dir()
target = _target_path(root)
target.parent.mkdir(parents=True, exist_ok=True)

if target.is_symlink():
if _is_ours(target, source):
print(f"already installed and up to date: {target}")
return 0
print(f"refusing to overwrite existing symlink not managed by {_SKILL_NAME}-skill: {target}")
return 1
if target.exists():
print(f"refusing to overwrite existing file or directory: {target}")
return 1

target.symlink_to(_link_target(source, target), target_is_directory=True)
print(f"installed: {target} -> {source}")
return 0


def _remove(root: str) -> int:
source = _source_dir()
target = _target_path(root)

if not target.exists() and not target.is_symlink():
print(f"not installed: {target}")
return 0
if not _is_ours(target, source):
print(f"refusing to remove {target}: not a symlink managed by {_SKILL_NAME}-skill")
return 1

target.unlink()
print(f"removed: {target}")
return 0


def main(argv: list[str] | None = None) -> int:
"""Entry point for the `itrx-skill` console script."""
parser = argparse.ArgumentParser(
prog=f"{_SKILL_NAME}-skill",
description=f"Install or remove the '{_SKILL_NAME}' agent skill as a symlink into a project.",
)
group = parser.add_mutually_exclusive_group(required=True)
group.add_argument(
"--install",
nargs="?",
const=_DEFAULT_PATH,
metavar="PATH",
help=f"Symlink the skill into PATH/skills/{_SKILL_NAME} (default PATH: {_DEFAULT_PATH}).",
)
group.add_argument(
"--remove",
nargs="?",
const=_DEFAULT_PATH,
metavar="PATH",
help=f"Remove the symlink from PATH/skills/{_SKILL_NAME} (default PATH: {_DEFAULT_PATH}).",
)
args = parser.parse_args(argv)

if args.install is not None:
return _install(args.install)
return _remove(args.remove)


if __name__ == "__main__":
raise SystemExit(main()) # pragma: no cover - console script entry point
Loading