From 145019ef28a7de94aed12fb94a51687fa2daec13 Mon Sep 17 00:00:00 2001 From: virgesmith Date: Wed, 30 Sep 2026 08:20:55 +0100 Subject: [PATCH 1/4] API consistency: no StopIteration from terminals, filter_map/unique/min_max and more A StopIteration escaping a map/filter callback silently ends the enclosing iteration, so e.g. Itr(groups).map(lambda g: Itr(g).reduce(max)) dropped every group after the first empty one without error. Terminal methods now report a missing item without StopIteration: - position returns None when nothing matches (like find, and Rust's Option) - peek, nth (out of range) and reduce (empty) raise ValueError, consistent with last/max/min; next() keeps StopIteration, as the builtin does partition now calls the predicate once per item, buffering items for the side not being read, instead of filtering a copy twice. New: filter_map, find_map, unique, min_max, compress, take_last, and a repr. Extended: chain/interleave take *others, zip(strict=), groupby(sort=False), optional sorted_by key, all/any default to truthiness, sum/prod start. flat_map uses chain.from_iterable. enumerate gains a doctest and test: it was previously covered only via the old position implementation. SKILL.md, README, relnotes, apidoc and the examples notebook are updated to match. Co-Authored-By: Claude Opus 5.5 --- README.md | 11 +- doc/apidoc.md | 256 +++++++++++++++++--- doc/examples.ipynb | 4 +- relnotes.md | 21 ++ src/itrx/itr.py | 379 +++++++++++++++++++++++++----- src/itrx/skill/SKILL.md | 85 ++++--- src/test/test_aggregation.py | 57 ++++- src/test/test_collection.py | 34 ++- src/test/test_combine_split.py | 48 +++- src/test/test_general.py | 6 + src/test/test_transform_filter.py | 55 +++++ 11 files changed, 813 insertions(+), 143 deletions(-) diff --git a/README.md b/README.md index 9b918ca..0ce626f 100644 --- a/README.md +++ b/README.md @@ -86,20 +86,20 @@ For reference, the equivalent using `itertools` directly (is fairly readable): Note: 1. Using `collect(dict)` requires an iterable that produces 2-tuples (key-value pairs). -2. `collect(set)` is equivalent to a "unique" method. +2. `collect(set)` gives the distinct items but loses their order; `unique()` lazily keeps the first occurrence of each, in order. ## How `Itr` Works: Lazy vs. Eager Most `Itr` methods are **lazy transformations**, meaning they return a new `Itr` instance without immediately processing any data. This allows for arbitrary chaining and efficient memory usage, as items are only processed as they are requested. In most cases, `Itr` simply acts as a convenient wrapper around `itertools`, enabling this left-to-right chaining syntax. -- **Combining and splitting:** `partition`, `copy`, `batched`, `pairwise`, `rolling`, `chain`, `cycle`, `repeat`, `product`, `inspect`, `intersperse`, `interleave`, `chunk_by`, `zip_longest` -- **Transformation and filtering:** `accumulate`, `filter`, `map`, `starmap`, `map_while`, `flatten`, `flat_map`, `skip_while`, `take_while`, `dedup`, `dedup_with_count`, `scan` +- **Combining and splitting:** `partition`, `copy`, `batched`, `pairwise`, `rolling`, `chain`, `cycle`, `repeat`, `product`, `inspect`, `intersperse`, `interleave`, `chunk_by`, `zip`, `zip_longest` +- **Transformation and filtering:** `accumulate`, `filter`, `filter_map`, `compress`, `map`, `starmap`, `map_while`, `flatten`, `flat_map`, `skip_while`, `take_while`, `dedup`, `dedup_with_count`, `unique`, `scan` However, some methods are **eager consumers**. These methods iterate over and consume the underlying data, returning concrete values, collections, or aggregates. Examples include: * **Collection methods:** `collect`, `last`, `next`, `next_chunk`, `next_if`, `nth`, `position` -* **Aggregation methods:** `count`, `reduce`, `max`, `min`, `sum`, `prod`, `all`, `any`, `consume`, `find`, `fold`, `eq`, `is_sorted` -* **Sorting/grouping:** `sorted_by` and `groupby` sort the entire input up front, and `value_counts` counts it (most common first, like pandas), so all three consume the whole iterator immediately and must not be used on infinite sources. Use the lazy `chunk_by` to group consecutive runs without sorting. +* **Aggregation methods:** `count`, `reduce`, `max`, `min`, `min_max`, `sum`, `prod`, `all`, `any`, `consume`, `find`, `find_map`, `fold`, `eq`, `is_sorted` +* **Sorting/grouping:** `sorted_by` and `groupby` sort the entire input up front, and `value_counts` counts it (most common first, like pandas), so all three consume the whole iterator immediately and must not be used on infinite sources. Use the lazy `chunk_by` to group consecutive runs without sorting. Likewise `rev` and `take_last` must read to the end of the input before yielding anything. ### Important Considerations @@ -107,6 +107,7 @@ When working with `Itr`, keep these points in mind: * **Single-Pass Iterators:** Like all Python iterators, `Itr` instances (and their underlying iterators) can generally only be consumed once. If you need to process the same sequence multiple times, use methods like `copy()`, `cycle()`, or `repeat()` as necessary. * **No Rewinding:** It's not possible to rewind an `Itr` to an earlier state. You can "preview" the next value using the `peek()` method, and conditionally consume it with `next_if()`. +* **Missing Items:** `find`, `find_map` and `position` return `None` when nothing matches, while `peek`, `nth`, `reduce`, `last`, `max`, `min` and `min_max` raise `ValueError` on an empty (or too short) iterator. Only `next()` raises `StopIteration`, like the builtin, so avoid calling it inside a `map` or `filter` callback: a `StopIteration` escaping a callback silently ends the enclosing iteration rather than raising. * **Infinite Iterators:** Be cautious with open-ended iterators (e.g., those from `itertools.count()` or custom generators). Eager evaluation methods (like `collect()`, `count()`, `reduce()`) will attempt to consume the entire sequence, potentially leading to infinite loops or out-of-memory errors if applied to an infinite source. ## Agent skill diff --git a/doc/apidoc.md b/doc/apidoc.md index 4cd2aa9..5b09f27 100644 --- a/doc/apidoc.md +++ b/doc/apidoc.md @@ -43,26 +43,40 @@ Example: ### `all` -Return True if all elements in the iterator satisfy the predicate. +Return True if all elements in the iterator satisfy the predicate (by default, if all are truthy). + +Short-circuits at the first failure. An empty iterator returns True. Args: - predicate (Callable[[T], bool]): A function to test each element. + predicate (Callable[[T], bool]): A function to test each element. Defaults to `bool` (truthiness). Returns: bool: True if all elements satisfy the predicate, False otherwise. +Example: + >>> Itr([1, 2, 0]).all() + False + >>> Itr([2, 4, 6]).all(lambda x: x % 2 == 0) + True ### `any` -Return True if any element in the iterator satisfies the predicate. +Return True if any element in the iterator satisfies the predicate (by default, if any is truthy). + +Short-circuits at the first success. An empty iterator returns False. Args: - predicate (Callable[[T], bool]): A function to test each element. + predicate (Callable[[T], bool]): A function to test each element. Defaults to `bool` (truthiness). Returns: bool: True if any element satisfies the predicate, False otherwise. +Example: + >>> Itr([0, "", None]).any() + False + >>> Itr([1, 2, 3]).any(lambda x: x > 2) + True ### `batched` @@ -86,14 +100,18 @@ Example: ### `chain` -Chain this iterator with another iterable, yielding all items from self followed by all items from other. +Chain this iterator with one or more other iterables, yielding all items from self followed by all items from +each of the others in turn. Args: - other (Iterable[U]): Another iterable to chain. + *others (Iterable[U]): The iterables to chain on. Returns: - Itr[T | U]: A new iterator yielding items from both iterables. + Itr[T | U]: A new iterator yielding items from all the iterables. +Example: + >>> Itr([1, 2]).chain([3], (4, 5)).collect() + (1, 2, 3, 4, 5) ### `chunk_by` @@ -124,6 +142,23 @@ Returns: +### `compress` + +Yield only the items whose corresponding selector is truthy (like `itertools.compress`). + +Stops when either the iterator or the selectors are exhausted. + +Args: + selectors (Iterable[object]): Values whose truthiness decides whether the item in the same position is kept. + +Returns: + Itr[T]: An iterator over the selected items. + +Example: + >>> Itr("abcde").compress([1, 0, 1, 0, 1]).collect() + ('a', 'c', 'e') + + ### `consume` Exhaust the iterator. Useful when only the side effects are required (see inspect) @@ -203,9 +238,15 @@ Example: Yield pairs of (index, item) for each item in the iterator, where index starts at 0 or the value provided +Args: + start (int): The index of the first item. Defaults to 0. + Returns: Itr[tuple[int, T]]: An iterator of (index, item) pairs. +Example: + >>> Itr("ab").enumerate(start=1).collect() + ((1, 'a'), (2, 'b')) ### `eq` @@ -242,6 +283,24 @@ Returns: +### `filter_map` + +Map each item, keeping only the results that are not None (like Rust's `Iterator::filter_map`). + +Equivalent to `map(mapper).filter(lambda x: x is not None)`, but typed as `Itr[U]` rather than +`Itr[U | None]`. Only None is dropped: other falsy results such as 0 or "" are kept. + +Args: + mapper (Callable[[T], U | None]): A function mapping each item to a result, or None to drop it. + +Returns: + Itr[U]: An iterator over the non-None results. + +Example: + >>> Itr(["1", "x", "0"]).filter_map(lambda s: int(s) if s.isdigit() else None).collect() + (1, 0) + + ### `find` Return the first item in the iterator that satisfies the predicate, or None if not found. @@ -254,6 +313,25 @@ Returns: +### `find_map` + +Return the first result of the mapper that is not None, or None if there is none (like Rust's +`Iterator::find_map`). + +Equivalent to `filter_map(mapper).next()`, except that it returns None rather than raising when there is no +result. Short-circuits at the first result. + +Args: + mapper (Callable[[T], U | None]): A function mapping each item to a result, or None to skip it. + +Returns: + U | None: The first non-None result, or None. + +Example: + >>> Itr(["a", "12", "3"]).find_map(lambda s: int(s) if s.isdigit() else None) + 12 + + ### `flat_map` Map each item to an iterable, then flatten one level. @@ -308,13 +386,24 @@ Because the input is sorted, this method is **eager**: it consumes and materiali immediately (so it must not be used on an infinite iterator), the output is ordered by key, and the keys must be mutually orderable. For lazy, order-preserving grouping of consecutive runs, see `chunk_by`. -Semantically this is equivalent to pandas' default `groupby` (i.e. `sort=True`): all items sharing a key are -collected into a single group regardless of their position, and groups are emitted in sorted-key order. It has -no `sort=False` (appearance-order) option, requires mutually-orderable keys, and does not drop `None` keys. +Semantically this is equivalent to pandas' `groupby`: all items sharing a key are collected into a single group +regardless of their position, and `None` keys are not dropped. By default (`sort=True`) groups are emitted in +sorted-key order, which requires mutually-orderable keys. With `sort=False` they are emitted in order of each +key's first appearance, and the keys need only be hashable. Either way, the items within each group keep +their original relative order. + +Args: + grouper (Callable[[T], U]): The key function applied to each element. + sort (bool): If True (the default), order the groups by key; otherwise by first appearance. Returns: Itr[tuple[U, tuple[T,...]]]: An iterator over the keys and tuples of values +Example: + >>> Itr(["bb", "a", "ccc", "dd"]).groupby(len).collect() + ((1, ('a',)), (2, ('bb', 'dd')), (3, ('ccc',))) + >>> Itr(["bb", "a", "ccc", "dd"]).groupby(len, sort=False).collect() + ((2, ('bb', 'dd')), (1, ('a',)), (3, ('ccc',))) ### `inspect` @@ -340,20 +429,23 @@ Example: ### `interleave` -Interleaves elements from this iterator with elements from another iterable, yielding alternately from each. -When one iterable is exhausted, the remaining elements of the other are yielded in order. +Interleaves elements from this iterator with elements from one or more other iterables, taking one from each +in turn (round-robin). Exhausted inputs are skipped, so the remaining elements of the longer ones are yielded +in order. Args: - other (Iterable[U]): Another iterable to interleave with. + *others (Iterable[U]): The iterables to interleave with. Returns: - Itr[T | U]: A new iterator yielding elements alternately from self and other. + Itr[T | U]: A new iterator yielding elements from self and the others in turn. Example: >>> Itr([1, 3, 5]).interleave([2, 4, 6]).collect() (1, 2, 3, 4, 5, 6) >>> Itr([1, 3, 5, 7]).interleave([2, 4]).collect() (1, 2, 3, 4, 5, 7) + >>> Itr([1, 4, 7]).interleave([2, 5], [3]).collect() + (1, 2, 3, 4, 5, 7) ### `intersperse` @@ -470,6 +562,30 @@ Raises: ValueError: If the iterator is empty. +### `min_max` + +Return the minimum and maximum elements in a single pass, optionally using a key function. + +Ties resolve as for the `min` and `max` builtins: the first minimal and the first maximal element are returned. +NB Consumes the iterator. + +Args: + key (Callable[[T], Any] | None, optional): A function to extract a comparison key from each element. + Defaults to comparing the elements themselves. + +Returns: + tuple[T, T]: The (minimum, maximum) elements. + +Raises: + ValueError: If the iterator is empty. + +Example: + >>> Itr([3, 1, 4, 1, 5]).min_max() + (1, 5) + >>> Itr(["bb", "a", "ccc", "dd"]).min_max(len) + ('a', 'ccc') + + ### `next` Return the next item from the iterator, if available. Otherwise raises StopIteration @@ -530,9 +646,11 @@ Returns: T: The n-th item. Raises: - StopIteration: if the iterator has fewer than n + 1 items. - ValueError: if n < 0 + ValueError: if n < 0, or the iterator has fewer than n + 1 items. +Example: + >>> Itr("abc").nth(1) + 'b' ### `pairwise` @@ -560,6 +678,15 @@ Returns: - The first iterator yields elements for which the predicate returns True. - The second iterator yields elements for which the predicate returns False. +Both iterators are lazy and share the source, and the predicate is called exactly once per item. Items destined +for one iterator are buffered while the other is being consumed, so memory can grow if one side lags far behind +(as with `tee`). + +Example: + >>> even, odd = Itr(range(6)).partition(lambda x: x % 2 == 0) + >>> even.collect(), odd.collect() + ((0, 2, 4), (1, 3, 5)) + ### `peek` @@ -569,7 +696,7 @@ Returns: T: The next element in the sequence. Raises: - StopIteration: If the iterator is exhausted. + ValueError: If the iterator is exhausted. Note: This method copies the iterator to avoid modifying the original iterator's state. The copy is cheap @@ -588,30 +715,39 @@ Example: ### `position` -Returns the index of the first element in the iterable that satisfies the given predicate. +Returns the index of the first element in the iterable that satisfies the given predicate, or None if there +is no such element (like Rust's `Iterator::position`). The index counts from the iterator's current position. Args: predicate (Callable[[T], bool]): A function that takes an element and returns True if the element matches the condition. Returns: - int: The index of the first matching element. + int | None: The index of the first matching element, or None. -Raises: - StopIteration: If no element satisfies the predicate. +Example: + >>> Itr("abc").position(lambda c: c == "b") + 1 + >>> Itr("abc").position(lambda c: c == "z") is None + True ### `prod` -Return the product of all items in the iterator (1 if empty). NB Consumes the iterator. +Return the product of `start` and all items in the iterator (so `start` if empty). NB Consumes the iterator. The items must support multiplication (e.g. numbers). +Args: + start (T | int): The initial value, multiplied by each item in turn. Defaults to 1. + Returns: T: The product of all items. Example: >>> Itr([2, 3, 4]).prod() 24 + >>> Itr([2, 3]).prod(10) + 60 ### `product` @@ -628,8 +764,7 @@ Returns: ### `reduce` -Reduce the iterator to a single value using a function. -Will raise StopIteration if the iterator is exhausted. +Reduce the iterator to a single value using a function, taking the first item as the initial value. Args: func (Callable[[T, T], T]): The function to combine values. @@ -637,6 +772,12 @@ Args: Returns: T: The final reduced value. +Raises: + ValueError: If the iterator is empty (use `fold` to supply an initial value instead). + +Example: + >>> Itr([3, 1, 4]).reduce(max) + 4 ### `repeat` @@ -719,13 +860,15 @@ Returns: ### `sorted_by` -Return an iterator over the items sorted by the given key function. +Return an iterator over the items sorted by the given key function, or by the items themselves if no key is +given. This method is **eager**: it consumes and materialises the whole iterator immediately (so it must not be used on an infinite iterator). The sort is stable: items that compare equal retain their relative order. Args: - key (Callable[[T], Any]): A function to extract a comparison key from each item. + key (Callable[[T], Any] | None): A function to extract a comparison key from each item. Defaults to + comparing the items themselves. reverse (bool): If True, sort in descending order. Defaults to False. Returns: @@ -734,6 +877,8 @@ Returns: Example: >>> Itr(["ccc", "a", "bb"]).sorted_by(len).collect() ('a', 'bb', 'ccc') + >>> Itr([3, 1, 2]).sorted_by(reverse=True).collect() + (3, 2, 1) ### `starmap` @@ -767,9 +912,13 @@ Returns: ### `sum` -Return the sum of all items in the iterator (0 if empty). NB Consumes the iterator. +Return the sum of `start` and all items in the iterator (so `start` if empty). NB Consumes the iterator. -The items must support addition with int (e.g. numbers; use `reduce` or `fold` for other types). +The items must support addition with `start`: with the default of 0 that means numbers. Pass e.g. `start=[]` to +concatenate lists (as with the `sum` builtin, strings are rejected: use `"".join` instead). + +Args: + start (T | int): The initial value, to which each item is added in turn. Defaults to 0. Returns: T: The sum of all items. @@ -777,6 +926,8 @@ Returns: Example: >>> Itr([1, 2, 3]).sum() 6 + >>> Itr([[1], [2, 3]]).sum([]) + [1, 2, 3] ### `take` @@ -791,6 +942,27 @@ Returns: +### `take_last` + +Return an iterator over the last n items (or all of them, if there are fewer than n). + +This method is **eager**: it consumes the whole iterator immediately to find its end (so it must not be used on +an infinite iterator), but only ever holds n items in memory. + +Args: + n (int): The number of items to keep. + +Returns: + Itr[T]: An iterator over the last n items. + +Raises: + ValueError: if n < 0 + +Example: + >>> Itr(range(10)).take_last(3).collect() + (7, 8, 9) + + ### `take_while` Yield items from the iterator as long as the given predicate is true. @@ -842,6 +1014,27 @@ Examples: [0, 1, 2] +### `unique` + +Lazily drop items that have been seen before, keeping the first occurrence of each and preserving order. + +Unlike `dedup`, duplicates need not be adjacent; unlike `collect(set)`, order is preserved and it works on an +infinite iterator. The seen items (or their keys) are held in a set, so they must be hashable, and memory +grows with the number of distinct values. + +Args: + key (Callable[[T], Any] | None): Applied to each item to decide uniqueness. Defaults to the item itself. + +Returns: + Itr[T]: An iterator over the first occurrence of each distinct item (or key). + +Example: + >>> Itr([3, 1, 3, 2, 1]).unique().collect() + (3, 1, 2) + >>> Itr(["apple", "avocado", "banana"]).unique(lambda s: s[0]).collect() + ('apple', 'banana') + + ### `unzip` Splits the iterator of pairs into two separate iterators, each containing the elements from one position of @@ -877,14 +1070,19 @@ Example: ### `zip` -Yield pairs of items from this iterator and another iterable. +Yield pairs of items from this iterator and another iterable, stopping at the end of the shorter one. Args: other (Iterable[U]): The other iterable. + strict (bool): If True, raise `ValueError` (when the shorter input runs out, since this is lazy) if the + inputs differ in length, as with the `zip` builtin. Defaults to False. Returns: Itr[tuple[T, U]]: An iterator of paired items. +Example: + >>> Itr([1, 2, 3]).zip("ab").collect() + ((1, 'a'), (2, 'b')) ### `zip_longest` diff --git a/doc/examples.ipynb b/doc/examples.ipynb index 5100707..648ff95 100644 --- a/doc/examples.ipynb +++ b/doc/examples.ipynb @@ -180,7 +180,7 @@ "metadata": {}, "source": [ "Some points to note:\n", - "- an `nth` call that overruns the sequence raises a `StopIteration`\n", + "- an `nth` call that overruns the sequence raises a `ValueError`\n", "- a `skip` call that overruns the sequence raises a `StopIteration`\n", "- `step_by` yields an item and then steps (same behaviour as `itetools.islice`)\n", "- constructing `Itr` directly from a dictionary captures only the keys, use the `items` method to caputure key, \n", @@ -224,7 +224,7 @@ } ], "source": [ - "with contextlib.suppress(StopIteration):\n", + "with contextlib.suppress(ValueError):\n", " Itr(collatz(4)).nth(10) # Collatz(4) = 4,2,1\n", "\n", "with contextlib.suppress(StopIteration):\n", diff --git a/relnotes.md b/relnotes.md index f05463b..10dd5d1 100644 --- a/relnotes.md +++ b/relnotes.md @@ -1,7 +1,28 @@ ## Unreleased +### Breaking changes + +- Terminal methods no longer raise `StopIteration` when an item is missing. A `StopIteration` escaping a `map`/`filter` callback silently ends the *enclosing* iteration, so e.g. `Itr(groups).map(lambda g: Itr(g).reduce(max))` used to drop every group after the first empty one without any error. Now: + - `position` returns `None` when nothing matches (like `find`, and Rust's `Option`). + - `peek`, `nth` (index out of range) and `reduce` (empty input) raise `ValueError`, consistent with `last`, `max` and `min`. + - `next()` still raises `StopIteration`, as the builtin does. +- `partition` calls the predicate once per item rather than twice, buffering items for whichever side is not being read. Side-effecting predicates now see each item exactly once. + ### New features +- `filter_map(f)` / `find_map(f)`: map and drop `None` results / return the first non-`None` result (Rust's `filter_map` and `find_map`). `filter_map` is typed `Itr[U]` rather than `Itr[U | None]`. +- `unique(key=None)`: lazy, order-preserving removal of duplicates anywhere in the input (not just adjacent ones, unlike `dedup`). Items or keys must be hashable. +- `min_max(key=None)`: the minimum and maximum in a single pass, with the builtins' tie-breaking. +- `compress(selectors)`: keep the items whose selector is truthy (`itertools.compress`). +- `take_last(n)`: the last `n` items, holding at most `n` in memory (eager, like `rev`). +- `chain(*others)` and `interleave(*others)` accept any number of iterables; `interleave` round-robins across them. +- `zip(other, *, strict=False)`: pass `strict=True` to raise `ValueError` on a length mismatch. +- `groupby(key, *, sort=True)`: `sort=False` emits groups in order of first appearance, and needs only hashable (not orderable) keys. +- `sorted_by(key=None, ...)`: the key is now optional, sorting by the items themselves. +- `all()` / `any()`: the predicate is now optional, defaulting to truthiness. +- `sum(start=0)` / `prod(start=1)`: optional start value, e.g. `sum([])` to concatenate lists. +- `Itr` now has a `repr` showing the wrapped iterator (without consuming it). + - Installable **agent skill**: the package now bundles a `SKILL.md` reference for AI coding agents, plus an `itrx-skill` console script to symlink it into a project (`itrx-skill --install [PATH]` / `--remove [PATH]`, default `PATH=.agents`, creating `PATH/skills/itrx`). The symlink points at the skill inside the installed `itrx`, so it always matches the version in use. See the "Agent skill" section of the README. - `dedup_with_count()`: the lazy, positional counterpart to `value_counts` — collapses each *consecutive* run of equal items into an `(item, count)` pair (run-length encoding), preserving order and working on infinite iterators. Note the `(item, count)` ordering matches `value_counts` and is the reverse of Rust's `dedup_with_count`. - `scan(init, func)`: lazily map items through a running state, optionally stopping early (Rust's `Iterator::scan`). Generalises `accumulate`: the state need not share the items' type, and returning `None` halts iteration. Returning `(new_state, None)` still yields `None` as an output, so the halt signal is never ambiguous. diff --git a/src/itrx/itr.py b/src/itrx/itr.py index 596abb5..493ba38 100644 --- a/src/itrx/itr.py +++ b/src/itrx/itr.py @@ -32,6 +32,10 @@ def __next__(self) -> T: "Implement the next method of the Iterator protocol" return next(self._it) + def __repr__(self) -> str: + "Show the wrapped iterator, without consuming it" + return f"Itr({self._it!r})" + def accumulate(self, func: Callable[[T, T], T] | None = None, *, initial: T | None = None) -> "Itr[T]": """ Return an iterator over the accumulated results of applying the function (or sum by default) to the items. Does @@ -53,29 +57,43 @@ def accumulate(self, func: Callable[[T, T], T] | None = None, *, initial: T | No """ return Itr(itertools.accumulate(self._it, func, initial=initial)) - def all(self, predicate: Predicate[T]) -> bool: - """Return True if all elements in the iterator satisfy the predicate. + def all(self, predicate: Predicate[T] = bool) -> bool: + """Return True if all elements in the iterator satisfy the predicate (by default, if all are truthy). + + Short-circuits at the first failure. An empty iterator returns True. Args: - predicate (Callable[[T], bool]): A function to test each element. + predicate (Callable[[T], bool]): A function to test each element. Defaults to `bool` (truthiness). Returns: bool: True if all elements satisfy the predicate, False otherwise. + Example: + >>> Itr([1, 2, 0]).all() + False + >>> Itr([2, 4, 6]).all(lambda x: x % 2 == 0) + True """ - return all(predicate(item) for item in self._it) + return all(map(predicate, self._it)) - def any(self, predicate: Predicate[T]) -> bool: - """Return True if any element in the iterator satisfies the predicate. + def any(self, predicate: Predicate[T] = bool) -> bool: + """Return True if any element in the iterator satisfies the predicate (by default, if any is truthy). + + Short-circuits at the first success. An empty iterator returns False. Args: - predicate (Callable[[T], bool]): A function to test each element. + predicate (Callable[[T], bool]): A function to test each element. Defaults to `bool` (truthiness). Returns: bool: True if any element satisfies the predicate, False otherwise. + Example: + >>> Itr([0, "", None]).any() + False + >>> Itr([1, 2, 3]).any(lambda x: x > 2) + True """ - return any(predicate(item) for item in self._it) + return any(map(predicate, self._it)) def batched(self, n: int) -> "Itr[tuple[T, ...]]": """ @@ -96,18 +114,21 @@ def batched(self, n: int) -> "Itr[tuple[T, ...]]": """ return cast("Itr[tuple[T, ...]]", Itr(itertools.batched(self._it, n))) - def chain[U](self, other: Iterable[U]) -> "Itr[T | U]": - """Chain this iterator with another iterable, yielding all items from self followed by all items from other. + def chain[U](self, *others: Iterable[U]) -> "Itr[T | U]": + """Chain this iterator with one or more other iterables, yielding all items from self followed by all items from + each of the others in turn. Args: - other (Iterable[U]): Another iterable to chain. + *others (Iterable[U]): The iterables to chain on. Returns: - Itr[T | U]: A new iterator yielding items from both iterables. + Itr[T | U]: A new iterator yielding items from all the iterables. + Example: + >>> Itr([1, 2]).chain([3], (4, 5)).collect() + (1, 2, 3, 4, 5) """ - - return cast("Itr[T | U]", Itr(itertools.chain(self._it, other))) + return cast("Itr[T | U]", Itr(itertools.chain(self._it, *others))) @overload def collect(self, container: type[tuple[T, ...]] = tuple) -> tuple[T, ...]: ... @@ -127,6 +148,23 @@ def collect(self, container: type[_CollectT] = tuple) -> _CollectT: # ty: ignor """ return container(self._it) + def compress(self, selectors: Iterable[object]) -> "Itr[T]": + """Yield only the items whose corresponding selector is truthy (like `itertools.compress`). + + Stops when either the iterator or the selectors are exhausted. + + Args: + selectors (Iterable[object]): Values whose truthiness decides whether the item in the same position is kept. + + Returns: + Itr[T]: An iterator over the selected items. + + Example: + >>> Itr("abcde").compress([1, 0, 1, 0, 1]).collect() + ('a', 'c', 'e') + """ + return Itr(itertools.compress(self._it, selectors)) + def consume(self) -> None: """Exhaust the iterator. Useful when only the side effects are required (see inspect) @@ -207,9 +245,15 @@ def dedup_with_count(self) -> "Itr[tuple[T, int]]": def enumerate(self, *, start: int = 0) -> "Itr[tuple[int, T]]": """Yield pairs of (index, item) for each item in the iterator, where index starts at 0 or the value provided + Args: + start (int): The index of the first item. Defaults to 0. + Returns: Itr[tuple[int, T]]: An iterator of (index, item) pairs. + Example: + >>> Itr("ab").enumerate(start=1).collect() + ((1, 'a'), (2, 'b')) """ return cast("Itr[tuple[int, T]]", Itr(enumerate(self._it, start))) @@ -248,6 +292,24 @@ def filter(self, predicate: Predicate[T]) -> "Itr[T]": """ return Itr(filter(predicate, self._it)) + def filter_map[U](self, mapper: Callable[[T], U | None]) -> "Itr[U]": + """Map each item, keeping only the results that are not None (like Rust's `Iterator::filter_map`). + + Equivalent to `map(mapper).filter(lambda x: x is not None)`, but typed as `Itr[U]` rather than + `Itr[U | None]`. Only None is dropped: other falsy results such as 0 or "" are kept. + + Args: + mapper (Callable[[T], U | None]): A function mapping each item to a result, or None to drop it. + + Returns: + Itr[U]: An iterator over the non-None results. + + Example: + >>> Itr(["1", "x", "0"]).filter_map(lambda s: int(s) if s.isdigit() else None).collect() + (1, 0) + """ + return Itr(y for y in map(mapper, self._it) if y is not None) + def find(self, predicate: Predicate[T]) -> T | None: """Return the first item in the iterator that satisfies the predicate, or None if not found. @@ -260,6 +322,25 @@ def find(self, predicate: Predicate[T]) -> T | None: """ return next(filter(predicate, self._it), None) + def find_map[U](self, mapper: Callable[[T], U | None]) -> U | None: + """Return the first result of the mapper that is not None, or None if there is none (like Rust's + `Iterator::find_map`). + + Equivalent to `filter_map(mapper).next()`, except that it returns None rather than raising when there is no + result. Short-circuits at the first result. + + Args: + mapper (Callable[[T], U | None]): A function mapping each item to a result, or None to skip it. + + Returns: + U | None: The first non-None result, or None. + + Example: + >>> Itr(["a", "12", "3"]).find_map(lambda s: int(s) if s.isdigit() else None) + 12 + """ + return next((y for y in map(mapper, self._it) if y is not None), None) + def flat_map[U](self, mapper: Callable[[T], Iterable[U]]) -> "Itr[U]": """Map each item to an iterable, then flatten one level. @@ -270,12 +351,7 @@ def flat_map[U](self, mapper: Callable[[T], Iterable[U]]) -> "Itr[U]": Itr[U]: An iterator over the mapped and flattened items. """ - - def gen() -> Iterable[U]: - for elem in self: - yield from mapper(elem) - - return Itr(gen()) + return Itr(itertools.chain.from_iterable(map(mapper, self._it))) def flatten[U](self) -> "Itr[U]": """Flatten one level of nesting in the iterator. Each item must itself be iterable. @@ -333,7 +409,7 @@ def chunk_by[U](self, grouper: Callable[[T], U]) -> "Itr[tuple[U, tuple[T, ...]] groups = ((k, tuple(v)) for k, v in itertools.groupby(self._it, key=key_fn)) return cast("Itr[tuple[U, tuple[T, ...]]]", Itr(groups)) - def groupby[U](self, grouper: Callable[[T], U]) -> "Itr[tuple[U, tuple[T,...]]]": + def groupby[U](self, grouper: Callable[[T], U], *, sort: bool = True) -> "Itr[tuple[U, tuple[T,...]]]": """ Sort and then group an iterable by the supplied key function. Note the following differences from itertools: - The iterable is pre-sorted because itertools.groupby only works correctly on sorted sequences @@ -343,16 +419,33 @@ def groupby[U](self, grouper: Callable[[T], U]) -> "Itr[tuple[U, tuple[T,...]]]" immediately (so it must not be used on an infinite iterator), the output is ordered by key, and the keys must be mutually orderable. For lazy, order-preserving grouping of consecutive runs, see `chunk_by`. - Semantically this is equivalent to pandas' default `groupby` (i.e. `sort=True`): all items sharing a key are - collected into a single group regardless of their position, and groups are emitted in sorted-key order. It has - no `sort=False` (appearance-order) option, requires mutually-orderable keys, and does not drop `None` keys. + Semantically this is equivalent to pandas' `groupby`: all items sharing a key are collected into a single group + regardless of their position, and `None` keys are not dropped. By default (`sort=True`) groups are emitted in + sorted-key order, which requires mutually-orderable keys. With `sort=False` they are emitted in order of each + key's first appearance, and the keys need only be hashable. Either way, the items within each group keep + their original relative order. + + Args: + grouper (Callable[[T], U]): The key function applied to each element. + sort (bool): If True (the default), order the groups by key; otherwise by first appearance. Returns: Itr[tuple[U, tuple[T,...]]]: An iterator over the keys and tuples of values - """ - key_fn = cast("Callable[[T], Any]", grouper) - groups = ((k, tuple(v)) for k, v in itertools.groupby(sorted(self._it, key=key_fn), key=key_fn)) + Example: + >>> Itr(["bb", "a", "ccc", "dd"]).groupby(len).collect() + ((1, ('a',)), (2, ('bb', 'dd')), (3, ('ccc',))) + >>> Itr(["bb", "a", "ccc", "dd"]).groupby(len, sort=False).collect() + ((2, ('bb', 'dd')), (1, ('a',)), (3, ('ccc',))) + """ + if sort: + key_fn = cast("Callable[[T], Any]", grouper) + groups = ((k, tuple(v)) for k, v in itertools.groupby(sorted(self._it, key=key_fn), key=key_fn)) + else: + groups_by_key: dict[U, list[T]] = {} + for item in self._it: + groups_by_key.setdefault(grouper(item), []).append(item) + groups = ((k, tuple(v)) for k, v in groups_by_key.items()) return cast("Itr[tuple[U, tuple[T, ...]]]", Itr(groups)) def inspect(self, func: Callable[[T], None]) -> "Itr[T]": @@ -404,28 +497,31 @@ def intersperser(item: U) -> Generator[T | U, None, None]: return cast("Itr[T | U]", Itr(intersperser(item))) - def interleave[U](self, other: Iterable[U]) -> "Itr[T | U]": + def interleave[U](self, *others: Iterable[U]) -> "Itr[T | U]": """ - Interleaves elements from this iterator with elements from another iterable, yielding alternately from each. - When one iterable is exhausted, the remaining elements of the other are yielded in order. + Interleaves elements from this iterator with elements from one or more other iterables, taking one from each + in turn (round-robin). Exhausted inputs are skipped, so the remaining elements of the longer ones are yielded + in order. Args: - other (Iterable[U]): Another iterable to interleave with. + *others (Iterable[U]): The iterables to interleave with. Returns: - Itr[T | U]: A new iterator yielding elements alternately from self and other. + Itr[T | U]: A new iterator yielding elements from self and the others in turn. Example: >>> Itr([1, 3, 5]).interleave([2, 4, 6]).collect() (1, 2, 3, 4, 5, 6) >>> Itr([1, 3, 5, 7]).interleave([2, 4]).collect() (1, 2, 3, 4, 5, 7) + >>> Itr([1, 4, 7]).interleave([2, 5], [3]).collect() + (1, 2, 3, 4, 5, 7) """ _sentinel = object() def interleaver() -> Generator[T | U, None, None]: - for pair in itertools.zip_longest(self._it, other, fillvalue=_sentinel): - for item in pair: + for row in itertools.zip_longest(self._it, *others, fillvalue=_sentinel): + for item in row: if item is not _sentinel: yield cast("T | U", item) @@ -536,6 +632,43 @@ def min(self, key: Callable[[T], Any] | None = None) -> T: """ return min(self._it, key=key) + def min_max(self, key: Callable[[T], Any] | None = None) -> tuple[T, T]: + """Return the minimum and maximum elements in a single pass, optionally using a key function. + + Ties resolve as for the `min` and `max` builtins: the first minimal and the first maximal element are returned. + NB Consumes the iterator. + + Args: + key (Callable[[T], Any] | None, optional): A function to extract a comparison key from each element. + Defaults to comparing the elements themselves. + + Returns: + tuple[T, T]: The (minimum, maximum) elements. + + Raises: + ValueError: If the iterator is empty. + + Example: + >>> Itr([3, 1, 4, 1, 5]).min_max() + (1, 5) + >>> Itr(["bb", "a", "ccc", "dd"]).min_max(len) + ('a', 'ccc') + """ + try: + lo = hi = next(self._it) + except StopIteration: + raise ValueError("min_max() of an empty iterator") from None + # T is unbounded so is not known to be orderable, as in sorted_by/groupby + key_fn = cast("Callable[[T], Any]", key if key is not None else lambda x: x) + lo_key = hi_key = key_fn(lo) + for item in self._it: + k = key_fn(item) + if k < lo_key: + lo, lo_key = item, k + elif k > hi_key: + hi, hi_key = item, k + return lo, hi + def next(self) -> T: """Return the next item from the iterator, if available. Otherwise raises StopIteration @@ -581,7 +714,7 @@ def next_if(self, predicate: Predicate[T]) -> T | None: 3 """ try: - item = self.peek() + item = self.copy().next() except StopIteration: return None if predicate(item): @@ -601,13 +734,18 @@ def nth(self, n: int) -> T: T: The n-th item. Raises: - StopIteration: if the iterator has fewer than n + 1 items. - ValueError: if n < 0 + ValueError: if n < 0, or the iterator has fewer than n + 1 items. + Example: + >>> Itr("abc").nth(1) + 'b' """ if n < 0: raise ValueError(f"nth index must be >= 0, got {n}") - return self.skip(n).next() + try: + return next(itertools.islice(self._it, n, None)) + except StopIteration: + raise ValueError(f"nth({n}) is out of range: the iterator has fewer than {n + 1} items") from None def pairwise(self) -> "Itr[tuple[T, T]]": """Returns an iterator that yields consecutive pairs of elements from the iterable. @@ -632,9 +770,30 @@ def partition(self, predicate: Predicate[T]) -> tuple["Itr[T]", "Itr[T]"]: tuple[Itr[T], Itr[T]]: A tuple containing two iterators: - The first iterator yields elements for which the predicate returns True. - The second iterator yields elements for which the predicate returns False. + + Both iterators are lazy and share the source, and the predicate is called exactly once per item. Items destined + for one iterator are buffered while the other is being consumed, so memory can grow if one side lags far behind + (as with `tee`). + + Example: + >>> even, odd = Itr(range(6)).partition(lambda x: x % 2 == 0) + >>> even.collect(), odd.collect() + ((0, 2, 4), (1, 3, 5)) """ - copy = self.copy() - return self.filter(predicate), copy.filter(lambda i: not predicate(i)) + matching: deque[T] = deque() + non_matching: deque[T] = deque() + + def side(mine: deque[T]) -> Generator[T, None, None]: + while True: + while not mine: + try: + item = next(self._it) + except StopIteration: + return + (matching if predicate(item) else non_matching).append(item) + yield mine.popleft() + + return Itr(side(matching)), Itr(side(non_matching)) def peek(self) -> T: """Returns the next element in the sequence without advancing the iterator. @@ -643,7 +802,7 @@ def peek(self) -> T: T: The next element in the sequence. Raises: - StopIteration: If the iterator is exhausted. + ValueError: If the iterator is exhausted. Note: This method copies the iterator to avoid modifying the original iterator's state. The copy is cheap @@ -658,36 +817,48 @@ def peek(self) -> T: >>> it.peek() 2 """ - return self.copy().next() + try: + return self.copy().next() + except StopIteration: + raise ValueError("peek() on an exhausted iterator") from None - def position(self, predicate: Predicate[T]) -> int: + def position(self, predicate: Predicate[T]) -> int | None: """ - Returns the index of the first element in the iterable that satisfies the given predicate. + Returns the index of the first element in the iterable that satisfies the given predicate, or None if there + is no such element (like Rust's `Iterator::position`). The index counts from the iterator's current position. Args: predicate (Callable[[T], bool]): A function that takes an element and returns True if the element matches the condition. Returns: - int: The index of the first matching element. + int | None: The index of the first matching element, or None. - Raises: - StopIteration: If no element satisfies the predicate. + Example: + >>> Itr("abc").position(lambda c: c == "b") + 1 + >>> Itr("abc").position(lambda c: c == "z") is None + True """ - return self.enumerate().skip_while(lambda x: not predicate(x[1])).next()[0] + return next((i for i, item in enumerate(self._it) if predicate(item)), None) - def prod(self) -> T: - """Return the product of all items in the iterator (1 if empty). NB Consumes the iterator. + def prod(self, start: T | int = 1) -> T: + """Return the product of `start` and all items in the iterator (so `start` if empty). NB Consumes the iterator. The items must support multiplication (e.g. numbers). + Args: + start (T | int): The initial value, multiplied by each item in turn. Defaults to 1. + Returns: T: The product of all items. Example: >>> Itr([2, 3, 4]).prod() 24 + >>> Itr([2, 3]).prod(10) + 60 """ - return cast("T", math.prod(cast("Iterable[Any]", self._it))) + return cast("T", math.prod(cast("Iterable[Any]", self._it), start=cast("Any", start))) def product[U](self, other: Iterable[U]) -> "Itr[tuple[T, U]]": """ @@ -702,8 +873,7 @@ def product[U](self, other: Iterable[U]) -> "Itr[tuple[T, U]]": return cast("Itr[tuple[T, U]]", Itr(itertools.product(self._it, other))) def reduce(self, func: Callable[[T, T], T]) -> T: - """Reduce the iterator to a single value using a function. - Will raise StopIteration if the iterator is exhausted. + """Reduce the iterator to a single value using a function, taking the first item as the initial value. Args: func (Callable[[T, T], T]): The function to combine values. @@ -711,8 +881,18 @@ def reduce(self, func: Callable[[T, T], T]) -> T: Returns: T: The final reduced value. + Raises: + ValueError: If the iterator is empty (use `fold` to supply an initial value instead). + + Example: + >>> Itr([3, 1, 4]).reduce(max) + 4 """ - return self.fold(next(self._it), func) + try: + first = next(self._it) + except StopIteration: + raise ValueError("reduce() of an empty iterator") from None + return self.fold(first, func) def repeat(self, n: int) -> "Itr[T]": """ @@ -809,14 +989,16 @@ def skip_while(self, predicate: Predicate[T]) -> "Itr[T]": """ return Itr(itertools.dropwhile(predicate, self._it)) - def sorted_by(self, key: Callable[[T], Any], *, reverse: bool = False) -> "Itr[T]": - """Return an iterator over the items sorted by the given key function. + def sorted_by(self, key: Callable[[T], Any] | None = None, *, reverse: bool = False) -> "Itr[T]": + """Return an iterator over the items sorted by the given key function, or by the items themselves if no key is + given. This method is **eager**: it consumes and materialises the whole iterator immediately (so it must not be used on an infinite iterator). The sort is stable: items that compare equal retain their relative order. Args: - key (Callable[[T], Any]): A function to extract a comparison key from each item. + key (Callable[[T], Any] | None): A function to extract a comparison key from each item. Defaults to + comparing the items themselves. reverse (bool): If True, sort in descending order. Defaults to False. Returns: @@ -825,8 +1007,11 @@ def sorted_by(self, key: Callable[[T], Any], *, reverse: bool = False) -> "Itr[T Example: >>> Itr(["ccc", "a", "bb"]).sorted_by(len).collect() ('a', 'bb', 'ccc') + >>> Itr([3, 1, 2]).sorted_by(reverse=True).collect() + (3, 2, 1) """ - return Itr(sorted(self._it, key=key, reverse=reverse)) + # T is unbounded so is not known to be orderable, as in groupby + return Itr(sorted(cast("Iterable[Any]", self._it), key=key, reverse=reverse)) def starmap[U](self, func: Callable[..., U]) -> "Itr[U]": """ @@ -857,10 +1042,14 @@ def step_by(self, n: int) -> "Itr[T]": """ return Itr(itertools.islice(self._it, 0, None, n)) - def sum(self) -> T: - """Return the sum of all items in the iterator (0 if empty). NB Consumes the iterator. + def sum(self, start: T | int = 0) -> T: + """Return the sum of `start` and all items in the iterator (so `start` if empty). NB Consumes the iterator. - The items must support addition with int (e.g. numbers; use `reduce` or `fold` for other types). + The items must support addition with `start`: with the default of 0 that means numbers. Pass e.g. `start=[]` to + concatenate lists (as with the `sum` builtin, strings are rejected: use `"".join` instead). + + Args: + start (T | int): The initial value, to which each item is added in turn. Defaults to 0. Returns: T: The sum of all items. @@ -868,8 +1057,10 @@ def sum(self) -> T: Example: >>> Itr([1, 2, 3]).sum() 6 + >>> Itr([[1], [2, 3]]).sum([]) + [1, 2, 3] """ - return cast("T", sum(cast("Iterable[Any]", self._it))) + return cast("T", sum(cast("Iterable[Any]", self._it), cast("Any", start))) def take(self, n: int) -> "Itr[T]": """Return an iterator over the next n items from the iterator. @@ -883,6 +1074,29 @@ def take(self, n: int) -> "Itr[T]": """ return Itr(itertools.islice(self._it, n)) + def take_last(self, n: int) -> "Itr[T]": + """Return an iterator over the last n items (or all of them, if there are fewer than n). + + This method is **eager**: it consumes the whole iterator immediately to find its end (so it must not be used on + an infinite iterator), but only ever holds n items in memory. + + Args: + n (int): The number of items to keep. + + Returns: + Itr[T]: An iterator over the last n items. + + Raises: + ValueError: if n < 0 + + Example: + >>> Itr(range(10)).take_last(3).collect() + (7, 8, 9) + """ + if n < 0: + raise ValueError(f"take_last requires n >= 0, got {n}") + return Itr(deque(self._it, maxlen=n)) + def take_while(self, predicate: Predicate[T]) -> "Itr[T]": """Yield items from the iterator as long as the given predicate is true. @@ -936,6 +1150,36 @@ def tee(self, n: int = 2) -> tuple["Itr[T]", ...]: raise ValueError(f"tee requires at least 1 iterator, got {n}") return tuple(Itr(t) for t in itertools.tee(self._it, n)) + def unique(self, key: Callable[[T], Any] | None = None) -> "Itr[T]": + """Lazily drop items that have been seen before, keeping the first occurrence of each and preserving order. + + Unlike `dedup`, duplicates need not be adjacent; unlike `collect(set)`, order is preserved and it works on an + infinite iterator. The seen items (or their keys) are held in a set, so they must be hashable, and memory + grows with the number of distinct values. + + Args: + key (Callable[[T], Any] | None): Applied to each item to decide uniqueness. Defaults to the item itself. + + Returns: + Itr[T]: An iterator over the first occurrence of each distinct item (or key). + + Example: + >>> Itr([3, 1, 3, 2, 1]).unique().collect() + (3, 1, 2) + >>> Itr(["apple", "avocado", "banana"]).unique(lambda s: s[0]).collect() + ('apple', 'banana') + """ + + def gen() -> Generator[T, None, None]: + seen: set[Any] = set() + for item in self._it: + k = item if key is None else key(item) + if k not in seen: + seen.add(k) + yield item + + return Itr(gen()) + def unzip[U, V](self: "Itr[tuple[U, V]]") -> tuple["Itr[U]", "Itr[V]"]: """Splits the iterator of pairs into two separate iterators, each containing the elements from one position of the pairs. @@ -970,17 +1214,22 @@ def value_counts(self) -> "Itr[tuple[T, int]]": """ return cast("Itr[tuple[T, int]]", Itr(Counter(self._it).most_common())) - def zip[U](self, other: Iterable[U]) -> "Itr[tuple[T, U]]": - """Yield pairs of items from this iterator and another iterable. + def zip[U](self, other: Iterable[U], *, strict: bool = False) -> "Itr[tuple[T, U]]": + """Yield pairs of items from this iterator and another iterable, stopping at the end of the shorter one. Args: other (Iterable[U]): The other iterable. + strict (bool): If True, raise `ValueError` (when the shorter input runs out, since this is lazy) if the + inputs differ in length, as with the `zip` builtin. Defaults to False. Returns: Itr[tuple[T, U]]: An iterator of paired items. + Example: + >>> Itr([1, 2, 3]).zip("ab").collect() + ((1, 'a'), (2, 'b')) """ - return cast("Itr[tuple[T, U]]", Itr(zip(self._it, other, strict=False))) + return cast("Itr[tuple[T, U]]", Itr(zip(self._it, other, strict=strict))) def zip_longest[U, V]( self, other: Iterable[U], *, fillvalue: V | None = None diff --git a/src/itrx/skill/SKILL.md b/src/itrx/skill/SKILL.md index ba638cc..1078e46 100644 --- a/src/itrx/skill/SKILL.md +++ b/src/itrx/skill/SKILL.md @@ -49,7 +49,8 @@ equivalent `itertools` code, because it *is* that code underneath. Reach for it terminal call. - **The operation exists in Rust's `Iterator` but not as a Python builtin** — `fold`, `inspect`, `partition`, `position`, `intersperse`, `interleave`, `dedup`, `dedup_with_count`, `chunk_by`, - `unzip`, `map_while`, `scan`, `is_sorted`, `eq`, `next_chunk`, `step_by`, `rolling`. + `unzip`, `map_while`, `filter_map`, `find_map`, `scan`, `is_sorted`, `eq`, `next_chunk`, + `step_by`, `rolling`, `min_max`, `unique`. Conversely, it is **not** worth it for a single `map`/`filter` (a comprehension is clearer), for code already dominated by numpy/pandas vectorised calls, or where the data is a materialised @@ -67,24 +68,25 @@ wrapping a **generator or iterator** hands over ownership: consuming the `Itr` c sources — with the exception of `product`, which materialises `other` up front like `itertools.product` does): -`accumulate`, `batched`, `chain`, `chunk_by`, `copy`, `cycle`, `dedup`, `dedup_with_count`, -`enumerate`, `filter`, -`flat_map`, `flatten`, `inspect`, `interleave`, `intersperse`, `map`, `map_dict`, `map_while`, -`pairwise`, `partition`, `product`, `repeat`, `rolling`, `scan`, `skip`, `skip_while`, -`step_by`, `take`, -`take_while`, `tee`, `unzip`, `zip`, `zip_longest` +`accumulate`, `batched`, `chain`, `chunk_by`, `compress`, `copy`, `cycle`, `dedup`, +`dedup_with_count`, `enumerate`, `filter`, `filter_map`, `flat_map`, `flatten`, `inspect`, +`interleave`, `intersperse`, `map`, `map_dict`, `map_while`, `pairwise`, `partition`, `product`, +`repeat`, `rolling`, `scan`, `skip`, `skip_while`, `starmap`, `step_by`, `take`, `take_while`, +`tee`, `unique`, `unzip`, `zip`, `zip_longest` **Eager** (consume the iterator, return a concrete value; **never** on an infinite source): - Collection: `collect`, `last`, `next`, `next_chunk`, `next_if`, `nth`, `position`, `peek` -- Aggregation: `all`, `any`, `consume`, `count`, `eq`, `find`, `fold`, `for_each`, `is_sorted`, - `max`, `min`, `prod`, `reduce`, `sum` -- Whole-input reordering: `groupby`, `sorted_by`, `value_counts`, `rev` +- Aggregation: `all`, `any`, `consume`, `count`, `eq`, `find`, `find_map`, `fold`, `for_each`, + `is_sorted`, `max`, `min`, `min_max`, `prod`, `reduce`, `sum` +- Whole-input: `groupby`, `sorted_by`, `value_counts`, `rev`, `take_last` (these return an `Itr`, + but read the entire input at call time) Note that some of these consume only as far as they need to: `next`, `next_chunk`, `nth`, -`next_if`, `peek`, `find`, `position`, `any` and `all` short-circuit, so they *are* safe on an -infinite source. `collect`, `count`, `last`, `consume`, `fold`, `reduce`, `sum`, `prod`, `max`, -`min`, `for_each`, `rev`, `groupby`, `sorted_by` and `value_counts` are not. `eq` and `is_sorted` +`next_if`, `peek`, `find`, `find_map`, `position`, `any` and `all` short-circuit, so they *are* +safe on an infinite source. `collect`, `count`, `last`, `consume`, `fold`, `reduce`, `sum`, +`prod`, `max`, `min`, `min_max`, `for_each`, `rev`, `take_last`, `groupby`, `sorted_by` and +`value_counts` are not. `eq` and `is_sorted` short-circuit on the first difference or inversion, so they too are safe on an infinite source. ## Outputs @@ -99,7 +101,8 @@ short-circuit on the first difference or inversion, so they too are safe on an i - `collect(dict)` requires the iterator to yield 2-tuples — pair up with `zip`, `enumerate`, `groupby`, `chunk_by`, `value_counts` or a `map` that returns pairs first. -- `collect(set)` is the "unique" operation (and drops order). +- `collect(set)` gives the distinct items but drops order; the lazy `unique()` keeps the first + occurrence of each, in order. - `for_each(func)` is the side-effecting terminal; `consume()` exhausts for side effects alone (pair it with `inspect`); `fold(init, func)` is the general terminal reduction. @@ -109,26 +112,34 @@ short-circuit on the first difference or inversion, so they too are safe on an i sequence twice, use `copy()` (splits at the current position into an independent `Itr`), `tee(n)`, or re-wrap the original source. There is no `reset`. - **`peek()` does not advance**, and returns the same value on repeated calls; it raises - `StopIteration` when exhausted. `next_if(predicate)` consumes the next item **only** if it + `ValueError` when exhausted. `next_if(predicate)` consumes the next item **only** if it satisfies the predicate, otherwise leaves it in place and returns `None` (also `None` when - exhausted) — the correct tool for conditional lookahead parsing. + exhausted) — the correct tool for conditional lookahead parsing. If the items can themselves be + `None`, a `None` result is ambiguous; `peek()` first to tell the cases apart. - **`tee(n)` and `copy()` share a buffer.** After `tee`, do not consume the original `Itr` — use the returned ones. If one tee lags far behind another the buffer grows to hold the gap, so memory can blow up on a large or infinite source. `tee(n)` raises `ValueError` for `n < 1`. - **`nth(n)` is 0-based** (like Rust's `Iterator::nth`): `nth(0)` is the first item. It raises - `ValueError` for `n < 0` and `StopIteration` if the iterator is shorter than `n + 1`. -- **`next()` and `last()` raise `StopIteration`** on an empty iterator, they do not return `None`. - `find(predicate)` *does* return `None` when nothing matches. -- **`groupby(key)` and `sorted_by(key)` sort the whole input up front**, so they reorder output, - require mutually-orderable keys, and must not touch an infinite source. Use the lazy - **`chunk_by(key)`** to group *consecutive* runs without sorting (the `itertools.groupby` - semantics) — it preserves order and works on infinite iterators. + `ValueError` for `n < 0` or if the iterator is shorter than `n + 1`. +- **Missing items: `None` or `ValueError`, and only `next()` raises `StopIteration`.** `find`, + `find_map` and `position` return `None` when nothing matches. `peek`, `nth`, `reduce`, `last`, + `max`, `min` and `min_max` raise `ValueError` on an empty (or too short) iterator. `next()` + raises `StopIteration`, like the builtin — so don't call `.next()` inside a `map`/`filter` + callback: a `StopIteration` escaping a callback silently ends the *enclosing* iteration instead + of raising. Use `find`, `peek` or `nth(0)` there. +- **`groupby(key)` and `sorted_by(key=None)` sort the whole input up front**, so they reorder + output, require mutually-orderable keys, and must not touch an infinite source. + `groupby(key, sort=False)` still reads everything, but emits groups in first-appearance order + and needs only hashable keys. Use the lazy **`chunk_by(key)`** to group *consecutive* runs + without sorting (the `itertools.groupby` semantics) — it preserves order and works on infinite + iterators. - **`value_counts()` is eager** and yields `(value, count)` most-common-first with ties by first appearance (pandas semantics, not sorted by key). Items must be hashable. Collect it into a `dict` for a lookup table. - **`dedup()` removes only *adjacent* duplicates**, keeping the first of each run. It compares by equality (items need not be hashable) and stays lazy — it is not "unique". For global - uniqueness use `collect(set)`, accepting the loss of order. + uniqueness use **`unique(key=None)`**, which is also lazy and order-preserving but needs + hashable items (or keys) and remembers every distinct one it has seen. - **`dedup_with_count()` is run-length encoding** — the same adjacent-run logic as `dedup`, but yielding `(item, count)` pairs. It is the lazy, positional counterpart to `value_counts`: same output shape, but counting adjacent runs in source order rather than occurrences overall. On @@ -138,15 +149,20 @@ short-circuit on the first difference or inversion, so they too are safe on an i Note the `(item, count)` order is the reverse of Rust's `dedup_with_count`. It stays lazy on an infinite source, but an infinite individual run (e.g. `itertools.repeat(1)`) will hang. - **`rev()` materialises the entire remaining sequence** into memory before yielding — unavoidable, - but never call it on an unbounded source. -- **`repeat(n)` tees the iterator `n` times**, so it buffers the whole sequence for large `n`; it - always returns a new `Itr` and leaves the original exhausted. `cycle()` repeats indefinitely. + but never call it on an unbounded source. For just the end of a sequence, `take_last(n)` also + reads everything but holds only `n` items. +- **`repeat(n)` tees the iterator `n` times**, so it buffers the whole sequence for large `n`. It + is lazy and shares the underlying iterator: do not use the original `Itr` after calling it, only + the returned one. `cycle()` repeats indefinitely. - **`flatten()` removes exactly one level** of nesting and every item must itself be iterable (note that `str` is, which flattens strings into characters). `flat_map(f)` is `map(f).flatten()`. -- **`zip(other)` stops at the shorter input** (no `strict=True` mode); `zip_longest(other, - fillvalue=...)` runs to the longer, padding. `interleave(other)` alternates and then yields the - tail of whichever is longer. +- **`filter_map(f)` drops only `None` results** (not `0` or `""`), and is typed `Itr[U]` rather + than `Itr[U | None]`, so prefer it to `map(f).filter(...)`. `find_map(f)` is its first result. +- **`zip(other, *, strict=False)` stops at the shorter input**; with `strict=True` it raises + `ValueError` on a length mismatch, but only once iteration reaches it. `zip_longest(other, + fillvalue=...)` runs to the longer, padding. `chain(*others)` and `interleave(*others)` take any + number of iterables; `interleave` round-robins and keeps going with the longer ones. - **`starmap(f)` unpacks each item as `*args`** — use it after `zip`, `enumerate`, `pairwise` or `product` instead of `map(lambda t: f(*t))`. - **`map_dict(mapping)`** looks each item up in a `dict` (a `defaultdict` works); a missing key @@ -154,8 +170,11 @@ short-circuit on the first difference or inversion, so they too are safe on an i - **`batched(n)` yields non-overlapping tuples** (the last possibly short); **`rolling(n)`** yields overlapping windows of exactly `n` (a generalisation of `pairwise`); **`step_by(n)`** keeps every n-th item starting with the first. -- **`partition(predicate)`** returns `(matching, non_matching)` as two lazy `Itr`s built on a - `copy()`, so the predicate runs twice per item and the same buffering caveat as `tee` applies. +- **`partition(predicate)`** returns `(matching, non_matching)` as two lazy `Itr`s sharing the + source. The predicate runs once per item; items for the side not being read are buffered, so + as with `tee`, memory grows if one side lags far behind the other. +- **`all()` / `any()` default to truthiness**, and **`sum(start=0)` / `prod(start=1)`** take a + start value (`sum([])` concatenates lists; strings are rejected, as by the builtin). - **`product(other)` materialises `other`** (as `itertools.product` does), so the *other* iterable must be finite even though the chain stays lazy in `self`. - **`inspect(func)` is the lazy debugging hook** — it calls `func` on each item and passes it @@ -169,6 +188,8 @@ short-circuit on the first difference or inversion, so they too are safe on an i - **`eq(other)` compares contents; `==` does not.** `Itr` defines no `__eq__`, so `itr_a == itr_b` is an identity check. Use `.eq(other)` for an element-wise comparison, which also short-circuits rather than materialising both sides. +- **`min_max(key=None)`** finds both extremes in one pass, with the builtins' tie-breaking (the + first minimal and first maximal item). ## Typing diff --git a/src/test/test_aggregation.py b/src/test/test_aggregation.py index 7610463..9a8d34f 100644 --- a/src/test/test_aggregation.py +++ b/src/test/test_aggregation.py @@ -73,7 +73,7 @@ def test_reduce_empty() -> None: # the lambda wont get called, result is just the single element assert it.reduce(lambda _a, _b: 0) == 1 # and again on exhausted iterator - with pytest.raises(StopIteration): + with pytest.raises(ValueError, match="empty"): it.reduce(lambda _a, _b: 0) @@ -270,3 +270,58 @@ def test_is_sorted_reverse() -> None: def test_is_sorted_agrees_with_sorted_by() -> None: data = ["ccc", "a", "bb"] assert Itr(data).sorted_by(len).is_sorted(len) + + +def test_all_any_default_truthiness() -> None: + assert Itr([1, "a", [0]]).all() + assert not Itr([1, 0, 2]).all() + assert Itr[int]([]).all() + assert Itr([0, "", None, 3]).any() + assert not Itr([0, "", None]).any() + assert not Itr[int]([]).any() + + +def test_all_any_short_circuit() -> None: + it = Itr(itertools.count()) + assert not it.all() # 0 is falsy + assert it.next() == 1 + assert Itr(itertools.count()).any() + + +def test_sum_prod_start() -> None: + assert Itr([1, 2, 3]).sum(10) == 16 + assert Itr[int]([]).sum(10) == 10 + assert Itr([[1], [2, 3]]).sum([]) == [1, 2, 3] + assert Itr([2, 3]).prod(10) == 60 + assert Itr[int]([]).prod(10) == 10 + + +def test_find_map() -> None: + def parse(s: str) -> int | None: + return int(s) if s.isdigit() else None + + assert Itr(["a", "12", "3"]).find_map(parse) == 12 + assert Itr(["a", "b"]).find_map(parse) is None + assert Itr[str]([]).find_map(parse) is None + # short-circuits, so is safe on an infinite iterator + assert Itr(itertools.count()).find_map(lambda n: n * n if n > 3 else None) == 16 + + +def test_min_max() -> None: + assert Itr([3, 1, 4, 1, 5, 9, 2, 6]).min_max() == (1, 9) + assert Itr([7]).min_max() == (7, 7) + assert Itr(["bb", "a", "ccc", "dd"]).min_max(len) == ("a", "ccc") + + +def test_min_max_ties_match_builtins() -> None: + data = [(1, "a"), (0, "b"), (0, "c"), (2, "d"), (2, "e")] + + def key(t: tuple[int, str]) -> int: + return t[0] + + assert Itr(data).min_max(key) == (min(data, key=key), max(data, key=key)) == ((0, "b"), (2, "d")) + + +def test_min_max_empty() -> None: + with pytest.raises(ValueError, match="empty"): + Itr[int]([]).min_max() diff --git a/src/test/test_collection.py b/src/test/test_collection.py index 55b02e7..ed9452b 100644 --- a/src/test/test_collection.py +++ b/src/test/test_collection.py @@ -56,20 +56,18 @@ def test_nth() -> None: assert it.nth(0) == 10 # consumes preceding items, so this advances from the current position assert it.nth(2) == 40 - with pytest.raises(StopIteration): + with pytest.raises(ValueError, match="out of range"): it.nth(10) def test_position() -> None: assert Itr("abcdefghijklmnopqrstuvwxyz").position(lambda x: x == "a") == 0 - with pytest.raises(StopIteration): - Itr("abcdefghijklmnopqrstuvwxyz").position(lambda x: x == "H") + assert Itr("abcdefghijklmnopqrstuvwxyz").position(lambda x: x == "H") is None a = Itr("abcdefghijklmnopqrstuvwxyz") assert a.position(lambda x: x == "h") == 7 assert a.position(lambda x: x == "z") == 17 # counter is reset - with pytest.raises(StopIteration): - assert a.position(lambda x: x == "a") # iterator is exhausted + assert a.position(lambda x: x == "a") is None # iterator is exhausted def test_peek_does_not_advance_iterator() -> None: @@ -85,7 +83,7 @@ def test_peek_does_not_advance_iterator() -> None: def test_peek_on_empty_iterator_raises() -> None: it = Itr[bool]([]) - with pytest.raises(StopIteration): + with pytest.raises(ValueError, match="exhausted"): it.peek() @@ -153,3 +151,27 @@ def test_next_if_after_peek() -> None: assert it.next_if(lambda x: x == 5) == 5 assert it.next_if(lambda x: x == 5) is None assert it.peek() == 6 + + +def test_take_last() -> None: + assert Itr(range(10)).take_last(3).collect() == (7, 8, 9) + assert Itr(range(2)).take_last(3).collect() == (0, 1) + assert Itr(range(10)).take_last(0).collect() == () + assert Itr[int]([]).take_last(3).collect() == () + + +def test_take_last_invalid() -> None: + with pytest.raises(ValueError, match="n >= 0"): + Itr(range(3)).take_last(-1) + + +def test_terminal_errors_do_not_truncate_enclosing_chain() -> None: + # a StopIteration escaping a map callback would silently end the outer iteration instead of raising + groups = [[1, 2], [], [3]] + with pytest.raises(ValueError): + Itr(groups).map(lambda g: Itr(g).reduce(max)).collect() + with pytest.raises(ValueError): + Itr(groups).map(lambda g: Itr(g).nth(0)).collect() + with pytest.raises(ValueError): + Itr(groups).map(lambda g: Itr(g).peek()).collect() + assert Itr(groups).map(lambda g: Itr(g).position(lambda x: x > 1)).collect() == (1, None, 0) diff --git a/src/test/test_combine_split.py b/src/test/test_combine_split.py index f7fbb61..61b1b0f 100644 --- a/src/test/test_combine_split.py +++ b/src/test/test_combine_split.py @@ -1,3 +1,4 @@ +import itertools from typing import Never import pytest @@ -10,6 +11,11 @@ def test_chain() -> None: assert it.collect() == (1, 2, 3, 4) +def test_chain_many() -> None: + assert Itr([1]).chain([2], (), range(3, 5), "a").collect() == (1, 2, 3, 4, "a") + assert Itr([1, 2]).chain().collect() == (1, 2) + + def test_intersperse() -> None: it = Itr([1, 2, 3]).intersperse(0) assert it.collect() == (1, 0, 2, 0, 3) @@ -141,11 +147,24 @@ def pred(x: int) -> bool: it = Itr([1, 2, 3]) a, b = it.partition(pred) - # Both iterators should be independent and call the predicate as needed + assert calls == [] # lazy assert a.collect() == (3,) assert b.collect() == (1, 2) - # The predicate should have been called for all elements twice (once per filter) - assert calls == [1, 2, 3, 1, 2, 3] + # each item is tested exactly once, the non-matching ones being buffered for b + assert calls == [1, 2, 3] + + +def test_partition_interleaved_consumption() -> None: + evens, odds = Itr(range(10)).partition(lambda x: x % 2 == 0) + assert (evens.next(), odds.next(), odds.next(), evens.next()) == (0, 1, 3, 2) + assert evens.collect() == (4, 6, 8) + assert odds.collect() == (5, 7, 9) + + +def test_partition_infinite() -> None: + small, big = Itr(itertools.count()).partition(lambda x: x < 3) + assert big.take(3).collect() == (3, 4, 5) + assert small.take(3).collect() == (0, 1, 2) def test_repeat_basic() -> None: @@ -447,3 +466,26 @@ def test_zip_longest_right_longer_with_fillvalue() -> None: def test_zip_longest_both_empty() -> None: assert Itr[int]([]).zip_longest([]).collect() == () + + +def test_interleave_many() -> None: + assert Itr([1, 4, 7]).interleave([2, 5], [3]).collect() == (1, 2, 3, 4, 5, 7) + assert Itr[int]([]).interleave([1, 2], [], [3]).collect() == (1, 3, 2) + assert Itr([1, 2]).interleave().collect() == (1, 2) + + +def test_zip_strict() -> None: + assert Itr([1, 2]).zip("ab", strict=True).collect() == ((1, "a"), (2, "b")) + it = Itr([1, 2, 3]).zip("ab", strict=True) + # lazy: the mismatch is only detected when the shorter input runs out + assert it.next() == (1, "a") + with pytest.raises(ValueError): + it.collect() + assert Itr([1, 2, 3]).zip("ab").collect() == ((1, "a"), (2, "b")) + + +def test_compress() -> None: + assert Itr("abcde").compress([1, 0, 1, 0, 1]).collect() == ("a", "c", "e") + # stops at the shorter input + assert Itr("abcde").compress([True, True]).collect() == ("a", "b") + assert Itr(itertools.count()).compress(itertools.cycle([0, 0, 1])).take(3).collect() == (2, 5, 8) diff --git a/src/test/test_general.py b/src/test/test_general.py index b15726a..c51bfd1 100644 --- a/src/test/test_general.py +++ b/src/test/test_general.py @@ -62,3 +62,9 @@ def test_consume_on_empty_iterator_no_error() -> None: itr: Itr[None] = Itr(()) itr.consume() # should not raise assert itr.collect() == () + + +def test_repr_does_not_consume() -> None: + it = Itr([1, 2]).map(str) + assert repr(it).startswith("Itr( None: assert d[6] == ("banana", "carrot") +def test_groupby_unsorted() -> None: + it = Itr(["bb", "a", "ccc", "dd", "e"]).groupby(len, sort=False) + assert it.collect() == ((2, ("bb", "dd")), (1, ("a", "e")), (3, ("ccc",))) + + +def test_groupby_unsorted_needs_only_hashable_keys() -> None: + # None and int keys can't be sorted together, but can be hashed + data = [1, None, 2, None] + with pytest.raises(TypeError): + Itr(data).groupby(lambda x: x) + assert Itr(data).groupby(lambda x: x, sort=False).collect(dict) == {1: (1,), None: (None, None), 2: (2,)} + + def test_chunk_by() -> None: # consecutive runs only, order preserved, no sorting (unlike groupby) it = Itr([1, 1, 2, 3, 3, 1]).chunk_by(lambda x: x) @@ -311,3 +324,45 @@ def test_scan_can_yield_none_outputs() -> None: def test_scan_lazy_on_infinite() -> None: running = Itr(itertools.count(1)).scan(0, lambda total, x: (total + x, total + x)) assert running.take(4).collect() == (1, 3, 6, 10) + + +def test_sorted_by_no_key() -> None: + assert Itr([3, 1, 2]).sorted_by().collect() == (1, 2, 3) + assert Itr("cab").sorted_by(reverse=True).collect() == ("c", "b", "a") + + +def test_filter_map() -> None: + def parse(s: str) -> int | None: + return int(s) if s.isdigit() else None + + # only None is dropped, not other falsy values + assert Itr(["1", "x", "0", ""]).filter_map(parse).collect() == (1, 0) + assert Itr[str]([]).filter_map(parse).collect() == () + assert Itr(itertools.count()).filter_map(lambda n: n if n % 3 == 0 else None).take(3).collect() == (0, 3, 6) + + +def test_flat_map_lazy() -> None: + assert Itr([1, 2]).flat_map(lambda n: [n] * n).collect() == (1, 2, 2) + assert Itr(itertools.count()).flat_map(lambda n: (n, -n)).take(4).collect() == (0, 0, 1, -1) + + +def test_unique() -> None: + assert Itr([3, 1, 3, 2, 1]).unique().collect() == (3, 1, 2) + assert Itr[int]([]).unique().collect() == () + assert Itr(["apple", "avocado", "banana", "blueberry"]).unique(lambda s: s[0]).collect() == ("apple", "banana") + + +def test_unique_lazy_on_infinite() -> None: + assert Itr(itertools.count()).map(lambda n: n // 3).unique().take(4).collect() == (0, 1, 2, 3) + + +def test_unique_unhashable() -> None: + with pytest.raises(TypeError): + Itr([[1], [1]]).unique().collect() + assert Itr([[1], [2], [1]]).unique(tuple).collect() == ([1], [2]) + + +def test_enumerate() -> None: + assert Itr("abc").enumerate().collect() == ((0, "a"), (1, "b"), (2, "c")) + assert Itr("ab").enumerate(start=5).collect() == ((5, "a"), (6, "b")) + assert Itr[str]([]).enumerate().collect() == () From 0b4919ef1cb474fe677447dcbcea082179021708 Mon Sep 17 00:00:00 2001 From: virgesmith Date: Wed, 30 Sep 2026 08:39:41 +0100 Subject: [PATCH 2/4] Match builtin exceptions: nth raises IndexError, reduce raises TypeError Follow the builtins these methods stand in for, rather than using ValueError across the board: - nth past the end raises IndexError, as indexing a sequence does (a negative n remains a ValueError, as for itertools.islice) - reduce on an empty iterator raises TypeError, as functools.reduce does peek keeps ValueError: it has no builtin counterpart, and matches last/max/min. Co-Authored-By: Claude Opus 5.5 --- README.md | 2 +- doc/apidoc.md | 6 ++++-- doc/examples.ipynb | 4 ++-- relnotes.md | 4 +++- src/itrx/itr.py | 10 ++++++---- src/itrx/skill/SKILL.md | 11 +++++++---- src/test/test_aggregation.py | 2 +- src/test/test_collection.py | 6 +++--- 8 files changed, 27 insertions(+), 18 deletions(-) diff --git a/README.md b/README.md index 0ce626f..8a0ea95 100644 --- a/README.md +++ b/README.md @@ -107,7 +107,7 @@ When working with `Itr`, keep these points in mind: * **Single-Pass Iterators:** Like all Python iterators, `Itr` instances (and their underlying iterators) can generally only be consumed once. If you need to process the same sequence multiple times, use methods like `copy()`, `cycle()`, or `repeat()` as necessary. * **No Rewinding:** It's not possible to rewind an `Itr` to an earlier state. You can "preview" the next value using the `peek()` method, and conditionally consume it with `next_if()`. -* **Missing Items:** `find`, `find_map` and `position` return `None` when nothing matches, while `peek`, `nth`, `reduce`, `last`, `max`, `min` and `min_max` raise `ValueError` on an empty (or too short) iterator. Only `next()` raises `StopIteration`, like the builtin, so avoid calling it inside a `map` or `filter` callback: a `StopIteration` escaping a callback silently ends the enclosing iteration rather than raising. +* **Missing Items:** `find`, `find_map` and `position` return `None` when nothing matches, while `peek`, `last`, `max`, `min` and `min_max` raise `ValueError` on an empty iterator. As with their builtin counterparts, `nth` raises `IndexError` when the iterator is too short (like indexing a sequence), and `reduce` raises `TypeError` when it is empty (like `functools.reduce`). Only `next()` raises `StopIteration`, like the builtin, so avoid calling it inside a `map` or `filter` callback: a `StopIteration` escaping a callback silently ends the enclosing iteration rather than raising. * **Infinite Iterators:** Be cautious with open-ended iterators (e.g., those from `itertools.count()` or custom generators). Eager evaluation methods (like `collect()`, `count()`, `reduce()`) will attempt to consume the entire sequence, potentially leading to infinite loops or out-of-memory errors if applied to an infinite source. ## Agent skill diff --git a/doc/apidoc.md b/doc/apidoc.md index 5b09f27..5d54b6e 100644 --- a/doc/apidoc.md +++ b/doc/apidoc.md @@ -646,7 +646,8 @@ Returns: T: The n-th item. Raises: - ValueError: if n < 0, or the iterator has fewer than n + 1 items. + ValueError: if n < 0 + IndexError: if the iterator has fewer than n + 1 items (as when indexing a sequence past its end). Example: >>> Itr("abc").nth(1) @@ -773,7 +774,8 @@ Returns: T: The final reduced value. Raises: - ValueError: If the iterator is empty (use `fold` to supply an initial value instead). + TypeError: If the iterator is empty, as with `functools.reduce` (use `fold` to supply an initial value + instead). Example: >>> Itr([3, 1, 4]).reduce(max) diff --git a/doc/examples.ipynb b/doc/examples.ipynb index 648ff95..6908eb6 100644 --- a/doc/examples.ipynb +++ b/doc/examples.ipynb @@ -180,7 +180,7 @@ "metadata": {}, "source": [ "Some points to note:\n", - "- an `nth` call that overruns the sequence raises a `ValueError`\n", + "- an `nth` call that overruns the sequence raises an `IndexError`\n", "- a `skip` call that overruns the sequence raises a `StopIteration`\n", "- `step_by` yields an item and then steps (same behaviour as `itetools.islice`)\n", "- constructing `Itr` directly from a dictionary captures only the keys, use the `items` method to caputure key, \n", @@ -224,7 +224,7 @@ } ], "source": [ - "with contextlib.suppress(ValueError):\n", + "with contextlib.suppress(IndexError):\n", " Itr(collatz(4)).nth(10) # Collatz(4) = 4,2,1\n", "\n", "with contextlib.suppress(StopIteration):\n", diff --git a/relnotes.md b/relnotes.md index 10dd5d1..80e8123 100644 --- a/relnotes.md +++ b/relnotes.md @@ -4,7 +4,9 @@ - Terminal methods no longer raise `StopIteration` when an item is missing. A `StopIteration` escaping a `map`/`filter` callback silently ends the *enclosing* iteration, so e.g. `Itr(groups).map(lambda g: Itr(g).reduce(max))` used to drop every group after the first empty one without any error. Now: - `position` returns `None` when nothing matches (like `find`, and Rust's `Option`). - - `peek`, `nth` (index out of range) and `reduce` (empty input) raise `ValueError`, consistent with `last`, `max` and `min`. + - `nth` raises `IndexError` when the iterator is too short, like indexing a sequence past its end (a negative `n` is still a `ValueError`). + - `reduce` raises `TypeError` on an empty iterator, like `functools.reduce`. + - `peek` raises `ValueError` on an exhausted iterator, consistent with `last`, `max` and `min`. - `next()` still raises `StopIteration`, as the builtin does. - `partition` calls the predicate once per item rather than twice, buffering items for whichever side is not being read. Side-effecting predicates now see each item exactly once. diff --git a/src/itrx/itr.py b/src/itrx/itr.py index 493ba38..9224273 100644 --- a/src/itrx/itr.py +++ b/src/itrx/itr.py @@ -734,7 +734,8 @@ def nth(self, n: int) -> T: T: The n-th item. Raises: - ValueError: if n < 0, or the iterator has fewer than n + 1 items. + ValueError: if n < 0 + IndexError: if the iterator has fewer than n + 1 items (as when indexing a sequence past its end). Example: >>> Itr("abc").nth(1) @@ -745,7 +746,7 @@ def nth(self, n: int) -> T: try: return next(itertools.islice(self._it, n, None)) except StopIteration: - raise ValueError(f"nth({n}) is out of range: the iterator has fewer than {n + 1} items") from None + raise IndexError(f"nth({n}) is out of range: the iterator has fewer than {n + 1} items") from None def pairwise(self) -> "Itr[tuple[T, T]]": """Returns an iterator that yields consecutive pairs of elements from the iterable. @@ -882,7 +883,8 @@ def reduce(self, func: Callable[[T, T], T]) -> T: T: The final reduced value. Raises: - ValueError: If the iterator is empty (use `fold` to supply an initial value instead). + TypeError: If the iterator is empty, as with `functools.reduce` (use `fold` to supply an initial value + instead). Example: >>> Itr([3, 1, 4]).reduce(max) @@ -891,7 +893,7 @@ def reduce(self, func: Callable[[T, T], T]) -> T: try: first = next(self._it) except StopIteration: - raise ValueError("reduce() of an empty iterator") from None + raise TypeError("reduce() of an empty iterator with no initial value") from None return self.fold(first, func) def repeat(self, n: int) -> "Itr[T]": diff --git a/src/itrx/skill/SKILL.md b/src/itrx/skill/SKILL.md index 1078e46..19248f9 100644 --- a/src/itrx/skill/SKILL.md +++ b/src/itrx/skill/SKILL.md @@ -120,10 +120,13 @@ short-circuit on the first difference or inversion, so they too are safe on an i the returned ones. If one tee lags far behind another the buffer grows to hold the gap, so memory can blow up on a large or infinite source. `tee(n)` raises `ValueError` for `n < 1`. - **`nth(n)` is 0-based** (like Rust's `Iterator::nth`): `nth(0)` is the first item. It raises - `ValueError` for `n < 0` or if the iterator is shorter than `n + 1`. -- **Missing items: `None` or `ValueError`, and only `next()` raises `StopIteration`.** `find`, - `find_map` and `position` return `None` when nothing matches. `peek`, `nth`, `reduce`, `last`, - `max`, `min` and `min_max` raise `ValueError` on an empty (or too short) iterator. `next()` + `ValueError` for `n < 0`, and `IndexError` (as sequence indexing does) if the iterator is + shorter than `n + 1`. +- **Missing items: `None` or an ordinary exception; only `next()` raises `StopIteration`.** + `find`, `find_map` and `position` return `None` when nothing matches. `peek`, `last`, `max`, + `min` and `min_max` raise `ValueError` on an empty iterator; following their builtin + counterparts, `nth` raises `IndexError` and `reduce` raises `TypeError` (as `functools.reduce` + does). `next()` raises `StopIteration`, like the builtin — so don't call `.next()` inside a `map`/`filter` callback: a `StopIteration` escaping a callback silently ends the *enclosing* iteration instead of raising. Use `find`, `peek` or `nth(0)` there. diff --git a/src/test/test_aggregation.py b/src/test/test_aggregation.py index 9a8d34f..e08f758 100644 --- a/src/test/test_aggregation.py +++ b/src/test/test_aggregation.py @@ -73,7 +73,7 @@ def test_reduce_empty() -> None: # the lambda wont get called, result is just the single element assert it.reduce(lambda _a, _b: 0) == 1 # and again on exhausted iterator - with pytest.raises(ValueError, match="empty"): + with pytest.raises(TypeError, match="empty"): it.reduce(lambda _a, _b: 0) diff --git a/src/test/test_collection.py b/src/test/test_collection.py index ed9452b..996daaa 100644 --- a/src/test/test_collection.py +++ b/src/test/test_collection.py @@ -56,7 +56,7 @@ def test_nth() -> None: assert it.nth(0) == 10 # consumes preceding items, so this advances from the current position assert it.nth(2) == 40 - with pytest.raises(ValueError, match="out of range"): + with pytest.raises(IndexError, match="out of range"): it.nth(10) @@ -168,9 +168,9 @@ def test_take_last_invalid() -> None: def test_terminal_errors_do_not_truncate_enclosing_chain() -> None: # a StopIteration escaping a map callback would silently end the outer iteration instead of raising groups = [[1, 2], [], [3]] - with pytest.raises(ValueError): + with pytest.raises(TypeError): Itr(groups).map(lambda g: Itr(g).reduce(max)).collect() - with pytest.raises(ValueError): + with pytest.raises(IndexError): Itr(groups).map(lambda g: Itr(g).nth(0)).collect() with pytest.raises(ValueError): Itr(groups).map(lambda g: Itr(g).peek()).collect() From 34f8b6aee2db39305960af7426416bc7bfff0dcc Mon Sep 17 00:00:00 2001 From: virgesmith Date: Wed, 30 Sep 2026 08:49:23 +0100 Subject: [PATCH 3/4] Examples notebook: cover every Itr method, fix stale statements The notebook demonstrated 39 of the 66 public methods. Add short examples for the other 27, under the existing section headings: count/last/take_last, zip_longest, pairwise, chunk_by, filter_map, compress, accumulate/scan, dedup/dedup_with_count/unique, sorted_by, sum/prod/min/min_max, value_counts, find/position/find_map, eq, is_sorted, map_while and peek/next_if. Also show the new chain and interleave variadics and groupby(sort=False). Fix statements that were wrong: - groupby was described as grouping consecutive elements; it sorts first (that's chunk_by) - interleave was said to stop at the shorter input (not since 0.3.0) - all was said to always consume the entire iterator; it short-circuits - an overrunning skip was said to raise StopIteration; it's lazy and yields nothing - the nth(10) comment still assumed 1-based indexing Includes a note that running the notebook needs ipykernel. All cells re-executed, so outputs are current. Co-Authored-By: Claude Opus 5.5 --- doc/examples.ipynb | 1186 +++++++++++++++++++++++++++++++++++++++----- 1 file changed, 1049 insertions(+), 137 deletions(-) diff --git a/doc/examples.ipynb b/doc/examples.ipynb index 6908eb6..32c3220 100644 --- a/doc/examples.ipynb +++ b/doc/examples.ipynb @@ -9,6 +9,8 @@ "\n", "All the examples here make use of the generators/iterators defined next. One is open-ended and the other (very probably) not.\n", "\n", + "> Note: to run this notebook you may need to manually install `ipykernel`, as `itrx` does not declare it as a dependency. \n", + "\n", "### Fibonacci Generator and Iterator\n", "\n", "Two implementations of Fibonacci sequences: a generator and an iterator class.\n" @@ -16,9 +18,16 @@ }, { "cell_type": "code", - "execution_count": 13, + "execution_count": 1, "id": "8b2541fb", - "metadata": {}, + "metadata": { + "execution": { + "iopub.execute_input": "2026-09-30T07:49:05.797286Z", + "iopub.status.busy": "2026-09-30T07:49:05.797003Z", + "iopub.status.idle": "2026-09-30T07:49:05.805295Z", + "shell.execute_reply": "2026-09-30T07:49:05.804056Z" + } + }, "outputs": [], "source": [ "import contextlib\n", @@ -31,9 +40,16 @@ }, { "cell_type": "code", - "execution_count": 14, + "execution_count": 2, "id": "fc89902b", - "metadata": {}, + "metadata": { + "execution": { + "iopub.execute_input": "2026-09-30T07:49:05.808276Z", + "iopub.status.busy": "2026-09-30T07:49:05.807960Z", + "iopub.status.idle": "2026-09-30T07:49:05.813772Z", + "shell.execute_reply": "2026-09-30T07:49:05.812888Z" + } + }, "outputs": [], "source": [ "def fibonacci() -> Generator[int, None, None]:\n", @@ -74,9 +90,16 @@ }, { "cell_type": "code", - "execution_count": 15, + "execution_count": 3, "id": "d9af20ad", - "metadata": {}, + "metadata": { + "execution": { + "iopub.execute_input": "2026-09-30T07:49:05.816054Z", + "iopub.status.busy": "2026-09-30T07:49:05.815844Z", + "iopub.status.idle": "2026-09-30T07:49:05.819988Z", + "shell.execute_reply": "2026-09-30T07:49:05.819055Z" + } + }, "outputs": [], "source": [ "def collatz(n: int) -> Generator[int, None, None]:\n", @@ -105,9 +128,16 @@ }, { "cell_type": "code", - "execution_count": 16, + "execution_count": 4, "id": "83d002b5", - "metadata": {}, + "metadata": { + "execution": { + "iopub.execute_input": "2026-09-30T07:49:05.822186Z", + "iopub.status.busy": "2026-09-30T07:49:05.821977Z", + "iopub.status.idle": "2026-09-30T07:49:05.830525Z", + "shell.execute_reply": "2026-09-30T07:49:05.829796Z" + } + }, "outputs": [ { "data": { @@ -130,10 +160,10 @@ { "data": { "text/plain": [ - "514229" + "832040" ] }, - "execution_count": 16, + "execution_count": 4, "metadata": {}, "output_type": "execute_result" } @@ -142,7 +172,7 @@ "f = Itr(Fibonacci())\n", "display(f.take(10).collect(set))\n", "display(f.next_chunk(10))\n", - "f.nth(10) # 30th value (1-indexed)" + "f.nth(10) # index 30 (0-based), as 20 items have already been consumed" ] }, { @@ -155,9 +185,16 @@ }, { "cell_type": "code", - "execution_count": 17, + "execution_count": 5, "id": "99c944c1", - "metadata": {}, + "metadata": { + "execution": { + "iopub.execute_input": "2026-09-30T07:49:05.832790Z", + "iopub.status.busy": "2026-09-30T07:49:05.832495Z", + "iopub.status.idle": "2026-09-30T07:49:05.837257Z", + "shell.execute_reply": "2026-09-30T07:49:05.836288Z" + } + }, "outputs": [ { "data": { @@ -165,7 +202,7 @@ "(19, 58, 29, 88, 44, 22, 11, 34, 17, 52, 26, 13, 40, 20, 10, 5, 16, 8, 4, 2, 1)" ] }, - "execution_count": 17, + "execution_count": 5, "metadata": {}, "output_type": "execute_result" } @@ -174,6 +211,62 @@ "Itr(collatz(19)).collect()" ] }, + { + "cell_type": "markdown", + "id": "7e660ef6", + "metadata": {}, + "source": [ + "`count`, `last` and `take_last` all have to reach the end of the iterator, so only use them on bounded ones. `take_last` only ever holds on to the last `n` items:" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "b97eef6e", + "metadata": { + "execution": { + "iopub.execute_input": "2026-09-30T07:49:05.839324Z", + "iopub.status.busy": "2026-09-30T07:49:05.839132Z", + "iopub.status.idle": "2026-09-30T07:49:05.846361Z", + "shell.execute_reply": "2026-09-30T07:49:05.845348Z" + } + }, + "outputs": [ + { + "data": { + "text/plain": [ + "21" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "1" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "(8, 4, 2, 1)" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "display(Itr(collatz(19)).count())\n", + "display(Itr(collatz(19)).last())\n", + "Itr(collatz(19)).take_last(4).collect()" + ] + }, { "cell_type": "markdown", "id": "f81c9465", @@ -181,20 +274,36 @@ "source": [ "Some points to note:\n", "- an `nth` call that overruns the sequence raises an `IndexError`\n", - "- a `skip` call that overruns the sequence raises a `StopIteration`\n", - "- `step_by` yields an item and then steps (same behaviour as `itetools.islice`)\n", - "- constructing `Itr` directly from a dictionary captures only the keys, use the `items` method to caputure key, \n", + "- `skip` is lazy, so a `skip` that overruns the sequence doesn't raise anything: it just yields nothing\n", + "- `step_by` yields an item and then steps (same behaviour as `itertools.islice`)\n", + "- constructing `Itr` directly from a dictionary captures only the keys, use the `items` method to capture key,\n", "value tuples.\n", - "- `iter` and `next`: `iter(itr)` is equivalent and interchangeable with `itr.__iter__()`. Same applies to `next(itr)` \n", - "and `itr.next()` or `itr.__next__()`\n" + "- `iter` and `next`: `iter(itr)` is equivalent and interchangeable with `itr.__iter__()`. Same applies to `next(itr)`\n", + "and `itr.next()` or `itr.__next__()`" ] }, { "cell_type": "code", - "execution_count": 18, + "execution_count": 7, "id": "fd47677f", - "metadata": {}, + "metadata": { + "execution": { + "iopub.execute_input": "2026-09-30T07:49:05.848729Z", + "iopub.status.busy": "2026-09-30T07:49:05.848506Z", + "iopub.status.idle": "2026-09-30T07:49:05.857960Z", + "shell.execute_reply": "2026-09-30T07:49:05.856772Z" + } + }, "outputs": [ + { + "data": { + "text/plain": [ + "()" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, { "data": { "text/plain": [ @@ -227,8 +336,7 @@ "with contextlib.suppress(IndexError):\n", " Itr(collatz(4)).nth(10) # Collatz(4) = 4,2,1\n", "\n", - "with contextlib.suppress(StopIteration):\n", - " Itr(collatz(4)).skip(10)\n", + "display(Itr(collatz(4)).skip(10).collect())\n", "\n", "display(Itr(collatz(19)).step_by(3).collect())\n", "\n", @@ -251,9 +359,16 @@ }, { "cell_type": "code", - "execution_count": 19, + "execution_count": 8, "id": "8bcc535f", - "metadata": {}, + "metadata": { + "execution": { + "iopub.execute_input": "2026-09-30T07:49:05.860240Z", + "iopub.status.busy": "2026-09-30T07:49:05.860040Z", + "iopub.status.idle": "2026-09-30T07:49:05.864731Z", + "shell.execute_reply": "2026-09-30T07:49:05.863996Z" + } + }, "outputs": [ { "name": "stdout", @@ -287,26 +402,28 @@ }, { "cell_type": "code", - "execution_count": 20, + "execution_count": 9, "id": "4c33c79c", - "metadata": {}, + "metadata": { + "execution": { + "iopub.execute_input": "2026-09-30T07:49:05.866327Z", + "iopub.status.busy": "2026-09-30T07:49:05.866182Z", + "iopub.status.idle": "2026-09-30T07:49:05.869308Z", + "shell.execute_reply": "2026-09-30T07:49:05.868736Z" + } + }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "1\n", - "8\n", - "10\n", - "13\n", - "17\n", "22\n", "29\n" ] } ], "source": [ - "Itr(collatz(19)).rev().step_by(3).inspect(print).skip(5).map(lambda x: x * x).filter(lambda x: x % 2 == 0).consume()" + "Itr(collatz(19)).rev().step_by(3).skip(5).inspect(print).map(lambda x: x * x).filter(lambda x: x % 2 == 0).consume()" ] }, { @@ -314,34 +431,39 @@ "id": "0c44a1cf", "metadata": {}, "source": [ - "\n", "### Combining and Splitting Iterators\n", "\n", "Methods for combining, splitting, or transforming multiple iterators.\n", "\n", - "`chain` concatenates two iterables:\n", - "\n" + "`chain` concatenates two or more iterables:" ] }, { "cell_type": "code", - "execution_count": 21, + "execution_count": 10, "id": "d9b906a6", - "metadata": {}, + "metadata": { + "execution": { + "iopub.execute_input": "2026-09-30T07:49:05.870629Z", + "iopub.status.busy": "2026-09-30T07:49:05.870505Z", + "iopub.status.idle": "2026-09-30T07:49:05.874471Z", + "shell.execute_reply": "2026-09-30T07:49:05.873649Z" + } + }, "outputs": [ { "data": { "text/plain": [ - "(5, 16, 8, 4, 2, 1, 6, 3, 10, 5, 16, 8, 4, 2, 1)" + "(5, 16, 8, 4, 2, 1, 6, 3, 10, 5, 16, 8, 4, 2, 1, 0)" ] }, - "execution_count": 21, + "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ - "Itr(collatz(5)).chain(collatz(6)).collect()" + "Itr(collatz(5)).chain(collatz(6), [0]).collect()" ] }, { @@ -354,9 +476,16 @@ }, { "cell_type": "code", - "execution_count": 22, + "execution_count": 11, "id": "377cd2c0", - "metadata": {}, + "metadata": { + "execution": { + "iopub.execute_input": "2026-09-30T07:49:05.875909Z", + "iopub.status.busy": "2026-09-30T07:49:05.875792Z", + "iopub.status.idle": "2026-09-30T07:49:05.878928Z", + "shell.execute_reply": "2026-09-30T07:49:05.878415Z" + } + }, "outputs": [ { "data": { @@ -364,7 +493,7 @@ "((5, 6), (16, 3), (8, 10), (4, 5), (2, 16), (1, 8))" ] }, - "execution_count": 22, + "execution_count": 11, "metadata": {}, "output_type": "execute_result" } @@ -373,6 +502,42 @@ "Itr(collatz(5)).zip(collatz(6)).collect()" ] }, + { + "cell_type": "markdown", + "id": "19db0063", + "metadata": {}, + "source": [ + "`zip_longest` carries on to the end of the longer input, padding the shorter one with `fillvalue`. (Alternatively, `zip(..., strict=True)` raises a `ValueError` if the lengths differ.)" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "id": "94bd9fb8", + "metadata": { + "execution": { + "iopub.execute_input": "2026-09-30T07:49:05.880225Z", + "iopub.status.busy": "2026-09-30T07:49:05.880118Z", + "iopub.status.idle": "2026-09-30T07:49:05.883376Z", + "shell.execute_reply": "2026-09-30T07:49:05.882724Z" + } + }, + "outputs": [ + { + "data": { + "text/plain": [ + "((5, 6), (16, 3), (8, 10), (4, 5), (2, 16), (1, 8), (0, 4), (0, 2), (0, 1))" + ] + }, + "execution_count": 12, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "Itr(collatz(5)).zip_longest(collatz(6), fillvalue=0).collect()" + ] + }, { "cell_type": "markdown", "id": "2e94d3db", @@ -384,9 +549,16 @@ }, { "cell_type": "code", - "execution_count": 23, + "execution_count": 13, "id": "03e8e659", - "metadata": {}, + "metadata": { + "execution": { + "iopub.execute_input": "2026-09-30T07:49:05.884898Z", + "iopub.status.busy": "2026-09-30T07:49:05.884776Z", + "iopub.status.idle": "2026-09-30T07:49:05.888310Z", + "shell.execute_reply": "2026-09-30T07:49:05.887551Z" + } + }, "outputs": [ { "data": { @@ -394,7 +566,7 @@ "((5, 16, 8, 4, 2, 1), (6, 3, 10, 5, 16, 8))" ] }, - "execution_count": 23, + "execution_count": 13, "metadata": {}, "output_type": "execute_result" } @@ -414,9 +586,16 @@ }, { "cell_type": "code", - "execution_count": 24, + "execution_count": 14, "id": "c3b77ea4", - "metadata": {}, + "metadata": { + "execution": { + "iopub.execute_input": "2026-09-30T07:49:05.889828Z", + "iopub.status.busy": "2026-09-30T07:49:05.889693Z", + "iopub.status.idle": "2026-09-30T07:49:05.893010Z", + "shell.execute_reply": "2026-09-30T07:49:05.892379Z" + } + }, "outputs": [ { "data": { @@ -424,7 +603,7 @@ "(8, 'abcde', 4, 'abcde', 2, 'abcde', 1)" ] }, - "execution_count": 24, + "execution_count": 14, "metadata": {}, "output_type": "execute_result" } @@ -438,14 +617,21 @@ "id": "cbf61517", "metadata": {}, "source": [ - "`interleave` alternates values from two sequences (and ends when either sequence is exhausted):" + "`interleave` takes one item from each of its inputs in turn. Once an input is exhausted it is skipped, so the remainder of the longer inputs is still yielded:" ] }, { "cell_type": "code", - "execution_count": 25, + "execution_count": 15, "id": "ed21cfb9", - "metadata": {}, + "metadata": { + "execution": { + "iopub.execute_input": "2026-09-30T07:49:05.894263Z", + "iopub.status.busy": "2026-09-30T07:49:05.894155Z", + "iopub.status.idle": "2026-09-30T07:49:05.898162Z", + "shell.execute_reply": "2026-09-30T07:49:05.897607Z" + } + }, "outputs": [ { "data": { @@ -453,13 +639,23 @@ "(5, 1, 16, 2, 8, 4, 4, 8, 2, 16, 1, 5)" ] }, - "execution_count": 25, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "(3, 4, 0, 10, 2, 5, 1, 16, 8, 4, 2, 1)" + ] + }, + "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ - "Itr(collatz(5)).interleave(Itr(collatz(5)).rev()).collect()" + "display(Itr(collatz(5)).interleave(Itr(collatz(5)).rev()).collect())\n", + "Itr(collatz(3)).interleave(collatz(4), [0]).collect()" ] }, { @@ -474,9 +670,16 @@ }, { "cell_type": "code", - "execution_count": 26, + "execution_count": 16, "id": "53be0e6a", - "metadata": {}, + "metadata": { + "execution": { + "iopub.execute_input": "2026-09-30T07:49:05.899637Z", + "iopub.status.busy": "2026-09-30T07:49:05.899499Z", + "iopub.status.idle": "2026-09-30T07:49:05.902886Z", + "shell.execute_reply": "2026-09-30T07:49:05.902391Z" + } + }, "outputs": [ { "data": { @@ -484,7 +687,7 @@ "((1, 5), (2, 16), (3, 8), (4, 4), (5, 2), (6, 1))" ] }, - "execution_count": 26, + "execution_count": 16, "metadata": {}, "output_type": "execute_result" } @@ -508,9 +711,16 @@ }, { "cell_type": "code", - "execution_count": 27, + "execution_count": 17, "id": "bb9a323a", - "metadata": {}, + "metadata": { + "execution": { + "iopub.execute_input": "2026-09-30T07:49:05.904131Z", + "iopub.status.busy": "2026-09-30T07:49:05.904010Z", + "iopub.status.idle": "2026-09-30T07:49:05.907779Z", + "shell.execute_reply": "2026-09-30T07:49:05.907102Z" + } + }, "outputs": [ { "data": { @@ -518,7 +728,7 @@ "(5, 8, 4)" ] }, - "execution_count": 27, + "execution_count": 17, "metadata": {}, "output_type": "execute_result" } @@ -537,9 +747,16 @@ }, { "cell_type": "code", - "execution_count": 28, + "execution_count": 18, "id": "87e4444d", - "metadata": {}, + "metadata": { + "execution": { + "iopub.execute_input": "2026-09-30T07:49:05.909208Z", + "iopub.status.busy": "2026-09-30T07:49:05.909095Z", + "iopub.status.idle": "2026-09-30T07:49:05.911884Z", + "shell.execute_reply": "2026-09-30T07:49:05.911393Z" + } + }, "outputs": [ { "name": "stdout", @@ -567,9 +784,16 @@ }, { "cell_type": "code", - "execution_count": 29, + "execution_count": 19, "id": "c2199b97", - "metadata": {}, + "metadata": { + "execution": { + "iopub.execute_input": "2026-09-30T07:49:05.913132Z", + "iopub.status.busy": "2026-09-30T07:49:05.913026Z", + "iopub.status.idle": "2026-09-30T07:49:05.917930Z", + "shell.execute_reply": "2026-09-30T07:49:05.917301Z" + } + }, "outputs": [ { "data": { @@ -618,9 +842,16 @@ }, { "cell_type": "code", - "execution_count": 30, + "execution_count": 20, "id": "1e6f6f75", - "metadata": {}, + "metadata": { + "execution": { + "iopub.execute_input": "2026-09-30T07:49:05.919122Z", + "iopub.status.busy": "2026-09-30T07:49:05.919002Z", + "iopub.status.idle": "2026-09-30T07:49:05.922559Z", + "shell.execute_reply": "2026-09-30T07:49:05.921718Z" + } + }, "outputs": [ { "data": { @@ -628,7 +859,7 @@ "(1, 2, 0, 0, 0, 0)" ] }, - "execution_count": 30, + "execution_count": 20, "metadata": {}, "output_type": "execute_result" } @@ -649,9 +880,16 @@ }, { "cell_type": "code", - "execution_count": 31, + "execution_count": 21, "id": "6c9cb5c1", - "metadata": {}, + "metadata": { + "execution": { + "iopub.execute_input": "2026-09-30T07:49:05.924610Z", + "iopub.status.busy": "2026-09-30T07:49:05.924478Z", + "iopub.status.idle": "2026-09-30T07:49:05.927637Z", + "shell.execute_reply": "2026-09-30T07:49:05.927194Z" + } + }, "outputs": [ { "data": { @@ -659,7 +897,7 @@ "(5, 17, 9, 6, 5, 6)" ] }, - "execution_count": 31, + "execution_count": 21, "metadata": {}, "output_type": "execute_result" } @@ -687,9 +925,16 @@ }, { "cell_type": "code", - "execution_count": 32, + "execution_count": 22, "id": "9d9bed75", - "metadata": {}, + "metadata": { + "execution": { + "iopub.execute_input": "2026-09-30T07:49:05.929194Z", + "iopub.status.busy": "2026-09-30T07:49:05.929093Z", + "iopub.status.idle": "2026-09-30T07:49:05.932961Z", + "shell.execute_reply": "2026-09-30T07:49:05.932421Z" + } + }, "outputs": [ { "data": { @@ -718,96 +963,323 @@ }, { "cell_type": "markdown", - "id": "55fe650d", + "id": "8a8be861", "metadata": {}, "source": [ - "`partition` splits an iterator into two based on a predicate. The original iterator is consumed." + "`filter_map` maps and drops any `None` results in one step (only `None`: other falsy values such as `0` are kept). `compress` keeps the items whose corresponding selector is truthy:" ] }, { "cell_type": "code", - "execution_count": 33, - "id": "ad83ad71", - "metadata": {}, + "execution_count": 23, + "id": "73492831", + "metadata": { + "execution": { + "iopub.execute_input": "2026-09-30T07:49:05.935057Z", + "iopub.status.busy": "2026-09-30T07:49:05.934922Z", + "iopub.status.idle": "2026-09-30T07:49:05.939277Z", + "shell.execute_reply": "2026-09-30T07:49:05.938632Z" + } + }, "outputs": [ { "data": { "text/plain": [ - "((0, 2, 8, 34, 144, 610, 2584),\n", - " (1, 1, 3, 5, 13, 21, 55, 89, 233, 377, 987, 1597, 4181),\n", - " ())" + "(5, 8, 4, 3, 5, 4)" ] }, - "execution_count": 33, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "('b', 'c', 'e', 'f', 'h')" + ] + }, + "execution_count": 23, "metadata": {}, "output_type": "execute_result" } ], "source": [ - "it0 = Itr(fibonacci()).take(20)\n", - "it1, it2 = it0.partition(lambda x: x % 2 == 0)\n", - "# it0 is consumed when it1 or it2 is accessed\n", - "it1.collect(), it2.collect(), it0.collect()" + "display(Itr(collatz(19)).filter_map(lambda n: n // 10 if n > 30 else None).collect())\n", + "Itr(\"abcdefgh\").compress(Itr(fibonacci()).map(lambda n: n % 2)).collect()" ] }, { "cell_type": "markdown", - "id": "64087520", + "id": "8cf50200", "metadata": {}, "source": [ - "`copy` creates a shallow copy of the iterator's state:" + "`accumulate` yields running totals (or the running result of any binary function). `scan` generalises it: the function carries a separate state, and returning `None` stops the iteration:" ] }, { "cell_type": "code", - "execution_count": 34, - "id": "b52225da", - "metadata": {}, + "execution_count": 24, + "id": "e13c1934", + "metadata": { + "execution": { + "iopub.execute_input": "2026-09-30T07:49:05.940937Z", + "iopub.status.busy": "2026-09-30T07:49:05.940828Z", + "iopub.status.idle": "2026-09-30T07:49:05.945310Z", + "shell.execute_reply": "2026-09-30T07:49:05.944823Z" + } + }, "outputs": [ { "data": { "text/plain": [ - "(10, 5)" + "(6, 9, 19, 24, 40, 48, 52, 54, 55)" ] }, - "execution_count": 34, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "(6, 6, 10, 10, 16, 16, 16, 16, 16)" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "(6, 9, 19, 24, 40, 48)" + ] + }, + "execution_count": 24, "metadata": {}, "output_type": "execute_result" } ], "source": [ - "it0 = Itr(collatz(10))\n", - "it1 = it0.copy()\n", - "next(it0) # Advance it0 (only)\n", - "next(it1), next(it0)" + "display(Itr(collatz(6)).accumulate().collect())\n", + "display(Itr(collatz(6)).accumulate(max).collect())\n", + "# running total, stopping before it exceeds 50\n", + "Itr(collatz(6)).scan(0, lambda total, n: None if total + n > 50 else (total + n, total + n)).collect()" ] }, { "cell_type": "markdown", - "id": "07b9bc9d", + "id": "7b27821f", "metadata": {}, "source": [ - "`tee` can be used to create multiple independent copies of the input iterator. But note that consuming *any* of the outputs consumes the original iterator." + "`dedup` removes *consecutive* duplicates, `dedup_with_count` also counts them (run-length encoding), and `unique` removes duplicates wherever they occur. All three are lazy and preserve order:" ] }, { "cell_type": "code", - "execution_count": 35, - "id": "ace74e50", - "metadata": {}, + "execution_count": 25, + "id": "1c35f403", + "metadata": { + "execution": { + "iopub.execute_input": "2026-09-30T07:49:05.946568Z", + "iopub.status.busy": "2026-09-30T07:49:05.946449Z", + "iopub.status.idle": "2026-09-30T07:49:05.951115Z", + "shell.execute_reply": "2026-09-30T07:49:05.950579Z" + } + }, "outputs": [ { "data": { "text/plain": [ - "1" + "(1, 2, 3, 1, 2)" ] }, - "execution_count": 35, "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "((1, 2), (2, 1), (3, 3), (1, 1), (2, 2))" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "(1, 2, 3)" + ] + }, + "execution_count": 25, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "data = [1, 1, 2, 3, 3, 3, 1, 2, 2]\n", + "display(Itr(data).dedup().collect())\n", + "display(Itr(data).dedup_with_count().collect())\n", + "Itr(data).unique().collect()" + ] + }, + { + "cell_type": "markdown", + "id": "28e5c0d4", + "metadata": {}, + "source": [ + "`sorted_by` sorts the items (eagerly) by a key, or by the items themselves if no key is given:" + ] + }, + { + "cell_type": "code", + "execution_count": 26, + "id": "f23f3b36", + "metadata": { + "execution": { + "iopub.execute_input": "2026-09-30T07:49:05.952818Z", + "iopub.status.busy": "2026-09-30T07:49:05.952686Z", + "iopub.status.idle": "2026-09-30T07:49:05.957488Z", + "shell.execute_reply": "2026-09-30T07:49:05.956936Z" + } + }, + "outputs": [ + { + "data": { + "text/plain": [ + "(1, 2, 4, 5, 7, 8, 10, 11, 13, 16, 17, 20, 22, 26, 34, 40, 52)" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "('ccc', 'bb', 'a')" + ] + }, + "execution_count": 26, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "display(Itr(collatz(7)).sorted_by().collect())\n", + "Itr([\"ccc\", \"a\", \"bb\"]).sorted_by(len, reverse=True).collect()" + ] + }, + { + "cell_type": "markdown", + "id": "55fe650d", + "metadata": {}, + "source": [ + "`partition` splits an iterator into two based on a predicate. The original iterator is consumed." + ] + }, + { + "cell_type": "code", + "execution_count": 27, + "id": "ad83ad71", + "metadata": { + "execution": { + "iopub.execute_input": "2026-09-30T07:49:05.959584Z", + "iopub.status.busy": "2026-09-30T07:49:05.959447Z", + "iopub.status.idle": "2026-09-30T07:49:05.962931Z", + "shell.execute_reply": "2026-09-30T07:49:05.962486Z" + } + }, + "outputs": [ + { + "data": { + "text/plain": [ + "((0, 2, 8, 34, 144, 610, 2584),\n", + " (1, 1, 3, 5, 13, 21, 55, 89, 233, 377, 987, 1597, 4181),\n", + " ())" + ] + }, + "execution_count": 27, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "it0 = Itr(fibonacci()).take(20)\n", + "it1, it2 = it0.partition(lambda x: x % 2 == 0)\n", + "# it0 is consumed when it1 or it2 is accessed\n", + "it1.collect(), it2.collect(), it0.collect()" + ] + }, + { + "cell_type": "markdown", + "id": "64087520", + "metadata": {}, + "source": [ + "`copy` creates a shallow copy of the iterator's state:" + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "id": "b52225da", + "metadata": { + "execution": { + "iopub.execute_input": "2026-09-30T07:49:05.964709Z", + "iopub.status.busy": "2026-09-30T07:49:05.964604Z", + "iopub.status.idle": "2026-09-30T07:49:05.967580Z", + "shell.execute_reply": "2026-09-30T07:49:05.967022Z" + } + }, + "outputs": [ + { + "data": { + "text/plain": [ + "(10, 5)" + ] + }, + "execution_count": 28, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "it0 = Itr(collatz(10))\n", + "it1 = it0.copy()\n", + "next(it0) # Advance it0 (only)\n", + "next(it1), next(it0)" + ] + }, + { + "cell_type": "markdown", + "id": "07b9bc9d", + "metadata": {}, + "source": [ + "`tee` can be used to create multiple independent copies of the input iterator. But note that consuming *any* of the outputs consumes the original iterator." + ] + }, + { + "cell_type": "code", + "execution_count": 29, + "id": "ace74e50", + "metadata": { + "execution": { + "iopub.execute_input": "2026-09-30T07:49:05.968852Z", + "iopub.status.busy": "2026-09-30T07:49:05.968724Z", + "iopub.status.idle": "2026-09-30T07:49:05.972356Z", + "shell.execute_reply": "2026-09-30T07:49:05.971618Z" + } + }, + "outputs": [ + { + "data": { + "text/plain": [ + "1" + ] + }, + "execution_count": 29, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ "original = Itr([1, 2, 3])\n", "\n", "a, b = original.tee(2)\n", @@ -830,9 +1302,16 @@ }, { "cell_type": "code", - "execution_count": 36, + "execution_count": 30, "id": "7fcc7a00", - "metadata": {}, + "metadata": { + "execution": { + "iopub.execute_input": "2026-09-30T07:49:05.974020Z", + "iopub.status.busy": "2026-09-30T07:49:05.973902Z", + "iopub.status.idle": "2026-09-30T07:49:05.977265Z", + "shell.execute_reply": "2026-09-30T07:49:05.976758Z" + } + }, "outputs": [ { "data": { @@ -843,7 +1322,7 @@ " (4, 2, 1))" ] }, - "execution_count": 36, + "execution_count": 30, "metadata": {}, "output_type": "execute_result" } @@ -858,27 +1337,44 @@ "id": "0382ee2b", "metadata": {}, "source": [ - "`pairwise` and `rolling`: generate rolling windows of elements:" + "`pairwise` and `rolling` generate (overlapping) windows of consecutive elements:" ] }, { "cell_type": "code", - "execution_count": 37, + "execution_count": 31, "id": "6f2d5115", - "metadata": {}, + "metadata": { + "execution": { + "iopub.execute_input": "2026-09-30T07:49:05.978635Z", + "iopub.status.busy": "2026-09-30T07:49:05.978512Z", + "iopub.status.idle": "2026-09-30T07:49:05.982404Z", + "shell.execute_reply": "2026-09-30T07:49:05.981950Z" + } + }, "outputs": [ + { + "data": { + "text/plain": [ + "((10, 5), (5, 16), (16, 8), (8, 4), (4, 2), (2, 1))" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, { "data": { "text/plain": [ "((10, 5, 16), (5, 16, 8), (16, 8, 4), (8, 4, 2), (4, 2, 1))" ] }, - "execution_count": 37, + "execution_count": 31, "metadata": {}, "output_type": "execute_result" } ], "source": [ + "display(Itr(collatz(10)).pairwise().collect())\n", "Itr(collatz(10)).rolling(3).collect()" ] }, @@ -887,14 +1383,21 @@ "id": "bd3cced9", "metadata": {}, "source": [ - "`groupby` groups consecutive elements by a key. It's often convenient to `collect` into a `dict`:" + "`groupby` sorts the input by the key and then groups *all* the items sharing a key (like pandas' `groupby`), so it consumes the whole iterator. It's often convenient to `collect` into a `dict`. Pass `sort=False` to order the groups by first appearance instead:" ] }, { "cell_type": "code", - "execution_count": 38, + "execution_count": 32, "id": "cce14801", - "metadata": {}, + "metadata": { + "execution": { + "iopub.execute_input": "2026-09-30T07:49:05.984148Z", + "iopub.status.busy": "2026-09-30T07:49:05.984010Z", + "iopub.status.idle": "2026-09-30T07:49:05.987935Z", + "shell.execute_reply": "2026-09-30T07:49:05.987395Z" + } + }, "outputs": [ { "data": { @@ -903,13 +1406,64 @@ " 2: (29, 44, 11, 17, 26, 20, 5, 8, 2)}" ] }, - "execution_count": 38, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "((0, 1, 2, 3, 4), (4, 3, 2, 1, 0))" + ] + }, + "execution_count": 32, "metadata": {}, "output_type": "execute_result" } ], "source": [ - "Itr(collatz(19)).groupby(lambda n: n % 3).collect(dict)" + "display(Itr(collatz(19)).groupby(lambda n: n % 3).collect(dict))\n", + "# the group keys in sorted order, and in order of first appearance\n", + "keys = Itr(collatz(19)).map(lambda n: n % 5)\n", + "(\n", + " keys.copy().groupby(lambda k: k).map(lambda kv: kv[0]).collect(),\n", + " keys.groupby(lambda k: k, sort=False).map(lambda kv: kv[0]).collect(),\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "a7201ec2", + "metadata": {}, + "source": [ + "`chunk_by` instead groups *consecutive* runs of items sharing a key, without sorting. This is what `itertools.groupby` does, and it's lazy, so works on unbounded sequences:" + ] + }, + { + "cell_type": "code", + "execution_count": 33, + "id": "8b7b9bb2", + "metadata": { + "execution": { + "iopub.execute_input": "2026-09-30T07:49:05.990006Z", + "iopub.status.busy": "2026-09-30T07:49:05.989850Z", + "iopub.status.idle": "2026-09-30T07:49:05.992683Z", + "shell.execute_reply": "2026-09-30T07:49:05.992181Z" + } + }, + "outputs": [ + { + "data": { + "text/plain": [ + "((0, (0,)), (1, (1, 1)), (0, (2,)), (1, (3, 5)), (0, (8,)), (1, (13, 21)))" + ] + }, + "execution_count": 33, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "Itr(fibonacci()).chunk_by(lambda n: n % 2).take(6).collect()" ] }, { @@ -928,9 +1482,16 @@ }, { "cell_type": "code", - "execution_count": 39, + "execution_count": 34, "id": "2ba56157", - "metadata": {}, + "metadata": { + "execution": { + "iopub.execute_input": "2026-09-30T07:49:05.993986Z", + "iopub.status.busy": "2026-09-30T07:49:05.993882Z", + "iopub.status.idle": "2026-09-30T07:49:05.996669Z", + "shell.execute_reply": "2026-09-30T07:49:05.996150Z" + } + }, "outputs": [ { "data": { @@ -938,7 +1499,7 @@ "(46, 46)" ] }, - "execution_count": 39, + "execution_count": 34, "metadata": {}, "output_type": "execute_result" } @@ -947,29 +1508,154 @@ "Itr(collatz(10)).fold(0, add), Itr(collatz(10)).reduce(add)" ] }, + { + "cell_type": "markdown", + "id": "d879d189", + "metadata": {}, + "source": [ + "`sum` and `prod` add or multiply the elements (both accept a `start` value), `count` counts them, and `min_max` finds both extremes in a single pass:" + ] + }, + { + "cell_type": "code", + "execution_count": 35, + "id": "37068d97", + "metadata": { + "execution": { + "iopub.execute_input": "2026-09-30T07:49:05.998276Z", + "iopub.status.busy": "2026-09-30T07:49:05.998121Z", + "iopub.status.idle": "2026-09-30T07:49:06.004349Z", + "shell.execute_reply": "2026-09-30T07:49:06.003870Z" + } + }, + "outputs": [ + { + "data": { + "text/plain": [ + "46" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "51200" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "7" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "1" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "9232" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "(1, 9232)" + ] + }, + "execution_count": 35, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "display(Itr(collatz(10)).sum(), Itr(collatz(10)).prod(), Itr(collatz(10)).count())\n", + "display(Itr(collatz(27)).min(), Itr(collatz(27)).max())\n", + "Itr(collatz(27)).min_max()" + ] + }, + { + "cell_type": "markdown", + "id": "e25e2f12", + "metadata": {}, + "source": [ + "`value_counts` counts how often each distinct value occurs, most common first (like pandas):" + ] + }, + { + "cell_type": "code", + "execution_count": 36, + "id": "4092d397", + "metadata": { + "execution": { + "iopub.execute_input": "2026-09-30T07:49:06.006578Z", + "iopub.status.busy": "2026-09-30T07:49:06.006368Z", + "iopub.status.idle": "2026-09-30T07:49:06.009955Z", + "shell.execute_reply": "2026-09-30T07:49:06.009387Z" + } + }, + "outputs": [ + { + "data": { + "text/plain": [ + "((1, 4), (3, 3), (4, 3), (7, 3), (0, 2), (5, 2), (2, 1), (8, 1), (9, 1))" + ] + }, + "execution_count": 36, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "Itr(fibonacci()).take(20).map(lambda n: n % 10).value_counts().collect()" + ] + }, { "cell_type": "markdown", "id": "16bae27d", "metadata": {}, "source": [ - "\n", "### Predicates and Terminal Operations\n", "\n", "Methods that return a boolean or consume the iterator for side effects.\n", "\n", - "`all` checks if all elements satisfy a predicate; `any` check if at least one element satisfies the predicate:\n", + "`all` checks if all elements satisfy a predicate; `any` checks if at least one element satisfies the predicate.\n", "\n", - "Note that `all` will always consume the entire iterator (and should never be using on unbounded sequences), whereas \n", - "`any` will stop when the predicate is satified\n", + "Both stop as soon as the answer is known: `all` at the first element that fails the predicate, `any` at the first\n", + "that satisfies it. The predicate is optional, and defaults to testing truthiness.\n", "\n", "`max` and `min` consume elements and return the maximum and minimum elements respectively." ] }, { "cell_type": "code", - "execution_count": 40, + "execution_count": 37, "id": "fab7b5c0", - "metadata": {}, + "metadata": { + "execution": { + "iopub.execute_input": "2026-09-30T07:49:06.011361Z", + "iopub.status.busy": "2026-09-30T07:49:06.011258Z", + "iopub.status.idle": "2026-09-30T07:49:06.016121Z", + "shell.execute_reply": "2026-09-30T07:49:06.015634Z" + } + }, "outputs": [ { "data": { @@ -1013,7 +1699,7 @@ "8" ] }, - "execution_count": 40, + "execution_count": 37, "metadata": {}, "output_type": "execute_result" } @@ -1031,6 +1717,128 @@ "itr.max() # now the fourth element, itr is now exhausted" ] }, + { + "cell_type": "markdown", + "id": "552084f6", + "metadata": {}, + "source": [ + "`find` returns the first element satisfying a predicate, `position` returns its index, and `find_map` returns the first non-`None` result of a function. All three return `None` if nothing matches, and stop as soon as something does, so they work on unbounded sequences:" + ] + }, + { + "cell_type": "code", + "execution_count": 38, + "id": "04a7bed5", + "metadata": { + "execution": { + "iopub.execute_input": "2026-09-30T07:49:06.018297Z", + "iopub.status.busy": "2026-09-30T07:49:06.018173Z", + "iopub.status.idle": "2026-09-30T07:49:06.023300Z", + "shell.execute_reply": "2026-09-30T07:49:06.022785Z" + } + }, + "outputs": [ + { + "data": { + "text/plain": [ + "1597" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "17" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "42" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "True" + ] + }, + "execution_count": 38, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "display(Itr(fibonacci()).find(lambda n: n > 1000))\n", + "display(Itr(fibonacci()).position(lambda n: n > 1000))\n", + "display(Itr([\"x\", \"y\", \"42\", \"7\"]).find_map(lambda s: int(s) if s.isdigit() else None))\n", + "Itr(collatz(4)).find(lambda n: n > 1000) is None" + ] + }, + { + "cell_type": "markdown", + "id": "53da7f3f", + "metadata": {}, + "source": [ + "`eq` compares two sequences element by element, and `is_sorted` checks their order. Both stop at the first mismatch:" + ] + }, + { + "cell_type": "code", + "execution_count": 39, + "id": "6641e710", + "metadata": { + "execution": { + "iopub.execute_input": "2026-09-30T07:49:06.024685Z", + "iopub.status.busy": "2026-09-30T07:49:06.024578Z", + "iopub.status.idle": "2026-09-30T07:49:06.028354Z", + "shell.execute_reply": "2026-09-30T07:49:06.027951Z" + } + }, + "outputs": [ + { + "data": { + "text/plain": [ + "True" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "True" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "False" + ] + }, + "execution_count": 39, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "display(Itr(collatz(5)).eq([5, 16, 8, 4, 2, 1]))\n", + "display(Itr(collatz(16)).is_sorted(reverse=True))\n", + "Itr(collatz(5)).is_sorted(reverse=True)" + ] + }, { "cell_type": "markdown", "id": "aba84990", @@ -1044,9 +1852,16 @@ }, { "cell_type": "code", - "execution_count": 41, + "execution_count": 40, "id": "2ac1c105", - "metadata": {}, + "metadata": { + "execution": { + "iopub.execute_input": "2026-09-30T07:49:06.029555Z", + "iopub.status.busy": "2026-09-30T07:49:06.029455Z", + "iopub.status.idle": "2026-09-30T07:49:06.032619Z", + "shell.execute_reply": "2026-09-30T07:49:06.032163Z" + } + }, "outputs": [ { "data": { @@ -1063,7 +1878,7 @@ "(19, 58, 29)" ] }, - "execution_count": 41, + "execution_count": 40, "metadata": {}, "output_type": "execute_result" } @@ -1073,6 +1888,42 @@ "Itr(collatz(19)).take_while(lambda i: i < 80).collect() # takes the first 3" ] }, + { + "cell_type": "markdown", + "id": "acac8819", + "metadata": {}, + "source": [ + "`map_while` combines the two: it maps elements for as long as the predicate holds:" + ] + }, + { + "cell_type": "code", + "execution_count": 41, + "id": "75ad52ea", + "metadata": { + "execution": { + "iopub.execute_input": "2026-09-30T07:49:06.034131Z", + "iopub.status.busy": "2026-09-30T07:49:06.033960Z", + "iopub.status.idle": "2026-09-30T07:49:06.037451Z", + "shell.execute_reply": "2026-09-30T07:49:06.036872Z" + } + }, + "outputs": [ + { + "data": { + "text/plain": [ + "(190, 580, 290)" + ] + }, + "execution_count": 41, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "Itr(collatz(19)).map_while(lambda i: i < 80, lambda i: i * 10).collect()" + ] + }, { "cell_type": "markdown", "id": "0ee62dec", @@ -1085,7 +1936,14 @@ "cell_type": "code", "execution_count": 42, "id": "3be2d7d0", - "metadata": {}, + "metadata": { + "execution": { + "iopub.execute_input": "2026-09-30T07:49:06.039154Z", + "iopub.status.busy": "2026-09-30T07:49:06.039041Z", + "iopub.status.idle": "2026-09-30T07:49:06.042088Z", + "shell.execute_reply": "2026-09-30T07:49:06.041604Z" + } + }, "outputs": [ { "data": { @@ -1117,7 +1975,14 @@ "cell_type": "code", "execution_count": 43, "id": "dac3efaa", - "metadata": {}, + "metadata": { + "execution": { + "iopub.execute_input": "2026-09-30T07:49:06.043068Z", + "iopub.status.busy": "2026-09-30T07:49:06.042965Z", + "iopub.status.idle": "2026-09-30T07:49:06.045837Z", + "shell.execute_reply": "2026-09-30T07:49:06.045363Z" + } + }, "outputs": [ { "data": { @@ -1146,7 +2011,14 @@ "cell_type": "code", "execution_count": 44, "id": "1be027d4", - "metadata": {}, + "metadata": { + "execution": { + "iopub.execute_input": "2026-09-30T07:49:06.046899Z", + "iopub.status.busy": "2026-09-30T07:49:06.046799Z", + "iopub.status.idle": "2026-09-30T07:49:06.049786Z", + "shell.execute_reply": "2026-09-30T07:49:06.049203Z" + } + }, "outputs": [ { "data": { @@ -1173,11 +2045,51 @@ "source": [ "Itr(collatz(5)).product(collatz(2)).collect()" ] + }, + { + "cell_type": "markdown", + "id": "0a121843", + "metadata": {}, + "source": [ + "`peek` returns the next item without consuming it, and `next_if` consumes the next item only if it satisfies a predicate (returning `None` otherwise). Together they make simple parsers straightforward:" + ] + }, + { + "cell_type": "code", + "execution_count": 45, + "id": "96213b84", + "metadata": { + "execution": { + "iopub.execute_input": "2026-09-30T07:49:06.050971Z", + "iopub.status.busy": "2026-09-30T07:49:06.050860Z", + "iopub.status.idle": "2026-09-30T07:49:06.054092Z", + "shell.execute_reply": "2026-09-30T07:49:06.053440Z" + } + }, + "outputs": [ + { + "data": { + "text/plain": [ + "('123', 'a')" + ] + }, + "execution_count": 45, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "chars = Itr(\"123abc45\")\n", + "digits = []\n", + "while (c := chars.next_if(str.isdigit)) is not None:\n", + " digits.append(c)\n", + "\"\".join(digits), chars.peek()" + ] } ], "metadata": { "kernelspec": { - "display_name": "itrx", + "display_name": ".venv", "language": "python", "name": "python3" }, From 683a62f1097f06880660acfd416f82740593bebe Mon Sep 17 00:00:00 2001 From: virgesmith Date: Wed, 30 Sep 2026 08:52:14 +0100 Subject: [PATCH 4/4] Release 0.5.0 Co-Authored-By: Claude Opus 5.5 --- doc/apidoc.md | 2 +- pyproject.toml | 2 +- relnotes.md | 2 +- uv.lock | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/doc/apidoc.md b/doc/apidoc.md index 5d54b6e..36b01e9 100644 --- a/doc/apidoc.md +++ b/doc/apidoc.md @@ -1,4 +1,4 @@ -# `Itr` v0.4.0 class documentation +# `Itr` v0.5.0 class documentation A generic iterator adaptor class inspired by Rust's Iterator trait, providing a composable API for functional-style iteration and transformation over Python iterables. ## Public methods diff --git a/pyproject.toml b/pyproject.toml index 2379c62..47d8102 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "itrx" -version = "0.4.0" +version = "0.5.0" description = "A chainable iterator adapter" readme = "README.md" authors = [ diff --git a/relnotes.md b/relnotes.md index 80e8123..8da8f25 100644 --- a/relnotes.md +++ b/relnotes.md @@ -1,4 +1,4 @@ -## Unreleased +## 0.5.0 ### Breaking changes diff --git a/uv.lock b/uv.lock index 2207d13..30f4df0 100644 --- a/uv.lock +++ b/uv.lock @@ -323,7 +323,7 @@ wheels = [ [[package]] name = "itrx" -version = "0.4.0" +version = "0.5.0" source = { editable = "." } [package.optional-dependencies]