Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 6 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,27 +86,28 @@ For reference, the equivalent using `itertools` directly (is fairly readable):

Note:
1. Using `collect(dict)` requires an iterable that produces 2-tuples (key-value pairs).
2. `collect(set)` is equivalent to a "unique" method.
2. `collect(set)` gives the distinct items but loses their order; `unique()` lazily keeps the first occurrence of each, in order.

## How `Itr` Works: Lazy vs. Eager

Most `Itr` methods are **lazy transformations**, meaning they return a new `Itr` instance without immediately processing any data. This allows for arbitrary chaining and efficient memory usage, as items are only processed as they are requested. In most cases, `Itr` simply acts as a convenient wrapper around `itertools`, enabling this left-to-right chaining syntax.

- **Combining and splitting:** `partition`, `copy`, `batched`, `pairwise`, `rolling`, `chain`, `cycle`, `repeat`, `product`, `inspect`, `intersperse`, `interleave`, `chunk_by`, `zip_longest`
- **Transformation and filtering:** `accumulate`, `filter`, `map`, `starmap`, `map_while`, `flatten`, `flat_map`, `skip_while`, `take_while`, `dedup`, `dedup_with_count`, `scan`
- **Combining and splitting:** `partition`, `copy`, `batched`, `pairwise`, `rolling`, `chain`, `cycle`, `repeat`, `product`, `inspect`, `intersperse`, `interleave`, `chunk_by`, `zip`, `zip_longest`
- **Transformation and filtering:** `accumulate`, `filter`, `filter_map`, `compress`, `map`, `starmap`, `map_while`, `flatten`, `flat_map`, `skip_while`, `take_while`, `dedup`, `dedup_with_count`, `unique`, `scan`

However, some methods are **eager consumers**. These methods iterate over and consume the underlying data, returning concrete values, collections, or aggregates. Examples include:

* **Collection methods:** `collect`, `last`, `next`, `next_chunk`, `next_if`, `nth`, `position`
* **Aggregation methods:** `count`, `reduce`, `max`, `min`, `sum`, `prod`, `all`, `any`, `consume`, `find`, `fold`, `eq`, `is_sorted`
* **Sorting/grouping:** `sorted_by` and `groupby` sort the entire input up front, and `value_counts` counts it (most common first, like pandas), so all three consume the whole iterator immediately and must not be used on infinite sources. Use the lazy `chunk_by` to group consecutive runs without sorting.
* **Aggregation methods:** `count`, `reduce`, `max`, `min`, `min_max`, `sum`, `prod`, `all`, `any`, `consume`, `find`, `find_map`, `fold`, `eq`, `is_sorted`
* **Sorting/grouping:** `sorted_by` and `groupby` sort the entire input up front, and `value_counts` counts it (most common first, like pandas), so all three consume the whole iterator immediately and must not be used on infinite sources. Use the lazy `chunk_by` to group consecutive runs without sorting. Likewise `rev` and `take_last` must read to the end of the input before yielding anything.

### Important Considerations

When working with `Itr`, keep these points in mind:

* **Single-Pass Iterators:** Like all Python iterators, `Itr` instances (and their underlying iterators) can generally only be consumed once. If you need to process the same sequence multiple times, use methods like `copy()`, `cycle()`, or `repeat()` as necessary.
* **No Rewinding:** It's not possible to rewind an `Itr` to an earlier state. You can "preview" the next value using the `peek()` method, and conditionally consume it with `next_if()`.
* **Missing Items:** `find`, `find_map` and `position` return `None` when nothing matches, while `peek`, `last`, `max`, `min` and `min_max` raise `ValueError` on an empty iterator. As with their builtin counterparts, `nth` raises `IndexError` when the iterator is too short (like indexing a sequence), and `reduce` raises `TypeError` when it is empty (like `functools.reduce`). Only `next()` raises `StopIteration`, like the builtin, so avoid calling it inside a `map` or `filter` callback: a `StopIteration` escaping a callback silently ends the enclosing iteration rather than raising.
* **Infinite Iterators:** Be cautious with open-ended iterators (e.g., those from `itertools.count()` or custom generators). Eager evaluation methods (like `collect()`, `count()`, `reduce()`) will attempt to consume the entire sequence, potentially leading to infinite loops or out-of-memory errors if applied to an infinite source.

## Agent skill
Expand Down
Loading
Loading