Skip to content

feat: v0.3 query ownership and complexity fixes #4

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
OrlovEvgeny wants to merge 1 commit into from
Closed

feat: v0.3 query ownership and complexity fixes #4

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 24 additions & 1 deletion .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ jobs:
version: 0.16.0

- name: Check formatting
run: zig fmt --check src/ test/
run: zig fmt --check src/ test/ bench/

build-and-test:
needs: lint
Expand Down Expand Up @@ -74,3 +74,26 @@ jobs:
# tests without emitting a binary to keep source compatibility covered.
- name: Semantic compatibility check
run: zig test src/root.zig -fno-emit-bin

bench:
name: Benchmarks
runs-on: ubuntu-latest
needs: build-and-test
steps:
- name: Checkout
uses: actions/checkout@v7.0.0

- name: Setup Zig
uses: mlugg/setup-zig@v2.2.1
with:
version: 0.16.0

# Wall-clock numbers are too noisy on shared runners to gate on, so this
# checks the two things that are not: no benchmark may retain memory in
# the document arena, and no workload may scale superlinearly.
#
# Run at full corpus size deliberately. The --quick corpora are small
# enough that fixed costs dominate and every workload measures as linear,
# which would make this gate useless.
- name: Complexity and retention gates
run: zig build bench -- --strict --scaling --filter find/
2 changes: 1 addition & 1 deletion .github/workflows/release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ jobs:
[[ "$RELEASE_TAG" =~ ^v[0-9]+\.[0-9]+\.[0-9]+$ ]] || { echo "Invalid release tag: $RELEASE_TAG" >&2; exit 1; }
ZON_VERSION=$(sed -n 's/.*\.version = "\([^"]*\)".*/\1/p' build.zig.zon)
test "${RELEASE_TAG#v}" = "$ZON_VERSION"
zig fmt --check src/ test/
zig fmt --check src/ test/ bench/
zig build
zig build test

Expand Down
132 changes: 107 additions & 25 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,16 +28,20 @@ pub fn main() !void {
);
defer doc.deinit();

// A Query owns everything the queries below allocate.
var q = doc.query(allocator);
defer q.deinit();

// Find all paragraphs.
const paragraphs = try doc.find("p");
const paragraphs = try q.find("p");
std.debug.print("Found {} paragraphs\n", .{paragraphs.len()});

// Get text content.
const title = try (try doc.find("h1")).text();
const title = try (try q.find("h1")).text();
std.debug.print("Title: {s}\n", .{title});

// Read attributes.
const link = try doc.find("a.active");
const link = try q.find("a.active");
const href = link.attr("href") orelse "";
std.debug.print("Link: {s}\n", .{href});
}
Expand Down Expand Up @@ -96,25 +100,56 @@ Supported selector syntax:

## API overview

### Document
### Document and Query

A `Document` owns the parsed DOM. A `Query` owns everything a query produces:
node slices, chained `Selection` states, parsed selectors, and the strings
returned by `html()`, `text()` and `outerHtml()`.

```zig
// Parse HTML into a document. All allocations use an internal arena.
// Parse HTML into a document. The DOM lives in the document's arena.
var doc = try zq.Document.initFromSlice(allocator, html);
defer doc.deinit();

// Query from the document root.
const sel = try doc.find("div.content");
// Open a query scope. Results live here, not in the document.
var q = doc.query(allocator);
defer q.deinit();

const sel = try q.find("div.content");

// Deep-clone the entire document.
var copy = try doc.clone(allocator);
defer copy.deinit();
```

Keeping the two apart is what makes a long-lived document practical. When you
run many queries against one document -- a scraper looping over pages, or a
server handling requests -- reclaim between rounds:

```zig
var q = doc.query(allocator);
defer q.deinit();

for (jobs) |job| {
const rows = try q.find(job.selector);
try handle(rows);
q.reset(); // releases this round's results, keeps the capacity
}
```

`q.reset()` and `q.deinit()` invalidate every `Selection` and every string
obtained through that query. Anything you need to keep, copy out first.

Changes you make to the DOM are not query data and are unaffected: attributes,
inserted nodes and parsed fragments belong to the document and outlive the
query that created them.

All the examples below assume a `q` in scope, as above.

### Selection — Traversal

```zig
const sel = try doc.find("div");
const sel = try q.find("div");

// Descendants matching a selector.
const links = try sel.find("a");
Expand All @@ -139,7 +174,7 @@ const prev_all = try sel.prevAll();
### Selection — Filtering

```zig
const items = try doc.find("li");
const items = try q.find("li");

const active = try items.filter(".active");
const inactive = try items.not(".active");
Expand All @@ -158,7 +193,7 @@ const is_active = try items.is(".active");
### Selection — Properties

```zig
const el = try doc.find("a.nav");
const el = try q.find("a.nav");

// Attributes.
const href = el.attr("href");
Expand All @@ -182,14 +217,14 @@ const name = zq.nodeName(el);
### Selection — Manipulation

```zig
const div = try doc.find("div");
const div = try q.find("div");

// Insert content.
try div.appendHtml("<p>appended</p>");
try div.prependHtml("<p>prepended</p>");

// Insert around selection.
const p = try doc.find("p");
const p = try q.find("p");
try p.afterHtml("<hr/>");
try p.beforeHtml("<!-- marker -->");

Expand All @@ -210,7 +245,7 @@ try div.unwrap();
### Selection — Iteration

```zig
const rows = try doc.find("tr");
const rows = try q.find("tr");

// Iterator (idiomatic Zig).
var it = rows.iterator();
Expand All @@ -231,8 +266,8 @@ rows.each(struct {
### Selection — Set operations

```zig
const a = try doc.find(".foo");
const b = try doc.find(".bar");
const a = try q.find(".foo");
const b = try q.find(".bar");

const combined = try a.add(".bar");
const merged = try a.addSelection(b);
Expand All @@ -248,22 +283,56 @@ Compile a selector once when it is reused across queries or documents:
var active_links = try zq.CompiledSelector.init(allocator, "a.active");
defer active_links.deinit();

const links = try doc.findCompiled(&active_links);
const links = try q.findCompiled(&active_links);
const matches = links.isCompiled(&active_links);
```

## Ownership and errors

`Document.initFromSlice`, `Document.initFromNode`, `Document.clone`, parsed fragments,
attributes, and inserted nodes own their data through the document arena. Input buffers
and source documents may be released after these operations complete. Use
`Document.initBorrowedNode` only when the source tree is guaranteed to outlive the
document.
Two arenas, with a clear division:

| Lives in the **document** arena | Lives in the **query** arena |
|---|---|
| Parsed nodes and their data | `Selection.nodes` slices |
| Attribute keys and values you set | Chained selection state (`end()`, `addBack()`) |
| Nodes from `appendHtml`, `setHtml`, `wrapHtml`, … | Parsed selector ASTs |
| Clones from `cloneSel` | Strings from `html()`, `text()`, `outerHtml()` |
| Released by `doc.deinit()` | Released by `q.deinit()` or `q.reset()` |

Input buffers and source documents may be released once parsing completes. Use
`Document.initBorrowedNode` only when the source tree is guaranteed to outlive
the document. A `Query` borrows its `Document`, so the document must outlive it.

Operations that allocate return an error union. In v0.2 this includes traversal methods
such as `children`, positional methods such as `first`, attribute/class updates, and DOM
mutations. Mutations parse or clone all required data before changing the tree, so an
allocation failure does not leave a partially updated selection.
Operations that allocate return an error union: traversal methods such as
`children`, positional methods such as `first`, attribute and class updates, and
DOM mutations. Mutations parse or clone all required data before changing the
tree, so an allocation failure does not leave a partially updated selection.

## v0.3 migration

Queries now run through a `Query` rather than the `Document`:

```zig
// v0.2
const links = try doc.find("a.active");

// v0.3
var q = doc.query(allocator);
defer q.deinit();
const links = try q.find("a.active");
```

- `Document.find`, `findMatcher`, `findCompiled` and `select` move to `Query`
(`select` is now `Query.root`).
- `Document.allocator` is renamed `Document.domAllocator` and is reserved for
content that joins the DOM.
- `Selection.document` is now the method `Selection.document()`; the struct
field is `Selection.q`.
- Strings from `html()`, `text()` and `outerHtml()` belong to the query and do
not survive `q.deinit()` or `q.reset()`. Duplicate anything you need to keep.

Previously every query allocated into the document's arena and nothing was ever
released, so repeated queries against one document grew without bound.

## v0.2 migration

Expand All @@ -285,6 +354,19 @@ full named entity table remain roadmap items. See [ROADMAP.md](ROADMAP.md).
zig build test
```

## Benchmarks

```sh
zig build bench # the standard suite, always ReleaseFast
zig build bench -- --scaling # empirical complexity at 1x / 2x / 4x input
zig build bench -- --filter find/ # a subset
zig build bench -- --corpus <dir> # also run against local .html files
```

Corpora are generated deterministically, so results are comparable across
machines. `--scaling` reports an exponent per workload: about 1.0 for linear,
2.0 for quadratic. `bench/baseline.json` records a reference run.

Inspired by Go's [goquery](https://github.com/PuerkitoBio/goquery) and, by extension, jQuery

## License
Expand Down
30 changes: 26 additions & 4 deletions ROADMAP.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,24 @@ dependencies remain at zero, and supported Zig versions are tested explicitly.
- Relative `:has()`, selector lists in `:not()`/`:is()`/`:where()`, and strict parsing.
- Reusable `CompiledSelector`, tree validation, and Zig 0.15.2/0.16.x CI.

## v0.3 - parser conformance
## v0.3 - ownership and complexity (current)

Done:

- `Query` scratch arena: query results, chained selections, parsed selectors and
returned strings live and die with the query, not the document. Repeated
queries against a long-lived document no longer grow it.
- Removed the quadratic paths found by the benchmark harness:
`wouldCreateCycle` under an `assert` (evaluated in every build mode) made tree
building O(nodes x depth); `autoClose` scanned the whole open-element stack per
start tag; `:has()` re-scanned the entire document per anchor; `:nth-child()`
recounted a sibling list per candidate.
- Iterative traversal for render, clone, validate, text extraction and matching,
so deeply nested documents no longer overflow the stack.
- Benchmark harness (`zig build bench`) with deterministic corpora, per-operation
memory accounting, and an empirical complexity check (`--scaling`).

Remaining for parser conformance, deferred to v0.4:

- Replace ad hoc tree construction with explicit WHATWG insertion modes.
- Add foster parenting, template modes, foreign SVG/MathML content, and adoption agency.
Expand All @@ -27,12 +44,17 @@ dependencies remain at zero, and supported Zig versions are tested explicitly.
- Selector specificity metadata and bounded matching for untrusted selectors.
- Conformance fixtures derived from Web Platform Tests.

## v0.5 - performance and tooling
## v0.5 - memory and tooling

- Benchmarks for parsing, matching, rendering, cloning, and bulk mutation.
- Reduce per-node footprint: pooled node allocation, exact-size attribute
arrays, an interned namespace field, and coalesced text nodes. On a
node-dense document the arena currently holds about 108 bytes per node
against a 96-byte `Node`, so node size is essentially the whole cost there.
- Fuzz targets for HTML/CSS parsers and mutation sequences with tree validation.
- Selector planning indexes for IDs/classes/tags and reduced temporary allocations.
- Streaming writer APIs and configurable serialization modes.
- Selector planning indexes for IDs/classes/tags, if profiling justifies them.
Measured selector times are currently in the low hundreds of microseconds on
a 2 MB document, so this is not yet the bottleneck it was assumed to be.

## v1.0 - stable contract

Expand Down
36 changes: 36 additions & 0 deletions bench/baseline-v0.2.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
{
"benchmarks": [
{"name": "parse/small", "ns_p50": 99703, "ns_min": 91528, "ns_p99": 126536, "bytes_in": 65623, "peak_bytes": 564888, "growth_per_op": 0, "growth_capped": false, "allocs_per_op": 3, "iters": 1968},
{"name": "parse/medium", "ns_p50": 4697167, "ns_min": 4497666, "ns_p99": 4784291, "bytes_in": 2200074, "peak_bytes": 15720190, "growth_per_op": 0, "growth_capped": false, "allocs_per_op": 3, "iters": 43},
{"name": "parse/large", "ns_p50": 23316417, "ns_min": 23153625, "ns_p99": 23455666, "bytes_in": 10485790, "peak_bytes": 74809384, "growth_per_op": 0, "growth_capped": false, "allocs_per_op": 3, "iters": 9},
{"name": "parse/attr_heavy", "ns_p50": 3773000, "ns_min": 3563250, "ns_p99": 3829375, "bytes_in": 2200308, "peak_bytes": 8273720, "growth_per_op": 0, "growth_capped": false, "allocs_per_op": 2, "iters": 54},
{"name": "parse/entity_heavy", "ns_p50": 4690667, "ns_min": 4490417, "ns_p99": 4727833, "bytes_in": 2200099, "peak_bytes": 15720434, "growth_per_op": 0, "growth_capped": false, "allocs_per_op": 3, "iters": 43},
{"name": "parse/script_heavy", "ns_p50": 4696875, "ns_min": 4620625, "ns_p99": 5025750, "bytes_in": 2200063, "peak_bytes": 15720342, "growth_per_op": 0, "growth_capped": false, "allocs_per_op": 3, "iters": 43},
{"name": "parse/dense", "ns_p50": 12112042, "ns_min": 11718042, "ns_p99": 12238750, "bytes_in": 2200052, "peak_bytes": 50126640, "growth_per_op": 0, "growth_capped": false, "allocs_per_op": 4, "iters": 17},
{"name": "parse/shallow_wide", "ns_p50": 3970041, "ns_min": 3764417, "ns_p99": 4043041, "bytes_in": 2200091, "peak_bytes": 15720314, "growth_per_op": 0, "growth_capped": false, "allocs_per_op": 3, "iters": 51},
{"name": "find/tag", "ns_p50": 357770, "ns_min": 321531, "ns_p99": 378271, "bytes_in": 0, "peak_bytes": 199760678, "growth_per_op": 324014, "growth_capped": true, "allocs_per_op": 0, "iters": 568},
{"name": "find/class", "ns_p50": 623958, "ns_min": 566333, "ns_p99": 717021, "bytes_in": 0, "peak_bytes": 55076258, "growth_per_op": 122223, "growth_capped": true, "allocs_per_op": 0, "iters": 322},
{"name": "find/compound", "ns_p50": 348521, "ns_min": 328760, "ns_p99": 360135, "bytes_in": 0, "peak_bytes": 55080428, "growth_per_op": 68811, "growth_capped": true, "allocs_per_op": 0, "iters": 572},
{"name": "find/descendant", "ns_p50": 396031, "ns_min": 374750, "ns_p99": 428718, "bytes_in": 0, "peak_bytes": 26914344, "growth_per_op": 22210, "growth_capped": true, "allocs_per_op": 0, "iters": 504},
{"name": "find/child", "ns_p50": 372896, "ns_min": 344718, "ns_p99": 404489, "bytes_in": 0, "peak_bytes": 15720190, "growth_per_op": 0, "growth_capped": false, "allocs_per_op": 0, "iters": 544},
{"name": "find/attr_exists", "ns_p50": 521416, "ns_min": 497041, "ns_p99": 599437, "bytes_in": 0, "peak_bytes": 307982506, "growth_per_op": 849599, "growth_capped": true, "allocs_per_op": 0, "iters": 344},
{"name": "find/attr_prefix", "ns_p50": 316271, "ns_min": 301218, "ns_p99": 375968, "bytes_in": 0, "peak_bytes": 15720190, "growth_per_op": 0, "growth_capped": false, "allocs_per_op": 0, "iters": 616},
{"name": "find/nth_child", "ns_p50": 1213958, "ns_min": 1178583, "ns_p99": 1422541, "bytes_in": 0, "peak_bytes": 26917838, "growth_per_op": 69121, "growth_capped": true, "allocs_per_op": 0, "iters": 162},
{"name": "find/has_child", "ns_p50": 1063179792, "ns_min": 999960458, "ns_p99": 1077509666, "bytes_in": 0, "peak_bytes": 15720190, "growth_per_op": 0, "growth_capped": false, "allocs_per_op": 0, "iters": 4},
{"name": "find/has_descendant", "ns_p50": 1479490542, "ns_min": 1474244167, "ns_p99": 1527702459, "bytes_in": 0, "peak_bytes": 15720190, "growth_per_op": 0, "growth_capped": false, "allocs_per_op": 0, "iters": 3},
{"name": "find/contains", "ns_p50": 358333, "ns_min": 306156, "ns_p99": 368593, "bytes_in": 0, "peak_bytes": 26910620, "growth_per_op": 19982, "growth_capped": true, "allocs_per_op": 0, "iters": 560},
{"name": "find/group", "ns_p50": 770562, "ns_min": 720354, "ns_p99": 808250, "bytes_in": 0, "peak_bytes": 27469646, "growth_per_op": 45190, "growth_capped": true, "allocs_per_op": 0, "iters": 260},
{"name": "find/not", "ns_p50": 309271, "ns_min": 287958, "ns_p99": 366291, "bytes_in": 0, "peak_bytes": 31058508, "growth_per_op": 23966, "growth_capped": true, "allocs_per_op": 0, "iters": 640},
{"name": "find/universal", "ns_p50": 1198083, "ns_min": 1118541, "ns_p99": 1561833, "bytes_in": 0, "peak_bytes": 442710746, "growth_per_op": 3234775, "growth_capped": true, "allocs_per_op": 0, "iters": 132},
{"name": "leak/find_repeat", "ns_p50": 3329819, "ns_min": 3329819, "ns_p99": 3329819, "bytes_in": 0, "peak_bytes": 285026700, "growth_per_op": 117449, "growth_capped": true, "allocs_per_op": 0, "iters": 2000},
{"name": "render/document", "ns_p50": 3863292, "ns_min": 3736708, "ns_p99": 3903083, "bytes_in": 2200074, "peak_bytes": 18731094, "growth_per_op": 0, "growth_capped": false, "allocs_per_op": 1, "iters": 52},
{"name": "clone/document", "ns_p50": 2377542, "ns_min": 2314000, "ns_p99": 2795042, "bytes_in": 2200074, "peak_bytes": 27564896, "growth_per_op": 0, "growth_capped": false, "allocs_per_op": 20, "iters": 84},
{"name": "text/document", "ns_p50": 525250, "ns_min": 460000, "ns_p99": 816875, "bytes_in": 0, "peak_bytes": 266572964, "growth_per_op": 670729, "growth_capped": true, "allocs_per_op": 0, "iters": 374},
{"name": "traverse/children", "ns_p50": 224812, "ns_min": 184854, "ns_p99": 317736, "bytes_in": 0, "peak_bytes": 307219546, "growth_per_op": 883331, "growth_capped": true, "allocs_per_op": 0, "iters": 330},
{"name": "traverse/parent", "ns_p50": 83505, "ns_min": 78205, "ns_p99": 131913, "bytes_in": 0, "peak_bytes": 307501030, "growth_per_op": 393235, "growth_capped": true, "allocs_per_op": 0, "iters": 742},
{"name": "traverse/siblings", "ns_p50": 11742166, "ns_min": 10991333, "ns_p99": 12354250, "bytes_in": 0, "peak_bytes": 470076422, "growth_per_op": 25242004, "growth_capped": true, "allocs_per_op": 0, "iters": 18},
{"name": "traverse/next_all", "ns_p50": 4297708, "ns_min": 4141625, "ns_p99": 4776959, "bytes_in": 0, "peak_bytes": 470123848, "growth_per_op": 11651372, "growth_capped": true, "allocs_per_op": 0, "iters": 39},
{"name": "traverse/filter", "ns_p50": 115133, "ns_min": 103642, "ns_p99": 140514, "bytes_in": 0, "peak_bytes": 55049894, "growth_per_op": 23026, "growth_capped": true, "allocs_per_op": 0, "iters": 1708},
{"name": "traverse/first", "ns_p50": 5, "ns_min": 5, "ns_p99": 10, "bytes_in": 0, "peak_bytes": 153196428, "growth_per_op": 32, "growth_capped": true, "allocs_per_op": 0, "iters": 4194304}
]
}
Loading
Loading