Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 27 additions & 14 deletions database/src/main/java/io/hstore/db/hora/Hora.java
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ public record OverlapPair(long left, long right, long overlap) {
}

public record Expansion(List<Long> path, long atom, Status status) {
public enum Status { EMIT, CYCLE, BUDGET_EXHAUSTED }
public enum Status { EMIT, CYCLE, DEPTH_LIMIT, BUDGET_EXHAUSTED }
}

private final Reader reader;
Expand Down Expand Up @@ -304,31 +304,44 @@ public Stream<Expansion> expand(long atom, int depth, Budget budget) {
reader.require(atom);
Budget.Meter meter = budget.start();
List<Expansion> out = new ArrayList<>();
expand(atom, Math.min(depth, budget.maxDepth()), new HashSet<>(), new ArrayList<>(List.of(atom)), meter, out);
Expansion.Status bottom = budget.maxDepth() < depth ? Expansion.Status.BUDGET_EXHAUSTED : Expansion.Status.DEPTH_LIMIT;
expand(atom, Math.min(depth, budget.maxDepth()), bottom, new HashSet<>(), new ArrayList<>(List.of(atom)), meter, out);
return out.stream();
}

private void expand(long atom, int depth, Set<Long> visited, List<Long> path, Budget.Meter meter, List<Expansion> out) {
if (depth == 0 || !meter.tryConsume(1)) {
private boolean expand(long atom, int depth, Expansion.Status bottom, Set<Long> ancestors, List<Long> path, Budget.Meter meter,
List<Expansion> out) {
if (!meter.tryConsume(1)) {
out.add(new Expansion(List.copyOf(path), atom, Expansion.Status.BUDGET_EXHAUSTED));
return;
return false;
}
if (!visited.add(atom)) {
if (ancestors.contains(atom)) {
out.add(new Expansion(List.copyOf(path), atom, Expansion.Status.CYCLE));
return;
return true;
}
for (Incidence member : reader.edge(atom).stream().toList()) {
if (!reader.owns(member.member())) {
if (depth == 0) {
out.add(new Expansion(List.copyOf(path), atom, bottom));
return true;
}
ancestors.add(atom);
boolean proceed = true;
for (Iterator<Incidence> members = reader.edge(atom).stream().iterator(); proceed && members.hasNext(); ) {
long member = members.next().member();
if (!reader.owns(member)) {
continue;
}
path.add(member.member());
boolean nested = reader.atom(member.member()).filter(Atom::isEdge).isPresent();
if (nested) {
expand(member.member(), depth - 1, visited, path, meter, out);
path.add(member);
if (reader.atom(member).filter(Atom::isEdge).isPresent()) {
proceed = expand(member, depth - 1, bottom, ancestors, path, meter, out);
} else if (meter.tryConsume(1)) {
out.add(new Expansion(List.copyOf(path), member, Expansion.Status.EMIT));
} else {
out.add(new Expansion(List.copyOf(path), member.member(), Expansion.Status.EMIT));
out.add(new Expansion(List.copyOf(path), member, Expansion.Status.BUDGET_EXHAUSTED));
proceed = false;
}
path.removeLast();
}
ancestors.remove(atom);
return proceed;
}
}
65 changes: 65 additions & 0 deletions database/src/test/java/io/hstore/db/DatabaseTest.java
Original file line number Diff line number Diff line change
Expand Up @@ -430,4 +430,69 @@ void writesOnBranchesStayIsolated() {
assertTrue(database.read(reader -> reader.find("Person", "ghost")).isEmpty());
assertTrue(database.read(branch.id(), reader -> reader.find("Person", "ghost")).isPresent());
}

@Test
void expandFollowsEveryPathAndStaysWithinItsBudget() {
long[] ids = database.write(writer -> {
long leaf = writer.node("Person", "expand-leaf", Map.of("name", "Leaf"));
long bottom = writer.edge("Coauthorship");
writer.add(bottom, leaf, MemberSpec.PLAIN);
long left = writer.edge("Coauthorship");
long right = writer.edge("Coauthorship");
writer.add(left, bottom, MemberSpec.PLAIN);
writer.add(right, bottom, MemberSpec.PLAIN);
long top = writer.edge("Coauthorship");
writer.add(top, left, MemberSpec.PLAIN);
writer.add(top, right, MemberSpec.PLAIN);
long loop = writer.edge("Coauthorship");
long back = writer.edge("Coauthorship");
writer.add(loop, back, MemberSpec.PLAIN);
writer.add(back, loop, MemberSpec.PLAIN);
long wide = writer.edge("Coauthorship");
for (int i = 0; i < 20; i++) {
writer.add(wide, writer.node("Person", "expand-wide-" + i, Map.of("name", "W" + i)), MemberSpec.PLAIN);
}
long previous = bottom;
for (int level = 0; level < 12; level++) {
long a = writer.edge("Coauthorship");
long b = writer.edge("Coauthorship");
writer.add(a, previous, MemberSpec.PLAIN);
writer.add(b, previous, MemberSpec.PLAIN);
previous = writer.edge("Coauthorship");
writer.add(previous, a, MemberSpec.PLAIN);
writer.add(previous, b, MemberSpec.PLAIN);
}
return new long[]{top, left, right, bottom, leaf, loop, back, wide, previous};
});
long top = ids[0], left = ids[1], right = ids[2], bottom = ids[3], leaf = ids[4], loop = ids[5], back = ids[6];
database.read(reader -> {
Hora hora = Hora.on(reader);
assertEquals(List.of(
new Hora.Expansion(List.of(top, left, bottom, leaf), leaf, Hora.Expansion.Status.EMIT),
new Hora.Expansion(List.of(top, right, bottom, leaf), leaf, Hora.Expansion.Status.EMIT)),
hora.expand(top, 8, Budget.DEFAULT).toList());
assertEquals(List.of(new Hora.Expansion(List.of(loop, back, loop), loop, Hora.Expansion.Status.CYCLE)),
hora.expand(loop, 8, Budget.DEFAULT).toList());
assertEquals(List.of(new Hora.Expansion(List.of(loop, back, loop), loop, Hora.Expansion.Status.CYCLE)),
hora.expand(loop, 2, Budget.DEFAULT).toList());
assertEquals(List.of(new Hora.Expansion(List.of(loop, back), back, Hora.Expansion.Status.DEPTH_LIMIT)),
hora.expand(loop, 1, Budget.DEFAULT).toList());
assertEquals(List.of(
new Hora.Expansion(List.of(top, left, bottom), bottom, Hora.Expansion.Status.DEPTH_LIMIT),
new Hora.Expansion(List.of(top, right, bottom), bottom, Hora.Expansion.Status.DEPTH_LIMIT)),
hora.expand(top, 2, Budget.DEFAULT).toList());
assertEquals(List.of(
new Hora.Expansion(List.of(top, left, bottom), bottom, Hora.Expansion.Status.BUDGET_EXHAUSTED),
new Hora.Expansion(List.of(top, right, bottom), bottom, Hora.Expansion.Status.BUDGET_EXHAUSTED)),
hora.expand(top, 8, Budget.DEFAULT.withDepth(2)).toList());
List<Hora.Expansion> wide = hora.expand(ids[7], 8, Budget.DEFAULT.withRows(5)).toList();
assertEquals(5, wide.size());
assertEquals(Hora.Expansion.Status.BUDGET_EXHAUSTED, wide.getLast().status());
assertEquals(4, wide.stream().filter(row -> row.status() == Hora.Expansion.Status.EMIT).count());
List<Hora.Expansion> layered = hora.expand(ids[8], 64, Budget.DEFAULT.withRows(1_000).withDepth(64)).toList();
assertTrue(layered.size() <= 1_001, "rows " + layered.size());
assertEquals(1, layered.stream().filter(row -> row.status() == Hora.Expansion.Status.BUDGET_EXHAUSTED).count());
return null;
});
}
}
12 changes: 8 additions & 4 deletions docs/database/hora.md
Original file line number Diff line number Diff line change
Expand Up @@ -170,10 +170,14 @@ descends into members that are themselves hyperedges and emits leaves.
| Status | Meaning |
|---|---|
| `EMIT` | a non-edge member reached through `path` |
| `CYCLE` | a hyperedge already visited on this traversal (cycles through membership are possible because hyperedges can contain hyperedges) |
| `BUDGET_EXHAUSTED` | depth `0` reached or the row budget denied the next step |
| `CYCLE` | a hyperedge that is already on `path` (cycles through membership are possible because hyperedges can contain hyperedges). A hyperedge reached again through a different parent is expanded again |
| `DEPTH_LIMIT` | a hyperedge at the depth the query asked for; its members were not expanded, and everything above it is complete |
| `BUDGET_EXHAUSTED` | the budget cut the traversal short: the row budget denied the next step and the traversal stopped, or the budget's depth cap (`budget.maxDepth`, lower than the requested depth) was reached on this path |

The effective depth is `min(d, budget.maxDepth)` (default 8); `LIMIT` sets the row budget.
Through the Java API the effective depth is `min(d, budget.maxDepth)` (default 8). HQL `EXPAND … DEPTH d` sets both to `d` (1 to 64), so its bottom rows are always `DEPTH_LIMIT`; `LIMIT` sets the row budget. A hyperedge already on the path is reported as `CYCLE` even at the bottom, so a client that continues from `DEPTH_LIMIT` rows never walks into a cycle. Every row the traversal
emits and every descent into a nested hyperedge takes one unit of the budget. When the budget refuses, one
`BUDGET_EXHAUSTED` row marks where the traversal stopped and nothing more is emitted, so the output never exceeds the
budget by more than that row, whatever the shape of the nesting.

```sql
EXPAND @120 DEPTH 2 LIMIT 20;
Expand Down Expand Up @@ -273,5 +277,5 @@ compared with `main` to test whether observed overlaps exceed chance; the branch
| `OVERLAP JOIN T THRESHOLD t` | `O(Σ_{a∈T} Σ_{v∈M(a)} |I(v)| + #pairs · t log n)` | `O(#partners of one edge + #results)` |
| set algebra | `O(min(|A|,|B|) · log max(|A|,|B|))` for intersections via leapfrog | result size |
| `CLOSURE e DIM d` | `O(Σ_{k≤d+1} C(|M(e)|, k))` | `O(d)` per subset (lazy) |
| `EXPAND e DEPTH d` | `O(rows emitted)` | `O(depth + visited)` |
| `EXPAND e DEPTH d` | `O(rows emitted)`, bounded by the row budget | `O(depth + rows emitted)` |
| `PATTERN` | bounded by the product of candidate domains after pruning | `O(partial bindings)` |
Loading