Skip to content

api.md

Latest commit

 

History

History
251 lines (200 loc) · 16.5 KB

api.md

File metadata and controls

251 lines (200 loc) · 16.5 KB

API specification

Single structural description of the Node.js ↔ Rust/C++ graph and DB API. Checkbox: [x] implemented, [ ] not implemented.

Principles

  • Real async: Methods returning Promise run heavy work off the main thread (addon worker or libuv pool). No Promise.resolve(syncCall()) on the main thread.
  • Sync = blocking: Sync methods block the calling thread; they are documented as such.
  • Zero-copy where possible: Topology and bulk data are exposed as TypedArrays (External/Shared when the native layer supports it). Arrow chunks are the single result path; no row-by-row string parsing on the hot path.

1. Types

  • NodeID — Dense integer node identifier (graph overlay). type NodeID = number.
  • LbugValue — Value from Ladybug (scalar or composite). Currently unknown (type-level stub).
  • QueryOptions — { timeoutMs?: number; signal?: AbortSignal; progressCallback?: (progress: unknown) => void }. Reserved; not yet wired.
  • PoolOptions — { databasePath: string; maxSize?: number; numThreads?: number; initRetries?: number; initBackoffMs?: number; initBackoffMaxMs?: number; idleTimeoutMs?: number }.
  • QuerySummary — { numTuples?: number }. Filled from QueryResult.getNumTuples().
  • IngestColumnType — "STRING" | "INT64" | "INT32" | "DOUBLE" | "BOOL".
  • IngestColumnSchema — { name: string; type: IngestColumnType }.
  • IngestColumnBatch — { name: string; values: (string | number | boolean | null)[] }.
  • LoadArrowOptions — { table: string; columns: IngestColumnSchema[] }.
  • TopologicalGraph — Topology snapshot: nodeCount: number, edges: Int32Array (flat [from, to, ...]), edgeTypes?: Int8Array, dictionary: string[]. New code should prefer split sources: Int32Array, targets: Int32Array as the canonical representation when topology comes from the native layer; edges can be derived or kept for in-memory GraphQuery.
  • ZeroCopyTopology — Disposable topology backed by native-owned buffers: { sources: Int32Array; targets: Int32Array; edgeTypes?: Int8Array; dictionary: string[]; [Symbol.dispose](): void }.
  • GraphNodeCursor — Mapping-backed node cursor: yields { id, index } for dense node IDs via nextBatch(batchSize), stream(), close(), [Symbol.dispose]() (alias of close() for Explicit Resource Management).
  • DatabaseOptions — { path: string } (and optional future fields).

2. Graph adapter (NativeGraphDB)

High-level graph-first API for Overlay/Build. Builds on top of the addon (Database, Connection, QueryResult, Arrow).

  • NativeGraphDB — Class; single entry for graph operations.

    • exportTopology(edgeLabels: string[], filterIds?: Int32Array): TopologicalGraph — Derives a topology snapshot from the underlying DB via Cypher/Arrow; current implementation ignores labels and filters until Ladybug graph API is wired.
    • fetchTopology(): TopologicalGraph — Convenience wrapper returning the current topology snapshot as TopologicalGraph for traversal in user code.
    • importTopology(edgeLabel: string, graph: TopologicalGraph): void — Bulk import of topology via Cypher UNWIND batches using a Connection.
    • getProperties(ids: Int32Array | string[], properties?: string[]): GraphNodeCursor — Mapping-backed cursor over node ids. Accepts dense indices (Int32Array) or original string ids (string[]); properties is currently ignored and only id/index are exposed.
    • V(label?: string | string[]): GraphQuery — Starts a vertex query over the exported topology; labels currently ignored until Ladybug graph API is wired.
    • E(label?: string | string[]): EdgeQuery — Starts an edge query over the exported topology; labels currently ignored until Ladybug graph API is wired.
    • readTransaction(): GraphTransaction — Returns a transaction-like handle for Fluent graph queries; commit/rollback are no-ops until native transactions are available.
    • writeTransaction(): GraphTransaction — Same as readTransaction; placeholder for future write semantics.

  • GraphQuery — Fluent builder over an in-memory TopologicalGraph; execution only on terminators.

    • Filtering: [x] has(propertyKey, value) (id only), [x] hasNot(propertyKey, value?) (id only), [x] hasIn(propertyKey, values) (id only), [x] hasLabel(label) (no-op).
    • Navigation: [x] out(edgeLabel?), [x] in(edgeLabel?), [x] both(edgeLabel?), [x] outE(edgeLabel?), [x] inE(edgeLabel?), [x] bothE(edgeLabel?) (edge labels are currently ignored).
    • Terminators: [x] fetchTopology(): TopologicalGraph, [x] fetchIds(): Int32Array, [x] fetchCursor(): GraphNodeCursor.

  • EdgeQuery — Fluent builder for edge context over an in-memory TopologicalGraph.

2.1 GraphTransaction

  • GraphTransaction — Disposable transaction-like handle for graph operations.
    • V(label?: string | string[]): GraphQuery — Start a vertex query within the transaction.
    • E(label?: string | string[]): EdgeQuery — Start an edge query within the transaction.
    • commit(): void — Placeholder for commit semantics (no-op until native transactions are available).
    • rollback(): void — Placeholder for rollback semantics (no-op until native transactions are available).
    • [Symbol.dispose](): void — Calls rollback(); enables using tx = db.readTransaction(); pattern.
    • has(propertyKey, value): EdgeQuery — Implemented for propertyKey = "id"; filters edges where either endpoint node has the given id.
    • outV(): GraphQuery, [x] inV(): GraphQuery, [x] otherV(): GraphQuery
    • Terminators: [x] fetchTopology(): TopologicalGraph, [x] fetchCursor(): GraphNodeCursor.

3. Low-level DB API (Ladybug-style)

Handle-based Database, Connection, QueryResult, Pool. Result path is Arrow-only (schema + chunks).

3.0 Connecting to a database (quick example)

import { Database, Connection } from "ladybug-node-zero-rs";

// 1) Open or create a database (file path or :memory:)
const db = new Database("example.lbug");
db.initSync(); // or: await db.initAsync();

// 2) Open a connection
const conn = new Connection(db, 1);
conn.initSync(); // or: await conn.initAsync();

// 3) Run a Cypher query
const result = conn.querySync("RETURN 1 AS value");
console.log(result.getNumTuples()); // 1
result.closeSync();

// 4) Close connection and database
conn.closeSync();
db.closeSync();

Using the pool:

import { createPool } from "ladybug-node-zero-rs";

const pool = createPool({ databasePath: "example.lbug", maxSize: 4 });
await pool.initAsync();

const value = await pool.runAsync(async (conn) => {
  const result = await conn.queryAsync("RETURN 1 AS value");
  const all = result.getAllSync();
  result.closeSync();
  return all;
});

console.log(value);
pool.closeSync();

3.1 Database

  • Database — Class. Handle to native DB; must be initialized before use.
    • constructor(path: string, options?: DatabaseOptions)
    • path: string (readonly)
    • handle: number | null
    • initAsync(): Promise — Real async: addon runs create off main thread.
    • initSync(): void — Blocking.
    • closeAsync(): Promise — Currently delegates to closeSync (implementation is sync).
    • closeSync(): void

3.2 Connection

  • Connection — Class. One connection per Database; used for Cypher execution.
    • constructor(database: Database, numThreads?: number)
    • handle: number | null
    • initAsync(): Promise — Currently delegates to initSync (implementation is sync).
    • initSync(): void — Blocking.
    • queryAsync(statement: string): Promise — Async wrapper around sync query using setImmediate to avoid blocking the event loop.
    • querySync(statement: string): QueryResult — Blocking.
    • scanNodeTableSync(nodeLabel: string, columns: string[]): QueryResult — MATCH (n:Label) RETURN n.col1, ...; safe identifiers only.
    • scanRelSync(relType: string, columns?: string[]): QueryResult — MATCH (a)-[r:RelType]->(b) RETURN ...; default columns source, target.
    • scanRelsSync(relTypes: string[], columns?: string[]): QueryResult — One Cypher with UNION of MATCHes for each rel type; same columns for all. Requires Ladybug Cypher UNION support.
    • closeAsync(): Promise — Currently delegates to closeSync (implementation is sync).
    • closeSync(): void
    • prepareAsync(statement): Promise, [x] prepareSync(statement): PreparedStatement — Async wrapper around prepareSync using setImmediate.
    • executeAsync(ps, params?, options?): Promise, [x] executeSync(ps, params?): QueryResult — Async wrapper around executeSync using setImmediate; options (QueryOptions) accepted but currently ignored.
    • ping() — Health check; runs a trivial Cypher statement and throws on failure.
    • explain(statement) — Returns engine-specific textual/JSON plan via EXPLAIN statement.
    • getNumNodes(nodeName), [x] getNumRels(relName) — Implemented via COUNT Cypher queries over node labels and relationship types.
    • registerStream(name, source, options), [ ] unregisterStream(name) — Removed from the public API surface for now. Use loadArrow directly for ingest.
    • loadArrowSync(batches: IngestColumnBatch[], options: LoadArrowOptions): void — Synchronous bulk ingest helper implemented in the native addon (connectionLoadArrowSync) via a batched UNWIND Cypher under the hood. Intended for fixtures and small/medium batches on the current thread.
    • loadArrowAsync(batches: IngestColumnBatch[], options: LoadArrowOptions): Promise — Async bulk ingest that runs work in the addon’s worker pool (connectionLoadArrowAsync); does not block the Node.js event loop and is recommended for large ingests.

3.3 QueryResult

  • QueryResult — Class. Wraps addon result handle; Arrow schema and chunks, plus basic row API.
    • constructor(resultHandle: number) — Internal; from Connection.query / querySync.
    • handle: number
    • getArrowSchemaSync(): string — Arrow schema as JSON string (compatibility path).
    • getNextArrowChunkSync(chunkSize?: number): string — Next chunk as JSON; empty string when done. Default chunkSize 8192.
    • getArrowSchemaBinarySync(): Uint8Array — Arrow IPC schema message (binary).
    • getNextArrowChunkBinarySync(chunkSize?: number): Uint8Array — Next Arrow IPC chunk (schema + one record batch); empty when done.
    • getAllChunksBinaryAsync(chunkSize?: number): Promise<Uint8Array[]> — Async; all IPC chunks. Prefer over JSON for bulk/index builds.
    • recordBatches(chunkSize?: number): AsyncGenerator — Parse IPC chunks with apache-arrow and yield RecordBatches; columnar access without manual parse. Depends on apache-arrow.
    • getAllChunksAsync(chunkSize?: number): Promise<string[]> — Async wrapper around sync getNextArrowChunkSync using setImmediate (compatibility).
    • Symbol.asyncIterator: AsyncGenerator — for-await over chunks.
    • toStream(chunkSize?: number): ReadableStream
    • closeSync(): void
    • getNumTuples(), [x] getColumnNames(), [x] getColumnDataTypes(), [x] getQuerySummary()
    • hasNext, [x] getNextSync(), [x] getAllSync(), [ ] resetIterator() — Not implemented; currently throws, no native iterator reset.
    • toString()

3.4 PreparedStatement

  • PreparedStatement — From Connection.prepare / prepareSync.
    • isSuccess(): boolean
    • getErrorMessage(): string

3.5 Pool

  • createPool(options: PoolOptions): Pool
  • Pool — Class. One shared Database; up to maxSize Connection instances.
    • initSync(): void, [x] initAsync(): Promise
    • acquireAsync(): Promise
    • release(conn: Connection): void
    • runAsync(fn: (conn: Connection) => Promise): Promise
    • closeSync(): void
    • closeAsync(): Promise — Currently delegates to closeSync (implementation is sync).

3.6 Constants and errors

  • LBUG_DATABASE_LOCKED — Error code when DB file is locked.
  • VERSION — Adapter library version derived from package.json (Node layer). Currently reflects the NPM package version, not the underlying Ladybug C++ library version.
  • QueryOptions.signal (AbortSignal) — Reserved in type; not wired.
  • QueryOptions.progressCallback — Reserved in type; not wired.

4. Addon binding (low-level)

Direct use of the native addon (tools/rust_api). All sync addon methods block the calling thread; async addon methods run work in addon worker and do not block the main thread.

  • getAddon(): AddonBinding
  • AddonBinding
    • databaseCreateSync(path): number
    • databaseCreateAsync(path): Promise
    • databaseCloseSync(dbHandle): void
    • databaseConnectSync(dbHandle, numThreads): number
    • connectionCloseSync(connHandle): void
    • connectionQuerySync(connHandle, statement): number
    • connectionQueryAsync(connHandle, statement): Promise
    • connectionPrepareSync(connHandle, statement): number
    • connectionExecuteSync(connHandle, psHandle, paramsJson): number
    • preparedStatementCloseSync(psHandle): void
    • preparedStatementIsSuccessSync(psHandle): boolean
    • preparedStatementGetErrorMessageSync(psHandle): string
    • queryResultGetArrowSchemaSync(resultHandle): string
    • queryResultGetNextArrowChunkSync(resultHandle, chunkSize): string
    • queryResultGetArrowSchemaBinarySync(resultHandle): Uint8Array
    • queryResultGetNextArrowChunkBinarySync(resultHandle, chunkSize): Uint8Array
    • getAllArrowChunksBinaryAsync(resultHandle, chunkSize): Promise<Uint8Array[]>
    • queryResultCloseSync(resultHandle): void
    • queryResultGetNumTuplesSync(resultHandle): number
    • queryResultGetColumnNamesSync(resultHandle): string[]
    • queryResultGetColumnDataTypesSync(resultHandle): string[]
    • queryResultHasNextSync(resultHandle): boolean
    • queryResultGetNextRowSync(resultHandle): string
    • getAllArrowChunksAsync(resultHandle, chunkSize): Promise<string[]>

5. Package exports

  • package.json — main, types, exports pointing at the API entry.
  • src/api/index.ts (or entry) exports:
    • Database, DatabaseOptions
    • Connection, QueryResult, Pool, createPool
    • NodeID, LbugValue, QueryOptions, PoolOptions, QuerySummary
    • getAddon, AddonBinding
    • PreparedStatement, LBUG_DATABASE_LOCKED, VERSION, STORAGE_VERSION
    • NativeGraphDB, TopologicalGraph, GraphNodeCursor, GraphQuery, EdgeQuery

Async contract summary

Method Returns Promise Work runs off main thread
Database.initAsync Yes Yes (addon)
Database.closeAsync Yes No (currently sync)
Connection.initAsync Yes No (currently sync)
Connection.queryAsync Yes Yes (addon)
Connection.closeAsync Yes No (currently sync)
Connection.loadArrowAsync Yes Yes (addon AsyncTask)
QueryResult.getAllChunksAsync Yes Yes (addon)
Pool.initAsync, acquireAsync, runAsync Yes acquire/run may block in pool thread; init currently sync
Pool.closeAsync Yes No (currently sync)

Any method marked "currently sync" must not be documented as non-blocking; prefer adding real async later or renaming to make sync explicit.