Skip to content

SPEC.md

Latest commit

 

History

History
418 lines (322 loc) · 14.3 KB

SPEC.md

File metadata and controls

418 lines (322 loc) · 14.3 KB

.table/ format specification

Spike-status. Versioned by formatVersion: 1 in meta.json. Will move before 1.0.

A .table/ is a directory that IS a file. The extension is .table. The directory contains plain-text files designed for line-diffable storage in git, plus an optional rebuildable SQLite cache.

1. Directory layout

my-data.table/
├── schema.json          REQUIRED
├── rows.ndjson          REQUIRED
├── views.json           OPTIONAL
├── meta.json            OPTIONAL — implicit empty if missing
├── attachments/         OPTIONAL
├── bodies/              OPTIONAL
│   └── {row.id}.md
└── index.sqlite         OPTIONAL — gitignored rebuildable cache

Readers MUST tolerate any of the optional members being absent. Readers MUST ignore unknown files at the directory root. Writers SHOULD preserve unknown files on round-trip (treat them as the user's domain).

2. schema.json

Defines the fields of every row, their types, and any constraints.

{
  "fields": [
    {
      "name": "title",
      "type": "string",
      "constraints": { "required": true }
    },
    {
      "name": "status",
      "type": "string",
      "constraints": { "enum": ["planning", "active", "done"] }
    },
    {
      "name": "summary",
      "type": "string",
      "format": "markdown"
    }
  ],
  "primaryKey": ["title"],
  "schema-version": 1
}

Field types

string, number, integer, boolean, date, datetime, time, year, array, object, duration, geopoint, geojson.

Field type names follow standard conventions used across text-first data formats. .table/ is not aligned with any specific format — borrowing the names is for readability, not compatibility.

Field constraints

required, unique, enum, minimum, maximum, minLength, maxLength, pattern.

Field annotations

  • format: "markdown" — the value is markdown content. Renderer hint; no validator effect.
  • attachment: true — the value is a filename inside attachments/.
  • relation: { table: "<name>", field: "id" } — the value points at a row in a sibling .table/ directory by that table's system id. Resolution is the app's concern (scan workspace, walk parent, etc.).
  • deprecated: true — the field is kept for backwards compatibility but should not be shown in new UIs.

Schema evolution

Schema evolution is append-only. Never remove or rename a field. Add new fields; mark old ones deprecated: true to retire them. Reordering enum values changes sort/group semantics and SHOULD bump schema-version.

Manifest declaration

Schema files MAY include third-party extension keys with the x- prefix. Readers MUST ignore unknown keys; writers MUST preserve them on round-trip.

3. rows.ndjson

One JSON object per line. Each row is independently parseable. Blank lines are skipped. The file ends with a trailing \n (POSIX-correct).

{"id":"p1","title":"Workspace v1","status":"active","priority":1}
{"id":"p2","title":"Table file format spike","status":"done","priority":2}

System id

Every row MUST have an id field at the top level — a stable nanoid, 21 chars, URL-safe alphabet. The id is system-level: it is minted by the writer, never edited by the user, and never declared in schema.json (it is implicit on every row).

id is what cross-table relation references resolve against. primaryKey (if present) is a separate domain-level uniqueness constraint over user-facing fields, not row identity.

Row ordering

Row order in rows.ndjson is not semantically meaningful. Display order belongs in views (sort and group). Append-friendly: appending a new row to the end MUST be a valid edit, even mid-document.

4. views.json

A list of saved views. Views are shared/team views only — personal view state (last opened, scroll position, etc.) lives in app-local storage outside the .table/ directory.

[
  {
    "id": "v1",
    "name": "Active by priority",
    "layout": "table",
    "fields": ["title", "status", "priority"],
    "filter": [{ "field": "status", "operator": "neq", "value": "done" }],
    "sort": [{ "field": "priority", "direction": "asc" }]
  }
]

Layouts

table, board, gallery, list, calendar. The layout field is a fixed enum — apps render based on this value.

The name field is free-form and user-facing — it's what shows up in view switchers and lists. Apps must not parse it; it can be in any language and contain any Unicode text. The pair (layout, name) lets the same layout type appear multiple times with different names (e.g. two "table" views named "Active" and "Done").

Filter operators

eq, neq, gt, gte, lt, lte, contains, not_contains, starts_with, ends_with, empty, not_empty, in, not_in.

Sort behaviour

  • For enum fields, sort by the enum's declared order (not alphabetical).
  • Null/undefined values sort last regardless of direction (asc and desc).

Group behaviour

  • Returns rows bucketed by the group field value.
  • Keys are ordered by the field's enum declaration when an enum constraint is present; otherwise by row arrival.
  • Rows with null/undefined/empty group value go into the literal "(empty)" bucket, which always sorts last.

Layout-specific fields

  • board_field — column field for board layouts.
  • gallery_field — hero/lead field for gallery cards.
  • calendar_field — date field for calendar layouts.
  • calendar_range — optional {start, end} (YYYY-MM-DD) bound for layout: "calendar" views. When present, calendar navigation is locked to this window: prev / next disable at the bounds and the initial cursor snaps inside the range. Useful for project calendars (locked to project duration), sprint cycles, event-specific calendars. Apps that don't recognise the field should still render the calendar correctly — they'll just allow free navigation.

5. meta.json

The manifest. Identifies the file as a .table/ and carries descriptive metadata.

{
  "format": "table",
  "formatVersion": 1,
  "title": "Projects",
  "description": "Workspace product roadmap",
  "created_at": "2026-04-01T00:00:00Z",
  "modified_at": "2026-04-27T00:00:00Z",
  "generator": "table-file-format spike fixture"
}

format and formatVersion are stamped on every write. Readers MAY use them to validate that a directory is a .table/.

6. attachments/

Files referenced by row values whose field declares attachment: true. The row stores the filename only (e.g. "avatar": "headshot.png"); the reader resolves under attachments/.

To avoid filename collisions across rows, the recommended write convention is {nanoid}-{original-name}.ext, but the format does not enforce a specific naming scheme — readers MUST resolve the filename verbatim against attachments/.

Orphan-attachment cleanup is an app concern. The format does not specify size limits.

7. bodies/

Optional long-form markdown bodies, one file per row. Filename is the row's stable system id plus .md:

bodies/
├── p1.md
└── p2.md

Each body is a standalone, readable markdown file. Bodies are intended for the canonical long-form content of a row — the Notion every-row-is-also-a-page pattern.

For shorter inline markdown content (a description, a one-paragraph summary), use a string field with format: "markdown" instead.

Rules:

  • A body file MUST have a corresponding row in rows.ndjson. Orphan body files (file present, no row) are a validator warning.
  • A row WITHOUT a body file is valid; bodies are optional per row.
  • Writers wholesale-replace bodies/ on writeTable: bodies not present in the input are removed from disk. Partial-update APIs belong elsewhere (writeBody(dir, id, content) is intended but not yet implemented).

8. index.sqlite

OPTIONAL rebuildable cache for filtered queries and full-text search. Never the source of truth.

  • Excluded from git via .gitignore.
  • When present and fresh, readers SHOULD use it for fast queries.
  • When absent or stale, readers MUST fall back to scanning rows.ndjson.
  • Apps decide their own caching strategy; the format prescribes only the fallback rule.

The interface (buildIndex, queryIndex, isIndexStale, dropIndex) is locked; the implementation is currently a stub.

Field-type → SQLite storage class mapping

When implemented, the cache will materialise rows into a SQLite table typed per-column. Mapping:

Field type SQLite class Notes
string / date / datetime / time / geojson TEXT dates as ISO-8601
number REAL
integer / year INTEGER
boolean INTEGER 0/1
array / object TEXT JSON-encoded
geopoint TEXT or two REAL columns "lat,lon" or split
(any when missing) NULL

9. Validation

A .table/ is valid when:

  1. Every row has a non-empty system id, and ids are unique.
  2. Every row's field values match the declared types and constraints.
  3. If primaryKey is declared, no two rows share the same composite key.
  4. Every body file in bodies/ has a matching row.

Validators MAY surface warnings for: orphaned attachments, dangling relations (cross-table reference to a missing id), unused enum values. None of these block validity.

10. Addressing

Stable addressing for rows, views, and tables — used by:

  • Cross-table relations (per-field relation references one row in another .table/)
  • Cross-format references (a markdown body inside bodies/{id}.md linking to a row elsewhere, an external markdown file linking into a .table/, etc.)
  • Application-level deep links (URL hashes, share links)

Address grammar

An address is a path to a .table/ followed by an optional URL fragment:

<path>[#<key>=<value>[&<key>=<value>]*]
  • <path> — a file-system path to a .table/ directory. Relative or absolute, app-resolved. Format files SHOULD use relative paths (typically against the containing workspace root) so the address survives directory moves.
  • <key>=<value> — a key-value pair. Keys defined by this spec:
    • row=<id> — the row with this system id
    • view=<id> — the view with this id
    • field=<name> — a specific field on the row (cell-level addressing for future affordances; reserved)

Multiple pairs join with & (the same convention as URL query strings). Order is not significant. Unknown keys MUST be tolerated by readers — apps MAY define additional keys (e.g. query=, highlight=) but consumers that don't recognise them should silently ignore.

Examples

docs/projects.table                              # whole table
docs/projects.table#row=p1                       # specific row
docs/projects.table#view=v3                      # specific view
docs/projects.table#row=p1&view=v3               # row pinned to view
../suppliers.table#row=ACME_CORP                 # cross-directory

Resolution

Resolution is the app's concern. The format does not prescribe how a path resolves to a .table/ directory — apps choose (filesystem scan, in-memory map, fetch over HTTP, etc.). Reference helpers in @workspace.sh/table-core (parseAddress, formatAddress, resolveRow) implement the grammar but accept a caller-supplied lookup function.

row= MUST resolve against the system id field. If a target row's id is not found, the address is dangling — apps SHOULD surface this visibly rather than silently rendering nothing.

Relation interop

Per-field relation (§2) references a row by id but does NOT use the address grammar literally — relations are structured as {table, field} on the field declaration plus the bare id value on the row, so apps can resolve them efficiently without parsing a string. Conceptually, a relation {table: "tasks", field: "id"} with row value "t_42" corresponds to the address <path-to-tasks.table>#row=t_42 — the grammar is the serialisation; the relation declaration is the structured form.

Reverse direction (markdown → row)

A markdown body inside bodies/{id}.md MAY contain links that use the address grammar:

See [the active sprint](../sprints.table#row=sp_current) for the
plan.

How those links are rendered, opened, or scrolled is the consuming app's concern. The format only standardises the grammar.

11. Interop

.table/ belongs to an open coalition of text-first data interchange formats (CSVW, Frictionless Data, Obsidian Bases, etc.) — none gets top billing. Direct converters in core: CSV (lossy export, lossless import with schema). Format-specific exporters belong in separate optional packages (@workspace.sh/table-frictionless, @workspace.sh/table-csvw, etc.) if and when there's demand.

12. Versioning

The spec covers two version axes only — the spec itself, and the schema. Data versioning (edit history, undo, audit, real-time collaboration) is explicitly the consuming app's concern (see docs/DECISIONS.md §D14).

Spec version — formatVersion

The spec itself is versioned by formatVersion in meta.json. The current value is 1. Breaking changes bump the major; additive changes do not. Readers SHOULD warn on formatVersion higher than they recognise but MAY still attempt to read.

Schema version — schema-version

The schema is independently versioned via the schema-version field on schema.json, which the app increments when it changes the schema in ways the app considers significant (typically: reordering enums, changing constraints).

Data versioning (NOT in the format)

The format does not specify a per-row history, edit log, concurrency semantics, or conflict-resolution model. Every NDJSON-row design choice (line-diffability, per-row bodies in separate files, append-friendly ordering) exists so git is the version-control substrate — any consumer that uses git gets full history, diffing, merging, branching, and authorship for free.

Apps that need versioning beyond what git provides (in-app undo, "what did this row look like yesterday," audit trail, real-time collaboration) implement it themselves. A future optional history.ndjson extension is reserved at the directory root for a portable append-only edit log, but it is not yet specified — its shape will be designed when a consumer's UX actually motivates it.