From d75344dc123b5daa0950c63f119f79bf47690d32 Mon Sep 17 00:00:00 2001 From: zlareb1 Date: Fri, 20 Nov 2026 22:04:07 +0000 Subject: [PATCH 1/2] Document a focused first contribution and local lab path Signed-off-by: zlareb1 --- CONTRIBUTING.md | 50 ++++++++++++++++++++++++++++++- docs/guides/teaching.md | 41 ++++++++++++++++++++----- examples/course-starter/README.md | 26 ++++++++++++---- 3 files changed, 103 insertions(+), 14 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index a7fba601c2..77a3c831f7 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -10,6 +10,48 @@ The [development guide](./docs/development/repository-guide.md) goes deeper: [re - **Search existing issues.** The bug may already be tracked. - **Keep changes focused.** One concern per pull request. A refactor, a bug fix and a feature do not share a commit. +## Choose a first contribution + +Start with [unassigned good first issues](https://github.com/ClickHouse/loom/issues?q=is%3Aissue%20is%3Aopen%20label%3A%22good%20first%20issue%22%20no%3Aassignee%20-label%3Ablocked%20-label%3Ahold) +or [unassigned help wanted issues](https://github.com/ClickHouse/loom/issues?q=is%3Aissue%20is%3Aopen%20label%3A%22help%20wanted%22%20no%3Aassignee%20-label%3Ablocked%20-label%3Ahold). +Read the issue's latest comments and linked pull requests before starting: someone can +be working on it without an assignee. Check [open pull requests](https://github.com/ClickHouse/loom/pulls) +for the same change. Help wanted includes larger design work; the label alone does not +make a task suitable for a beginner. + +Choose one reproducible problem and a small acceptance check. If the issue has no active +claim or pull request, describe the part you intend to work on in its thread. A question, +a documentation correction and a failing example are useful contributions too; use the +[issue forms](#filing-an-issue) when you need to report a problem. + +| Start from | Read | First check | +| -------------------------------------- | -------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------- | +| A broken instruction or link | [Documentation style](./docs/development/documentation-style.md) | Run the link check below, replacing the page parameter with the page you edited. | +| A Python bug with a small reproduction | [Repository map](./docs/development/repository-guide.md), [testing](./docs/development/testing.md) | Run the affected test file or one test with `python -m pytest path/to/test.py::test_name -q`. | +| A lab or course project | [Teaching guide](./docs/guides/teaching.md), [course starter](./examples/course-starter/README.md) | Run the starter's grader against your own local stack; it fails until the TODOs are completed. | + +After the Python environment in [Development setup](#development-setup), this checks one +page without starting a database, server or model provider: + +```bash +python -m pytest 'tests/unit/test_doc_links.py::test_relative_links_resolve[CONTRIBUTING.md]' -q +``` + +A successful run reports `1 passed`. If it fails, fix the reported target or heading. +The link check verifies local targets and headings, not external sites or the accuracy of +instructions. Check the changed commands and explanations too. Unit tests use isolated +configuration and test doubles; integration tests need a private stack +([testing](./docs/development/testing.md)). Keep tokens, personal data and customer data +out of reproductions. Security reports follow [SECURITY.md](./SECURITY.md). + +Finish with the [branch and checks workflow](#branch-change-check) and the +[commit sign-off](#commits-and-sign-offs-dco). In the pull request, say what reproduces the +problem, what changes and which focused checks you ran. CI runs the wider checks; read +each failed job rather than treating an unrun check as passing. After a merge, use the +same issue searches to choose a related test, example or small fix for your next contribution. +The repository's [contributor history](https://github.com/ClickHouse/loom/graphs/contributors) +and merged pull requests record contributions. + ## Filing an issue Every issue goes through a form in `.github/ISSUE_TEMPLATE/`; pick the one that fits: @@ -53,7 +95,13 @@ cd loom python3 -m venv .venv . .venv/bin/activate pip install -e ".[dev]" +``` + +Documentation checks and unit tests need only this environment. No database, running +server or provider account is needed for the focused check above. Continue below when +your change needs a running server: +```bash # Start ClickHouse locally, with the config and lemmatizer dictionary Loom needs # (a bare clickhouse-server container fails every memory_embeddings insert) bash dev/ci_start_clickhouse.sh @@ -79,7 +127,7 @@ docker compose --env-file .env -f compose.yaml -f dev/docker-compose.dev.yml --p 1. Branch from `main`. CI runs on pull requests into `main` or a `release/X.Y` line and on pushes to them. A pull request stacked on another branch gets no checks until it targets one of them. 2. Make the change with its tests ([testing](./docs/development/testing.md)). If it moves a source that something is generated from, regenerate it ([code generation](./docs/development/code-generation.md)). If it changes the schema, follow [migrations](./docs/development/migrations.md). -3. Run what CI runs: +3. Run focused checks for the changed behavior first. CI runs the following wider gates; use [testing](./docs/development/testing.md) to choose the relevant local checks: ```bash make format # ruff, Biome, Prettier, Taplo, shfmt (CI: make format-check) diff --git a/docs/guides/teaching.md b/docs/guides/teaching.md index 5ff57fa6e6..fe4bc57236 100644 --- a/docs/guides/teaching.md +++ b/docs/guides/teaching.md @@ -2,15 +2,15 @@ Loom can serve as the lab environment for a course on AI agents, information retrieval, databases or software engineering. Each student runs their own single-user stack, on a -laptop with Docker or in a GitHub codespace with nothing installed on their machine. Each -lab links the page that does its work. +laptop with Docker. A hosted codespace is optional and has its own billing. Each lab +links the page that does its work. ## Setup -| Students have | Start with | -| ------------------------------ | ------------------------------------------------------------------------------------------- | -| A GitHub account and a browser | [Try it in GitHub Codespaces](../getting-started/quickstart.md#try-it-in-github-codespaces) | -| Docker on their own machine | [Quickstart: fast path](../getting-started/quickstart.md#fast-path-released-image-no-clone) | +| Students have | Start with | +| ------------------------------ | ------------------------------------------------------------------------------------------------------------------------- | +| Docker on their own machine | [Quickstart: fast path](../getting-started/quickstart.md#fast-path-released-image-no-clone) | +| A GitHub account and a browser | [Try it in GitHub Codespaces](../getting-started/quickstart.md#try-it-in-github-codespaces), after checking billing below | A codespace uses the student's Codespaces quota while it runs and while its disk is kept, and GitHub deletes one left unused for its retention period, with its data @@ -81,7 +81,10 @@ graded by hand: their exercises ask for explanations. A grading run takes about two minutes, most of it starting the stack. In a private repository those minutes count against the Actions quota of the account or organization that owns it ([Actions billing](https://docs.github.com/en/billing/concepts/product-billing/github-actions)); -codespaces cost what [Setup](#setup) says. +codespaces cost what [Setup](#setup) says. Standard hosted compute in public repositories +is free under those terms; artifact and cache storage have separate allowances. Local +grading avoids hosted compute charges. Keep student submissions, grades and personal +data private when deciding how to distribute and grade the work. ## Lab 1: an agent that remembers @@ -183,3 +186,27 @@ Exercises: | Information retrieval | 1, 2 | [Search and ask](../features/memory/search-and-ask.md), [evaluate Loom](../getting-started/evaluate.md) | | Databases | 1, 4 | [Storage authority](../architecture/storage-authority.md), [SQL surface](../reference/sql-surface.md) | | Software engineering | 3 | [Code index](../features/code-index/overview.md), [PR history](../features/code-index/history-and-prs.md) | + +## Bounded projects after the labs + +Use the existing labs as the starting point. Each project produces a small reproducible +artifact and an explanation of its limits, without generated answers or paid model calls. + +| Project | Deliverable | Acceptance | +| ----------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Retrieval evaluation from Lab 2 | A synthetic document, answerable questions with labeled passages, unanswerable questions and the search script. | Separate cutoff selection from held-out questions. Report recall@k and false positives and false negatives for the chosen similarity cutoff; retain the raw hits and labels. | +| Call graph investigation from Lab 3 | A small repository fixture you may redistribute, expected caller relationships and commands for both index and literal search. | Explain each missed or extra relationship, including dynamic calls; do not present static results as a complete runtime graph. | +| Memory history from Lab 1 | A script that writes, revises and reads one synthetic fact and its versions. | Show the before/after values and provenance, then repeat on a fresh namespace without affecting another student's data. | + +An instructor can assess reproducibility, correctness of the labels or expected results, +and the explanation of failure cases separately. Preserve the repository commit, released +image digest, SDK version, configuration, commands and raw results with the submission. +License original fixtures explicitly; preserve the license and notices for reused material. +Use synthetic data rather than student records or private documents. A local retrieval +exercise does not establish general answer quality or a published benchmark result. + +A student who finds a reproducible defect can follow the +[first contribution path](../../CONTRIBUTING.md#choose-a-first-contribution). Search for an +existing issue and active work first; a course project does not reserve upstream work or +promise acceptance. Use the repository's [software citation metadata](../../CITATION.cff) +and record the exact source commit or release used in a report. diff --git a/examples/course-starter/README.md b/examples/course-starter/README.md index 959e084736..d729632178 100644 --- a/examples/course-starter/README.md +++ b/examples/course-starter/README.md @@ -4,15 +4,10 @@ Four labs on [Loom](https://github.com/ClickHouse/loom), a memory and search sto agents. You run your own Loom, in a GitHub codespace or on your laptop, and write small Python programs and a SQL query against it. Each lab's text, exercises and background are in [the course guide](https://github.com/ClickHouse/loom/blob/main/docs/guides/teaching.md). -No API key and no paid service is needed. +The labs need no model API key. The local laptop route needs no hosted service. ## Start Loom -**In a codespace:** on this repository's page, choose **Code**, then **Codespaces**, then -**Create codespace**. Setting it up takes several minutes; when the terminal says Loom -is running, the dashboard is on the forwarded port 7777 (the **Ports** tab). In each new -terminal, run `. .venv/bin/activate`. - **On a laptop with Docker:** install Loom with the [quickstart's fast path](https://github.com/ClickHouse/loom/blob/main/docs/getting-started/quickstart.md#fast-path-released-image-no-clone), then, in this repository's directory, with Python 3.11 or newer: @@ -21,6 +16,19 @@ then, in this repository's directory, with Python 3.11 or newer: python3 -m venv .venv && . .venv/bin/activate && pip install -r requirements.txt ``` +Use your own machine for a route without cloud compute charges. Docker, the stack's +images and data still need local memory and disk space; check the +[setup requirements](https://github.com/ClickHouse/loom/blob/main/docs/getting-started/quickstart.md#prerequisites) +before starting. + +**In a codespace (optional):** check your +[compute and storage billing](https://docs.github.com/en/billing/concepts/product-billing/github-codespaces) +first. A codespace uses its owner's quota and can incur charges; the lab does not grant +an allowance. On this repository's page, choose **Code**, then **Codespaces**, then +**Create codespace**. Setting it up takes several minutes; when the terminal says Loom +is running, the dashboard is on the forwarded port 7777 (the **Ports** tab). In each new +terminal, run `. .venv/bin/activate`. + ## The labs | Lab | You write | Checked by | @@ -46,3 +54,9 @@ reason, and exits non-zero while any check fails. On a laptop it finds the stack Every push runs the same checks on GitHub Actions, on a fresh Loom: open the **Actions** tab, then the latest **Grade** run, whose summary lists each check. + +Standard hosted runner compute for a public repository is free under +[Actions billing](https://docs.github.com/en/billing/concepts/product-billing/github-actions). +Private repository compute and artifact or cache storage have separate allowances. You +can run the grader locally without enabling hosted grading; choose repository visibility +for your work's privacy, not to publish private answers or data for free compute. From 20b98805643e6d840d95917a56f24bcbb204ca96 Mon Sep 17 00:00:00 2001 From: zlareb1 Date: Fri, 20 Nov 2026 21:40:07 +0000 Subject: [PATCH 2/2] Replicated erasure fixtures provide isolated write leases Fenced memory inserts require a lease table in the selected schema. Give each fixture database its own Keeper root while preserving the replicated content checks. Signed-off-by: zlareb1 --- .../integration/test_erasure_replicated_ch.py | 5 + tests/unit/test_erasure_replicated_fixture.py | 96 +++++++++++++++++++ 2 files changed, 101 insertions(+) create mode 100644 tests/unit/test_erasure_replicated_fixture.py diff --git a/tests/integration/test_erasure_replicated_ch.py b/tests/integration/test_erasure_replicated_ch.py index a1fa3f60c9..611f07969d 100644 --- a/tests/integration/test_erasure_replicated_ch.py +++ b/tests/integration/test_erasure_replicated_ch.py @@ -74,6 +74,11 @@ def db(raw: Any, monkeypatch: pytest.MonkeyPatch) -> Iterator[str]: parameters={"db": name}, ).result_rows assert engines == [(True,)] + # Fenced memory inserts need leases isolated from the source schema's keys. + raw.command( + f"CREATE TABLE {name}._migration_lock AS {source}._migration_lock " + f"ENGINE = KeeperMap('/{name}/_migration_lock') PRIMARY KEY name" + ) monkeypatch.setattr(settings, "ch_database", name) yield name finally: diff --git a/tests/unit/test_erasure_replicated_fixture.py b/tests/unit/test_erasure_replicated_fixture.py new file mode 100644 index 0000000000..3147871591 --- /dev/null +++ b/tests/unit/test_erasure_replicated_fixture.py @@ -0,0 +1,96 @@ +"""The replicated erasure fixture supplies an isolated write-lease store.""" + +from __future__ import annotations + +from types import SimpleNamespace + +import pytest + +from server.core import leases +from server.settings import settings +from tests.integration import test_erasure_replicated_ch as replicated +from tests.unit.claims_fake import FakeLockTable + + +class Schema: + def __init__(self): + self.tables = {} + self.commands = [] + + def command(self, sql): + self.commands.append(sql) + if sql.startswith("CREATE TABLE "): + self.tables[sql.split()[2]] = sql + + def query(self, sql, parameters=None): + if "engine_full" in sql: + assert parameters["db"] == "source_schema" + return SimpleNamespace(result_rows=[("ReplacingMergeTree(version) ORDER BY tuple()",)]) + assert "SELECT DISTINCT engine" in sql + flags = {"ENGINE = Replicated" in ddl for ddl in self.tables.values()} + return SimpleNamespace(result_rows=[(flag,) for flag in sorted(flags)]) + + +class FixtureLocks(FakeLockTable): + def __init__(self, schema): + super().__init__() + self.schema = schema + + async def query_one(self, sql, params=None): + if "loom._migration_lock" in sql: + if f"{settings.ch_database}._migration_lock" not in self.schema.tables: + raise RuntimeError("Unknown table _migration_lock") + return await super().query_one(sql, params) + + +@pytest.fixture +def isolated_schema(monkeypatch): + monkeypatch.setattr(settings, "ch_database", "source_schema") + schema = Schema() + stream = replicated.db.__wrapped__(schema, monkeypatch) + name = next(stream) + try: + yield schema, name + finally: + stream.close() + assert schema.commands[-1] == f"DROP DATABASE IF EXISTS {name} SYNC" + + +async def test_replicated_erasure_fixture_can_take_and_release_a_write_lease(isolated_schema): + schema, name = isolated_schema + keeper = FixtureLocks(schema) + lease = await leases.take("memory_insert/fixture", ttl_s=60, ch=keeper) + assert lease is not None and lease.held + assert settings.ch_database == name + await lease.release() + assert keeper.keys == {} + + +async def test_a_missing_fixture_lease_table_still_refuses(isolated_schema): + schema, name = isolated_schema + schema.tables.pop(f"{name}._migration_lock", None) + with pytest.raises(leases.LeaseUnavailable): + await leases.take("memory_insert/fixture", ttl_s=60, ch=FixtureLocks(schema)) + + +def test_fixture_lease_paths_are_separate_from_source_and_each_other(monkeypatch): + paths = [] + for _ in range(2): + with monkeypatch.context() as patch: + patch.setattr(settings, "ch_database", "source_schema") + schema = Schema() + stream = replicated.db.__wrapped__(schema, patch) + name = next(stream) + try: + ddl = schema.tables[f"{name}._migration_lock"] + assert "AS source_schema._migration_lock" in ddl + assert f"KeeperMap('/{name}/_migration_lock')" in ddl + assert "PRIMARY KEY name" in ddl + assert all( + "ENGINE = Replicated" in schema.tables[f"{name}.{table}"] + for table in replicated._TABLES + ) + paths.append(name) + finally: + stream.close() + assert paths[0] != paths[1]