Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 49 additions & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,48 @@ The [development guide](./docs/development/repository-guide.md) goes deeper: [re
- **Search existing issues.** The bug may already be tracked.
- **Keep changes focused.** One concern per pull request. A refactor, a bug fix and a feature do not share a commit.

## Choose a first contribution

Start with [unassigned good first issues](https://github.com/ClickHouse/loom/issues?q=is%3Aissue%20is%3Aopen%20label%3A%22good%20first%20issue%22%20no%3Aassignee%20-label%3Ablocked%20-label%3Ahold)
or [unassigned help wanted issues](https://github.com/ClickHouse/loom/issues?q=is%3Aissue%20is%3Aopen%20label%3A%22help%20wanted%22%20no%3Aassignee%20-label%3Ablocked%20-label%3Ahold).
Read the issue's latest comments and linked pull requests before starting: someone can
be working on it without an assignee. Check [open pull requests](https://github.com/ClickHouse/loom/pulls)
for the same change. Help wanted includes larger design work; the label alone does not
make a task suitable for a beginner.

Choose one reproducible problem and a small acceptance check. If the issue has no active
claim or pull request, describe the part you intend to work on in its thread. A question,
a documentation correction and a failing example are useful contributions too; use the
[issue forms](#filing-an-issue) when you need to report a problem.

| Start from | Read | First check |
| -------------------------------------- | -------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------- |
| A broken instruction or link | [Documentation style](./docs/development/documentation-style.md) | Run the link check below, replacing the page parameter with the page you edited. |
| A Python bug with a small reproduction | [Repository map](./docs/development/repository-guide.md), [testing](./docs/development/testing.md) | Run the affected test file or one test with `python -m pytest path/to/test.py::test_name -q`. |
| A lab or course project | [Teaching guide](./docs/guides/teaching.md), [course starter](./examples/course-starter/README.md) | Run the starter's grader against your own local stack; it fails until the TODOs are completed. |

After the Python environment in [Development setup](#development-setup), this checks one
page without starting a database, server or model provider:

```bash
python -m pytest 'tests/unit/test_doc_links.py::test_relative_links_resolve[CONTRIBUTING.md]' -q
```

A successful run reports `1 passed`. If it fails, fix the reported target or heading.
The link check verifies local targets and headings, not external sites or the accuracy of
instructions. Check the changed commands and explanations too. Unit tests use isolated
configuration and test doubles; integration tests need a private stack
([testing](./docs/development/testing.md)). Keep tokens, personal data and customer data
out of reproductions. Security reports follow [SECURITY.md](./SECURITY.md).

Finish with the [branch and checks workflow](#branch-change-check) and the
[commit sign-off](#commits-and-sign-offs-dco). In the pull request, say what reproduces the
problem, what changes and which focused checks you ran. CI runs the wider checks; read
each failed job rather than treating an unrun check as passing. After a merge, use the
same issue searches to choose a related test, example or small fix for your next contribution.
The repository's [contributor history](https://github.com/ClickHouse/loom/graphs/contributors)
and merged pull requests record contributions.

## Filing an issue

Every issue goes through a form in `.github/ISSUE_TEMPLATE/`; pick the one that fits:
Expand Down Expand Up @@ -53,7 +95,13 @@ cd loom
python3 -m venv .venv
. .venv/bin/activate
pip install -e ".[dev]"
```

Documentation checks and unit tests need only this environment. No database, running
server or provider account is needed for the focused check above. Continue below when
your change needs a running server:

```bash
# Start ClickHouse locally, with the config and lemmatizer dictionary Loom needs
# (a bare clickhouse-server container fails every memory_embeddings insert)
bash dev/ci_start_clickhouse.sh
Expand All @@ -79,7 +127,7 @@ docker compose --env-file .env -f compose.yaml -f dev/docker-compose.dev.yml --p

1. Branch from `main`. CI runs on pull requests into `main` or a `release/X.Y` line and on pushes to them. A pull request stacked on another branch gets no checks until it targets one of them.
2. Make the change with its tests ([testing](./docs/development/testing.md)). If it moves a source that something is generated from, regenerate it ([code generation](./docs/development/code-generation.md)). If it changes the schema, follow [migrations](./docs/development/migrations.md).
3. Run what CI runs:
3. Run focused checks for the changed behavior first. CI runs the following wider gates; use [testing](./docs/development/testing.md) to choose the relevant local checks:

```bash
make format # ruff, Biome, Prettier, Taplo, shfmt (CI: make format-check)
Expand Down
41 changes: 34 additions & 7 deletions docs/guides/teaching.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,15 @@

Loom can serve as the lab environment for a course on AI agents, information retrieval,
databases or software engineering. Each student runs their own single-user stack, on a
laptop with Docker or in a GitHub codespace with nothing installed on their machine. Each
lab links the page that does its work.
laptop with Docker. A hosted codespace is optional and has its own billing. Each lab
links the page that does its work.

## Setup

| Students have | Start with |
| ------------------------------ | ------------------------------------------------------------------------------------------- |
| A GitHub account and a browser | [Try it in GitHub Codespaces](../getting-started/quickstart.md#try-it-in-github-codespaces) |
| Docker on their own machine | [Quickstart: fast path](../getting-started/quickstart.md#fast-path-released-image-no-clone) |
| Students have | Start with |
| ------------------------------ | ------------------------------------------------------------------------------------------------------------------------- |
| Docker on their own machine | [Quickstart: fast path](../getting-started/quickstart.md#fast-path-released-image-no-clone) |
| A GitHub account and a browser | [Try it in GitHub Codespaces](../getting-started/quickstart.md#try-it-in-github-codespaces), after checking billing below |

A codespace uses the student's Codespaces quota while it runs and while its disk is kept, and
GitHub deletes one left unused for its retention period, with its data
Expand Down Expand Up @@ -81,7 +81,10 @@ graded by hand: their exercises ask for explanations.
A grading run takes about two minutes, most of it starting the stack. In a private
repository those minutes count against the Actions quota of the account or organization
that owns it ([Actions billing](https://docs.github.com/en/billing/concepts/product-billing/github-actions));
codespaces cost what [Setup](#setup) says.
codespaces cost what [Setup](#setup) says. Standard hosted compute in public repositories
is free under those terms; artifact and cache storage have separate allowances. Local
grading avoids hosted compute charges. Keep student submissions, grades and personal
data private when deciding how to distribute and grade the work.

## Lab 1: an agent that remembers

Expand Down Expand Up @@ -183,3 +186,27 @@ Exercises:
| Information retrieval | 1, 2 | [Search and ask](../features/memory/search-and-ask.md), [evaluate Loom](../getting-started/evaluate.md) |
| Databases | 1, 4 | [Storage authority](../architecture/storage-authority.md), [SQL surface](../reference/sql-surface.md) |
| Software engineering | 3 | [Code index](../features/code-index/overview.md), [PR history](../features/code-index/history-and-prs.md) |

## Bounded projects after the labs

Use the existing labs as the starting point. Each project produces a small reproducible
artifact and an explanation of its limits, without generated answers or paid model calls.

| Project | Deliverable | Acceptance |
| ----------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Retrieval evaluation from Lab 2 | A synthetic document, answerable questions with labeled passages, unanswerable questions and the search script. | Separate cutoff selection from held-out questions. Report recall@k and false positives and false negatives for the chosen similarity cutoff; retain the raw hits and labels. |
| Call graph investigation from Lab 3 | A small repository fixture you may redistribute, expected caller relationships and commands for both index and literal search. | Explain each missed or extra relationship, including dynamic calls; do not present static results as a complete runtime graph. |
| Memory history from Lab 1 | A script that writes, revises and reads one synthetic fact and its versions. | Show the before/after values and provenance, then repeat on a fresh namespace without affecting another student's data. |

An instructor can assess reproducibility, correctness of the labels or expected results,
and the explanation of failure cases separately. Preserve the repository commit, released
image digest, SDK version, configuration, commands and raw results with the submission.
License original fixtures explicitly; preserve the license and notices for reused material.
Use synthetic data rather than student records or private documents. A local retrieval
exercise does not establish general answer quality or a published benchmark result.

A student who finds a reproducible defect can follow the
[first contribution path](../../CONTRIBUTING.md#choose-a-first-contribution). Search for an
existing issue and active work first; a course project does not reserve upstream work or
promise acceptance. Use the repository's [software citation metadata](../../CITATION.cff)
and record the exact source commit or release used in a report.
26 changes: 20 additions & 6 deletions examples/course-starter/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,10 @@ Four labs on [Loom](https://github.com/ClickHouse/loom), a memory and search sto
agents. You run your own Loom, in a GitHub codespace or on your laptop, and write small
Python programs and a SQL query against it. Each lab's text, exercises and background
are in [the course guide](https://github.com/ClickHouse/loom/blob/main/docs/guides/teaching.md).
No API key and no paid service is needed.
The labs need no model API key. The local laptop route needs no hosted service.

## Start Loom

**In a codespace:** on this repository's page, choose **Code**, then **Codespaces**, then
**Create codespace**. Setting it up takes several minutes; when the terminal says Loom
is running, the dashboard is on the forwarded port 7777 (the **Ports** tab). In each new
terminal, run `. .venv/bin/activate`.

**On a laptop with Docker:** install Loom with the
[quickstart's fast path](https://github.com/ClickHouse/loom/blob/main/docs/getting-started/quickstart.md#fast-path-released-image-no-clone),
then, in this repository's directory, with Python 3.11 or newer:
Expand All @@ -21,6 +16,19 @@ then, in this repository's directory, with Python 3.11 or newer:
python3 -m venv .venv && . .venv/bin/activate && pip install -r requirements.txt
```

Use your own machine for a route without cloud compute charges. Docker, the stack's
images and data still need local memory and disk space; check the
[setup requirements](https://github.com/ClickHouse/loom/blob/main/docs/getting-started/quickstart.md#prerequisites)
before starting.

**In a codespace (optional):** check your
[compute and storage billing](https://docs.github.com/en/billing/concepts/product-billing/github-codespaces)
first. A codespace uses its owner's quota and can incur charges; the lab does not grant
an allowance. On this repository's page, choose **Code**, then **Codespaces**, then
**Create codespace**. Setting it up takes several minutes; when the terminal says Loom
is running, the dashboard is on the forwarded port 7777 (the **Ports** tab). In each new
terminal, run `. .venv/bin/activate`.

## The labs

| Lab | You write | Checked by |
Expand All @@ -46,3 +54,9 @@ reason, and exits non-zero while any check fails. On a laptop it finds the stack

Every push runs the same checks on GitHub Actions, on a fresh Loom: open the **Actions**
tab, then the latest **Grade** run, whose summary lists each check.

Standard hosted runner compute for a public repository is free under
[Actions billing](https://docs.github.com/en/billing/concepts/product-billing/github-actions).
Private repository compute and artifact or cache storage have separate allowances. You
can run the grader locally without enabling hosted grading; choose repository visibility
for your work's privacy, not to publish private answers or data for free compute.