Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 46 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# Both Dockerfiles build from the repo root and do `COPY . .`, so without this
# every image carried the whole working tree -- including `.env`, which holds a
# real GitHub client secret and session secret. A secret in a layer stays in
# that layer whatever the running container does with it.

# Secrets first, because this is the line that matters.
.env
.env.*
!.env.example

# Reinstalled inside the image, and a host node_modules is the wrong platform
# anyway -- the parser's tree-sitter bindings are compiled.
node_modules
**/node_modules

# Build output. The image builds its own; copying a stale one in is how a
# container serves last week's bundle.
dist
**/dist
apps/web/dist
services/parser/bin

# History and tooling that never affect the build, but do bust the cache on
# every commit.
.git
.github
.claude
.agents
**/*.log

# Tests do not run in the image. This also keeps compiled test files out of
# `apps/api/dist`, which CLAUDE.md lists as harmless locally and wrong here.
**/*.test.ts
**/*.test.tsx
**/*_test.go
**/__tests__

# Documentation and plans.
docs
*.md
!README.md

# Editor and OS noise.
.vscode
.idea
.DS_Store
28 changes: 22 additions & 6 deletions .env.example
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
# Local development environment for funcatlas.
# Copy to .env and fill in values. .env is gitignored; this file is committed.
# Copy to .env and fill in values -- or just run `make setup`, which does it
# and generates the secrets for you. .env is gitignored; this file is committed.
#
# Values are left empty rather than <placeholder>: the Makefile sources this
# file with `set -a && . ./.env`, and a shell reads `<` as a redirection, so a
# placeholder made every env-using target fail with "newline unexpected".

# --- App (Fastify API) ---
NODE_ENV=development
Expand Down Expand Up @@ -35,18 +40,29 @@ REDIS_URL=redis://localhost:6379
# Queue name must match between API (producer) and parser (consumer).
QUEUE_NAME=funcatlas-parse

# --- Running without a GitHub OAuth app ---
# Set this to any name and the API registers no /auth/login, /auth/callback or
# /auth/logout, and treats every request as that user. `make setup` writes it
# for you, which is what makes `docker compose up` work from a fresh clone.
#
# It means the instance has NO AUTHENTICATION. Compose publishes on 127.0.0.1
# so nothing off-box can reach it; do not expose the port. Blank means off.
# See docs/RISKS.md R39.
FUNCATLAS_SINGLE_USER=

# --- GitHub OAuth ---
# Register an OAuth app; set its redirect URI to the value below.
GITHUB_CLIENT_ID=<client-id>
GITHUB_CLIENT_SECRET=<client-secret>
# Only needed when FUNCATLAS_SINGLE_USER is blank. Register an OAuth app and
# set its callback to GITHUB_REDIRECT_URI below.
GITHUB_CLIENT_ID=
GITHUB_CLIENT_SECRET=
GITHUB_REDIRECT_URI=http://localhost:3000/auth/callback

# --- Webhooks ---
# HMAC secret for verifying GitHub webhook payloads.
GITHUB_WEBHOOK_SECRET=<webhook-secret>
GITHUB_WEBHOOK_SECRET=

# --- Sessions ---
SESSION_SECRET=<random-32-byte-string>
SESSION_SECRET=
SESSION_COOKIE_NAME=funcatlas_session
SESSION_TTL=604800

Expand Down
11 changes: 8 additions & 3 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -200,9 +200,14 @@ Phase 3b is closed. `TASKLIST.md` is the chunk-level truth; this is what outlive
the insert. Write edges for a file whose functions you did not delete and they silently double.
- **Public repositories only.** The OAuth scope is `read:user`, and the parser clones over public
HTTPS. See R26 for why `repo` was not the answer.
- **`pnpm start` cannot run the API.** `packages/shared` exports point at `./src/*.ts`, so plain
`node dist/index.js` cannot follow them. `pnpm dev` (tsx) works. Fix before containerising.
- **Compiled test files land in `apps/api/dist`.** Harmless locally, wrong in an image.
- **`docker compose up` is the front door, and everything runs under `tsx`.** `packages/shared`
exports point at `./src/*.ts`, so `node dist/index.js` cannot follow them -- the image ships
source and runs tsx, and `pnpm start` does the same. Building shared to `dist` and repointing
`exports` is the cleaner endpoint; it was not done because it puts a build step in front of the
dev loop. `.dockerignore` keeps compiled tests, `node_modules` and `.env` out of every image.
- **`make setup` writes `FUNCATLAS_SINGLE_USER`, so the default stack has no authentication.**
One branch in `requireSession`, no OAuth routes registered, every compose port on `127.0.0.1`,
and a warning on every start. R39 -- read it before assuming this is safe to deploy.
- **Resolution limits that are honest, not broken:** barrel re-export chains, default imports, and
`tsconfig` path aliases all resolve to `unresolved`. So does every per-language construct in
"Per-language extraction limits" — Go's single-type-argument generic call (ambiguous with a
Expand Down
35 changes: 34 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# Run `make <target>`. Most targets shell out to pnpm/turbo or docker.
# `make start` is the one that brings the whole thing up; `make help` lists all.

.PHONY: start stop wait-infra migrate-test install dev build lint typecheck test \
.PHONY: setup start stop wait-infra migrate-test install dev build lint typecheck test \
migrate up down health parser-isolated \
go-build go-build-bin go-test go-vet go-run go-tidy go-lint clean

Expand All @@ -11,6 +11,31 @@
# need a variable from it write `$$DATABASE_URL`, not `$(DATABASE_URL)`.
ENV := set -a && . ./.env && set +a

setup: ## One-time: write .env, generate secrets, create the test database
@test -f .env && echo ".env already exists, leaving it alone." || { \
cp .env.example .env; \
sed -i "s|^SESSION_SECRET=.*|SESSION_SECRET=$$(openssl rand -hex 32)|" .env; \
sed -i "s|^GITHUB_WEBHOOK_SECRET=.*|GITHUB_WEBHOOK_SECRET=$$(openssl rand -hex 32)|" .env; \
sed -i "s|^FUNCATLAS_SINGLE_USER=.*|FUNCATLAS_SINGLE_USER=$$(id -un)|" .env; \
echo "Wrote .env with generated secrets."; \
echo ""; \
echo " FUNCATLAS_SINGLE_USER=$$(id -un) -- the API will run with NO"; \
echo " AUTHENTICATION so you do not need a GitHub OAuth app. Compose"; \
echo " publishes on 127.0.0.1 only. Do not expose the port."; \
echo " Blank the value in .env to use real GitHub sign-in instead."; \
echo ""; \
}
$(MAKE) up
$(MAKE) wait-infra
@# Idempotent: the second run finds the database and says so.
@# No $(ENV) here: the credentials are compose's own, and sourcing .env
@# before it has been filled in is how this used to fail.
docker compose exec -T postgres psql -U funcatlas -d postgres \
-c "CREATE DATABASE funcatlas_test OWNER funcatlas" 2>/dev/null \
&& echo "Created funcatlas_test." || echo "funcatlas_test already exists."
@echo ""
@echo "Ready. Run 'docker compose up' and open http://localhost:5173"

start: ## Bring up EVERYTHING: Postgres, Redis, migrations, parser binary, API, web
@test -f .env || { echo "No .env — copy .env.example to .env first."; exit 1; }
$(MAKE) up
Expand Down Expand Up @@ -56,6 +81,14 @@ typecheck: ## Type-check all packages
pnpm typecheck

test: ## Run all tests (TS + Go)
@# A running compose worker consumes the very jobs queue/parse.test.ts
@# enqueues and asserts on, so the failures read as a broken queue rather
@# than as two things sharing one Redis.
@docker compose ps --services --filter status=running 2>/dev/null | grep -qx worker && { \
echo "The compose worker is running; it will consume the jobs these tests assert on."; \
echo "Run 'docker compose stop worker' first."; \
exit 1; \
} || true
pnpm test
# Sourced, or dbtest finds no DATABASE_URL and every integration test skips
# -- a green run that never touched Postgres.
Expand Down
2 changes: 1 addition & 1 deletion PRD.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ database, saved canvas layouts, real-time multi-user editing. Rationale for each
| NFR-1 | **Performance** — 300-file TypeScript repo parsed and resolved in under 90 s; `functions-for-file` p95 under 150 ms; 5-hop traversal over 10k edges under 500 ms; 60 fps at 2,000 visible nodes. |
| NFR-2 | **Security** — no network egress during parse; an untrusted repo cannot read host files; webhooks are replay- and flood-safe; every graph endpoint is session-gated; secrets come from the environment. |
| NFR-3 | **Correctness** — re-parsing a renamed or deleted function leaves no orphan edges, and resolution never claims certainty it does not have. |
| NFR-4 | **Operability** — one `docker compose up` brings the whole stack up; `/healthz` on api and parser; structured logs (zap in the parser, pino in the api). |
| NFR-4 | **Operability** — one `docker compose up` brings the whole stack up; `/healthz` on api and parser; structured logs (zap in the parser, pino in the api). **Met on the clone-and-run branch**, verified from a clean clone with `make start` never run. |
| NFR-5 | **Maintainability** — shared TypeScript types only in `packages/shared`; one SQL migration source at `services/parser/migrations/`; explicit SQL via sqlx in Go, Drizzle in the API, no full ORM anywhere. |
| NFR-6 | **UX** — dark-mode-first with accent tokens; motion that explains rather than decorates; skeleton loading; actionable errors; `prefers-reduced-motion` respected. |

Expand Down
81 changes: 36 additions & 45 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,69 +45,57 @@ produces *less* reads as one that worked. See [`docs/PARSING_STRATEGY.md`](docs/

## Running it

### Prerequisites
Two commands. You need **Docker**, plus `make` and `openssl` — both already present on macOS and
any Linux.

- **Node 20+** and **pnpm** (`npm i -g pnpm`)
- **Go 1.24+** with a C toolchain (`gcc`) — tree-sitter uses cgo
- **Docker** and **Docker Compose** — for Postgres and Redis
- **golang-migrate** CLI — or the `migrate/migrate` Docker image, as CI does
- A **GitHub OAuth App** — the app has no other way to know who you are
```bash
git clone https://github.com/ARCoder181105/funcatlas.git
cd funcatlas
make setup # writes .env, generates secrets, creates the test database
docker compose up # then open http://localhost:5173
```

Paste a public repository URL and explore it. `⌘K` finds any function by name.

> `docker compose up` does **not** work yet and is the next piece of work: `apps/web` has no
> Dockerfile, there is no worker service, and the API image cannot start. Until then the four
> prerequisites above are all required. See NFR-4 in [`PRD.md`](PRD.md).
> **`make setup` sets `FUNCATLAS_SINGLE_USER`, which means the API runs with no authentication.**
> That is what lets you skip registering a GitHub OAuth app. Compose publishes every port on
> `127.0.0.1`, so nothing off-box can reach it — but do not expose these ports, and do not run it
> this way on a server. Blank the value in `.env` to use real GitHub sign-in instead
> ([`docs/RISKS.md`](docs/RISKS.md) R39).

### Register a GitHub OAuth App
### With real GitHub sign-in

<https://github.com/settings/developers> → **New OAuth App**. The exact values:
Blank `FUNCATLAS_SINGLE_USER` in `.env` and register an OAuth app at
<https://github.com/settings/developers> → **New OAuth App**:

| Field | Value |
|---|---|
| Application name | anything — `funcatlas (local)` |
| Homepage URL | `http://localhost:5173` |
| Authorization callback URL | `http://localhost:3000/auth/callback` |

Generate a client secret and keep both values for the next step. The scope requested is
`read:user` and nothing more — funcatlas never writes to your account, and the token is read once
for your username and then never used again. It clones over public HTTPS, which is why only public
repositories work: the only scope that reads private ones is `repo`, and that also grants **write**
access to every private repository you can reach.
Put the client id and secret into `.env` as `GITHUB_CLIENT_ID` and `GITHUB_CLIENT_SECRET`. The
scope requested is `read:user` and nothing more — funcatlas never writes to your account, and the
token is read once for your username and then never used again. It clones over public HTTPS, which
is why only public repositories work: the only scope that reads private ones is `repo`, and that
also grants **write** access to every private repository you can reach.

### Set up and start
### Working on the code

```bash
git clone https://github.com/ARCoder181105/funcatlas.git
cd funcatlas
pnpm install
cp .env.example .env
```
`docker compose up` has no hot reload. For that, run the services natively — which needs three more
things installed:

Then edit `.env` and fill in four values:

```bash
GITHUB_CLIENT_ID=<from the OAuth app>
GITHUB_CLIENT_SECRET=<from the OAuth app>
GITHUB_WEBHOOK_SECRET=$(openssl rand -hex 32) # unused locally, but required
SESSION_SECRET=$(openssl rand -hex 32)
```

Create the test database once, or `make test` truncates the one you develop against:

```bash
docker compose up -d postgres redis
docker compose exec postgres psql -U funcatlas -d postgres \
-c "CREATE DATABASE funcatlas_test OWNER funcatlas;"
```

Then bring the whole thing up — infra, migrations, the parser binary, the API, the web app and the
parse worker:
- **Node 24+** and **pnpm** (`npm i -g pnpm`) — pnpm 11 does not run on Node 20
- **Go 1.25+** with a C toolchain (`gcc`) — tree-sitter uses cgo
- **golang-migrate** CLI — or the `migrate/migrate` Docker image, as CI does

```bash
make start # then open http://localhost:5173
pnpm install
make start # infra in compose, api + web + worker natively, with watch
```

Sign in with GitHub, paste a public repository URL, and explore it. `⌘K` finds any function by
name.
Stop the compose worker first if it is running: it consumes the queue the tests assert on, and
`make test` will tell you so rather than failing obscurely.

### Without the app

Expand Down Expand Up @@ -140,6 +128,9 @@ Stated here rather than discovered later:

- **Public repositories only.** See the OAuth section above for why.
- **No hosted instance.** You run it.
- **No authentication by default.** `make setup` turns on single-user mode so you can skip
registering an OAuth app. Fine on a laptop behind a loopback-only port map, wrong anywhere else
([`docs/RISKS.md`](docs/RISKS.md) R39).
- **An incremental re-parse still re-parses everything.** The *write* is scoped to changed files;
the clone, extract and resolve are not. Resolution is whole-repo, and a partial symbol table
would emit a confident edge where the whole repository would correctly say ambiguous
Expand Down
36 changes: 36 additions & 0 deletions TASKLIST.md
Original file line number Diff line number Diff line change
Expand Up @@ -184,3 +184,39 @@ available browser window and `resize_window` was ignored, so neither could be ob
dist/index.js` against `packages/shared` exports that point at `.ts`; and the `parser` service
sets both `network_mode: none` and `depends_on` Postgres health. Do **not** resolve the last
by giving the parser a network — that undoes a Phase 1 guarantee.

---

## NFR-4 — `docker compose up` (`clone-and-run`, not a phase)

The promise in `PRD.md` since Phase 0, never once tested. Eight faults, not the six on record:
the `api` build context and the missing `.dockerignore` turned up on re-reading, and three more
only appeared when the images were actually built and run.

- [x] **`.dockerignore`.** First and alone: `COPY . .` with no ignore file put `.env` — a real
client secret — into every image layer.
- [x] **API and worker image.** Root build context, `packages/shared` copied, runs under `tsx`.
Go stage builds the parser with cgo; `git` installed, because `clone.go` shells out to it.
- [x] **Worker service.** Nothing consumed the queue: a webhook answered 202 and the graph never
moved.
- [x] **`migrate` service**, gating api and worker on `service_completed_successfully`.
- [x] **Web image** — Vite build behind nginx with a `try_files` SPA fallback.
- [x] **Parser service** — vestigial `depends_on` dropped, isolation kept, `tools` profile so
`docker compose up` does not start it.
- [x] **`make setup`** and **`FUNCATLAS_SINGLE_USER`**, so a clone needs no GitHub OAuth app.
- [x] **Every port on `127.0.0.1`**, Postgres and Redis included.

**Exit test passed** from a clean clone in /tmp with `make start` never run: `make setup`,
`docker compose up`, register `sindresorhus/p-limit` through the API with no cookie, worker parses
it to `ready` — 6 files, 36 functions, commit SHA recorded.

Found by running it rather than reading it: pnpm 11 will not start on Node 20; `pnpm run` verifies
the workspace before executing, so a build stage needs every manifest; `.env`'s `localhost` means
the container itself; `<placeholder>` values broke every `set -a && . ./.env`; and `.optional()`
rejects a present-but-empty variable, which is the state every fresh `.env` is in.

## Next

- [ ] **NFR-1 — the performance targets have never been measured.** `honojs/hono` is the fixed
benchmark (R3); the timings are not.
- [ ] **FR-7 — several file cards at once.** Cut deliberately in 3b (`UI_GUIDE.md` §6).
Loading
Loading