Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# Existing Photo Workflow Repair Plan

## Goal

Make the active kernelCAD Studio generation path accept a sourced, scaled photo
reference for simple consumer-device CAD, then prove the result through the
current runtime and physical checks.

## Tasks

1. Repair the e-reader acceptance artifact: verified scale anchor, valid image
dimensions, and parametric static-assembly metadata. Add regressions first.
2. Add an explicit photo-reference request contract to the active Studio UI:
file, MIME/name/hash, and known dimension. Write client tests before UI code.
3. Add a matching hosted API/server contract that validates and materializes
the reference, then supplies a concise photo-device brief to the existing
authoring-and-gate loop. Write route/orchestrator tests before implementation.
4. Make trace confidence fail closed for the light-device/internal-screen case;
tracing is evidence, not a device generator.
5. Run focused client/server/runtime tests plus e-reader evaluate, interference,
and hidden-reference render inspection. Review every changed boundary before
handoff.

## Out of scope

- No client-side direct vision-chat bypass or legacy HeadlessKernel repair.
- No public Meshy/Tripo integration.
- No claim that a single image yields an exact branded, manufacturable, or
electronically functional device.
Original file line number Diff line number Diff line change
@@ -0,0 +1,226 @@
# Photo-to-Parametric Consumer Device Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Make KernelCAD's existing reference-image capability usable for a simple consumer-electronics photo by shipping a tested, parameterized e-reader reconstruction and its agent workflow.

**Architecture:** A new child skill under the existing `kernelcad-from-reference` orchestrator constrains what an agent may infer from a photo and points to the appropriate CAD checks. A checked-in public-domain reference image, provenance note, and `.kcad.ts` acceptance model demonstrate the flow. An integration test executes the actual script and guards the source-level contract.

**Tech Stack:** TypeScript, KernelCAD authoring API, Vitest, Sharp-backed `referenceImage()` validation, KernelCAD CLI rendering.

---

### Task 1: Lock the acceptance contract with a failing integration test

**Files:**
- Create: `tests/integration/examples/photoReferenceEreader.test.ts`
- Create later: `examples/from-reference/e-reader/kindle-2-e-reader.kcad.ts`
- Create later: `examples/from-reference/e-reader/kindle-2-reference.jpg`
- Create later: `examples/from-reference/e-reader/PROVENANCE.md`

- [ ] **Step 1: Write the failing test**

```ts
import { existsSync, readFileSync } from 'node:fs';
import { dirname, resolve } from 'node:path';
import { describe, expect, it } from 'vitest';
import { evaluateAndBuildScript } from '../../../src/agent/cli/commands/evaluate';
import { checkInterference } from '../../../src/agent/script-runtime/checkInterference';

const EXAMPLE = 'examples/from-reference/e-reader/kindle-2-e-reader.kcad.ts';
const REFERENCE = 'examples/from-reference/e-reader/kindle-2-reference.jpg';

describe('photo-reference e-reader example', () => {
it('turns a sourced photo into a parametric, evaluatable device model', async () => {
expect(existsSync(REFERENCE)).toBe(true);
const source = readFileSync(EXAMPLE, 'utf8');
expect(source).toContain('// Real Object Brief');
expect(source).toContain("referenceImage('./kindle-2-reference.jpg'");
for (const name of ['bodyWidth', 'bodyHeight', 'bodyThickness', 'screenWidth', 'screenHeight']) {
expect(source).toContain(`param('${name}'`);
}

const built = await evaluateAndBuildScript({ file: EXAMPLE });
expect(built.evaluation.exitCode).toBe(0);
expect(built.evaluation.diagnostics.filter((d) => d.severity === 'error')).toEqual([]);
expect(built.evaluation.featureCount).toBeGreaterThanOrEqual(7);
const referenceRecord = built.model?.records.find((record) => record.kind === 'referenceImage');
expect(referenceRecord).toBeDefined();
expect((referenceRecord?.metadata?.path as string | undefined)?.endsWith('kindle-2-reference.jpg')).toBe(true);
expect(referenceRecord?.metadata?.diagnostics ?? []).toEqual([]);

const interference = await checkInterference({
code: source,
fileName: EXAMPLE,
scriptDir: dirname(resolve(EXAMPLE)),
epsilonMm3: 0.01,
});
expect(interference.partCount).toBe(4);
expect(interference.pairs).toEqual([]);
});
});
```

- [ ] **Step 2: Run it to verify it fails for missing acceptance artifacts**

Run:

```bash
/home/andrii/.nvm/versions/node/v22.22.0/bin/node ./node_modules/vitest/vitest.mjs run tests/integration/examples/photoReferenceEreader.test.ts --reporter=dot
```

Expected: FAIL because the e-reader source and reference image do not yet exist.

- [ ] **Step 3: Commit the red test only**

```bash
git add tests/integration/examples/photoReferenceEreader.test.ts
git commit -m "test: define photo-reference e-reader contract"
```

### Task 2: Add source-owned reference evidence and the e-reader model

**Files:**
- Create: `examples/from-reference/e-reader/kindle-2-reference.jpg`
- Create: `examples/from-reference/e-reader/PROVENANCE.md`
- Create: `examples/from-reference/e-reader/kindle-2-e-reader.kcad.ts`

- [ ] **Step 1: Download the public-domain Kindle 2 front image from Wikimedia Commons**

Run:

```bash
curl -L --fail --silent --show-error \
'https://commons.wikimedia.org/wiki/Special:FilePath/Amazon-kindle-gen2.jpg' \
-o examples/from-reference/e-reader/kindle-2-reference.jpg
```

- [ ] **Step 2: Record immutable source facts in `PROVENANCE.md`**

Include the Wikimedia file page, author Evan-Amos, public-domain license, download date, SHA-256, intended internal benchmark use, and the explicit trade-dress limitation.

- [ ] **Step 3: Implement the minimal parameterized model**

Use a Real Object Brief followed by these named parameters: `bodyWidth`, `bodyHeight`, `bodyThickness`, `cornerRadius`, `bezelWidth`, `screenWidth`, `screenHeight`, `screenRecess`, `controlDiameter`, `usbPortWidth`, and `usbPortHeight`. Use `referenceImage()` on the XZ plane, an `extrudeRoundedRect()` body, a shallow screen recess, a seated display plate, a recessed circular control, status LED, and a through USB-C opening. Apply PBR material to leaf geometry before every boolean or union. Return `assembly('photo-reference-e-reader').model()` with exactly four named, non-overlapping parts: housing, display, navigation control, and status LED.

- [ ] **Step 4: Run the test and make it pass**

Run the Task 1 command again.

Expected: PASS with one file and no error diagnostics.

- [ ] **Step 5: Commit source, reference, provenance, and green test**

```bash
git add examples/from-reference/e-reader tests/integration/examples/photoReferenceEreader.test.ts
git commit -m "feat: add photo-reference e-reader example"
```

### Task 3: Make the workflow reusable for agents

**Files:**
- Create: `src/agent/skills/kernelcad-from-reference/photo-to-device/SKILL.md`
- Modify: `src/agent/skills/kernelcad-from-reference/SKILL.md`
- Create: `tests/unit/skill/photoToDeviceSkill.test.ts`

- [ ] **Step 1: Write the failing skill contract test**

```ts
import { readFileSync } from 'node:fs';
import { describe, expect, it } from 'vitest';

const SKILL = 'src/agent/skills/kernelcad-from-reference/photo-to-device/SKILL.md';

describe('photo-to-device skill', () => {
it('requires a scale anchor and labels mesh output as visual reference only', () => {
const source = readFileSync(SKILL, 'utf8');
expect(source).toContain('known dimension');
expect(source).toContain('visual_mesh_reference');
expect(source).toContain('kernelcad evaluate');
expect(source).toContain('kernelcad interference');
});
});
```

- [ ] **Step 2: Run it to verify it fails for the missing skill**

Run:

```bash
/home/andrii/.nvm/versions/node/v22.22.0/bin/node ./node_modules/vitest/vitest.mjs run tests/unit/skill/photoToDeviceSkill.test.ts --reporter=dot
```

Expected: FAIL with `ENOENT` for the new skill path.

- [ ] **Step 3: Implement the skill and route to it from the orchestrator**

The new skill must cover: suitable categories, required scale anchor, visible-versus-inferred facts, e-reader/handheld enclosure feature checklist, required provenance, optional tracing, `visual_mesh_reference` terminology, and the deterministic evaluate/interference/render-inspect gates. Add it as the right-angled consumer-device path after `blockout-model` in the orchestrator.

- [ ] **Step 4: Run the skill test and distribution regression tests**

Run:

```bash
/home/andrii/.nvm/versions/node/v22.22.0/bin/node ./node_modules/vitest/vitest.mjs run \
tests/unit/skill/photoToDeviceSkill.test.ts \
tests/unit/cli/walkSkillTree.test.ts \
tests/unit/cli/skillInstallRecursion.test.ts \
--reporter=dot
```

Expected: PASS; recursive discovery makes the new child available to installed clients.

- [ ] **Step 5: Commit the reusable workflow**

```bash
git add src/agent/skills/kernelcad-from-reference tests/unit/skill/photoToDeviceSkill.test.ts
git commit -m "feat: guide photo-to-parametric device builds"
```

### Task 4: Run the physical and visual proof packet

**Files:**
- Read only: `examples/from-reference/e-reader/kindle-2-e-reader.kcad.ts`
- Generated outside the repository: `/tmp/kernelcad-e-reader-inspect/`

- [ ] **Step 1: Build the CLI**

```bash
/home/andrii/.nvm/versions/node/v22.22.0/bin/node ./node_modules/typescript/bin/tsc -p tsconfig.cli.json --noEmit
/home/andrii/.nvm/versions/node/v22.22.0/bin/node scripts/build-cli.mjs
```

- [ ] **Step 2: Run deterministic gates**

```bash
/home/andrii/.nvm/versions/node/v22.22.0/bin/node dist/cli/index.js evaluate examples/from-reference/e-reader/kindle-2-e-reader.kcad.ts --json
/home/andrii/.nvm/versions/node/v22.22.0/bin/node dist/cli/index.js interference examples/from-reference/e-reader/kindle-2-e-reader.kcad.ts --json
```

Expected: zero error diagnostics, four assembly parts, and zero interference pairs.

- [ ] **Step 3: Create and inspect the render packet**

```bash
/home/andrii/.nvm/versions/node/v22.22.0/bin/node dist/cli/index.js render inspect \
examples/from-reference/e-reader/kindle-2-e-reader.kcad.ts \
/tmp/kernelcad-e-reader-inspect \
--channels rgb,mask,depth,normals \
--hide-reference-images
```

Open a canonical RGB output and verify the body, screen recess, circular control, LED, and port are visibly present and the object has non-trivial depth.

- [ ] **Step 4: Run focused regression tests and typecheck**

```bash
/home/andrii/.nvm/versions/node/v22.22.0/bin/node ./node_modules/vitest/vitest.mjs run \
tests/integration/examples/photoReferenceEreader.test.ts \
tests/unit/skill/photoToDeviceSkill.test.ts \
tests/integration/agent/trace-from-image-smoke.test.ts \
tests/unit/cli/skillInstallRecursion.test.ts \
--reporter=dot
/home/andrii/.nvm/versions/node/v22.22.0/bin/node ./node_modules/typescript/bin/tsc -b --noEmit
```

Expected: all focused tests and typecheck pass.
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
# Existing Photo Workflow Repair

## Status

Re-scoped on 2026-07-14 after direct reproduction showed that kernelCAD already
has a photo button, but that button is not connected to the current generated
CAD and verification path.

## Observed failure

The legacy AI Assistant uploads a data URL directly to a vision chat request,
then returns markdown. It does not retain a reference asset, run
`trace_from_image`, or enter the current agent loop. Its Apply and Preview
actions run `.kcad.ts` source through the obsolete `HeadlessKernel`, which only
injects `replicad` and `console`; ordinary `param()`-based kernelCAD source
therefore fails before the Studio runtime can evaluate it.

The active Studio surface (`StudioGenerate`) has the right staged-review and
modern-runtime path but currently accepts only text/mesh context, not a photo.

## Decision

Repair the active Studio generation path rather than revive the legacy panel.
A photo-assisted build must include:

- a bounded, typed reference asset and its SHA-256 provenance;
- at least one `known dimension` supplied by the user;
- a restricted first-use scope: simple, front-on consumer electronics and
passive hobby enclosures;
- a photo-aware brief passed to the hosted authoring agent;
- the existing current-runtime evaluate/gate loop before a staged edit is
offered; and
- a clear distinction between observed image facts and inferred depth or
internals.

`trace_from_image` remains an optional coordinate aid. It is not a device
generator and must not return a confident outer silhouette when it has only
found an internal dark screen. Any Meshy/Tripo-style result remains a
`visual_mesh_reference`, not authoritative CAD.

## Architecture

```text
Studio photo + known dimension
|
v
typed reference request (mime, filename, hash, scale)
|
v
hosted generate route materializes a managed reference asset
|
+--> optional trace_from_image evidence
|
v
photo-device brief + existing authoring agent
|
v
current evaluate / assembly-interference gates
|
v
staged edit -> setCode -> current Studio runtime
```

## Acceptance criteria

- A selected image and known dimension reach the active `StudioGenerate`
request as structured reference input, not an untracked chat attachment.
- The server validates and materializes that input, records its hash, and
passes a photo-specific brief into the verified authoring loop.
- The first consumer-device acceptance source has a real cited scale anchor,
a reference image with positive decoded dimensions, connected parametric
assembly metadata, zero interference, and a reviewed render.
- The system fails honestly when a photo cannot establish a usable silhouette
or scale; it does not label an internal display as the outer device.
- Robots, mechanisms, electronics internals, and manufacture-ready claims
remain outside the one-photo consumer-device path.
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# Photo-to-Parametric Consumer Device Design

## Status

Approved for implementation by the user's explicit overnight request on 2026-07-14. This document records the bounded first milestone rather than waiting for an interactive review.

## Problem

KernelCAD already exposes `referenceImage()` and `trace_from_image`, but an agent receiving a product photograph has no focused, evidence-backed recipe for turning a simple consumer-electronics reference into editable CAD. A raw image-to-mesh result is useful visual evidence but cannot establish hidden geometry, dimensions, manufacturability, or electronics fit.

## Decision

Ship a native photo-to-parametric path for simple, slab-like consumer devices. The acceptance artifact is a public-domain Kindle 2 front photograph reconstructed as an editable e-reader enclosure with a display recess, navigation control, status LED, and USB-C opening. The model will explicitly state its inferred dimensions and hidden-side assumptions.

The path is deliberately provider-neutral:

- A photo plus at least one known dimension is the required input.
- `referenceImage()` stays available as the visual alignment aid.
- `trace_from_image` is optional for organic silhouettes, not required for rectangular devices.
- Any later Meshy, Tripo, Rodin, or photogrammetry output is named `visual_mesh_reference`; it must never be represented as a KernelCAD, STEP, BREP, or manufacturable result.

## Scope

1. Add a `photo-to-device` child skill beneath `kernelcad-from-reference` with the decision rules, provenance requirements, and deterministic gates for consumer electronics.
2. Add a checked-in, public-domain e-reader reference photograph and provenance file.
3. Add one parameterized e-reader `.kcad.ts` artifact with a complete Real Object Brief and conservative physical assumptions.
4. Add an integration test that proves the example, its reference asset, and its key editable parameters remain valid.
5. Produce an explicit render-inspection packet during verification.

## Non-goals

- No claim that one image recovers a real product's PCB, battery, fasteners, material stack, or exact dimensions.
- No public proto.cat upload surface or direct handoff to its current mesh endpoint.
- No external image-to-mesh provider integration, keys, billing, or network dependency.
- No attempt to use the visual acceptance model as a branded production clone; it is an internal reference-reconstruction benchmark.

## Architecture

```text
reference photo + known width
|
v
Real Object Brief and provenance
|
v
parametric KernelCAD source (.kcad.ts)
|
+--> evaluate / interference / DFM
|
+--> render inspect packet and visual review
```

The skill owns agent behavior. The example owns the concrete first-use-case source and proves the API surface. The integration test protects both from drift without making a network request.

## Acceptance criteria

- The e-reader source has named, bounded dimensions for body, display, bezel, control, and USB-C opening.
- It contains a Real Object Brief with known facts, inferred facts, and validation focus.
- The local reference photo is validated by `referenceImage()` during evaluation.
- `kernelcad evaluate` has no errors, the four-part static assembly reports no interference pairs, and the rendered model visibly reads as a real, layered e-reader rather than a flat card.
- The reusable skill instructs agents to ask for a known dimension, model visible geometry first, label inferred internals, and route mechanisms to the assembly/physical-review workflow.

## Follow-on integration boundary

proto.cat should eventually accept an authenticated, stored reference identifier rather than arbitrary image URLs, derive a device brief, and invoke the same parametric KernelCAD path. Its current visual mesh route is not the handoff contract: it needs a separate security and provenance hardening pass before it can be used for this feature.
Loading
Loading