diff --git a/.zenodo.json b/.zenodo.json index d9896f3a..43a0bdc0 100644 --- a/.zenodo.json +++ b/.zenodo.json @@ -1,8 +1,8 @@ { - "title": "Catalog of Second Chances (CSC) - Digital Database and Corresponding Interfaces for ReUse of Architectural Components (v0.6.3.1)", - "version": "0.6.3.1", + "title": "Catalog of Second Chances (CSC) - Digital Database and Corresponding Interfaces for ReUse of Architectural Components (v0.6.4.0)", + "version": "0.6.4.0", "upload_type": "software", - "description": "

The Catalog of Second Chances (CSC) is a prototypical digital component database. It catalogs digital representations of uniquely identifiable architectural components for reuse. Furthermore, it provides corresponding interfaces and tools to leverage them via a backend REST-API, a web-frontend as well as a interface for direct interaction within Rhino Grasshopper.

Changes in v0.6.3.1 (2026-10-21)

A hardening release after 0.6.3.0.

Changed

Security

Server notes

Full history: CHANGELOG.md

", + "description": "

The Catalog of Second Chances (CSC) is a prototypical digital component database. It catalogs digital representations of uniquely identifiable architectural components for reuse. Furthermore, it provides corresponding interfaces and tools to leverage them via a backend REST-API, a web-frontend as well as a interface for direct interaction within Rhino Grasshopper.

Changes in v0.6.4.0 (2026-10-23)

Search and the location map.

Added

Changed

Fixed

Security

Server notes

Full history: CHANGELOG.md

", "creators": [ { "name": "Eschenbach, Max Benjamin", diff --git a/CHANGELOG.md b/CHANGELOG.md index c682bc0e..31eb3f49 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,97 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [0.6.4.0] - 2026-10-23 + +Search and the location map. + +### Added + +- **"What fits?":** a Fits section in Browse's More filters finds pieces by size, whatever their + orientation: *about this size* (within a tolerance in mm or %, default 10 %, closest first) or + *can hold this size* (every side at least the asked one, least waste first). Each row shows how + far off it is or how much it wastes, as a share of the piece and in m3 (per piece on batches). The + list can also sort by volume, and the card view has a sort menu. +- **Near:** a Near section in Browse's More filters finds pieces around a point (a pasted "lat, + lon"), by their current location or by their origin, within a radius in km or simply nearest + first. Each row shows its distance, and the list sorts by it. A component page links to "Near + this piece". +- **Origin coordinates:** the origin form of a piece takes the latitude and longitude of its + origin place; until now they could be set only when the piece was created. In every location + form a pasted "lat, lon" fills both boxes, and a comma after the latitude moves on to the + longitude. +- **Location Map:** a new page next to the Similarity Map shows the pieces on a map, by their + current location or by their origin, in clusters with counts. It takes the filters of Browse + (with a "Show as list" link back), colours the pieces like the Similarity Map, opens a piece's + preview on a click, and offers "Search near here" on a click into the map. The map images come + from the tile server of FOSSGIS e.V. (tile.openstreetmap.de); only this page loads them. +- **Similar pieces:** a card on the component page shows the six pieces nearest in shape, over + the whole shape descriptor rather than on the two-dimensional Similarity Map; size counts as + well as the outline. "Show more" brings up to 24. The nearest pieces are also available at + `GET /identities/{id}/similar`, with every Browse filter applied before the ranking. +- **MCP:** `search_components` takes `fit_mode`, `size_mm` and `tolerance`, `near_lat`, + `near_lon`, `near_layer` and `radius_km` (each result with its distance), and `sort` and + `order`; its description now says what the text query matches (name, id prefix, catalog + number). The new tool `catalog_overview` counts the pieces of a search by material, original + function, dataset, shape class and circulation, with the range of each side in mm, how many + have a location and how many are not public. `similar_components` now ranks by the shape + descriptor (with `basis`, `count` and the filters of `search_components`) instead of the + distance on the Similarity Map. +- **Grasshopper:** the new `FetchSimilarComponents` returns the pieces nearest in shape to a + piece, with their distance, and takes the filters, the dimension search and the near search of + `FetchFilteredComponents`. `FetchFilteredComponents` gains the dimension search (`FitMode`, + `FitSize`, `FitTolerance`) and the near search (`NearPoint`, `NearLayer`, `RadiusKm`), with the + deviation or waste (`Fit`) and the distance (`DistanceKm`) per result. +- **Grasshopper, texture colours:** `CreateComponentIdentity` and `CreateComponentSnapshot` give + a mesh the colours of its texture when it is staged: when the mesh comes from a Rhino object + whose material has a base-colour image, each vertex takes the colour at its texture + coordinate, on every level of detail. The texture replaces vertex colours the mesh already + has, with a remark. Meshes without such a texture are staged as before. Grasshopper library + 261022: run `CSC_Update`. + +### Changed + +- **Registration:** without open domains the catalog is invite only, and the register page then + asks for an invitation code and suggests no open sign-up. With open domains the page says that + registration is open to certain domains and lists them behind a "?"; no page names a fixed + university any more. `CSC_OPEN_REGISTRATION_DOMAINS` is now empty by default: a server that + keeps open registration must set it explicitly. +- **Privacy page and imprint:** they state that the maintainer runs the catalog in person and is + its sole controller, and that TU Darmstadt does not operate it and is not responsible for it. + The privacy page names the tile server of the Location Map and what it receives, and no longer + claims that nothing is loaded from other sites; the imprint cites the DDG instead of the + repealed TMG. The Content-Security-Policy allows images from that one tile host in addition. +- **No Google map preview:** pointing at a location in the Browse table no longer loads an + embedded Google map; the location is a link that opens Google Maps in a new tab on a click. + With no embed left, the Content-Security-Policy allows no frames (`frame-src 'none'`). + +### Fixed + +- **Browse table:** at middle widths its columns no longer run into each other; the table keeps + a minimum width for the columns shown and scrolls sideways inside its frame. The same for the + admin Materials table. +- **Statistics:** one piece with a malformed size in the database can no longer make the + statistics (and with them Analytics) fail for everyone. + +### Security + +- **Next.js 16.4.1:** the web app moves from Next.js 16.4.0 to 16.4.1, which fixes two critical + advisories (GHSA-2464-fhr4-w9xw, code execution through the image optimizer with SVG files; + GHSA-p3m5-59mp-jv5r, denial of service through Server Actions). The catalog was not reachable + through either: it does not use the image optimizer and has no Server Actions. Upgraded as a + precaution; `npm audit` is clean again. + +### Server notes + +- After the deploy, run `scripts/db_maintenance/backfill_geo_points.py --apply` once: it stores + the location points the near search uses for the existing pieces (new and edited pieces get + them on their own). Without `--apply` it only reports, and it exits with 4 while points are + missing or out of date. +- After the deploy, run the map job once (`main_component_map.py`, or `--knn-only` for the + similarity data alone): until it has run, similar pieces answer 409 and the card stays hidden. + The similarity data is then rebuilt with the map on every scheduled run; a new or reshaped + piece joins at the next run. + ## [0.6.3.1] - 2026-10-21 A hardening release after 0.6.3.0. diff --git a/README.md b/README.md index 1b856b1f..5ab02022 100644 --- a/README.md +++ b/README.md @@ -29,8 +29,8 @@ framework ## Current Versions -- **CSC**: 0.6.3.1 --- backend, web frontend and Grasshopper interface are released together - under one version (tag `v0.6.3.1`); the single source is the `VERSION` file. +- **CSC**: 0.6.4.0 --- backend, web frontend and Grasshopper interface are released together + under one version (tag `v0.6.4.0`); the single source is the `VERSION` file. See `CHANGELOG.md` for release notes. @@ -547,11 +547,16 @@ and send you the link, so look at the large address line (the host you will be sent back to) before you press Allow, and press Deny when it is not the tool you just connected. -**What it can do.** Nine read-only tools: `search_components` (the filters of -Browse, a page at a time), `get_component`, `list_versions`, `get_version` -(geometry as counts and a link, never the meshes), `get_evidence`, -`list_datasets`, `get_dataset`, `get_timeline` and `similar_components` (the -nearest neighbours on the similarity map, with their distance). Every answer +**What it can do.** Ten read-only tools: `search_components` (the filters of +Browse, a page at a time), `catalog_overview` (how many pieces, counted by +material, original function, dataset, shape class and circulation, the range of +each sorted side in millimetres, and how many have a location; it takes the same +filters), `get_component`, `list_versions`, `get_version` (geometry as counts +and a link, never the meshes), `get_evidence`, `list_datasets`, `get_dataset`, +`get_timeline` and `similar_components` (the pieces nearest in shape over the +whole descriptor, not on the two-dimensional similarity map, with their distance +in standardised units; it takes the same filters, which apply before the +ranking). Every answer carries the web link of the record, and a record that is not public is marked (`visibility` private, `status` other than published). There are no write tools and no tool for users, invitations or tokens; no answer holds an e-mail address diff --git a/VERSION b/VERSION index a50a1dcf..31d34d20 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.6.3.1 +0.6.4.0 diff --git a/docs/adr/DATA_MODEL_SPEC.md b/docs/adr/DATA_MODEL_SPEC.md index 92dfda9d..66cd903a 100644 --- a/docs/adr/DATA_MODEL_SPEC.md +++ b/docs/adr/DATA_MODEL_SPEC.md @@ -1121,8 +1121,8 @@ shown anywhere on its own: it appears through the records it fills (section 3.3. well (the account and its `default_role` stay). **Registration (decision 8.14).** Self-registration is open to addresses of -`CSC_OPEN_REGISTRATION_DOMAINS` (env, comma-separated, default `tu-darmstadt.de`, subdomains -included), with the verification email as in 0.5. Every other address needs an **invitation**: +`CSC_OPEN_REGISTRATION_DOMAINS` (env, comma-separated, subdomains included; empty by default, which +means invite only, 8.172), with the verification email as in 0.5. Every other address needs an **invitation**: ```jsonc // invitations diff --git a/docs/adr/DESIGN_DECISIONS.md b/docs/adr/DESIGN_DECISIONS.md index 2a57b90f..ee0dbe0e 100644 --- a/docs/adr/DESIGN_DECISIONS.md +++ b/docs/adr/DESIGN_DECISIONS.md @@ -148,7 +148,7 @@ into one **metadata object**. | 8.11 | **The client header is required for writes and every logged-in request; anonymous GETs may omit it; a scanned non-public piece shows a "not public" page** (review item 5, user 2026-09-29). Writes and requests with a login token: header required, known client, version >= minimum, else 426. Anonymous GET / HEAD: header optional (the `/id/{uuid}` resolver for any scanner, curl / Python readers of public pieces --- CPR Art 76(2)(e) free access). A known client below its minimum: 426 always. User: an anonymous scan of a non-public piece shows a page saying the component is not public, with a sign-in link. | Every GH UserObject logs in (`CSC_Session` sends the token), so old ones --- also those before 0.5.1.0 without any header --- can neither write nor misread 0.6 data. Enforcement is on iff `CSC_MIN_CLIENT_VERSIONS` is set (production sets it in the cutover deploy; dev and tests stay log-only). Exempt: CORS preflight (`OPTIONS`), all of `/ghinterface/` (the updater also calls version / download / name lists), `/docs`, `/redoc`, `/openapi.json`, `/health`, `/version`. `/id/{uuid}` no longer hides existence behind a 404-equivalent: whoever scans a tag holds the piece. Anonymous --> "not public" + sign-in (callback to the piece); logged in without access --> "no access" page, no dataset details. Spec section 7.4, section 7.5. | | 8.12 | **Verification has owners: `self_attested` is the recorder's own claim, `reviewed` and `accredited` need a second person** (review item 6, user 2026-09-30). (1) `self_attested`: set by the recorder (evidence form checkbox "I performed this and stand by the result"), only when `performed_by` contains an actor with `user_id == recorded_by_user_id`; an outside lab's result stays `unverified` until reviewed. The recorder toggles `unverified` <--> `self_attested` and never overrides a reviewer's state. (2) `reviewed` / `accredited`: `reviewer(D)` or admin who is neither the recorder nor a `performed_by` user --- **no admin exception**; server sets `verification.by` (the reviewer as a user actor) and `at`. (3) The accreditation block stays typed in; `accredited` is a reviewer's act with I22 as precondition and a **required `verification.note`** saying what was checked (the reviewer view links the body's public register, e.g. DAkkS). No curated organisation registry in 0.6. (4) Changing a result field (`summary`, `payload`, `derived`, `performed_by`, `standard`, `method*`, `observed_at*`, `sampled_at*`, `position.snapshot_id` / `point`) while draft / pending resets `reviewed` / `accredited` to `unverified`. | Closes: a reviewer could mark their own record `reviewed` (factor 1.00); anyone could type an ISO 17025 block and satisfy I22; nobody could set `self_attested`. New **I27** (document check). A one-person dataset cannot reach `reviewed` without a second person. Spec section 3.3.3, section 5, section 7.0, section 7.2. | | 8.13 | **Attachment files are downloaded only by signed-in users** (review item 7, user 2026-09-30: "to reduce moderation overhead"). Anonymous readers of an `is_public` piece see the attachment list (`media_type`, `size`, `sha256`, `uploaded_at`) and "sign in to download"; signed-in users who can see the record download as before. Snapshot photos stay public (general impression, EXIF-stripped, 7.13). The upload control carries a "?" notice: professional roles are fine; private persons, faces, number plates are not --- redact before upload. `POST /actors/redact` also returns a worklist of attachments on records naming the person (`performed_by`, `recorded_by`, `uploaded_by`); a moderator adds a redacted copy and removes the original with reason `gdpr`, which also blanks the tombstone's `name` (`sha256` kept). No content scanning. | Actor redaction and the organisation-only projection could not reach inside files (lab reports name technicians, photos show people); a per-attachment public flag was rejected (a review step per file). Spec section 3.1.4, section 3.3, section 7.2, section 7.3. | -| 8.14 | **Registration without an open-domain address is by email-bound invitation only** (user 2026-09-30: 0.6 needs non-TU registration; "only email bound invitations, no open codes"). `admin` invites anyone; `moderator(D)` invites into its own dataset. An invitation names one email address, holds a single-use code (16-char base32, 80 bit; only `code_sha256` stored), expires after 14 days by default, is sent by mail as `/auth/register?code=...` and can be revoked while unused. Registration with a code must use the invited address; the address then counts as verified (the code proves receipt). Optional dataset + roles are granted on redemption (a moderator only in its own dataset). Batches = a list of addresses, one invitation each. Self-registration stays open for `CSC_OPEN_REGISTRATION_DOMAINS` (env, default `tu-darmstadt.de`, subdomains included). | New collection `invitations` (spec section 3.7), routes in section 7.7, permission rows in section 7.0, `/admin` + dataset member editor gain "invite" (plan P3). Rejected: open codes handed over in person (a leaked code admits anyone). | +| 8.14 | **Registration without an open-domain address is by email-bound invitation only** (user 2026-09-30: 0.6 needs non-TU registration; "only email bound invitations, no open codes"). `admin` invites anyone; `moderator(D)` invites into its own dataset. An invitation names one email address, holds a single-use code (16-char base32, 80 bit; only `code_sha256` stored), expires after 14 days by default, is sent by mail as `/auth/register?code=...` and can be revoked while unused. Registration with a code must use the invited address; the address then counts as verified (the code proves receipt). Optional dataset + roles are granted on redemption (a moderator only in its own dataset). Batches = a list of addresses, one invitation each. Self-registration stays open for `CSC_OPEN_REGISTRATION_DOMAINS` (env, default `tu-darmstadt.de`, subdomains included). Amended by 8.172: no open domain set means invite only, and the web names the open domains instead of TU Darmstadt. | New collection `invitations` (spec section 3.7), routes in section 7.7, permission rows in section 7.0, `/admin` + dataset member editor gain "invite" (plan P3). Rejected: open codes handed over in person (a leaked code admits anyone). | | 8.15 | **The frame is the valid frame closest to the stored coordinates** (review item 8, user 2026-09-30). Candidates: the OBB axes assigned to x / y / z in extent order (7.10, columns standing), extents within `max(2 %, 3 mm)` of each other count as tied and may swap, either sign per axis, right-handed only; the candidate with the smallest rotation from the stored axes (max `trace(R)`) wins, exact ties by a fixed candidate order. Round sections: trimesh's angle about the long axis is accepted (deterministic on the same input; `bbx` unaffected). | One rule gives both the sign rule and the tie-break; re-runs are stable; a beam uploaded along world X keeps +X and DDU stones stay near their gripper-plane pose. Signs are not intrinsic: rescans in other poses may differ by a 180 degree flip (`bbx`, filters, shape class unaffected; snapshot comparison needs registration, not in 0.6). Rejected: third-moment (skew) signs --- flip on near-symmetric pieces. Spec section 4.3. | | 8.16 | **Minor review items b--d** (user 2026-09-30). (b) At most one open correction per record: at most one `draft` / `pending` record with `supersedes == X` (I14 extended; a second --> 409 naming the open one). (c) Evidence on a withdrawn identity --> 409 (with `duplicate_of`, the 409 names the canonical identity); on a terminal exit (`split`, `merged`, `recycled`, `disposed`) only with `observed_at <= exit.at`; on a non-terminal exit (`installed`, `returned`, `lost`) allowed (resolves `after_exit`, 8.10). (d) Positioned evidence keeps its `position.snapshot_id` and stored-coordinate point when that snapshot is corrected in new coordinates; the viewer draws it only on that snapshot; the correction dialog warns "N evidence records are positioned on the previous version" and offers to supersede each (re-placed point / re-imported bars). No stored transform between versions in 0.6. | New I28 (evidence vs. withdrawn identity / terminal exit; corpus check). Spec section 3.3.2, section 5, section 7.2. | | 8.17 | **"Withdrawn" means one thing at every level: the full record for the dataset, a tombstone for everyone else** (minor review item a, user's proposal refined, 2026-09-30). Identity, snapshot and evidence alike: members of D and admin see content, files and the reason; everyone else who can see the component (anonymous on `is_public`, logged-in non-members of a `catalog` dataset) gets **200 with a tombstone** --- id, kind, `version`, `status: withdrawn`, withdrawal date, link to the identity's current snapshot (or `duplicate_of`). The reason is never public; files of withdrawn records are members only; the public timeline shows "v2 --- withdrawn (date)". 410 stays for purged records. Withdrawing the current snapshot falls back automatically to the latest remaining published, non-superseded snapshot (naming a replacement is optional); none left --> `current_snapshot_id = null`, identity out of default lists, `/id` shows "no current state". Reinstate restores. | Before, withdrawn records followed the component's visibility, so a public piece kept showing exactly what a moderator had withdrawn (wrong piece, bad scan, person in a photo). I3b: "if not null, references a published snapshot". `can_see_record` returns a projection `full | tombstone | none`. Content that must vanish for members too: photo / attachment removal (tombstone entry) or purge. Spec section 3.1.4, section 5, section 7.0, section 7.1, section 7.5. | @@ -298,11 +298,19 @@ into one **metadata object**. | 8.161 | **Username character set** (user 2026-10-17; 0.6.2.0 T5, from 8.144 a; amends 8.28). A username matches `^[a-z0-9][a-z0-9._-]{2,31}$`: 3 to 32 characters, lowercase ASCII letters, digits, `.`, `_`, `-`, starting with a letter or digit; no `@`, so a username never looks like an e-mail address and the sign-in identifier is never ambiguous. Registration lowercases the input before the check (8.28); the web form checks the same pattern and shows it as a hint. No non-ASCII letters and no Unicode normalisation. Checked against the 31 stored names (local dump 261016: lowercase ASCII, length 3--14, `-` and `_` only): every one passes, so no migration and no rename; a test runs the pattern against the newest local dump when one is present (skipped otherwise, nothing committed). | The length was the only rule (`models.py`, 3--50); look-alike letters from other scripts and names that look like addresses are excluded at the source. | | 8.162 | **Per-account limit of failed sign-ins** (user 2026-10-17; 0.6.2.0 T5, from 8.144 a). `POST /auth/token` counts failed sign-ins per identifier: at most 10 in 15 minutes, then 429 with `Retry-After` until the window moves on; a successful sign-in is not counted; no permanent lock, nothing for an admin to unlock. The key is an HMAC-SHA256 of the lowercased identifier with a key derived from `JWT_SECRET`, so the counter collection holds no username or address (8.144 h). An unknown identifier is counted and answered exactly like a known one (no account test). The counters live in the shared MongoDB store of 8.152. The per-address limit (10 a minute) stays. The 429 text points to the password reset, which keeps its own limit (3 an hour per address). | Rotating addresses defeat a per-address limit alone; a targeted 15-minute lock-out of one account is accepted at CSC's size. | | 8.163 | **Request bodies rejected before they are read** (user 2026-10-17; 0.6.2.0 T5, from 8.144 a). An ASGI middleware in front of every route caps the body per route group from the settings: the mesh and point cloud uploads `GEOMETRY_UPLOAD_LIMIT_MB`, evidence attachments `EVIDENCE_UPLOAD_LIMIT_MB`, photos `SNAPSHOT_PHOTO_UPLOAD_LIMIT_MB`, each plus 1 MB for the multipart overhead; every other route (JSON) 16 MB through a new `CSC_JSON_BODY_LIMIT_MB`; the remote runner's result route keeps its own limits. A `Content-Length` above the cap answers 413 before any byte is read; without one (chunked) the middleware counts the bytes and answers 413 as soon as the cap is passed. `read_upload_limited` stays as the second check. | Starlette spools a whole multipart body to a temporary file before the route runs, and JSON bodies had no cap; Uberspace's web backend passes requests straight to gunicorn, so the web server cannot cap them per route. | -| 8.164 | **Content-Security-Policy: scripts by nonce, styles inline allowed** (user 2026-10-17; 0.6.2.0 T6, completes 8.144 g). The web app sends one CSP header, built per request in `proxy.ts` (its matcher then covers every page path, auth gated inside the function): `default-src 'self'`; `script-src 'self' 'nonce-' 'strict-dynamic' 'wasm-unsafe-eval'` (wasm for the meshopt decoder drei always sets up; local development adds `'unsafe-eval'` and `ws:` for HMR); `style-src 'self' 'unsafe-inline'`; `img-src 'self' blob: data:` (plus the static host when `NEXT_PUBLIC_STATIC_BASE_URL` is set); `connect-src 'self'`; `font-src 'self'`; `worker-src 'self' blob:`; `media-src 'self' blob:`; `frame-src https://maps.google.com https://www.google.com`; `object-src 'none'`; `base-uri 'none'`; `form-action 'self'`; `frame-ancestors 'none'`. The nonce goes to Next's own scripts (automatic) and to next-themes (`nonce` prop from the request headers). The static `frame-ancestors` CSP of `lib/securityHeaders.ts` is merged into it (one CSP header); `X-Frame-Options` and the other headers of 8.144 g stay. Rollout (user, same day): enforced at once, no Report-Only phase and no report endpoint. Before the release the implementer walks the production build (`next build` + `next start`, not dev) through the main pages in a browser with the console checked for violations (home, Browse, map, component page with mesh, point cloud and proxy overlay, History with the swimlane PNG export, photos, QR scanner, the Google map, sign-in, registration, admin, mail preview, GH interface); a test checks one CSP header, a fresh nonce per request and the directives. Emergency switch: `CSC_CSP_REPORT_ONLY=1` in the frontend environment sends the same policy as `Content-Security-Policy-Report-Only` (restart of the frontend). | Every page already renders per request (`force-dynamic`), the app has no inline script of its own and talks to its own origin only, so a strict script policy costs little; styles stay inline-allowed because sonner, Radix's scroll lock, styled-jsx and the mail preview inject un-nonced `