feat(modernization): Next.js planning console + JWT/WS API + engine fixes + Helm staging#1
Open
dnplkndll wants to merge 82 commits into
Open
feat(modernization): Next.js planning console + JWT/WS API + engine fixes + Helm staging#1dnplkndll wants to merge 82 commits into
dnplkndll wants to merge 82 commits into
Conversation
…ss CI Modernization groundwork for the Next.js/API/Helm effort: - MODERNIZATION_PLAN.md: phased roadmap (0-4) with verification gates, resolved open questions (licensing=MIT, Django=thin 4.2 fork, scenario routing, same-origin+JWT deploy), and target architecture. - tools/modernization/gates.py: data-driven progress tracker mirroring the plan's gates; renders a live progress table to the CI step summary. - .github/workflows/modernization.yml: fast progress + lint CI (heavy C++ build stays in ubuntu24.yml).
Folds in the engine audit findings: - Rust stance: deferred to evidence-based Engine-track decision (E4 pilot), not assumed. C++ is modern but pointer-heavy with a real safety surface. - DDMRP: classic MRP today; partial primitives exist (decoupled lead time, IP_DATA flag); hybrid solver_ddmrp is a feature project, not a rewrite. - Licensing: confirmed complete ungated MIT Community Edition (no gate). - New gates E1-E4: code review + sanitizers, test hardening, DDMRP mode, Rust/PyO3 pilot decision.
Parallel subsystem review (solver/model/forecast/utils/Django) with file:line evidence, severity-ranked. Headline findings: - Production-reachable bugs: weight[] OOB read (forecast), per-callback PyObject refcount leak (utils), JWT path skips is_active (Django). - Real solver bug confirmed: a_penalty double-count (the in-code TODO). - Model/pegging is the strongest Rust case (UB-on-copy, double-free, iterator-after-erase) AND the least tested (2 pegging tests). - 10-item immediate-fix queue (isolated, high-value, low-risk). Flips E1 review-report gate to active/passing.
Immediate-fix-queue #2 (ENGINE_REVIEW.md). The static weight[500] SMAPE array was indexed weight[count - i] where count = history buckets, which exceeds 500 with the default 10-yr horizon (weekly ~520, daily ~3650) -> out-of-bounds read corrupting forecast method selection. - Add ForecastSolver::smapeWeight() clamping accessor; weights decay exponentially so weight[>=MAXBUCKETS] ~= 0 -> clamping is behavior- preserving and bounds-safe. Replace all 24 read sites in timeseries.cpp. - Fix the divergent runtime initializer (was 'i < 299', left weight[300..499] stale when SmapeAlfa is set at runtime) to fill the full MAXBUCKETS array, matching ForecastSolver::initialize. - Add test/forecast_11: a weekly forecast with >500 history buckets that forces the OOB; no golden .expect by design (passes iff frepple processes it without error -> ASan aborts pre-fix, clean post-fix).
Immediate-fix-queue #1 and #3 (ENGINE_REVIEW.md). #1 refcount leak: PythonData(const PyObject*) INCREFs a borrowed ref, so wrapping an owned (new) ref leaked one reference per Python callback inside the solve loop. Add PythonData::fromOwned() that adopts an owned ref without INCREF; use it at the 3 PythonFunction::call() variants + getDuration(), adopting under the GIL (also fixes a latent INCREF-after-release ordering bug). #3 GIL leak: PythonInterpreter::initialize() acquired the GIL but could throw (PyDateTime_IMPORT, PyErr_NewException, registerGlobalMethod, the nok check) without releasing it. Wrap the body in try/catch that honors the SAME intentional 'if (init)' conditional release on every path (verified: embedded mode deliberately retains the GIL for the process lifetime via main.cpp -> dllmain.cpp -> library.cpp). execute() was already correct.
Builds Debug (-fsanitize=address) and runs the engine golden suite incl. forecast_11. Path-filtered to engine changes so doc/gate commits don't trigger the heavy build. continue-on-error (informational) until the rest of the immediate-fix queue is cleared, then flip to blocking + activate the E2 sanitizer-ci gate.
Immediate-fix-queue #4 (ENGINE_REVIEW.md). The factory did Py_INCREF(s) on a freshly-created object that already has refcount 1, pinning it forever (one leaked solver_delete per construction). The sibling SolverCreate::create deliberately omits the INCREF so the object stays garbage-collectable; match it.
Immediate-fix-queue #5 (ENGINE_REVIEW.md). The copy constructor called this->~EntityIterator() on a just-constructed object, reading the still-uninitialized 'type' and deleting a garbage union pointer (UB / heap corruption). A fresh object has nothing to destroy; remove the call. The assignment operator legitimately keeps it (there 'type' is initialized).
Immediate-fix-queue #8 (ENGINE_REVIEW.md). 'else if (data_double > LONG_MIN)' is true for nearly all in-range doubles, so JSONData::getLong returned LONG_MIN instead of the value; getInt had the identical '> INT_MIN' bug. The lower clamp must be '< LONG_MIN' / '< INT_MIN'. Silent wrong integers from JSON input.
…imum Immediate-fix-queue #9 (ENGINE_REVIEW.md). 'delete &(*(oo++))' advanced the iterator after erase(), which nulls the node's next/prev -> oo++ read freed memory and jumped to end(), so setMaximumCalendar deleted only the first max event and setMaximum did a UB read (masked by an immediate return). Use the capture-advance-then-erase idiom already in setMinimumCalendar. Validated: buffer/safety-stock scenarios pass clean under a local AddressSanitizer build.
Immediate-fix-queue #6 (ENGINE_REVIEW.md). The token branch in MultiDBMiddleware resolves the user and calls login() directly, bypassing the auth backend, so user.is_active was never checked - a still-valid webtoken/API key for a deactivated account kept authenticating to the REST API and scenario data. Add an explicit is_active check before login().
Pegging had only 2 tests despite being the most pointer-heavy, least-covered engine code. Adds 10 scenarios derived from existing non-crashing models, covering pegging branches the review flagged as untested: split, alternate, routing sub-steps, distribution/transfer, flow-alternate, multi-level material chains, and offset flows. Each has a deterministic golden generated and verified stable under a local AddressSanitizer build (all ASan-clean). Cycle and dependency-edge pegging are deferred: they hit existing memory bugs (operation_dependency aborts under ASan) that must be fixed first. Flips the E2 pegging-tests gate to active (self-validating: counts >=12).
…tor-- Calendar::EventIterator::operator-- did '--cacheiter' even when cacheiter == eventlist.begin(), which is undefined behaviour for a std::map iterator (it trips AddressSanitizer with a heap-buffer-overflow in the red-black tree). The code below already assumed the step-before-begin case lands on end(), so make that explicit instead of relying on UB. Root cause of 8 of the suite's ASan crashes (calendar, operation_available, json, load_bucketized, constraints_combined_1/2, heuristic2, operation_ dependency all iterate calendar events during planning) -> all now ASan-clean. Validated: full engine suite has 0 ASan crashes (was 8) on a local ASan build.
Their alternate/routing pegging output is platform-sensitive (macOS vs the Linux CI), so a committed golden can't match across platforms. Drop the .expect and keep them as smoke/ASan regression tests (pass if frepple processes the model without error). The other 7 new pegging tests have platform-stable goldens that pass in CI.
pegging_4 (alternate), pegging_5 (routing), pegging_7 (flow-alternate) segfault in a Release build during pegging iteration over alternate/routing operationplans (ASan-Debug masks it - an unchecked-cast-class bug in followPegging). Removed for now; the crash is a real engine bug to fix separately, after which these can return. 9 pegging tests remain (up from 2), all passing with platform-stable goldens.
The Calendar::EventIterator operator-- UB fix cleared all 8 ASan crashes, so the engine golden suite now runs clean under AddressSanitizer in CI (verified on the last green run). Remove continue-on-error so memory regressions fail the build. Activates the E1 'sanitizers' and E2 'sanitizer-ci' gates.
followPegging dereferenced dynamic_cast<FlowPlan*>(&(*f))->getOperationPlan() at 4 sites with no null check. The buffer timeline holds mixed event types (flowplans + min/max/onhand events); a non-flowplan event in the scanned window makes the cast null and the deref crashes. Skip non-flowplan events. Provably a no-op for all non-crashing scenarios (the guard only diverges when the cast is null, which previously segfaulted), so goldens are unchanged; suite stays ASan-clean. NOTE: this hardens the predicted H4 sites but does NOT resolve the separate Release-only crash in pegging-over-alternate iteration (pegging_4/5/7) — that needs a backtrace the local macOS tooling couldn't produce; deferred.
structural_1/2/3 plan a material / distribution / resource model and assert universal invariants in-process: no operationplan has negative quantity, and none ends before it starts. Golden-free (they raise on violation -> non-zero exit), so they're platform-independent and catch a class of plan-corruption regressions the goldens might miss. Validated locally (INVARIANTS_OK, exit 0). Activates the E2 structural-asserts gate.
Begins the modernization main arc (REST track). Adds drf-spectacular for an
OpenAPI 3 schema over the existing DRF API, served at /api/schema/ (+ Swagger
at /api/doc/, ReDoc at /api/redoc/); schema versioned 1.0.0.
Adds plan/forecast OUTPUT JSON endpoints under /api/output/{forecast,inventory,
resource,demand,pegging}/. These are thin JSONStreamView wrappers
(common/api/output.py) that force ?format=json and delegate to each report's
own class-based view, reusing the existing raw-SQL + chunked-cursor streaming
path -- so they use NO DRF serializer (avoids serializer cost), with the
report's permission/bucket/filter/scenario handling intact.
NOTE: /api/v1/ URL-prefix versioning is deferred (the api routes are
intermixed with UI routes; a blind re-prefix is fragile). The schema is the
versioned contract for now. Auth/WS standardization is Increment 2.
Phase0SchemaTest: the drf-spectacular schema generates and is served at
/api/schema/ (+ swagger/redoc). Phase0OutputEndpointTest: each /api/output/*
endpoint is byte-identical to the legacy report's ?format=json response
(proving JSONStreamView reuses the same raw-SQL streaming path), and returns
the expected {total,page,records,rows} JSON envelope.
- frePPleListCreate/RetrieveUpdateDestroy get_queryset: guard on swagger_fake_view so drf-spectacular can introspect the CRUD endpoints (they used self.request.database, absent during schema generation, so those endpoints were dropped from the schema with warnings). - Phase 0 output tests: assert the streaming jqGrid envelope and that the new endpoint matches the legacy ?format=json envelope up to 'records' (the count varies with the planning horizon vs now(), so byte-identical parity was overspecified; empty uncomputed-plan output is also not strict JSON). - Add tools/modernization/gen_api_client.sh (TS client from the schema; runs where a Django runtime + Node are available).
CI green on the Phase 0 REST track: OpenAPI schema served, output endpoints stream correctly and match the legacy report path, no DRF serializer on that path. Flip openapi-schema, output-endpoints, no-drf-serializer-output to active. ts-client stays pending (the gen-client script needs a Django runtime + the Next.js repo to execute).
common/jwtauth.py consolidates the JWT secret resolution + decode logic that was duplicated across the HTTP middleware, the ASGI middleware and the token minting helper, plus an extract_scenario() that picks the scenario database from a URL prefix or X-Frepple-Scenario header (falling back to a default). This is the single source of truth the websocket layer will use to become scenario-aware (stage 2). Unit-tested: encode/decode round-trip, invalid -> None, expired -> raises, scenario from default/url-prefix/header.
… Increment 2, stage 2) Route the websocket protocol through the same cookie/session + token + permission stack as HTTP, with the per-connection auth gate in the consumer's connect() (AuthenticatedMiddleware is HTTP-only). - TokenMiddleware now resolves the scenario from the URL prefix / X-Frepple-Scenario header via the shared extract_scenario(), falling back to the FREPPLE_DATABASE env var (single-scenario deploys unchanged), and strips the prefix from scope[path] mirroring the WSGI middleware. - JWT decode goes through the shared decode_jwt(); credentials are accepted from the Authorization header, a Sec-WebSocket-Protocol subprotocol, or a ?token= query param (browser WS clients can't set request headers). - Minimal AsyncWebsocketConsumer at ws/ rejects anonymous/inactive users (4401) and echoes messages tagged with the resolved scenario. - Channels WebsocketCommunicator tests (reject no/bad token, accept via header + subprotocol, echo scenario); flip jwt-auth + ws-scenario-routing gates active (13/46).
The module-level service-loading loop re-raised ModuleNotFoundError when an app's services.py imports the C++ "frepple" engine module, so freppledb.asgi could only be imported inside the embedded-interpreter worker — not in a plain Django/test/schema process. The new websocket tests import asgi.application and hit exactly this. Tolerate a missing "frepple" engine module (alongside the existing no-services-module case); the engine-only services it would register are meaningless outside the worker anyway.
- Add a ?token= query-string websocket test (browser-usable carrier). - Assert the reject tests close with 4401, and surface the close code in the accept-test failure messages so a rejected handshake is diagnosable. - Normalize subprotocol entries to str (defensive against byte values).
Run tools/modernization/gen_api_client.sh in the ubuntu24 build job (where the Django runtime exists): emit the drf-spectacular OpenAPI schema, generate types with openapi-typescript, and tsc --strict --skipLibCheck them; upload the schema + types as a build artifact. Modernization-branch-guarded so upstream CI is unaffected. Fix the script's tsc invocation (npx -p typescript) and flip the ts-client gate active (14/46).
…hosts) Addresses the deploy review findings: - DEBUG: FREPPLE_DEBUG env override (chart sets false) so the public env isn't served with tracebacks even though it runs runserver --insecure. - SECRET_KEY: read FREPPLE_SECRETKEY; the chart generates one and preserves it across upgrades (templates/secret.yaml, lookup) instead of the public in-repo default. ALLOWED_HOSTS: FREPPLE_ALLOWED_HOSTS (chart sets the public host) instead of '*'. Web probes carry the public Host header so DEBUG-off + host allowlist doesn't reject them. - secretKey/allowedHosts/loadDemo values are now live (were dead config with misleading comments); FREPPLE_LOAD_DEMO gates the demo load in entrypoint. - entrypoint pg_isready uses $POSTGRES_USER (not a hardcoded 'frepple'). - asgi gets a liveness probe (was readiness-only). - frontend image: Next standalone output + non-root 'node' user + slim runtime (drops source + devDependencies). - Cross-reference the three routing tables (nginx.conf / Ingress / next.config) and note the env is single-scenario (default prefix only).
- gates.py: drop the unused render() 'failures' param; broaden the no-drf-serializer-output check (file_contains_any) so it also catches 'from rest_framework.serializers import XSerializer', not just the contiguous 'import serializ'. - asan_pegging_repro.sh: guard the cmake configure/build with '|| exit 1' so a build failure aborts before runtest runs against a stale binary, without a blanket 'set -e' (runtest is expected to fail and its exit code is printed).
dnplkndll
changed the title
The enriched wrapper (auth-gate-first, then prepend measures+buckets over
the report's unchanged {data}) is generic to any GridPivot, not just
forecast. Rename it PivotJSONStreamView (keep a ForecastJSONStreamView
alias for the forecast endpoint + tests) and point /api/output/inventory/
at it so the SPA gets a self-describing envelope. The 'data' payload is
byte-identical, so data-parity holds.
Extract the generic GridPivot core from forecast.ts into lib/pivot.ts (parsePivot/pivotRows/bucketOrder; measure names come from the envelope, no hardcoded list) and refactor forecast.ts to reuse it (its 12 tests unchanged). Add the read-only Inventory screen (lib/inventory.ts, useInventory.ts, app/inventory/page.tsx) reusing authedFetch + the design system, a nav entry, and pivot.test.ts. Playwright smoke + a11y for /inventory; flip the inventory-report gate active (21/47). Read-only, so no runwebservice/engine-interpreter dependency. demand/ resource/pegging screens are the next increments, now trivial via the shared parser + PivotJSONStreamView.
/api/output/inventory/ opted into the measures+buckets envelope, so it's no longer byte-identical to /buffer/?format=json. Drop it from the bare PARITY set and add test_inventory_output_enriched: assert the enriched header and that the wrapped 'data' stays byte-identical to the legacy buffer envelope (data-parity).
…View)
/api/output/demand/ and /api/output/resource/ move from the bare
JSONStreamView to the enriched wrapper so the SPA gets the self-describing
{measures,buckets,data} envelope (data stays byte-identical). Replace the
bare PARITY tests with a parametrized test_pivot_outputs_enriched over
inventory/demand/resource that asserts the enriched header AND data-parity
vs each legacy report; forecast keeps its shape-only test.
…Screen Extract a generic read-only pivot screen: usePivotReport(endpoint,keyField) + <PivotScreen config> (pagehead/auth/loading/empty + series x measures x buckets), and refactor Inventory onto it (delete useInventory.ts). Demand and Resource are then thin configs (lib/demand.ts, lib/resource.ts) + tiny pages + two nav entries. Playwright smoke + a11y for /demand and /resource; flip the resource-capacity gate active (utilization pivot; timeline Gantt deferred) and update the inventory-report gate marker. 22/47 gates.
…ildx registry cache) Two build-time wins, validated locally: - Dockerfile.engine: copy the engine SOURCE (CMakeLists/src/include/bin/doc/ contrib/requirements - everything add_subdirectory()/configure_file/the venv target references) and compile BEFORE copying the Django app. A Python/frontend-only change then reuses the cached cmake build layer instead of recompiling the ~4-min engine. (Verified: a Python-only edit rebuilds with the compile step CACHED, 0 C++ files recompiled.) - deploy-staging.yml: build via docker/build-push-action + buildx with a GHCR registry cache (type=registry ...:buildcache, mode=max), so the compiled layer persists across runs (the dind runners have no local layer cache). First run seeds the cache; subsequent Python-only builds skip the engine compile. No source change, so the produced image is functionally identical.
… E4) Evidence-based answer to 'does Rust prevent this bug class?'. Port the JSON number-conversion kernel (src/utils/json.cpp getLong/getInt/getUnsignedLong — the inverted-bound bug site) to a memory-safe PyO3 extension rust/frepple-num/ (saturating casts, #![forbid(unsafe_code)]). - Parity: a true Rust-vs-C++ diff against a verbatim reference (tools/rust-pilot/cxx_reference.cpp) over test/rust_parity/vectors.json — 24/24, incl. the regression case clamp_to_long(5.0)=5 (the C++ bug returned LONG_MIN) and Rust-safe cases the C++ leaves UB (NaN, neg->unsigned). - Measured LOC/perf/safety + go/no-go in tools/modernization/rust-pilot.md (decision: conditional GO for targeted numeric leaf modules; NO-GO for a wholesale rewrite). cargo test runs the logic with zero Python dep. - CI: .github/workflows/rust-pilot.yml (cargo test + maturin + parity), standalone and fast — no engine build, no deploy. Intentionally CI-only; shipping the wheel is a go-only fast-follow. - All three E4 gates active (25/47).
The real evidence step after the json clamp: port an actual forecasting method — MovingAverage::generateForecast (src/forecast/timeseries.cpp:294-384) + smapeWeight (forecast.h, the weight[] OOB site) — to a memory-safe PyO3 crate rust/frepple-forecast/ (saturating/bounds-checked, #![forbid(unsafe_code)]). - Parity: Rust-vs-C++ diff vs a verbatim reference (tools/rust-pilot/forecast_reference.cpp) over test/rust_parity/ forecast_vectors.json — 10/10, incl. two >MAXBUCKETS series (the OOB case); smape/stdev/avg within 1e-9 (same f64 op order), outlier indices exact. - Honest finding: LOC is comparable, not smaller (~109 Rust vs ~73 C++) for tight numeric code — the win is compile-enforced safety + the clean PyO3 linkage (no manual refcounting), not line count. Recorded in tools/modernization/rust-pilot.md; decision stays conditional-GO. - CI: rust-pilot.yml runs both crates' cargo tests + both parity suites, standalone (no engine build). rust-pilot-parity gate now covers both. cargo test + 34 parity tests (24 json + 10 forecast) green locally.
…e 3) Port SingleExponential::generateForecast (timeseries.cpp:420-593) — single exponential smoothing with 1D Levenberg-Marquardt on alfa + the two-pass outlier scan/filter — to rust/frepple-forecast/src/single_exp.rs. Extract a shared common.rs (smape_weight, weight table, constants, the Forecast result) and refactor MovingAverage onto it (its parity re-verified). Parity: the forecast C++ reference now dispatches by method; the Rust single_exponential is diffed against the verbatim C++ core over new vectors (constant/trend/outlier/noisy/too-short/>MAXBUCKETS). 40 parity tests (24 json + 10 MA + 6 SE) + cargo tests green; smape/stdev/forecast within 1e-9, outliers exact, DBL_MAX sentinel honored.
…e 4) Port DoubleExponential::generateForecast (timeseries.cpp:633-892) — Holt-Winters level+trend with a 2D Levenberg-Marquardt over (alfa,gamma) via a 2x2 Hessian. Factor a shared common::solve_2x2_marquardt (Cramer's rule + damping + singular-retry), written bit-for-bit with the C++ op order so parity is exact. Parity: forecast reference gains a verbatim double_exp; 46 parity tests (24 json + 10 MA + 6 SE + 6 DE) + cargo tests green; smape/stdev/forecast within 1e-9, outliers exact.
Port Croston::generateForecast (timeseries.cpp:1307-1463) — intermittent-demand smoothing (demand magnitude q_i / inter-demand period p_i) with an alfa grid-search and upper-only outlier clamping. Preserves the C++ quirk that between_demands persists across grid iterations. Verbatim C++ reference added. 52 parity tests (24 json + 10 MA + 6 SE + 6 DE + 6 Croston) + cargo tests green; smape/stdev/forecast within 1e-9, outliers exact. (Fixed a module/pyfunction name clash by aliasing the croston module import.)
…thods complete Port Seasonal::detectCycle + generateForecast (timeseries.cpp:942-1262) — the hardest method: autocorrelation cycle detection, Holt-Winters multiplicative with per-period seasonal factors, 2D Marquardt over (alfa,beta) reusing common::solve_2x2_marquardt. Returns a richer SeasonalResult (period, force, S_i[period]) so the seasonal state can be reconstructed at apply time. Verbatim C++ reference emits period/force/s_i; the parity test compares those element-wise. 57 parity tests (24 json + 10 MA + 6 SE + 6 DE + 6 Croston + 5 Seasonal) + cargo tests green; a period-7 cycle detects period=7/force=true. All five forecast methods now ported + parity-verified. Next: Phase 7 flag-gated engine integration (C-ABI staticlib + forecast_* golden parity).
… phase 7) Add a C-ABI staticlib so libfrepple can call the Rust forecast methods: rust/frepple-forecast now builds crate-type staticlib with src/capi.rs (extern "C" wrappers for all 5 methods) + tools/rust-pilot/frepple_forecast.h. capi.rs is the only unsafe in the crate (the FFI boundary); the numeric modules stay #![forbid(unsafe_code)]. A committed C harness (tools/rust-pilot/capi_harness.c) links the staticlib and calls the methods as the engine would (MovingAverage->8.0, Seasonal->period 7), run in CI. Key finding for the gated engine-link: Rust matches the C++ to ~1e-9 but NOT bit-for-bit (~14/33 vectors exact) - g++ -O2 uses -ffp-contract=fast (FMA), rustc doesn't. Byte-exact forecast_* golden parity will need the forecast TU built with -ffp-contract=off. Documented in rust-pilot.md; the remaining CMake link + flag-gated dispatch + golden CI leg is default-OFF and CI-gated (the engine build is Linux-only, not validatable on the dev box).
…te (#2) * ci(e2e): compose-based Playwright E2E guardrail with engine warmup gate Adds a CI job that brings up the full same-origin stack (Postgres + Django/wsgi + daphne/asgi on the C++ engine image + Next.js + nginx) via the e2e compose files and runs the Playwright suite (smoke + a11y across all five screens + the engine-backed live-progress run). The engine image is restored from the deploy-staging buildx registry cache, so the C++ engine is not recompiled here. Backward-compat net: new SPA features can no longer silently break an existing screen or the runplan -> Task.status -> Redis -> websocket -> React live loop. Hardens against the cold-start race that flaked live-progress: the engine overlay now computes a warmup plan on startup (FREPPLE_INIT_RUNPLAN), and CI waits for that plan to reach Done before running Playwright, so the C++ engine is warm and the broadcast path has fired once. Also runs on PRs into modernization, not just pushes. * ci(e2e): harden the e2e workflow + document the guardrail Review follow-ups on the compose E2E job: - concurrency group with cancel-in-progress so a newer push supersedes an in-flight (expensive) stack build instead of both running to completion. - timeout-minutes: 30 caps a hung stack/Playwright run. - warmup gate now fails fast if the startup plan ends 'Failed' (reads the latest runplan status) instead of burning the full 5-min poll on a known failure. - cache npm via setup-node (keyed on e2e/playwright/package-lock.json) and split dependency install from the test run so a failure is attributable. Also documents the no-backlog guarantee that keeps live-progress unambiguous: TaskProgressConsumer relays only live broadcasts (asgi.py sends no backlog on connect), so the Execute feed starts empty and the only task the test can match is the one it launched - the warmup plan finished pre-load and never appears. Adds a CI section to e2e/README.md.
* feat(web): read-only Demand Pegging Gantt screen (Phase 3-D1) Adds the modernization SPA's pegging screen: pick a sales order and trace the supply chain that pegs to it - every operationplan feeding the delivery - on one dated timeline. The marquee Phase 3 screen, read-only first (no engine writes); drag-reschedule + downstream preview follow in D2/D3. Backend - PeggingJSONView (freppledb/common/api/output.py): enriches the existing demand-pegging report stream for the SPA. The bare ?format=json drops the report's hidden columns, so the absolute horizon + due/current marker dates never reach a client; this prepends a "window" header (start/end/due/current, ISO) over the report's tree+bars UNCHANGED under "data". Mirrors the PivotJSONStreamView pattern; data stays byte-identical to the legacy stream. - Wires /api/output/pegging/<demand>/ to it (was the bare JSONStreamView, which nothing consumed). - Django test (test_api_phase0): window header present + data-parity vs the legacy /demandpegging/<demand>/ envelope. Frontend - lib/pegging.ts: typed parse of the enriched response + date->fraction geometry + day-snapped axis ticks (pure, unit-tested). - lib/usePegging.ts / lib/useDemandList.ts: fetch hooks (authedFetch), mirroring usePivotReport's loading/error/authError/reload contract. - app/pegging: demand typeahead picker (deep-linkable ?demand=) + PeggingGantt, an HTML/CSS positioned-bar Gantt (depth-indented lanes, status-colored bars, due/now markers) - not SVG, so D2 can add pointer-drag without re-plumbing. - design tokens reused; new .gantt/.picker classes in globals.css. Tests - pegging.test.ts (parse + geometry + axis), Playwright smoke + a11y (0 critical) + engine-backed pegging render added to the e2e suite (now 15 specs). * refactor(web): review follow-ups on the pegging Gantt Addressing self-review findings (no behaviour change for clients): - DRY: extract _run_report (the force-json + auth-gated inner run) and _wrap (the stream-prefix + close) onto JSONStreamView; PivotJSONStreamView and PeggingJSONView now share them instead of each re-implementing the streaming wrapper. Output-endpoint tests stay green (pivot + forecast + pegging). - Perf: PeggingJSONView reused the horizon the report's own get() already computed (report_startdate/enddate on the request) instead of re-running the heavy recursive pegging CTE via a second getBuckets() call. Falls back to getBuckets() only if the attrs aren't set. - Guard an empty demand segment (/pegging//) so args[0]=='' doesn't hit the DB. - Trim dead data: PeggingBar carried color/item/location that the Gantt never used; drop them and surface the kept criticality in the bar tooltip. - Docs: record the D1 delivery + the D2/D3 split in MODERNIZATION_PLAN, and add the pegging screen to the e2e/README scope. Rejected (verified, not a bug): the suggestion to unquote() the demand URL segment - Django already decodes PATH_INFO before routing (confirmed live: /api/output/pegging/Demand%2001/ returns the real 'Demand 01' plan), so unquote() would be a no-op at best and double-decode a literal '%' at worst.
* style(web): refine the planning-console shell A precision pass over the SPA chrome - evolves the existing planning-console design language (IBM Plex Mono/Sans, amber signal, near-black blueprint), no aesthetic replacement, so every screen benefits at once: - Status rail: a live UTC mission-clock (the console's heartbeat), hairline dividers between stats, and a faint amber underglow on the rail edge. - Nav: a tactile amber rail-bar that grows in on the active route + a 2px hover nudge; the nav cascades in on first paint. - Panels: a 1px top highlight to catch the light + a leading amber diamond tick on every panel title (the section marker). - Depth: a fixed low-opacity film-grain over the flat near-black for a printed- instrument texture. - Motion: each screen assembles top-to-bottom in a quick staggered reveal. - A11y + polish: one crisp amber :focus-visible ring on every interactive element (replaces the default outline). All entrance motion is disabled under prefers-reduced-motion (existing rule). Verified: tsc + next build clean, full Playwright suite 15/15, 0 critical axe violations on every screen. * fix(web): make the focus ring clip-proof + drop redundant entrance anims Review follow-ups on the shell refresh: - Focus ring: switch :focus-visible from box-shadow to outline. A box-shadow ring is clipped by overflow:hidden/auto ancestors - the launch console, the gantt, the table + picker scrollers all clip - so keyboard focus on the controls inside them was invisible (an a11y regression). An outline isn't clipped and follows each element's own border-radius, so the ring is always visible and correctly shaped. Removes the --ring token + the forced border-radius that mismatched larger/round elements. - Drop the now-redundant standalone reveal animations on .pagehead and .console: the .content > main > * entrance stagger already owns (and overrode) them, so they were dead, double-declared CSS.
Adds the write-path to the pegging Gantt: drag an operationplan bar to shift its start/end by the dragged time delta, persisting via the DRF operationplan API. - lib/reschedule.ts: the operationplan type -> detail-endpoint map (MO/WO/PO/DO/ DLVR; STCK not reschedulable), an editability rule (locks completed/closed), naive-ISO date shifting, and patchReschedule() (PATCH /api/input/<type>/<ref>/ via authedFetch -> Bearer + CSRF). Pure helpers unit-tested (9 cases). - PeggingGantt: pointer-drag on editable bars (px -> lane-fraction -> time delta), optimistic offset while dragging, pending pulse during the PATCH, snap-back on failure. Non-editable bars (wrong type / executed status) stay locked. A sub- threshold drag is treated as a click. - page.tsx: handleReschedule PATCHes then reloads (so the Gantt shows the persisted dates) and raises a 'peg is stale until you re-plan' banner - honest about the constraint that pegging is engine-computed (D3 closes that loop with a preview + re-plan). Pegging is read-only/engine-computed: a reschedule persists dates but does NOT recompute the peg until a plan runs - surfaced in the UI, not hidden. Tests: reschedule.test.ts (map/editability/date-math); engine-backed Playwright drag spec (drag -> PATCH -> persisted -> stale banner). Verified live: PATCH /api/input/manufacturingorder/<ref>/ returns 200 and persists; full Playwright suite 15/15, 0 critical a11y.
…se 7) (#7) * feat(engine): flag-gated Rust forecast link + golden-parity CI gate (Phase 7) Wires the already-ported, parity-verified Rust forecast methods (rust/frepple-forecast) into the C++ engine behind a default-OFF flag, and adds the CI gate that answers the open byte-parity question. - CMakeLists.txt: option(FREPPLE_RUST_FORECAST OFF). When ON, src/CMakeLists.txt cargo-builds libfrepple_forecast.a, links it into the forecast lib, defines FREPPLE_RUST_FORECAST=1, and compiles the forecast TU with -ffp-contract=off (match rustc's no-FMA — the documented requirement for byte-exact parity). - timeseries.cpp: MovingAverage::generateForecast dispatches to extern "C" frepple_moving_average behind the flag (the template). The engine passes timeseries.data()+count — confirmed to be the same [0..count-1] data points (trailing-0 at [count]) the parity reference + Rust consume. Outlier ProblemOutlier creation + applyForecast stay in C++; Rust returns numbers + outlier indices only. The other 4 methods stay C++ under the flag (mechanical follow-on once MovingAverage clears the gate). - forecast-phase7.yml: builds -DFREPPLE_RUST_FORECAST=ON and runs the forecast_1..11 golden tests byte-exact. Green => flip ON; red => the FP- contraction ULP gap is the recorded 'stop = success' datapoint. Default OFF => the shipping engine is byte-for-byte unchanged. The Rust staticlib builds clean locally; the in-engine golden gate runs in CI (Linux-only build). * feat(engine): wire SingleExponential + Croston to Rust (Phase 7) Both write a constant forecast (f_i), so the scalar C-ABI's single forecast value is sufficient for applyForecast - same template as MovingAverage. Params mapped from the engine statics (initial/min/max alfa, decay_rate) + the shared Forecast_maxDeviation / Forecast_SmapeAlfa / skip / iterations globals; outlier indices recreated as ProblemOutlier in C++. 3/5 methods now dispatch to Rust under the flag. DoubleExponential + Seasonal stay C++: their applyForecast extrapolates per bucket and needs decomposed state (constant_i+trend_i; L_i+T_i+S_i[]+cycleindex) that the parity-oriented C-ABI doesn't yet expose - documented in rust-pilot.md as a C-ABI extension to finish later. Gate stays green meanwhile (flag default OFF; the C++ path for those two is unchanged). * feat(engine): wire DoubleExponential to Rust via C-ABI state extension (Phase 7) DoubleExp's applyForecast extrapolates per bucket (constant_i += trend_i; trend_i *= dampenTrend), so it needs the decomposed level+trend, not just the one-step sum. Extend the C-ABI to return it: - double_exp.rs: factor double_exponential_state() returning DoubleExpState {base, constant, trend}; double_exponential() stays a thin wrapper for the PyO3/parity path (one-step forecast), so cargo/pytest parity is unaffected. - capi.rs: dedicated frepple_double_exponential with two trailing out-pointers (out_constant, out_trend); header updated to match. - timeseries.cpp: DoubleExp::generateForecast dispatches behind the flag and sets constant_i + trend_i so applyForecast extrapolates unchanged. 4/5 methods now dispatch to Rust under the flag (MA, SingleExp, Croston, DoubleExp). Seasonal still needs L_i/T_i/S_i[]/cycleindex exposed - the last ABI extension. Rust staticlib builds clean; gate validates byte parity. * docs(engine): rust-pilot phase 7 status — 4/5 methods green, Seasonal next
) * feat(engine): wire Seasonal to Rust — 5/5 forecast methods (Phase 7) Completes the forecast C++->Rust conversion. Seasonal's applyForecast extrapolates per bucket (L_i += T_i; T_i *= damp; fcst = L_i * S_i[cycleindex], cycleindex wrapping at period), so it needs the level/trend/cycle apply-state, not just the one-step forecast. Quality-first: the existing parity only pinned l_i + t_i/period (via the one-step forecast) and never checked cycleindex. So this adds a DEDICATED apply-state parity check FIRST -- the verbatim C++ reference + Rust both emit L_i/T_i/cycleindex and test_forecast_parity asserts they match (cycleindex = count%period; level/trend within 1e-9). They match (33/33), so wiring is verified-safe. - seasonal.rs: SeasonalResult gains l_i/t_i/cycleindex; lib.rs PyO3 tuple + the parity reference + test extended to cover them. - capi.rs/header: frepple_seasonal returns the three extra out-params; C-ABI harness updated (links + period 7 + cycleindex). - timeseries.cpp: Seasonal::generateForecast dispatches behind the flag, setting period/L_i/T_i/S_i[]/cycleindex (no outlier detection in this method). 5/5 methods now run in Rust under the flag. Default OFF; the golden gate (forecast_1..11) validates byte-exact in-engine parity. * docs(engine): clarify the Seasonal C-ABI (review follow-up) Self-review of the Seasonal 5/5 branch found no functional issues (the cycleindex=count%period + L_i/T_i handoff is parity-verified, DRY repetition is acceptable). Addressing the doc nits it surfaced — comments only: - frepple_forecast.h: inline param order + the no-outliers / apply-state semantics right on the frepple_seasonal declaration. - capi.rs: note Seasonal has no outlier indices (no ProblemOutlier), unlike the scalar methods. - timeseries.cpp: note period 0 -> smape=DBL_MAX -> never selected, so applyForecast is never reached with period 0 (no S_i[] OOB).
Evidence-gathering spike for a greenfield finite-capacity / DDMRP planning mode: can an advanced optimisation engine, driven from Rust, do what frePPLe's constructive MRP heuristic can't? rust/solver-spike/: a small capacitated multi-period production-planning LP (good_lp modelling layer, pure-Rust microlp backend) vs a lot-for-lot heuristic. On a capacity-tight instance the heuristic is INFEASIBLE (period-4 demand 75 > capacity 50); the LP finds the cheapest feasible plan by pre-building, at a quantified holding premium (187 vs the infeasible 180). That build-ahead/holding trade-off is exactly what a heuristic can't reason about and an optimiser nails. good_lp keeps it solver-portable (swap microlp -> HiGHS/CBC/SCIP via a feature flag for scale) with no model rewrite; microlp is the only pure-Rust path. Decision (tools/modernization/solver-spike.md): Conditional GO as an optional, flag-gated capacity-optimise mode (no parity tax — it's new capability, not a C++ behaviour to reproduce); NO-GO on replacing the battle-tested constructive solver. ~190 LOC, one dependency, ~5s build; wired into rust-pilot CI so it doesn't bitrot.
…e 3-D3) (#10) Closes the pegging loop the D2 banner only pointed at. A reschedule persists dates but pegging is engine-computed, so it stays stale until a plan runs. - downstreamChain (lib/pegging.ts, unit-tested): the rows whose timing depends on a moved op — the op + its ancestors toward the demand delivery (the pre-order tree is depth 1 = delivery, deeper = upstream supply). After a reschedule those rows get an 'impact pending' highlight (.gantt-row--affected). - useReplan (lib/useReplan.ts): launches runplan and resolves once it reaches a terminal state over the task websocket (subscribes BEFORE launching — the consumer sends no backlog, so the completion can't be missed). The page then re-fetches the now-authoritative pegging and clears the highlight. - page: the stale banner becomes actionable ('Re-plan now'); PeggingGantt takes the affected set + threads the moved row id through onReschedule. Deliberately NOT a precise client-side ghost-bar simulation of the downstream shift — it can't match the engine and would mislead. The highlight shows WHICH steps are affected; the re-plan shows the real result. Honest > flashy. Tests: downstreamChain units (11/11 pegging); engine-backed Playwright for the full drag -> highlight -> re-plan -> refresh loop. Suite 16/16, 0 critical a11y.
…RUD (Phase 3) (#11) * feat(web): Problems/Constraints + Orders list screens (Phase 3) Finishes the Phase 3 screen set with the two remaining views, both flat record lists (not time-bucket pivots), so they get a reusable list stack rather than PivotScreen: - Backend: /api/output/{problem,constraint}/ via the bare JSONStreamView (the reports' raw-SQL ?format=json stream). Django test asserts 200 + rows. - lib/records.ts: parseRecords (normalises a DRF array, a GridReport {rows}, or an enriched {data:{rows}} body) + cell/date/number formatters (unit-tested). - lib/useRecordList.ts: fetch hook mirroring usePivotReport's contract. - components/RecordTable.tsx: generic filterable table over a column config. - Problems screen: Problems/Constraints tabs (shared columns) over the new output endpoints. Orders screen: MO/PO/DO tabs over the input DRF lists, with per-type columns. New .tabbar/.tab design-system styles; nav entries added. Read-only — inline CRUD editing is a deliberate follow-on (the pegging Gantt already does the date-edit write path). Tests: records.test.ts (8); Playwright smoke + a11y (0 critical) for both screens; full suite 17/17. * refactor(web): DRY the Phase 3 list screens + harden formatters (review) Self-review follow-ups on the Problems/Orders branch (the column keys all matched the serializers — no silent '—' bugs): - DRY: the two near-identical screens collapse into a shared TabListScreen (header + tabbed RecordTable + auth/loading/error). problems/orders pages are now thin config (~15 lines each). TabListScreen also owns the proper tabs a11y pattern the per-page versions lacked: role=tabpanel + aria-controls/labelledby + arrow-key tab nav. - Formatters: fmtNum now renders non-finite/unparseable as '—' (was leaking 'NaN'); fmtDate rejects non-date strings instead of leaking partials like '2026'. Edge-case tests added (records.test.ts 10). - RecordTable: key rows by their natural id (reference/id) instead of array index, so client-side filtering can't reshuffle row identity. Verified: tsc + build clean, frontend unit green, full Playwright 21/21 (0 critical a11y on both screens incl. the new tabpanel wiring). * feat(web): inline CRUD on the Orders grid + editable-grid UX (Phase 3) Turns the read-only Orders grid into an editable instrument, within the planning-console design system: - Status PILLS (amber proposed / lime firm / muted done) replace plain text — scannable at a glance. - Per-row EDIT mode: a row's status/dates/quantity become inline inputs with an amber 'live' rail; Save (PATCH) / Cancel; saving pulses; optimistic + toast + reload; on failure the row stays open to retry. - DELETE with an inline confirm (Delete? Yes/No) -> DRF DELETE. - Hover-revealed row actions; executed orders (completed/closed) render 'locked'. - RecordTable gains an optional "edit" config (read-only problems screen unchanged); Column gains pill/edit/options flags (kept pure). orders.ts adds patchOrder/deleteOrder + canEditOrder + date normalisation (unit-tested). - TabListScreen threads the edit config + owns the toast/reload wiring. Create is a documented follow-on (needs an operation/item picker). Tests: orders.test.ts (canEditOrder, normalizeChange, tab config); engine-backed Playwright edit-persist + delete specs. Suite 19/19, 0 critical a11y.
…#12) Flip the staging frepple-app image to the Rust forecast methods, the last step of Engine track E4 phase 7 (proven 5/5 byte-parity in the forecast-phase7 gate). Default stays OFF everywhere else. - deploy-staging.yml: frepple-app matrix entry gains rust: "ON"; the build passes FREPPLE_RUST_FORECAST through as a build-arg (default OFF). - Dockerfile.engine: COPY the rust crate; when the flag is ON, install a minimal rustup toolchain before the C++ compile and pass -DFREPPLE_RUST_FORECAST to cmake so cargo builds + links the staticlib. - src/CMakeLists.txt: ranlib the cargo staticlib after the build. rustc emits the archive without a symbol-table index on some toolchains (seen on aarch64), which GNU ld rejects; ranlib adds it, a no-op where one already exists (x86 CI). - .dockerignore: exclude **/target. A host-arch target/ leaking into the build context shadowed the in-image cargo build (CMake saw the OUTPUT present and skipped it), linking a wrong-arch/index-less staticlib. Validated locally: the Rust-ON image compiles, links, and embeds all five extern "C" forecast wrappers (frepple_{moving_average,single_exponential, double_exponential,croston,seasonal}) in libfrepple.so.
#13) Close out the Rust forecast conversion: it is now the forecast source of truth on staging (helm REVISION 7, PR #12). Document the deploy-staging flag flip, the two aarch64 build fixes (ranlib + .dockerignore **/target), and the live in-pod golden verification (forecast_1..11 byte-exact, 11/11).
Add the UndefinedBehaviorSanitizer half of Engine track E1 (ASan was already
wired + blocking in engine-asan.yml). Establishes a documented UBSan baseline
over the golden test suite and fixes the one real undefined-behaviour bug it
surfaced.
- CMakeLists.txt: parameterise the Debug sanitizer via -DFREPPLE_SANITIZER
(address default = unchanged ASan gate; undefined = UBSan; both supported).
The UBSan build excludes -fsanitize=vptr: frePPLe's hand-rolled MetaClass
RTTI (downcast by type tag, not C++ polymorphism) is incompatible with the
vptr check, which would otherwise flood every run with by-design reports.
Sanitizer flags added to the Debug link line for the shared lib.
- .github/workflows/engine-ubsan.yml: advisory gate (halt_on_error=0) that
builds Debug+UBSan and runs the golden suite with -d so reports are visible,
summarising distinct findings in the step summary. Proves the UBSan build
keeps compiling/linking and tracks findings; flips to blocking in E2 once the
iterator-idiom null-bindings are retired (mirrors how engine-asan started).
- src/model/operationdependency.cpp: fix a symmetric null member-call in
set{Operation,BlockedBy} - both called addDependency() on a possibly-null
receiver. Behaviour-preserving (addDependency no-ops on an incomplete
dependency before touching the receiver), so the golden output is unchanged.
- tools/modernization/ubsan-baseline.md: full findings, root cause, and
severity for the three categories (vptr noise / iterator idiom / the real
fix), plus the path to a blocking gate.
Baseline: 96 type-2 golden tests under UBSan. After excluding vptr and fixing
operationdependency, the only remaining diagnostics are the two accepted
iterator operator* null-bindings (timeline.h, model.h) - STL-parallel UB,
documented.
…15) * feat(engine): UBSan gate blocking + clang-tidy baseline (E2 slice 1) Harden the engine CI gates - the mechanical close-out of Engine track E1's static-analysis gate. UBSan -> blocking: - The advisory baseline's only remaining findings were the iterator operator* null-binding idiom (timeline.h:293, model.h:8667) - a reference formed to a null end()-sentinel that is never dereferenced (same UB as *v.end()). Mark both with FREPPLE_NO_SANITIZE_NULL (new no_sanitize("null") macro in utils.h; g++/clang, no-op in normal builds), leaving the full golden suite UBSan-clean. - engine-ubsan.yml flips to halt_on_error=1 + abort - a new UB site now fails CI, the same contract as engine-asan. Verified locally: g++ accepts the attribute (no warning) and 0 findings remain across the timeline/problem/ forecast-heavy tests. clang-tidy baseline (E1 gate item 3): - .clang-tidy: bug-finder check set (clang-analyzer-* + high-signal bugprone-*), style/readability off; the two noisy checks (unhandled-self-assignment, exception-escape) excluded. - engine-clang-tidy.yml: advisory gate (parse-only, never fails) that reports the finding count + breakdown and uploads the report. Tightens to "no new findings on changed files" in E2. Docs: ubsan-baseline.md updated (Finding 2 resolved, gate now blocking); clang-tidy-baseline.md added; MODERNIZATION_PLAN E1 gate - sanitizer + tidy items checked (review report still open). * fix(ci): clang-tidy gate - per-file logs + distinct-finding dedup Self-review of the advisory clang-tidy gate caught two issues: - Parallel clang-tidy processes all redirected to the same clang-tidy.out; concurrent writes can interleave and garble lines. Write each TU's output to its own log under .tidy-logs/, then concatenate. - The reported "1076 warnings" was meaningless: with HeaderFilterRegex a header finding is emitted once per including TU. Dedup by the warning line (path:line:col + check, byte-identical across TUs) so the summary reports DISTINCT findings (54) alongside the raw count (~182), with a deduped per-check breakdown. Update clang-tidy-baseline.md with the real full-src numbers (54 distinct) and the actual breakdown - the ~10 high-signal triage items (NullDereference, CallAndMessage, integer-division, NewDelete, uninitialised) are the E2 work-list.
…ots (#16) Completes the last Engine track E1 gate item. A verification-backed review of the C++ engine: prioritized debt (99 TODO/FIXME markers triaged), a risk-hotspot map (solver state machine + manual-undo, memory ownership incl. ~all-raw with 16 smart pointers engine-wide, CPython coupling at 109 refcount sites, pegging), and a cross-reference to the clang-tidy (54) + UBSan findings. Method: four parallel surveys, then direct source verification of every load-bearing claim - which corrected several stale/wrong ones and is itself part of the value: - the solveroperation a_penalty "incorrectly???" bug is ALREADY FIXED (comment now documents the snapshot/reset); - pegging has 9 golden + 3 smoke-only tests, not "only 2"; - the pegging `visited` cycle-guard is a default-constructed set, not uninitialised UB; - the operatordelete "dangerous side effects" block is disabled (/* */); - a MAXSTATES overrun throws a catchable exception, not a crash. E1 is now complete (review report + ASan/UBSan blocking-clean + clang-tidy baseline). The report hands a prioritized work-list to E2 (pegging golden coverage, structural-invariant assertions, clang-tidy triage, stress baseline).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
Modernizes frePPLe while keeping the C++ planning engine and Django data layer, and replaces the AngularJS UI with a Next.js app. Adds same-origin JWT + websocket auth, a streaming REST/output layer, a batch of engine memory-safety fixes with regression tests, and a Helm chart + ARC→GHCR build that deploys a live staging review env. Tracked in
MODERNIZATION_PLAN.mdwith a gate harness (tools/modernization/gates.py, 22/47 active).Live review env: https://frepple-staging.hz.ledoweb.com — log in
admin/admin.By area
Engine (C++). Targeted memory-safety / correctness fixes, each with a regression scenario: SMAPE weight OOB read (
forecast.h/timeseries.cpp),EntityIteratorcopy-ctor UB andCalendar::EventIterator::operator--UB,Buffer::setMaximum*iterator-after-erase andfollowPeggingnull-deref guards,OperatorDeleterefcount leak, GIL-safeinitialize()+ owned-PyObject adoption inpython.cpp, inverted bound in JSONgetLong/getInt, anda_penalty/a_costdouble-counting across capacity rechecks. Newpegging_*/structural_*/forecast_11tests; the Debug+ASan engine job is blocking and ASan-clean.Backend (Django / API / WS).
/api/token/mints a short-lived JWT for the session user; sharedjwtauthdecode + scenario routing; ASGI app with authenticated websockets (live task progress + log tail). Streaming JSON output endpoints; the enriched, self-describingPivotJSONStreamView(measures + buckets header over the report's unchangeddata) backs the forecast, inventory, demand and resource screens. Inactive-user rejection on the token paths.Frontend (Next.js console). App Router SPA with an industrial "planning console" design system (IBM Plex Mono/Sans, amber signal, app shell + status rail). Screens: Execute (launch / live-progress), Forecast (pivot editor, bulk fill/±%, outliers, Recharts), and three read-only pivot reports — Demand, Inventory, Resource — rendered by one generic
<PivotScreen>+usePivotReport+lib/pivot.ts(measure names from the envelope). Same-origin data layer:authedFetch(Bearer + CSRF) with typedAuthError, websocket hooks, auth-aware sign-in states.Deploy.
deploy/helm/frepple(app web+asgi co-located, frontend, redis, optional builtin postgres, one TLS ingress) +.github/workflows/deploy-staging.yml(buildfrepple-app/frepple-frontendon the x86 ARC runners → GHCR). Runtime hardened:DEBUGoff, generated/persistedSECRET_KEY, host allowlist, non-root standalone frontend image.Review round
Ran an independent multi-dimension review (quality / completeness / simplicity / DRY / tests / docs) and addressed the findings in code rather than posting them — frontend (typed
AuthError+ sharedauthedFetch, hardened launch detection, unmount guards, +tests), backend (xframe footgun on/api/token/, auth-before-metadata, +inactive-user tests), deploy (DEBUGoff, real secret/hosts, non-root image), tooling.Verification
frepplectl test freppledbgreen in CI.npm test(38) +next buildtypecheck green.helm upgraderollout green; cert issued;/execute,/forecast,/demand,/inventory,/resource→ 200; the four pivot endpoints return the enriched envelope; HTTPS login + JWT mint; DEBUG confirmed off. All 12 Playwright specs (smoke + a11y on five screens + live-progress: Run plan → real engine → WS → terminal) pass against the staging URL.Known limitations / follow-ups
runserver --insecure(DEBUG off) serves static for the demo; production wants gunicorn + whitenoise.runwebservice(daphne inside the frepple interpreter)./api/input/operationplan/write endpoint).jwtauth(decode centralized for ASGI only).