From 19ab1574f2bb0d62948dea801c49fe468e345302 Mon Sep 17 00:00:00 2001 From: agrechenkov Date: Mon, 27 Jul 2026 17:31:55 -0400 Subject: [PATCH 1/2] =?UTF-8?q?2.4.1:=20model=20picker=20=E2=80=94=20hones?= =?UTF-8?q?t=20sizing,=20nothing=20hidden,=20upgrade-break=20detection?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Fixes three reported UI issues plus a silent-failure gap found while investigating them. - No model is hidden. The dashboard dropped every fits=="no" candidate, which made real catalog models invisible: qwen3.6-moe:35b-a3b never appeared and had to be pulled by hand. recommend_models now returns EVERY candidate with a fits tag + fit_reason; oversized ones render marked and confirm before pull. - Sizing uses real numbers. MoE tags were read as their total ('...35b-a3b' as 35B, not ~3B active). Fit now prefers FastFlowLM's measured footprint_gb (which also fixes gemma4-it:e4b: reads "4B" by name, actually 8B / 9.1 GB) and falls back to active-param counts for MoE. flm_list forwards the catalog metadata (footprint/installed/flm_min_version/parameter_size/context) that it previously discarded. Verified against all 37 live FLM 0.9.45 catalog names: zero false MoE detections. - Provider-upgrade breakage is surfaced. FLM 0.9.45 rejects weights stamped for 0.9.43 and reports those models as not installed; nothing detected that, so the next hotkey failed opaquely (no Ollama fallback available). New _active_model_health reports ok / not_installed / unknown, and Config → Models shows a warning naming the model and remedy. Listing errors stay "unknown" — never a false alarm. - One "Models" card. "Installed models" + "Pull a new model" merged into a single card with Installed / Add a model / FastFlowLM runtime subheads. - The picker is app-styled. Replaced a native (browsers draw the popup in their own chrome and ignore page CSS, so it could never match the app) and a native ` list box — browsers draw datalist popups in their own chrome and ignore page CSS entirely, so that control could never match the dashboard. The new combobox and list are ordinary elements: they follow the light/dark theme, and support arrow-key/Enter/Escape navigation and type-to-filter. + ## 2.4.0 **Streaming chat.** The dashboard Chat tab renders replies token-by-token instead of waiting for the whole answer, so the response starts appearing at warm time-to-first-token rather than after the full completion. diff --git a/README.md b/README.md index 657b254..6968086 100644 --- a/README.md +++ b/README.md @@ -4,10 +4,11 @@ Flowkey is a Windows desktop assistant that adds local-LLM hotkeys for grammar f Everything runs locally through [FastFlowLM](https://fastflowlm.com) (AMD Ryzen AI NPU) or, on machines without the NPU, through [Ollama](https://ollama.com) (CPU/GPU) as a secondary provider. No cloud service, analytics, or telemetry is used by the app. -Current version: `2.4.0` +Current version: `2.4.1` ## What's new in 2.4 +- **The model picker shows every model, sized honestly.** Config → **Models** is now one card, styled like the rest of the dashboard (the old picker used a native browser dropdown that ignored the app's theme). Nothing is hidden from it: models too big for your machine are marked rather than silently dropped, and Mixture-of-Experts models are sized by the parameters actually active per token plus FastFlowLM's own measured memory footprint — so a 35B-total / 3B-active model is no longer mistaken for a 35B one. If a FastFlowLM upgrade invalidates your active model, the card now says so instead of letting the next hotkey fail with an opaque error. - **Chat streams as it thinks.** The dashboard **Chat** tab now fills the reply in token-by-token over a live stream instead of waiting on a blank "Thinking…" until the whole answer is ready — the first words typically appear in ~1.6 s (warm) rather than after the full response. It works on both FastFlowLM and Ollama, and if anything interrupts the stream the partial reply is kept and the error is shown. Grammar, `prompt:`, and the other hotkeys are unchanged (they still paste the finished result). ## What's new in 2.3 diff --git a/SPEC.md b/SPEC.md index 8f3aedf..9a744fe 100644 --- a/SPEC.md +++ b/SPEC.md @@ -89,6 +89,9 @@ Caveman-encoded (compression, not amputation). Paths / ids / action names / numb - V35: default v2 → exactly 1 short LLM draft call; ⊥ anti-echo/rescue calls; V33 finalizer supplies surfaced 4-section contract - V36: pre-2.3 prompt_builder identity cfg (⊥ `prompt_version`, legacy default fields) → migrate v2+concise; any custom field → preserve - V37: `chat_send_stream` streams reply deltas as `text/event-stream` (SSE `data:`/`event:` frames), persists the full-or-partial turn exactly once under the daemon write-lock; CSRF `X-FFP-API` + Host gates still apply; dashboard falls back to `chat_send` iff the stream ⊥ opens (never after tokens arrive); `send()`/AHK/grammar/prompt paths unchanged (⊥ streamed) +- V38: `recommend_models` returns EVERY candidate tagged yes/tight/no/unknown + `fit_reason`; UI ⊥ hide any (oversized → marked + confirm-before-pull). Fit signal precedence: provider `footprint_gb` → effective params (MoE active `ab` < total, else total) vs budget +- V39: configured active model ⊥ installed → `model_recommendations.active_model.status="not_installed"` + dashboard warning naming model/remedy; listing error/exception → `"unknown"` (⊥ claim broken) +- V40: model picker + installed list = app-styled elements (⊥ native ``/``|V40; custom app-styled combobox + list +B32|2026-07-27|`app.js` set `models-title` = "Installed models — " ∴ overwrote merged card heading at runtime|V40 caught in browser; title → "Models — " +B33|2026-07-27|new CSS block referenced undefined `--muted`/`--card`/`--bad` (pre-existing pattern elsewhere in stylesheet: 6+2+3 refs, never defined ∴ inert)|use defined tokens `--text-muted`/`--surface`/`--warn`; pre-existing refs left for separate cleanup +B34|2026-07-27|self-caught: `_active_model_health` tested membership vs `_provider_list("all")`; ollama "all" = installed+suggested (`ffp_provider_runtime:80`) ∴ never-pulled model → false `installed=True` (⊥ warn)|V39; trust `details` (unfiltered, authoritative) else re-list w/ `installed` filter ``` diff --git a/installer/installer.iss b/installer/installer.iss index cd2e88b..6c67e32 100644 --- a/installer/installer.iss +++ b/installer/installer.iss @@ -41,7 +41,7 @@ #define AppURL "https://github.com/agr77one/Fastflow" #define AppExeName "Flowkey.exe" ; symbolic — actual launchers below ; Keep in lockstep with scripts\_version.py. -#define AppVersion "2.4.0" +#define AppVersion "2.4.1" [Setup] AppId={{8A4F1E6C-9B3D-4E62-9F7A-FASTFLOW140}} diff --git a/installer/sign.ps1 b/installer/sign.ps1 index 6248553..f2be445 100644 --- a/installer/sign.ps1 +++ b/installer/sign.ps1 @@ -52,7 +52,7 @@ .EXAMPLE # Sign the installer $env:FFP_SIGN_PFX_PASSWORD = "ChangeMe!" - .\sign.ps1 -FilePath ..\out\Flowkey-Setup-2.4.0.exe + .\sign.ps1 -FilePath ..\out\Flowkey-Setup-2.4.1.exe #> [CmdletBinding(DefaultParameterSetName = "Sign")] diff --git a/pyproject.toml b/pyproject.toml index 7d7682a..a1704a4 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -17,7 +17,7 @@ build-backend = "setuptools.build_meta" [project] name = "fastflowprompt" -version = "2.4.0" +version = "2.4.1" description = "Local-LLM-powered grammar fix, prompt rewrite, chat, and dashboard for Windows." readme = "README.md" requires-python = ">=3.11" diff --git a/scripts/_version.py b/scripts/_version.py index 9458797..5d04335 100644 --- a/scripts/_version.py +++ b/scripts/_version.py @@ -1,3 +1,3 @@ """Single source of truth for the app version. Read by grammar_fix.py.""" -__version__ = "2.4.0" +__version__ = "2.4.1" diff --git a/scripts/ffp_daemon.py b/scripts/ffp_daemon.py index b268e34..c4c44d2 100644 --- a/scripts/ffp_daemon.py +++ b/scripts/ffp_daemon.py @@ -340,16 +340,92 @@ def _act_models_not_installed(_args: dict) -> dict: def _act_model_recommendations(_args: dict) -> dict: """Hardware-aware model suggestions for the active provider: detected - RAM/VRAM, a size budget, and candidate models tagged fits yes/tight/no.""" + RAM/VRAM, a size budget, and candidate models tagged fits yes/tight/no. + + On FastFlowLM the catalog's own per-model metadata (measured footprint, + install state, minimum FLM version) is forwarded so fit is decided from real + numbers rather than guessed from the tag name.""" import ffp_hardware provider = grammar_fix.LLM_PROVIDER if provider == "ollama": - candidates = list(ffp_hardware.OLLAMA_CATALOG) + candidates: list = list(ffp_hardware.OLLAMA_CATALOG) else: listing = grammar_fix._provider_list("all") - names = listing.get("models") or [] - candidates = [(n, ffp_hardware.parse_params_b(n)) for n in names] - return ffp_hardware.recommend_models(provider, candidates) + details = listing.get("details") or [] + if details: + candidates = [dict(entry) for entry in details if isinstance(entry, dict)] + else: + # Older FLM without per-model JSON metadata: fall back to tag parsing. + candidates = [(n, ffp_hardware.parse_params_b(n)) for n in (listing.get("models") or [])] + out = ffp_hardware.recommend_models(provider, candidates) + out["active_model"] = _active_model_health(listing if provider != "ollama" else None) + return out + + +def _active_model_health(listing: dict | None = None) -> dict: + """Report whether the configured active model is actually usable. + + A FastFlowLM upgrade can invalidate an already-pulled model (FLM 0.9.45 + rejects models stamped for 0.9.43), and FLM then reports them as + NOT installed. Nothing used to surface that, so the next hotkey failed with + an opaque provider error. This gives the dashboard a specific, actionable + state: which model is configured, whether it is installed, and the remedy. + + ``listing`` may be a already-fetched provider listing to avoid a second CLI + call; it is only trusted when it carries per-model ``details`` (whose + ``installed`` flags are authoritative and filter-independent). Without + details we must re-list with the ``installed`` filter, because an + ``all`` listing includes not-yet-pulled suggestions and would report a + missing model as installed.""" + provider = grammar_fix.LLM_PROVIDER + model = str(grammar_fix.FLM_MODEL or "") + health = {"provider": provider, "model": model, "installed": None, "status": "unknown", "detail": ""} + if not model: + health.update(status="unknown", detail="no active model configured") + return health + + details = list((listing or {}).get("details") or []) + if not details: + try: + listing = grammar_fix._provider_list("installed") + except Exception as exc: + health.update(status="unknown", detail=f"could not list models: {exc}") + return health + if listing.get("error"): + health.update(status="unknown", detail=str(listing.get("error"))) + return health + details = list(listing.get("details") or []) + elif listing and listing.get("error"): + health.update(status="unknown", detail=str(listing.get("error"))) + return health + + entry = next((d for d in details if isinstance(d, dict) and d.get("name") == model), None) + if entry is not None: + installed = bool(entry.get("installed")) + health["installed"] = installed + health["flm_min_version"] = entry.get("flm_min_version") or "" + if installed: + health.update(status="ok", detail="") + else: + health.update( + status="not_installed", + detail=f"'{model}' is in the catalog but not installed for this " + f"{provider} version - run a pull to (re)install it.", + ) + return health + + # No per-model metadata (Ollama, or an older FLM): the names here are + # installed-only, so absence genuinely means "not installed". + names = set((listing or {}).get("models") or []) + if model in names: + health.update(installed=True, status="ok", detail="") + else: + health.update( + installed=False, + status="not_installed", + detail=f"'{model}' is not installed for {provider} - pull it or pick another model.", + ) + return health def _act_pull_model(args: dict) -> str: diff --git a/scripts/ffp_flm_server.py b/scripts/ffp_flm_server.py index 86ec758..9562878 100644 --- a/scripts/ffp_flm_server.py +++ b/scripts/ffp_flm_server.py @@ -252,6 +252,14 @@ def flm_list(filter_kind: str, model: str, no_window: int) -> dict: bare names. The JSON mode (FLM >= 0.9.x) carries an authoritative per-model `installed` boolean instead, so we parse that and filter client-side. We force UTF-8 decoding because the human-readable variant embeds emoji. + + Also returns ``details``: the per-model metadata the catalog already carries + — ``footprint_gb`` (FLM's own measured memory cost, the honest sizing signal + that name-parsing cannot supply: ``gemma4-it:e4b`` reads as "4B" but is 8B / + 9.1 GB, and ``qwen3.6-moe:35b-a3b`` is 35B total / 3B active / 24.3 GB), + ``installed``, ``flm_min_version``, ``parameter_size`` and + ``context_length``. ``details`` is unfiltered so callers can tell "not in + the catalog" from "in the catalog but not installed". """ if filter_kind not in {"installed", "not-installed", "all"}: return {"error": f"bad filter: {filter_kind}", "models": [], "active": model} @@ -276,6 +284,7 @@ def flm_list(filter_kind: str, model: str, no_window: int) -> dict: return {"error": "could not parse 'flm list --json' output", "models": [], "active": model} names: list[str] = [] + details: list[dict] = [] for entry in payload.get("models") or []: if not isinstance(entry, dict): continue @@ -283,12 +292,28 @@ def flm_list(filter_kind: str, model: str, no_window: int) -> dict: if not name: continue is_installed = bool(entry.get("installed")) + info = entry.get("details") if isinstance(entry.get("details"), dict) else {} + details.append({ + "name": name, + "installed": is_installed, + "footprint_gb": _as_float(entry.get("footprint")), + "flm_min_version": str(entry.get("flm_min_version") or "").strip(), + "parameter_size": str(info.get("parameter_size") or "").strip(), + "context_length": entry.get("default_context_length"), + }) if filter_kind == "installed" and not is_installed: continue if filter_kind == "not-installed" and is_installed: continue names.append(name) - return {"models": names, "active": model} + return {"models": names, "active": model, "details": details} + + +def _as_float(value: object) -> float | None: + try: + return float(value) # type: ignore[arg-type] + except (TypeError, ValueError): + return None def flm_version(no_window: int) -> str: diff --git a/scripts/ffp_hardware.py b/scripts/ffp_hardware.py index db6ea9b..ef67f08 100644 --- a/scripts/ffp_hardware.py +++ b/scripts/ffp_hardware.py @@ -55,11 +55,17 @@ _PARAMS_RE = re.compile(r"(\d+(?:\.\d+)?)\s*b\b", re.IGNORECASE) _PARAMS_M_RE = re.compile(r"(\d+(?:\.\d+)?)\s*m\b", re.IGNORECASE) +# Mixture-of-Experts tags publish ACTIVE params as "ab" alongside the total: +# "qwen3.6-moe:35b-a3b" is 35B total but only ~3B active per token, so it decodes +# like a 3B model. Sizing it by the 35B total is what hid it from the picker. +_ACTIVE_PARAMS_RE = re.compile(r"\ba(\d+(?:\.\d+)?)\s*b\b", re.IGNORECASE) def parse_params_b(name: str) -> float | None: - """Best-effort parameter count (in billions) from a model name/tag. - 'qwen3.5:4b' -> 4.0, 'gemma4-it:e4b' -> 4.0, 'embed-gemma:300m' -> 0.3.""" + """Best-effort TOTAL parameter count (in billions) from a model name/tag. + 'qwen3.5:4b' -> 4.0, 'gemma4-it:e4b' -> 4.0, 'embed-gemma:300m' -> 0.3. + For MoE tags this is the total ('...:35b-a3b' -> 35.0); see + :func:`parse_active_params_b` for the active count that governs speed.""" text = str(name or "") match = _PARAMS_RE.search(text) if match: @@ -70,6 +76,29 @@ def parse_params_b(name: str) -> float | None: return None +def parse_active_params_b(name: str) -> float | None: + """Active params for a MoE tag, or None when the tag isn't MoE-shaped. + + 'qwen3.6-moe:35b-a3b' -> 3.0; 'qwen3.5:4b' -> None. Only returns a value when + an 'ab' group is present AND smaller than the parsed total, so an ordinary + tag that merely happens to contain 'ab' can't be misread.""" + text = str(name or "") + match = _ACTIVE_PARAMS_RE.search(text) + if not match: + return None + active = float(match.group(1)) + total = parse_params_b(text) + if total is None or active >= total: + return None + return active + + +def effective_params_b(name: str) -> float | None: + """Params that govern how this model actually performs: active for MoE, + total otherwise. This is what a size budget should be compared against.""" + return parse_active_params_b(name) or parse_params_b(name) + + def system_memory_gb() -> float: if os.name == "nt": import ctypes @@ -209,22 +238,109 @@ def model_budget(provider: str, hw: dict | None = None) -> dict: return {"max_params_b": max_b, "basis": basis, "summary": summary} -def recommend_models(provider: str, candidates: list[tuple[str, float | None]], - hw: dict | None = None) -> dict: - """Tag candidate (name, params_b) pairs with whether they fit this - machine. Unparseable sizes are kept but marked unknown.""" +# Memory the OS, the app, and the browser need to keep running alongside a model. +_OS_HEADROOM_GB = 6.0 +# Below this share of usable memory a model is comfortable; above it, "tight". +_COMFORT_SHARE = 0.85 + + +def usable_memory_gb(hw: dict | None = None) -> float: + """Memory available for model weights, after OS/app headroom. + + On Ryzen AI laptops the iGPU's "dedicated" VRAM is a UMA carve-out of the + same DIMMs, so it is counted back in (matching :func:`model_budget`).""" + hw = hw or detect_hardware() + mem = float(hw.get("ram_gb") or 0) + float(hw.get("vram_gb") or 0) + return max(2.0, round(mem - _OS_HEADROOM_GB, 1)) + + +def _normalize_candidate(candidate: object) -> dict: + """Accept either a legacy ``(name, params_b)`` tuple or a metadata dict.""" + if isinstance(candidate, dict): + info = dict(candidate) + name = str(info.get("name") or "") + else: + name, params = candidate # type: ignore[misc] + name = str(name or "") + info = {"name": name, "params_b": params} + if "params_b" not in info or info.get("params_b") is None: + info["params_b"] = parse_params_b(name) + info["name"] = name + info["active_params_b"] = parse_active_params_b(name) + return info + + +def recommend_models(provider: str, candidates: list, hw: dict | None = None) -> dict: + """Tag candidates with whether they fit this machine. + + Candidates are either legacy ``(name, params_b)`` tuples or dicts carrying + provider metadata (``footprint_gb``, ``installed``, ``flm_min_version``, …). + + Fit is decided from the best signal available, in order: + + 1. **``footprint_gb``** — FastFlowLM's own measured memory cost. Name-parsing + is unreliable in both directions (``gemma4-it:e4b`` reads as 4B but is + 8B/9.1 GB; ``qwen3.6-moe:35b-a3b`` reads as 35B but is 3B-active/24.3 GB), + so when the provider tells us the real number we use it. + 2. **effective params** — active params for MoE, total otherwise, against the + provider size budget. + + Nothing is ever dropped: every candidate is returned with a ``fits`` tag of + yes/tight/no/unknown plus a human ``fit_reason``. Callers must render "no" + entries (disabled or warned), never hide them — silently filtering oversized + models is what made a catalog model invisible and forced a manual pull.""" hw = hw or detect_hardware() budget = model_budget(provider, hw) max_b = budget["max_params_b"] + usable_gb = usable_memory_gb(hw) models = [] - for name, params in candidates: - if params is None: - fits = "unknown" - elif params <= max_b: - fits = "yes" - elif params <= max_b * 1.5: - fits = "tight" + for candidate in candidates: + info = _normalize_candidate(candidate) + name = info["name"] + params = info.get("params_b") + active = info.get("active_params_b") + footprint = info.get("footprint_gb") + try: + footprint = float(footprint) if footprint is not None else None + except (TypeError, ValueError): + footprint = None + + if footprint and footprint > 0: + if footprint <= usable_gb * _COMFORT_SHARE: + fits, reason = "yes", f"~{footprint:g} GB of ~{usable_gb:g} GB usable" + elif footprint <= usable_gb: + fits, reason = "tight", f"~{footprint:g} GB needs most of ~{usable_gb:g} GB usable" + else: + fits, reason = "no", f"~{footprint:g} GB exceeds ~{usable_gb:g} GB usable" else: - fits = "no" - models.append({"name": name, "params_b": params, "fits": fits}) - return {"hardware": hw, "budget": budget, "provider": provider, "models": models} + effective = active or params + if effective is None: + fits, reason = "unknown", "size unknown" + elif effective <= max_b: + fits = "yes" + reason = f"~{effective:g}B active of ~{max_b:g}B budget" if active else f"~{effective:g}B of ~{max_b:g}B budget" + elif effective <= max_b * 1.5: + fits, reason = "tight", f"~{effective:g}B just over the ~{max_b:g}B budget" + else: + fits, reason = "no", f"~{effective:g}B exceeds the ~{max_b:g}B budget" + + entry = { + "name": name, + "params_b": params, + "fits": fits, + "fit_reason": reason, + "footprint_gb": footprint, + } + if active: + entry["active_params_b"] = active + for key in ("installed", "flm_min_version", "parameter_size", "context_length"): + if key in info: + entry[key] = info[key] + models.append(entry) + return { + "hardware": hw, + "budget": budget, + "usable_memory_gb": usable_gb, + "provider": provider, + "models": models, + } diff --git a/scripts/ui/web/app.js b/scripts/ui/web/app.js index 60e99e0..a58d8be 100644 --- a/scripts/ui/web/app.js +++ b/scripts/ui/web/app.js @@ -175,7 +175,9 @@ function applyProviderCaps() { $("cfg-warm-on-start").disabled = !isFlm; $("cfg-keep-warm").disabled = !isFlm; $("warm-model-note").hidden = !isFlm; - $("models-title").textContent = `Installed models — ${PROVIDER_LABELS[sel]}`; + // The card covers both the installed list and the pull form (each has its own + // subhead), so the heading is just "Models — ". + $("models-title").textContent = `Models — ${PROVIDER_LABELS[sel]}`; $("pull-hint").hidden = isFlm; $("pull-name").placeholder = isFlm ? "model name, e.g. qwen3.5:4b" : "model name, e.g. llama3.2:3b"; const note = $("provider-note"); @@ -954,68 +956,209 @@ async function loadServerStatus() { // Set by loadModels() from model_recommendations: {max_params_b, summary}. let modelBudget = null; +// Full candidate list [{name, fits, fit_reason, footprint_gb, installed, ...}]. +let modelOptions = []; +// Name selected in the installed-models list (replaces the old +

Models

+ + +

Installed

+
    - -
    -

    Pull a new model

    + +

    Add a model

    - - +
    + + + +

    -

    FastFlowLM runtime

    +

    FastFlowLM runtime

    FastFlowLM: checking…

    diff --git a/scripts/ui/web/styles.css b/scripts/ui/web/styles.css index fa501bd..93c30e1 100644 --- a/scripts/ui/web/styles.css +++ b/scripts/ui/web/styles.css @@ -479,6 +479,130 @@ input[type="text"], input[type="number"], textarea, select { } textarea { resize: vertical; font-family: Consolas, monospace; } select[size] { padding: 4px; } + +/* ---- Models card: styled list + combobox ---- + Replaces a native