feat: add client-workflow find-similar via immich-clip-probe

README.md

@@ -15,6 +15,8 @@
 I want to be able to manage my Immich server from the command line for my large photo collection.
 There are some bulk features that are not available in the web interface, and I want to be able to automate some tasks. Like compression, converting, repairing, and deleting photos.
 
+Another motivation is managing an overwhelming number of images with AI-agent support: the generated agent skill (`immich-admin-cli/SKILL.md`) describes every command for AI consumption, and the CLI surface itself is optimized for AI agents (e.g. Hermes Agent) — machine-readable `--json` output, pipeable `--ids-only` IDs, `--dry-run` previews, and non-interactive `--yes` flags. Agents can always retrieve the up-to-date skill straight from the binary via `immich-admin return-agent-skill`, so they never have to work from stale documentation about how the CLI works.
+
 ## Self-Update
 
 `immich-admin update` checks the latest [GitHub release](https://github.com/dhcgn/immich-admin-cli/releases) for a build matching the current OS/arch and, on confirmation, replaces the running executable in place and restarts it (via [`gh-update`](https://github.com/dhcgn/gh-update)).
@@ -42,6 +44,7 @@ All workflows follow the same safety model: `--dry-run` shows what would happen
 | ✅ done | `client-workflow repair-assets` | Repair corrupt JPEG (missing EOI marker) and TIFF (invalid zero-count IFD tag) assets and re-import them, keeping metadata |
 | ✅ done | `client-workflow fix-album-dates` | Check assets in date-named albums ("2025-07-04 Garten", "2010 USA") against the date implied by the album name, and offer to fix mismatches |
 | ✅ done | `client-workflow download-album` | Download all originals or all thumbnails from one album to a local folder, optionally excluding videos, optionally kept in sync |
+| ✅ done | `client-workflow find-similar` | Find assets in Immich that look like a local image file (via external clip-probe service) |
 | ⏳ planned | `client-workflow reencode-jxl` | Re-encode assets to JPEG XL (`cjxl`), then replace the originals |
 | ⏳ planned | `client-workflow reencode-jpegli` | Re-encode assets with jpegli (`cjpegli`), then replace the originals |
 
@@ -317,6 +320,24 @@ Flags: exactly one of `--album-id UUID` / `--album-name NAME`, `--target-dir DIR
 
 The underlying operations are also available standalone: `assets download-original` and `assets download-thumbnail` (`GET /assets/{id}/thumbnail`, supporting `fullsize|preview|thumbnail`, default `preview`, plus `--edited`; `original` is not accepted here — the OpenAPI spec deprecates `size=original` on this endpoint in favor of `assets download-original`).
 
+### `client-workflow find-similar`
+
+Finds assets already in your Immich library that look like a local image file — before you upload it. Uses the external [immich-clip-probe](https://github.com/dhcgn/immich-clip-probe/) service (runs next to Immich, read-only, token-authenticated). This is the first command that depends on a third-party service: configure `clip_probe.server` + `clip_probe.token` in the config file (or `IMMICH_CLIP_PROBE_SERVER` / `IMMICH_CLIP_PROBE_TOKEN` env vars).
+
+```sh
+# Nearest matches, duplicate verdict based on --max-distance (default 0.01, same as Immich's own duplicate default)
+immich-admin cw find-similar DSC_2031.jpg
+
+# Calibrate a threshold: nearest 5 regardless of distance
+immich-admin cw find-similar DSC_2031.jpg --all --limit 5
+
+# Pipeable IDs + full raw response
+immich-admin cw find-similar DSC_2031.jpg --ids-only
+immich-admin cw find-similar DSC_2031.jpg --json
+```
+
+Flags: `--limit 1-100` (default `10`), `--max-distance 0-2` (default `0.01`), `--all` (ignore the cut-off), `--type IMAGE|VIDEO|all` (default `IMAGE`), `--json`, `--ids-only` / `-q`. An empty match list is a normal result (nothing close enough), not an error.
+
 ## Sample Use Cases
 
 ### I imported some corrupt images, now I want to replace them, but keep all metadata and albums

config.example.yaml

@@ -17,3 +17,18 @@ api_key: YOUR_API_KEY_HERE
 #   # Used by `client-workflow download-album --resize-video-preset`.
 #   # Env var: IMMICH_FFMPEG_PATH
 #   ffmpeg_path: /usr/bin/ffmpeg
+
+# Optional: external visual-similarity service used by
+# `client-workflow find-similar`. This is the first command that depends on a
+# third-party service running next to Immich — setup:
+# https://github.com/dhcgn/immich-clip-probe/
+# Leave unset if unused; only `find-similar` requires it. Each field can
+# also be set via an environment variable instead (see comments below),
+# which takes precedence over this file.
+# clip_probe:
+#   # Origin of the clip-probe service (no /v1 suffix — the client appends
+#   # /v1/similar). Env var: IMMICH_CLIP_PROBE_SERVER
+#   server: https://clip-probe.example.com/
+#   # This service's own token (API_TOKEN), NOT an Immich API key.
+#   # Env var: IMMICH_CLIP_PROBE_TOKEN
+#   token: YOUR_CLIP_PROBE_TOKEN_HERE

immich-admin-cli/SKILL.md

@@ -378,6 +378,17 @@ Move every asset from one album into another, optionally deleting the emptied so
 - `--dry-run` (bool): print the merge plan without changing anything
 - `--yes` (bool): skip the confirmation prompt before moving assets
 
+### client-workflow find-similar
+
+Find assets in Immich that look like a local image file (via clip-probe)
+Args: `FILE`
+- `--limit` (int): maximum number of matches to return (1-100) [default: 10]
+- `--max-distance` (float): cosine-distance cut-off; matches above this are dropped (0-2) [default: 0.01]
+- `--all` (bool): return the limit nearest matches regardless of --max-distance (for calibrating a threshold)
+- `--type` (string): restrict to a single asset type: IMAGE, VIDEO, or all [default: "IMAGE"]
+- `--json` (bool): print the raw clip-probe response as JSON
+- `--ids-only, -q` (bool): print only matching asset IDs, one per line
+
 ### client-workflow watch-upload
 
 Watch a folder and auto-upload new stable files (POST /assets/bulk-upload-check + POST /assets)
@@ -569,6 +580,10 @@ albums separate, or rename instead of merging blindly. Always run
   `--sidecar`/`--filename` are single-file only; per-file byte bar on stderr, `--quiet` disables, `--json` implies quiet).
   `assets check-remote-exists FILE|DIR...` (alias `check-bulk-upload`) hashes locally and reports
   `uploaded <id>` / `missing` / `unsupported` without mutating (`--json`, `--ids-only -q` pipes duplicate IDs into `albums add-assets`/`tags tag`).
+- **Similarity**: `client-workflow find-similar FILE [--all] [--limit N] [--max-distance 0.01] [--type IMAGE] [--json] [-q]`
+  asks the external clip-probe service (https://github.com/dhcgn/immich-clip-probe/, configured via
+  `clip_probe.server`/`clip_probe.token` or `IMMICH_CLIP_PROBE_*` env) which library assets look like a local
+  file before uploading. Empty matches = nothing close enough (not an error).
 - **Watch**: `client-workflow watch-upload --watch-dir DIR --mode flat|by-subfolder [--tag-pattern "immich-admin-cli/watch/{yyyy-MM-dd}"] [--album-id|--album-name] --interval 60s --stable-for 10s [--once] [--dry-run] [--yes]`
   polls and uploads only stable files (bulk-check first; duplicates only linked; `--once` forces `--stable-for 0s`, i.e. uploads immediately).
   A finished folder that nothing is still writing into needs just `--once`; run the loop (no `--once`) only while files are still arriving.

internal/clipprobe/client.go

@@ -0,0 +1,138 @@
+package clipprobe
+
+import (
+	"bytes"
+	"context"
+	"encoding/json"
+	"fmt"
+	"io"
+	"mime/multipart"
+	"net/http"
+	"net/textproto"
+	"net/url"
+	"os"
+	"path/filepath"
+	"slices"
+	"strconv"
+	"strings"
+	"time"
+)
+
+// ValidTypes is the closed set accepted by POST /v1/similar ?type=.
+var ValidTypes = []string{"IMAGE", "VIDEO", "all"}
+
+// Options maps 1:1 to the /v1/similar query parameters.
+type Options struct {
+	Limit       int
+	MaxDistance float64
+	All         bool
+	Type        string
+}
+
+// DefaultOptions mirrors the service defaults (openapi.yaml).
+func DefaultOptions() Options {
+	return Options{Limit: 10, MaxDistance: 0.01, Type: "IMAGE"}
+}
+
+// Validate rejects values outside the service's closed set before any I/O.
+func (o Options) Validate() error {
+	if o.Limit < 1 || o.Limit > 100 {
+		return fmt.Errorf("invalid --limit %d: must be between 1 and 100", o.Limit)
+	}
+	if o.MaxDistance < 0 || o.MaxDistance > 2 {
+		return fmt.Errorf("invalid --max-distance %v: must be between 0 and 2", o.MaxDistance)
+	}
+	if slices.Contains(ValidTypes, o.Type) {
+		return nil
+	}
+	return fmt.Errorf("invalid --type %q: must be one of %s", o.Type, strings.Join(ValidTypes, ", "))
+}
+
+// Client talks to one clip-probe service. No Immich credentials are ever
+// sent here — the token is the service's own API_TOKEN (x-api-key header).
+type Client struct {
+	server string
+	token  string
+	http   *http.Client
+}
+
+// New builds a Client for the service origin (e.g. https://clip-probe.example.com/).
+func New(server, token string) *Client {
+	transport := http.DefaultTransport.(*http.Transport).Clone()
+	transport.ResponseHeaderTimeout = 30 * time.Second
+	return &Client{server: strings.TrimRight(server, "/"), token: token, http: &http.Client{Transport: transport}}
+}
+
+// FindSimilar posts one image file and returns the decoded response plus the
+// raw body (for --json output). An empty Matches is not an error.
+func (c *Client) FindSimilar(ctx context.Context, filePath string, opts Options) (*SimilarResponse, []byte, error) {
+	if err := opts.Validate(); err != nil {
+		return nil, nil, err
+	}
+	f, err := os.Open(filePath)
+	if err != nil {
+		return nil, nil, fmt.Errorf("opening file: %w", err)
+	}
+	defer f.Close()
+
+	q := url.Values{}
+	q.Set("limit", strconv.Itoa(opts.Limit))
+	q.Set("max_distance", strconv.FormatFloat(opts.MaxDistance, 'f', -1, 64))
+	if opts.All {
+		q.Set("all", "true")
+	}
+	if opts.Type != "" {
+		q.Set("type", opts.Type)
+	}
+	endpoint := c.server + "/v1/similar?" + q.Encode()
+
+	var body bytes.Buffer
+	w := multipart.NewWriter(&body)
+	h := textproto.MIMEHeader{}
+	h.Set("Content-Disposition", fmt.Sprintf(`form-data; name="file"; filename="%s"`, escapeQuotes(filepath.Base(filePath))))
+	h.Set("Content-Type", "application/octet-stream")
+	part, err := w.CreatePart(h)
+	if err != nil {
+		return nil, nil, fmt.Errorf("building multipart body: %w", err)
+	}
+	if _, err := io.Copy(part, f); err != nil {
+		return nil, nil, fmt.Errorf("reading file: %w", err)
+	}
+	if err := w.Close(); err != nil {
+		return nil, nil, fmt.Errorf("building multipart body: %w", err)
+	}
+
+	req, err := http.NewRequestWithContext(ctx, http.MethodPost, endpoint, &body)
+	if err != nil {
+		return nil, nil, fmt.Errorf("building request: %w", err)
+	}
+	req.Header.Set("Content-Type", w.FormDataContentType())
+	req.Header.Set("x-api-key", c.token)
+
+	resp, err := c.http.Do(req)
+	if err != nil {
+		return nil, nil, fmt.Errorf("calling POST /v1/similar: %w", err)
+	}
+	defer resp.Body.Close()
+	raw, err := io.ReadAll(io.LimitReader(resp.Body, 8<<20))
+	if err != nil {
+		return nil, nil, fmt.Errorf("reading response: %w", err)
+	}
+	if resp.StatusCode != http.StatusOK {
+		var eb ErrorBody
+		if json.Unmarshal(raw, &eb) == nil && eb.Error != "" {
+			return nil, nil, fmt.Errorf("clip-probe returned %s (%s): %s", resp.Status, eb.Error, eb.Message)
+		}
+		return nil, nil, fmt.Errorf("clip-probe returned %s: %s", resp.Status, strings.TrimSpace(string(raw)))
+	}
+	var out SimilarResponse
+	if err := json.Unmarshal(raw, &out); err != nil {
+		return nil, nil, fmt.Errorf("decoding response: %w", err)
+	}
+	return &out, raw, nil
+}
+
+// escapeQuotes keeps a hostile file name inside the multipart header.
+func escapeQuotes(s string) string {
+	return strings.ReplaceAll(s, `"`, "_")
+}

internal/clipprobe/client_test.go

@@ -0,0 +1,112 @@
+package clipprobe
+
+import (
+	"context"
+	"encoding/json"
+	"mime/multipart"
+	"net/http"
+	"net/http/httptest"
+	"os"
+	"path/filepath"
+	"strings"
+	"testing"
+)
+
+func TestOptionsValidate(t *testing.T) {
+	if err := DefaultOptions().Validate(); err != nil {
+		t.Fatalf("defaults must validate: %v", err)
+	}
+	for _, tc := range []Options{
+		{Limit: 0, MaxDistance: 0.01, Type: "IMAGE"},
+		{Limit: 101, MaxDistance: 0.01, Type: "IMAGE"},
+		{Limit: 10, MaxDistance: -0.1, Type: "IMAGE"},
+		{Limit: 10, MaxDistance: 2.1, Type: "IMAGE"},
+		{Limit: 10, MaxDistance: 0.01, Type: "BOGUS"},
+	} {
+		if err := tc.Validate(); err == nil {
+			t.Errorf("expected error for %+v", tc)
+		}
+	}
+}
+
+func TestFindSimilarRoundtrip(t *testing.T) {
+	var gotKey, gotQuery, gotFileName string
+	var gotFileBytes []byte
+	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		gotKey = r.Header.Get("x-api-key")
+		gotQuery = r.URL.RawQuery
+		if err := r.ParseMultipartForm(1 << 20); err != nil {
+			t.Errorf("parsing multipart: %v", err)
+		}
+		f, hdr, err := r.FormFile("file")
+		if err != nil {
+			t.Errorf("missing file part: %v", err)
+		} else {
+			gotFileName = hdr.Filename
+			buf := make([]byte, 4)
+			n, _ := f.Read(buf)
+			gotFileBytes = buf[:n]
+			f.Close()
+		}
+		w.Header().Set("Content-Type", "application/json")
+		_ = json.NewEncoder(w).Encode(SimilarResponse{
+			Duplicate:   true,
+			MaxDistance: 0.01,
+			Query:       Query{FileName: "q.jpg", Bytes: 4, Width: 2, Height: 2, Model: "ViT-B-32__openai", Dimensions: 512},
+			Matches:     []Match{{AssetID: "0421250e-23a9-4a5b-a9ac-23c9bd7cafdb", OriginalFileName: "DSC_2031.jpg", Type: "IMAGE", Distance: 0.0028, Similarity: 0.9971}},
+		})
+	}))
+	defer srv.Close()
+
+	dir := t.TempDir()
+	img := filepath.Join(dir, "q.jpg")
+	if err := os.WriteFile(img, []byte{0xFF, 0xD8, 0xFF, 0xD9}, 0o644); err != nil {
+		t.Fatal(err)
+	}
+
+	c := New(srv.URL, "sekret")
+	resp, raw, err := c.FindSimilar(context.Background(), img, DefaultOptions())
+	if err != nil {
+		t.Fatalf("FindSimilar: %v", err)
+	}
+	if gotKey != "sekret" {
+		t.Errorf("x-api-key = %q", gotKey)
+	}
+	for _, want := range []string{"limit=10", "max_distance=0.01", "type=IMAGE"} {
+		if !strings.Contains(gotQuery, want) {
+			t.Errorf("query %q missing %q", gotQuery, want)
+		}
+	}
+	if gotFileName != "q.jpg" {
+		t.Errorf("filename = %q", gotFileName)
+	}
+	if len(gotFileBytes) != 4 {
+		t.Errorf("file bytes = %d", len(gotFileBytes))
+	}
+	if !resp.Duplicate || len(resp.Matches) != 1 || resp.Matches[0].AssetID != "0421250e-23a9-4a5b-a9ac-23c9bd7cafdb" {
+		t.Errorf("unexpected response: %+v", resp)
+	}
+	var check SimilarResponse
+	if err := json.Unmarshal(raw, &check); err != nil {
+		t.Errorf("raw body is not JSON: %v", err)
+	}
+}
+
+func TestFindSimilarErrorMapping(t *testing.T) {
+	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		_ = multipart.NewWriter(nil)
+		w.WriteHeader(http.StatusUnauthorized)
+		_, _ = w.Write([]byte(`{"error":"unauthorized","message":"bad token"}`))
+	}))
+	defer srv.Close()
+
+	dir := t.TempDir()
+	img := filepath.Join(dir, "q.jpg")
+	if err := os.WriteFile(img, []byte{1, 2}, 0o644); err != nil {
+		t.Fatal(err)
+	}
+	_, _, err := New(srv.URL, "wrong").FindSimilar(context.Background(), img, DefaultOptions())
+	if err == nil || !strings.Contains(err.Error(), "unauthorized") {
+		t.Errorf("expected unauthorized error, got %v", err)
+	}
+}