GitHub - whit3rabbit/openkind: A Jev compatible rust client and custom local model with server. Custom model documented and researched.

OpenKind is a Rust decision engine with a CLI and server that implement the TypeSafe Jev protocol. The server runs locally hosted models, while the project researches custom Qwen 3.5 decision models and shared input processing across questions. For a more polished, related implementation, see Ollaya.

Install

Install the openkind CLI and openkindd server using one of these options:

Homebrew: macOS and Linux.
Cargo: install the published 0.4.0 crates. Requires Rust 1.90+, a native C/C++ build toolchain, and protoc on PATH.
Prebuilt archives: macOS, Linux, and Windows (x86_64).

After installation, keep both binaries on PATH; run openkind serve to start the server. Standard releases include MLX FP32 on Apple silicon and native CUDA on Linux glibc and Windows. Model selectors default to auto, with CPU fallback during loading. See backend releases and diagnostics.

Homebrew (macOS and Linux)

brew install whit3rabbit/tap/openkind
openkind version

Cargo

The currently published CLI and server crates are version 0.4.0. Install them from crates.io with Rust 1.90+, a native C/C++ build toolchain, and protoc on PATH. The server compiles its Protobuf interface during installation. Install protobuf with Homebrew on macOS or protobuf-compiler on Debian/Ubuntu. On Windows, use the MSVC build tools and a Protobuf compiler installation.

cargo install --locked --version 0.4.0 openkind-cli openkind-server
openkind version

Releases

Download an archive from GitHub Releases:

macOS: Apple silicon or Intel.
Linux: x86_64 glibc or portable musl.
Windows: x86_64 MSVC.

Separate -onnx archives bundle ONNX Runtime. Unpack the tarball or Windows zip, and keep both binaries together. CUDA user libraries and cuDNN remain operator-installed.

Run a model

Browse the catalog, pull a profile, and serve it locally. This example uses the Laya English decision encoder:

openkind catalog
openkind pull laya-english:c8ea29bf1e33a343c4b7
openkind serve \
  --installed-models laya-english:c8ea29bf1e33a343c4b7 \
  --http-addr 127.0.0.1:18080 \
  --grpc-addr 0

Use openkind doctor --json to inspect backend readiness after installation. Runtime availability does not establish model qualification. See the backend guide for MLX, CUDA, and ONNX details.

pull downloads pinned model files and verifies their SHA-256 digests. Builds and tests do not download weights. See the model guide for download sizes, memory requirements, and available backends.

In another terminal, save a request as request.json:

{
  "state": "My card was charged twice. Please refund the extra payment.",
  "model": "laya-english:c8ea29bf1e33a343c4b7",
  "questions": {
    "department": {
      "type": "choice",
      "instructions": "Which team should handle this request?",
      "criteria": {
        "billing": "Payments, charges, and refunds",
        "technical": "Bugs, outages, and integrations",
        "__none__": "None of these teams"
      }
    },
    "refund_requested": {
      "type": "noul",
      "instructions": "Is the customer asking for a refund?"
    },
    "frustration": {
      "type": "score",
      "instructions": "How frustrated is the customer?",
      "criteria": ["Calm", "Frustrated", "Very angry"]
    }
  }
}

Submit it with the CLI:

openkind evaluate request.json --server http://127.0.0.1:18080 --format text --verbose

The output shows each answer and its available probabilities. Omit --format text --verbose to get JSON for scripts.

Question	Returns	Example use
`choice`	A selected option and probability distribution	Route a ticket to a team
`noul`	A yes/no probability	Detect a refund request
`score`	A numeric score against an ordered rubric	Rate urgency or frustration

Native Qwen Choice questions require a non-empty __none__ option. Keeping it in your requests gives that model a way to reject the offered choices.

Try the API without downloading a model

From the checkout, start a mock server:

openkindd --models jev-latest --http-addr 127.0.0.1:18080 --grpc-addr 0

In another terminal:

openkind evaluate examples/04_mixed.json --server http://127.0.0.1:18080 --pretty

Stop any server already using that port first. In this setup, jev-latest returns deterministic mock answers for integration testing.

Choose a local model

For an Apple silicon Mac, start with Laya Typed Decisions (laya:typed-decisions) for mixed Choice, Noul, and Score requests. It balances memory, speed, and the two classification panels below. GLiClass (encoder-instruct-label) is the smaller, faster option for label classification.

Keep Laya Choice questions under about 20 options, and validate its confidence on your own data before using probability thresholds.

For CPU-only machines, try Kev 0.6B for short Choice requests. DistilBERT NLI (encoder-nli) uses much less memory, with lower accuracy on these panels. These are models to try on an 8-16 GB machine, not capacity guarantees: the measurements came from a 36 GiB Mac. Leave memory for the OS and other applications.

The first three rows are suggested starting points. The rest are grouped by measured MLX, CPU, and GGUF execution, sorted by catalog loader name, followed by auxiliary models. Model links lead to Hugging Face; short pull aliases appear where the catalog defines one. Use the model guide for full pinned pull names.

Numbers come from the local suite: Apple M4 Max, 14 cores, 36 GiB RAM, macOS arm64. Timing uses one warm repetition of the same 12-decision, four-request smoke fixture unless marked. Model loading is excluded. Peak RSS is measured process memory in decimal GB, not download size, dedicated VRAM, or a maximum-context memory estimate.

Accuracy is SST-2 / AG News, 50 eval rows each, using pinned zero-shot templates and the injected __none__ option. These small public-dataset samples establish limited task evidence, not general accuracy or release qualification. Small percentage differences are not stable rankings. Accuracy links open reports with confidence intervals; speed links open timing summaries.

Context means the current runtime's encoded input budget, including state, instructions, and options unless a state-only limit is shown. It is often smaller than the checkpoint's advertised window. Laya uses 512 or 1,024 tokens despite an 8,192-token backbone. Input tokens/s is harness-reported input throughput; OpenKind does not generate output tokens.

The measured backend is bold. CPU is available for every checkpoint row. MLX requires an Apple silicon source build.

CUDA* and ONNX* are optional candidates without comparable hardware measurements or completed backend qualification here. ONNX requires a separate pinned export and can use CPU, CUDA, or Linux ROCm providers. See the CUDA, ONNX, and ROCm guides. GGUF describes the weights, not GPU acceleration.

Model / alias (Hugging Face)	Execution support	Peak RSS (GB)	Runtime context (tokens)	Decisions/s	Input tokens/s	Accuracy: SST-2 / AG News
`laya:typed-decisions`	CPU, MLX, CUDA, ONNX	2.12	1,024	34.74	7,116	92% / 90%
GLiClass (`encoder-instruct-label`)	CPU, MLX, CUDA, ONNX	1.01	1,024	72.32	15,536	92% / 78%
`kev`	CPU, CUDA*	4.21	1,024 (state 384)	5.48	844	86% / 88%
`jevk5:4b`	CPU, MLX, CUDA*	14.75	512	0.55	149	98% / 86%
`laya:en`	CPU, MLX, CUDA, ONNX	2.12	512	33.42	6,845	82% / 86%
`laya:multilingual`	CPU, MLX, CUDA, ONNX	2.94	1,024	71.11	14,494	30% / 88%
`plumb:4b`	CPU, MLX, CUDA*	11.99	16,384	0.54 (777)	146	96% / 86%
`qwen35-state-first`	CPU, MLX, CUDA*	12.31	1,792	0.33	36	2% / 70%
`clef:flash`	CPU (BF16 weights, FP32 compute)	15.59	16,384	0.22	37	96% / 90%
`decider:4b`	CPU, CUDA*	8.05	32,768 state	0.18	36	96% / 90%
`decoder-logit-letter`	CPU, CUDA, ONNX	3.34	8,192	6.63	1,794	72% / 72%
`decoder-logit-qwen3-06b`	CPU, CUDA, ONNX	4.21	512	3.55 (777)	967	88% / 72%
`decoder-logit-qwen3-17b`	CPU, CUDA, ONNX	11.46	512	2.09	570	96% / 6%
`qwen3:4b`	CPU, CUDA, ONNX	22.88	512	0.85	230	98% / 86%
DistilBERT NLI (`encoder-nli`)	CPU, CUDA, ONNX	0.56	512	26.04	5,306	74% / 64%
`qwen3guard`	CPU, CUDA, ONNX	4.21	8,192	5.12	949	n/a (safety head)
`schema-scorer`	CPU, CUDA, ONNX	0.26	512	12.19	5,260	50% / 16%
`strands-decider-2b`	CPU, CUDA*	9.15	4,096	0.42	117	96% / 88%
`von:1.1`	CPU, CUDA, ONNX	1.81	8,192	2.43	534	94% / 66%
`winnow` (in-house LoRA router)	CPU, CUDA*	n/a	8,192	n/a (routing pass)	n/a	n/a (router)
`winnow:e4b`	CPU, CUDA*	8.42	8,192	0.10	27	94% / 80%
`clef:27b` (Q4_K_M + head)	CPU	pending	16,384	pending	pending	pending / pending
`clef:flash-gguf` (Q4_K_M + head)	CPU	pending	16,384	pending	pending	96% / pending
`decoder-logit-llm` (GGUF Q8_0)	CPU, CUDA*	1.56	8,192	0.44	120	72% / 66%
BGE-small (`encoder-embedding`)	CPU, MLX	pending	256	289.98 emb/s	n/a	n/a (embedder)
`router-script`	Rules only	n/a	n/a	n/a	n/a	n/a (router)

(777) marks the 777-decision shape fixture for Plumb and Qwen3 0.6B. Those speed and memory rows are not directly comparable to the 12-decision runs. Plumb accuracy was measured on CPU, while its timing uses MLX. The native state-first timing uses repeated_full; earlier scheduler-selected results measure a different execution strategy.
BGE is a proxy-cache embedding component, measured separately with 30 calls after five warmups. Its rate is embeddings/s. Winnow and router-script compose other engines; a routing-pass rate does not measure the complete decision path.
Pending cells have no completed, checked-in measurement. GGUF can reduce weight storage without improving these CPU kernels' speed; do not transfer Clef BF16 memory or accuracy to a quantized profile.
Laya Multilingual's 30% SST-2 result and the native state-first profile's 2% result make them poor defaults for that task. For wide-label classification, the suite's Banking77 and CLINC150 panels favor Clef Flash (98% / 94%, 50 rows each), at substantial CPU latency and memory cost. Test your actual labels before choosing a larger model.

The benchmark guide explains scoring, calibration metrics, and evidence limits. Public model-card and leaderboard scores use different harnesses and are not included in this table.

CLI and playground

Manage local models and inspect the running server:

openkind list
openkind show laya-english:c8ea29bf1e33a343c4b7
openkind status --server http://127.0.0.1:18080
openkind inspect request.json

Process JSONL requests as a resumable batch, with optional inclusive source row bounds:

openkind batch run requests.jsonl --job-dir ./job --server http://127.0.0.1:18080 --start-row 100 --end-row 500
openkind batch resume ./job

The batch guide covers stdin pipes, pacing, stop/status, the last successful row, and result export.

API keys

API key checking is optional and off by default. Generate a key with openkind keygen, then configure the server to require it. The key is printed once and is not saved automatically:

export OPENKIND_API_KEY="$(openkind keygen)"
openkind serve --http-addr 127.0.0.1:18080 --grpc-addr 0

You can also set the key explicitly with openkind serve --api-key <KEY>. Pass it to CLI commands that call a protected server, such as:

openkind status --server http://127.0.0.1:18080 --api-key "$OPENKIND_API_KEY"
openkind evaluate request.json --server http://127.0.0.1:18080 --api-key "$OPENKIND_API_KEY"

OPENKIND_API_KEY supplies the key to these commands automatically. See the CLI key guide for PowerShell examples, and language bindings for client libraries.

For a browser interface, start the playground with an installed model:

openkind playground --installed-models laya-english:c8ea29bf1e33a343c4b7

It opens a local page where you can edit requests, inspect probabilities, and load or unload installed models. Running openkind playground alone uses mock models. The daemon rejects --playground on unless its HTTP listener is bound to a loopback address, keeping model lifecycle controls local even when inference authentication is disabled or an API key is shared. See the CLI guide for commands and the registry guide for model storage and lifecycle.

Use from Rust

Use openkind-client to call the local server. For an application beside the cloned openkind directory, add these dependencies to Cargo.toml:

[dependencies]
openkind-client = "0.4.0"
tokio = { version = "1", features = ["macros", "rt-multi-thread"] }

With the Laya server above running, put this in src/main.rs and run cargo run.

For an authenticated daemon, replace .allow_unauthenticated() with .api_key("server-configured-key"), or remove it to read OPENKIND_API_KEY.

use std::time::Duration;

use openkind_client::{question, Client, RetryPolicy};

#[tokio::main]
async fn main() -> Result<(), openkind_client::Error> {
    let client = Client::builder()
        .allow_unauthenticated() // This local server has auth disabled.
        .base_url("http://127.0.0.1:18080")
        .default_model("laya-english:c8ea29bf1e33a343c4b7")
        .timeout(Duration::from_secs(600))
        .retry(RetryPolicy::new().total_timeout(Some(Duration::from_secs(600))))
        .build()?;

    let response = client
        .system_one(
            "My card was charged twice. Please refund the extra payment.",
            [(
                "refund_requested",
                question::noul("Is the customer asking for a refund?"),
            )],
        )
        .await?;

    println!("{:?}", response.answers);
    Ok(())
}

For inference inside your Rust process, use openkind-backends through the DecisionEngine interface. openkind-core provides the shared request types and validation. The HTTP client connects to a server; the backend crates load and execute models.

TypeScript, Python, and Swift HTTP clients and local server wrappers are also available. See the bindings guide.

API compatibility

openkindd serves Jev's typed Noul, Choice, and Score contract over HTTP and gRPC. The HTTP API includes POST /v1/systemone and GET /v1/models. The same request works with curl:

curl -sS http://127.0.0.1:18080/v1/systemone \
  -H 'Content-Type: application/json' \
  --data-binary @request.json

Existing TypeSafe clients can target the local server with a supported request and a registered model name. The Rust client also supports TypeSafe, OpenRouter System One, and Cloudflare Workers AI. Accepted fields and provider routes differ; the compatibility matrix records those limits.

API compatibility does not imply the same predictions as TypeSafe's Jev. OpenKind implements its own models and inference paths. Your application still decides which actions an answer may trigger.

Apple silicon and MLX

Enable the optional MLX/Metal backend on macOS arm64 to run supported models on the GPU:

export SDKROOT=$(xcrun --show-sdk-path)
cargo build --release --locked -p openkind-cli -p openkind-server --features openkind-server/mlx
openkindd \
  --installed-models laya-english:c8ea29bf1e33a343c4b7 \
  --laya-backend mlx-fp32 \
  --http-addr 127.0.0.1:18080 \
  --grpc-addr 0

Stop the CPU server before starting this one on the same port. FP32 MLX paths are available for the native Qwen3.5 profile, the three Laya profiles, GLiClass (encoder-instruct-label), and JevK5 (decoder-logit-qwen35). Other profiles use the CPU backend.

The MLX guide covers build requirements and parity checks. The model guide lists measured performance and memory use. BF16 Qwen execution remains experimental and has not passed the FP32 probability tolerance.

AMD GPUs (ROCm)

On Linux, the eight ONNX-capable surveyed families can execute on AMD GPUs through the ONNX Runtime ROCm execution provider:

cargo build --release --locked -p openkind-server --features openkind-server/onnx-rocm
openkindd --encoder-nli-backend onnx-rocm --onnx-runtime /path/to/libonnxruntime.so --http-addr 127.0.0.1:18080 --grpc-addr 0

This requires a ROCm-enabled ONNX Runtime build placed by the operator; loads fail closed when the provider is unavailable, and there is no fallback to CPU. These paths are unpromoted candidates — no parity gates have run on a ROCm host yet. The ROCm guide covers build flags, detection, and evidence gates.

Models and research

The catalog includes Laya decision encoders, NLI and GLiClass classifiers, Qwen-based decision decoders, and the native qwen35-state-first research profile. Use openkind catalog for pull names and the model guide for each profile's purpose and limits.

OpenKind's custom model work explores learned decision readouts and training recipes over open Qwen backbones. Its native Rust engine can process shared state once, then branch question and candidate work while isolating attention and recurrent state. The research tests calibration, precision, cache reuse, and whether those changes preserve decisions.

The new mixed-task Qwen3.5 decision LoRA trainer has been checked with tiny models; full 4B training and Mac qualification are still unrun. Fixture parity and throughput measurements establish specific implementation behavior. They do not establish general task accuracy.

Read more	What it covers
Architecture	Rust crates, inference paths, and shared-state execution
Research dossier	Evidence, related systems, and reproduction limits
Whitepaper	Methods and measured results
Research notebooks	Experiments, training recipe, and supporting artifacts
Benchmarks	Measurement methods and recorded runs

Development

Build from source

To build from source, install Rust 1.90 or newer and the Protocol Buffers compiler (protoc):

git clone https://github.com/whit3rabbit/openkind.git
cd openkind
cargo build --release --locked -p openkind-cli -p openkind-server
export PATH="$PWD/target/release:$PATH"
openkind version

Apple silicon users can enable the optional GPU-accelerated backend with --features openkind-server/mlx (see Apple silicon and MLX).

Verification

Start with the architecture guide and family integration guide. The repository's agent guide lists the full verification battery:

cargo fmt --check
cargo clippy --workspace --all-targets -- -D warnings
env -u RUST_LOG cargo test --workspace

Report bugs or propose changes through GitHub issues.

License

See the MIT license. Each model retains its own license, recorded in the model catalog and linked manifests.

Maintainers: see the release checklist and changelog.

Name		Name	Last commit message	Last commit date
.claude/docs		.claude/docs
.github		.github
assets		assets
benchmarks		benchmarks
bindings		bindings
crates		crates
docs		docs
examples		examples
packaging		packaging
proto		proto
registry/v1		registry/v1
research		research
scripts		scripts
.gitattributes		.gitattributes
.gitignore		.gitignore
AGENTS.md		AGENTS.md
CHANGELOG.md		CHANGELOG.md
Cargo.lock		Cargo.lock
Cargo.toml		Cargo.toml
LICENSE		LICENSE
Package.swift		Package.swift
README.md		README.md

Navigation Menu

Repository navigation

About

Repository files navigation

Install

Homebrew (macOS and Linux)

Cargo

Releases

Run a model

Choose a local model

CLI and playground

API keys

Use from Rust

API compatibility

Apple silicon and MLX

AMD GPUs (ROCm)

Models and research

Development

Build from source

Verification

License

About

Releases

Packages

Used by

Contributors

Languages

Latest commit

History

Folders and files

Repository files navigation

Install

Homebrew (macOS and Linux)

Cargo

Releases

Run a model

Choose a local model

CLI and playground

API keys

Use from Rust

API compatibility

Apple silicon and MLX

AMD GPUs (ROCm)

Models and research

Development

Build from source

Verification

License

About

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Used by

Contributors

Languages