Guide to Running Local LLMs

A plain local LLM usually only does this:

input text -> model -> output text

It does not automatically know how to edit files, run commands, search your project, call APIs, use your terminal, or browse documentation.

Those abilities come from the application around the model, for example:

VS Code Copilot / Continue / Claude Code / Codex / Cline

These apps provide the "skills" or tools, such as:

• read files
• edit files
• search workspace
• run terminal commands
• run tests
• inspect errors
• call external APIs

The model decides what to do, but the app actually provides and controls the tools.

So the layers are:

Model:
Phi-4, Llama, Qwen, Mistral, Claude, GPT, etc.

Runtime/backend:
Ollama, Foundry Local, LM Studio, llama.cpp, vLLM, etc.

Agent/app/tool layer:
VS Code Copilot agent mode, Claude Code, Codex, Continue, Cline, Aider, etc.

There are many ways to run and interact with an LLM locally, for example through VS Code with eg the codex og claude extension, the Codex desktop app, or the Claude Code desktop app.

For desktop applications, you can often change the model by switching the backend. One common backend is Ollama(optimized for CPU use), a framework for running models locally.

For example, you can use Ollama to run a Llama model and connect it to applications that support local model backends:

https://ollama.com/library/llama3.1

You need to install Ollama first before using it as a backend.

LLM Skills And Instructions

Skills are Markdown-based reusable instruction packages for agent workflows. They sit above the model and inside the agent/app/tool layer: the model still reasons, but the skill gives it task-specific steps, references, and sometimes helper scripts.

Plain .md files are useful only when an agent knows to read them. Different coding agents discover skills and instructions in different ways.

Codex Skills

Codex supports both global skills and repo-specific skills.

Use global skills for personal workflows you want available across projects:

~/.codex/skills/<skill-name>/SKILL.md

On Windows, that is usually:

C:\Users\<you>\.codex\skills\<skill-name>\SKILL.md

Use repo-specific skills for workflows that should travel with a project:

<repo>/.agents/skills/<skill-name>/SKILL.md

A Codex skill is a folder, not just a loose Markdown file. The required file is SKILL.md, and it should start with metadata:

---
name: research-summarizer
description: Summarize research from multiple sources with citations, uncertainty notes, and clear conclusions. Use when the user asks for literature summaries, evidence reviews, source comparisons, paper digests, or research briefings.
---

This repository uses repo-specific Codex skills:

local_llm/
|-- README.md
|-- AGENTS.md
`-- .agents/
    `-- skills/
        |-- readme/
        |   `-- SKILL.md
        |-- contributing/
        |   `-- SKILL.md
        |-- architecture/
        |   `-- SKILL.md
        |-- development/
        |   `-- SKILL.md
        |-- coding-standards/
        |   `-- SKILL.md
        |-- ai-instructions/
        |   `-- SKILL.md
        |-- security/
        |   `-- SKILL.md
        |-- testing/
        |   `-- SKILL.md
        |-- deployment/
        |   `-- SKILL.md
        |-- changelog/
        |   `-- SKILL.md
        |-- api/
        |   `-- SKILL.md
        `-- troubleshooting/
            `-- SKILL.md

Each skill explains one important project documentation file, how it is used, and gives an example of what that file can contain. README.md remains the human-facing overview, and AGENTS.md remains broad repo guidance for coding agents.

Codex can select a skill automatically from the prompt, or you can invoke one explicitly. In Codex, explicit skill invocation is usually done with $skill-name, such as $research-summarizer. In the Codex CLI, you can also use /skills to pick a skill from a menu.

Claude Code Skills

Claude Code also uses skill folders with a SKILL.md file.

Common locations are:

~/.claude/skills/<skill-name>/SKILL.md

or project-local:

<repo>/.claude/skills/<skill-name>/SKILL.md

The same basic pattern applies: one folder per skill, with SKILL.md as the entrypoint and optional supporting files beside it.

Claude Code skills can be selected automatically, and they can also be invoked manually with slash commands such as /research-summarizer. The command name usually comes from the skill folder location, not only from the frontmatter name.

AGENTS.md

Use AGENTS.md, plural, for general repository guidance for coding agents. AGENT.md, singular, is not the common convention.

AGENTS.md is useful for rules that apply to all work in the repo, such as setup commands, test commands, code style, architecture notes, and project gotchas. It is not the same thing as a Codex or Claude skill.

Common support:

Tool	Uses AGENTS.md?	Notes
Codex	Yes	Codex reads global and project AGENTS.md files before doing work.
Claude Code	Not as the main convention	Claude Code primarily uses CLAUDE.md for repo instructions and SKILL.md for skills.
Other coding agents	Sometimes	AGENTS.md is an emerging shared convention, but each tool must explicitly support it.

Quick Rule

Codex skill = folder + SKILL.md in ~/.codex/skills or .agents/skills
Codex manual invocation = $skill-name, or /skills in the CLI
Claude Code skill = folder + SKILL.md in ~/.claude/skills or .claude/skills
Claude Code manual invocation = /skill-name
AGENTS.md = broad repo guidance for coding agents
Plain .md file = only useful if the agent is told to read it; it is not a skill by itself