Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

Thanks for contributing! This repo powers [create-awesome-python-app](https://github.com/Create-Python-App/create-python-app).

For a full explanation of how templates, extensions, and the file system work, read [docs/AUTHORING.md](./docs/AUTHORING.md).
For a full explanation of how templates, extensions, and the file system work, read [docs/AUTHORING.md](./docs/AUTHORING.md). A Spanish translation is also available: [docs/AUTHORING.es.md](./docs/AUTHORING.es.md) (English remains canonical).

## Adding an extension

Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ CI=true uvx create-awesome-python-app my-api \
|---|---|
| [docs/ARCHITECTURE.md](./docs/ARCHITECTURE.md) | System overview, type system, generation flow |
| [docs/AUTHORING.md](./docs/AUTHORING.md) | Directory layout, `cpa.config.json`, extensions |
| [docs/AUTHORING.es.md](./docs/AUTHORING.es.md) | Spanish translation of AUTHORING.md |
| [docs/TESTING.md](./docs/TESTING.md) | Local testing and CI workflow |
| [CONTRIBUTING.md](./CONTRIBUTING.md) | How to add templates and extensions |
| [docs/MAINTENANCE_RUNBOOK.md](./docs/MAINTENANCE_RUNBOOK.md) | Operating runbook: decision trees, checklists, and procedures for maintaining the CLI and templates |
Expand Down
143 changes: 143 additions & 0 deletions docs/AI_ML_AUTHORING.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,149 @@ documents the rule to apply once the first one lands (tracked in
- Ship tests for generated paths the extension adds (or document mount steps + unit tests).
- Do **not** embed `.github/workflows` in FastAPI AI extensions — compose `github-setup` or `all-mlops-github-actions`.

## AI span primitive contract (#112)

Describes the standard span kinds and attribute schema AI extensions must use
when emitting LLM/tool/retrieval/guardrail spans via the primitives from
`fastapi-mlflow-tracing` (#81). The helper API (`maybe_start_span`,
`set_attribute`) is owned by #81; this section defines **what** to emit and
**how**, not the helper signatures.

### Related issues

| Issue | Role |
|-------|------|
| [#81](https://github.com/Create-Python-App/cpa-templates/issues/81) | Primitive API owner (`maybe_start_span`, `set_attribute`) |
| [#77](https://github.com/Create-Python-App/cpa-templates/issues/77) | `fastapi-ai-chat` — first consumer |
| [#78](https://github.com/Create-Python-App/cpa-templates/issues/78) | `fastapi-rag-pgvector` — retrieval consumer |
| [#79](https://github.com/Create-Python-App/cpa-templates/issues/79) | `fastapi-langgraph-chat` — agent/tool consumer |
| [#80](https://github.com/Create-Python-App/cpa-templates/issues/80) | `fastapi-mcp-client` — tool_call consumer |
| [#82](https://github.com/Create-Python-App/cpa-templates/issues/82) | `fastapi-ai-guardrails` — guardrail_check consumer |
| [#91](https://github.com/Create-Python-App/cpa-templates/issues/91) | `incompatibleWith` matrix (combination policy) |
| [#112](https://github.com/Create-Python-App/cpa-templates/issues/112) | This contract |

### Span kinds

Every AI extension MUST use one of these four span kinds — never invent new
ones. The span name should be a human-readable identifier (e.g.
`"chat-completion"` or `"vector-search"`).

| Kind | Meaning | Owner issue |
|------|---------|-------------|
| `llm_inference` | A single model completion (chat, embeddings, completion). | #77, #79 |
| `tool_call` | An MCP/agent tool invocation. | #80, #79 |
| `retrieval` | RAG fetch from a vector or knowledge store. | #78 |
| `guardrail_check` | Input/output guardrail evaluation. | #82 |

### Required attributes

#### `llm_inference`

| Attribute | Type | Semantics |
|-----------|------|-----------|
| `llm.provider` | `str` | `"openai"` \| `"anthropic"` \| `"ollama"` \| ... |
| `llm.model` | `str` | Exact model id, e.g. `"gpt-4o-mini"` |
| `llm.input_tokens` | `int` | Token count in |
| `llm.output_tokens` | `int` | Token count out |
| `llm.latency_ms` | `float` | Wall-clock from request to last chunk |
| `llm.error` | `str \| None` | Exception type if failed, `None` on success |
| `llm.stream` | `bool` | `True` if streaming response |
| `llm.temperature` | `float` | _(optional)_ sampling temperature |
| `llm.tool_name` | `str \| None` | _(optional)_ set when the LLM call resolved to a tool |

#### `tool_call`

| Attribute | Type | Semantics |
|-----------|------|-----------|
| `tool.name` | `str` | Tool / function name |
| `tool.input` | `str \| None` | Serialized input, behind `LLM_TRACE_PAYLOAD` opt-in only |
| `tool.output` | `str \| None` | Serialized output, behind `LLM_TRACE_PAYLOAD` opt-in only |
| `tool.error` | `str \| None` | Exception type if failed, `None` on success |

#### `retrieval`

| Attribute | Type | Semantics |
|-----------|------|-----------|
| `retrieval.query` | `str \| None` | Query text, behind `LLM_TRACE_PAYLOAD` opt-in only |
| `retrieval.top_k` | `int` | Number of results requested |
| `retrieval.results_count` | `int` | Number of results returned |
| `retrieval.index` | `str` | Index / store identifier |
| `retrieval.error` | `str \| None` | Exception type if failed, `None` on success |

#### `guardrail_check`

| Attribute | Type | Semantics |
|-----------|------|-----------|
| `guardrail.name` | `str` | Guardrail identifier |
| `guardrail.blocked` | `bool` | `True` if the check rejected the input/output |
| `guardrail.reason` | `str \| None` | Short reason when `blocked=True` |

### Span shape

1. Each kind opens with `maybe_start_span(kind, name="...")` from the
`fastapi-mlflow-tracing` extension and closes with `.end()`. Latency is
recorded automatically by the span context manager (#81).
2. **Guardrail rejections** set `llm.error = "guardrail_blocked"` and
`guardrail.reason` on the **same** `llm_inference` span — not a separate
span. The span tree stays linear, and the parent `llm_inference` span
records the error.

Example (illustrative — the helper API is owned by #81):

```python
from app.core.mlflow_tracing import maybe_start_span

def chat(messages: list[dict]) -> str:
with maybe_start_span(
"llm_inference",
name="chat-completion",
**{
"llm.provider": "openai",
"llm.model": "gpt-4o-mini",
"llm.stream": True,
}
) as span:
try:
response = call_openai(messages)
span.set_attribute("llm.input_tokens", response.usage.prompt_tokens)
span.set_attribute("llm.output_tokens", response.usage.completion_tokens)
span.set_attribute("llm.latency_ms", response.latency_ms)
span.set_attribute("llm.error", None)
return response
except Exception as exc:
span.set_attribute("llm.error", type(exc).__name__)
raise
```

### Privacy

- **Default is no payload logging.** Raw prompts, completions, tool inputs,
and tool outputs are never recorded unless an explicit opt-in env var is set.
- `LLM_TRACE_PAYLOAD=true` enables recording of `llm.input_text`,
`llm.output_text`, `tool.input`, `tool.output`, and `retrieval.query`.
This is **off in CI** by default and is documented in
`docs/MLFLOW_TRACING_GUIDE.md`.
- PII redaction surface stays in `fastapi-ai-guardrails` (#82), not in the
tracing layers or this contract.

### Acceptance criteria

- [x] The 4 span kinds above are documented with their required attributes.
- [x] The privacy rule (`LLM_TRACE_PAYLOAD=false` default, no raw
prompt/completion logging) is documented.
- [x] The contract links #81 (primitives owner), #77, #78, #79, #80, #82
(consumers).
- [ ] First consumer (#77 or #82) implements against this contract, not an
ad-hoc schema.

## Extension constraints

- Use `template/` so bank `README.md` does not overwrite the project README.
- Ship `template/docs/<TOPIC>_GUIDE.md` and `template/docs/README.md.append`.
- Partial `pyproject.toml` overlays for dependencies.
- Ship tests for generated paths the extension adds (or document mount steps + unit tests).
- Do **not** embed `.github/workflows` in FastAPI AI extensions — compose `github-setup` or `all-mlops-github-actions`.

## Related docs

- [AUTHORING.md](./AUTHORING.md)
Expand Down
28 changes: 28 additions & 0 deletions docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,34 @@ cpa-templates/
└── docs/ # Authoring and testing guides
```

## Architecture Diagram

```mermaid
graph TD
UVX[UVX CLI] --> TEMPLATE[Template Registry (templates.json)]
TEMPLATE --> EXT_FASTAPI[FastAPI Starter]
TEMPLATE --> EXT_FAIACHAT[FastAPI AI Chat Extension]
EXT_FASTAPI --> GENERATION[Generation Flow]
GENERATION --> SYNC[Sync & Install]
SYNC --> GIT[Initialize Git Repo]

subgraph "Layer Order"
BASE[Base Template]
ADDON[Extension Layer]
OUTPUT[Final Project]
end
BASE --> ADDON
ADDON --> OUTPUT

style UVX fill:#f9f,stroke:#333,stroke-width:2px
style TEMPLATE fill:#e1f5fe,stroke:#333,stroke-width:2px
style EXT_FASTAPI fill:#e8f5e9,stroke:#333,stroke-width:2px
style EXT_FAIACHAT fill:#e8f5e9,stroke:#333,stroke-width:2px
style GENERATION fill:#fff3e0,stroke:#333,stroke-width:2px
style SYNC fill:#ffe0b2,stroke:#333,stroke-width:2px
style GIT fill:#f3e5f5,stroke:#333,stroke-width:2px
```

## Related repositories

- **create-python-app** — CLI monorepo (`create-awesome-python-app`, `create-python-app-core`)
Expand Down
2 changes: 2 additions & 0 deletions docs/AUTHORING.es.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

Guía para colaboradores que quieran añadir o actualizar plantillas y extensiones en `cpa-templates`. Esta guía sigue la misma estructura que [cna-templates AUTHORING.md](https://github.com/Create-Node-App/cna-templates/blob/main/docs/AUTHORING.md) y se mantiene en paridad con ella.

> **Nota sobre CI:** El repositorio define cuatro niveles de integración continua (CI). CI Integrity (L0) valida la integridad del registro y perfiles curvados; CI Templates (L1) prueba cada plantilla individualmente con `uvx`; CI Extensions (L2) prueba cada extensión con una plantilla canónica; y CI Profiles (L3) ejecuta pilas completas curadas en `ci/profiles/`. Consulta [docs/TESTING.md](./TESTING.md) para más detalles.

## Estructura del directorio de plantillas

```text
Expand Down
43 changes: 43 additions & 0 deletions extensions/flower-docker/.dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# Flower Monitoring Extension

## Adding Flower monitoring to a Celery worker

Flower is a real-time monitoring dashboard for Celery. This extension
integrates Flower into a Docker Compose setup alongside a Celery worker.

## Files added

- `compose.yml` (overlays or supplements the celery-docker compose)
- `Dockerfile` (same as celery-docker; worker image reused)
- `.env.example` (Flower-specific environment variables)
- `docs/FLOWER_GUIDE.md` (usage guide)

## Usage

```sh
uvx create-awesome-python-app my-worker \
--template celery-worker \
--addons celery-docker flower-docker \
--yes
```

Then:

```sh
cp .env.example .env
# Edit .env with your broker URL
docker compose up --build
# Dashboard at http://localhost:5555
```

## Configuration

All configuration is via environment variables loaded from `.env`.
No secrets are hardcoded in any source file.

## Security

- Add `.env` to `.gitignore`
- Flower can expose task arguments — use `FLOWER_BASIC_PASSWORD` in
production-like environments
- Consider a reverse proxy with TLS for non-local deployments
53 changes: 53 additions & 0 deletions extensions/flower-docker/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# Flower Monitoring Extension for Celery

Adds Flower monitoring dashboard for Celery workers.

## Features

- **Flower Integration**: Provides a Flask-based monitoring dashboard for Celery workers
- **Real-time Metrics**: Live stats on task queues, worker performance, and task execution
- **Docker Compose Support**: Includes a dedicated `flower` service in the compose configuration
- **Environment Variables**: Secrets (Redis URLs, Flower port) are loaded from `.env` files only

## Installation

```sh
uvx create-awesome-python-app my-worker \
--template celery-worker \
--addons flower-docker \
--yes
```

## Configuration

The extension requires:
- `FLOWER_PORT` (default: 5555) - Port for the Flower dashboard
- `FLOWER_HOST` (default: "0.0.0.0") - Host to bind the Flower server
- `REDIS_URL` - Redis connection string (must match celery worker config)

## Usage

1. Start the worker with the flower-docker addon:
```sh
uvx create-awesome-python-app my-worker \
--template celery-worker \
--addons flower-docker \
--yes
```

2. Access the Flower dashboard at `http://localhost:5555`

3. Configure `FLOWER_PORT` and `FLOWER_HOST` in your `.env` file.

## Requirements

- Python 3.12+
- Celery (>=5.0)
- Flower (>=2.0)
- Redis (for broker)

## Security

- All secrets (Redis URLs, Flower port) are loaded from environment variables only
- No hardcoded credentials in source code
- Follows the same security patterns as the celery-docker extension
12 changes: 12 additions & 0 deletions extensions/flower-docker/template/.dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
.venv
__pycache__/
*.pyc
.git
.gitignore
*.egg-info
dist
build
.mypy_cache
.pytest_cache
.ruff_cache
.terraform
4 changes: 4 additions & 0 deletions extensions/flower-docker/template/.env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
FLOWER_PORT=5555
FLOWER_HOST=0.0.0.0
# REDIS_URL is typically set by the celery-docker extension
# Add FLOWER_BASIC_PASSWORD if you want HTTP basic auth
15 changes: 15 additions & 0 deletions extensions/flower-docker/template/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
FROM ghcr.io/astral-sh/uv:python3.12-bookworm-slim

WORKDIR /app

ENV UV_COMPILE_BYTECODE=1
ENV UV_LINK_MODE=copy
ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1

COPY pyproject.toml README.md ./
COPY worker ./worker

RUN uv sync --no-dev

CMD ["uv", "run", "celery", "-A", "worker.celery_app", "worker", "--loglevel=INFO"]
55 changes: 55 additions & 0 deletions extensions/flower-docker/template/README.md.append
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# Flower Monitoring Extension

Adds Flower monitoring dashboard for Celery workers via Docker Compose.

## What's included

| Path | Purpose |
|------|---------|
| `Dockerfile` | uv-based image; Celery worker CMD |
| `compose.yml` | Compose with worker + flower services |
| `.env.example` | Example environment variables for Flower |
| `docs/README.md.append` | Index bullet for docs |
| `docs/FLOWER_GUIDE.md` | Long-form guide for Flower monitoring |

## Docker Compose overview

The `compose.yml` adds a Flower service that:
- Runs the official `flower` Docker image
- Exposes the dashboard on port 5555
- Connects to the Redis broker used by the Celery worker
- Reads configuration from environment variables (never hardcoded)

### Environment variables

| Variable | Default | Description |
|----------|---------|-------------|
| `FLOWER_PORT` | `5555` | Port for the Flower dashboard |
| `FLOWER_HOST` | `0.0.0.0` | Host to bind the Flower server |
| `REDIS_URL` | `redis://redis:6379/0` | Redis connection for broker |
| `BASIC_PASSWORD` | — | Optional HTTP basic auth password |

All secrets are loaded from `.env` files at runtime. No credentials are
stored in the repository.

## Apply

```sh
uvx create-awesome-python-app my-worker \
--template celery-worker \
--addons flower-docker \
--yes
```

## Verify

```sh
docker compose up --build
# Flower dashboard: http://localhost:5555
```

## Compatibility

- Compatible with the `celery-worker` template (L2)
- Requires `celery-docker` extension for Docker Compose support
- Works alongside Redis-based brokers
Loading