Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions ci/profiles/mlops-sklearn-default.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
{
"id": "mlops-sklearn-default",
"description": "MLOps sklearn starter with GitHub CI",
"templateDir": "mlops-sklearn-starter",
"addons": [
"github-setup"
]
}
68 changes: 68 additions & 0 deletions docs/AI_ML_AUTHORING.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,12 +69,15 @@ Declare conflicts in `templates.json` before merging conflicting pairs.
| Extension A | Extension B | Reason / resolution |
|-------------|-------------|---------------------|
| `fastapi-ai-chat` | `fastapi-langgraph-chat` | Both may own `/chat` — either set `incompatibleWith` or document non-overlapping routes before shipping LangGraph |
| `fastapi-mlflow-tracing` | `fastapi-opentelemetry` | Both emit `llm_inference` spans on same FastAPI request — declare `incompatibleWith` to avoid double-instrumentation |
| Competing `all-mlops-*-data` packs that overwrite the same data paths | each other | Prefer one modality pack per profile |

Neither `fastapi-ai-chat` nor `fastapi-langgraph-chat` exists yet — this row
documents the rule to apply once the first one lands (tracked in
[#77](https://github.com/Create-Python-App/cpa-templates/issues/77)).

Neither `fastapi-mlflow-tracing` nor `fastapi-opentelemetry` exist as stable releases yet — but this matrix entry documents the rule for when they ship.

## Extension constraints

- Use `template/` so bank `README.md` does not overwrite the project README.
Expand All @@ -89,3 +92,68 @@ documents the rule to apply once the first one lands (tracked in
- [TEMPLATE_QUALITY_M1.md](./TEMPLATE_QUALITY_M1.md)
- [TESTING.md](./TESTING.md)
- [MLOPS_CONTRACT.md](./MLOPS_CONTRACT.md)

## AI span primitive contract

This section defines the contract that all FastAPI AI extensions must follow when emitting MLflow/observability spans. The contract was landed in #71 and is consumed by #81 (primitives) and #77/#78/#79/#80/#82 (consumers).

### Span kinds

Extensions must use exactly one of the following span kinds when recording AI activity:

| Kind | Description |
|------|-------------|
| `llm_inference` | A single LLM model completion (chat, embeddings, text completion). |
| `tool_call` | An MCP/agent tool invocation (extension #80). |
| `retrieval` | A RAG fetch operation (extension #78). |
| `guardrail_check` | An input/output guardrail evaluation (extension #82). |

### `llm_inference` required attributes

Each `llm_inference` span must include the following attributes. These are recorded automatically by the primitives in #81 when using `record_mlflow_span("llm_inference", {...})`.

| Attribute | Type | Semantics |
|-----------|------|-----------|
| `llm.provider` | `str` | Provider name: `"openai"` \| `"anthropic"` \| `"ollama"` \| custom provider id |
| `llm.model` | `str` | Exact model id, e.g. `"gpt-4o-mini"` |
| `llm.input_tokens` | `int` | Token count in the request |
| `llm.output_tokens` | `int` | Token count in the response |
| `llm.latency_ms` | `float` | Wall-clock time from request to last chunk (or end of non-streaming) |
| `llm.error` | `str \| None` | Exception type if failed, `None` on success |
| `llm.stream` | `bool` | `true` if streaming response |
| `llm.temperature` | `float` | Optional — if set, recorded as-is |
| `llm.tool_name` | `str \| None` | Optional — set when the LLM call resolved to a tool execution |

### Span shape

- Each span opens with `start_span(kind, name="__main__")` from the base helper (primitives in #81) and closes with `.end()`, recording latency automatically.
- Guardrail rejections set `llm.error = "guardrail_blocked"` and `guardrail.reason` on the **same** `llm_inference` span, not a separate one — the span tree stays linear.
- When a tool is involved, `tool_call` spans wrap the `llm_inference` span, keeping the tree: `llm_inference` → `tool_call`.

### Privacy

- **Never** log `llm.input_text` / `llm.output_text` / raw messages by default.
- Add an explicit `LLM_TRACE_PAYLOAD=true` env opt-in (off in CI, documented in `MLFLOW_TRACING_GUIDE.md`).
- PII redaction surface stays in `fastapi-ai-guardrails` (#82), not here.

### Consumer links

- **#81** owns the primitive API surface (`record_mlflow_span`, `start_span` helpers).
- **#77** (`fastapi-ai-chat`) must call `record_mlflow_span("llm_inference", {...})` instead of ad-hoc schemas.
- **#78** (`fastapi-rag-pgvector`) must use `retrieval` kind when emitting RAG fetch spans.
- **#79** (`fastapi-langgraph-chat`) must emit `tool_call` spans for agent tool invocations.
- **#80** (`fastapi-mcp-client`) must emit `tool_call` spans for MCP client calls.
- **#82** (`fastapi-ai-guardrails`) must set `llm.error = "guardrail_blocked"` on the same `llm_inference` span when a guardrail rejects the input.

### Example (illustrative — owned by #81)

```python
from mlflow.tracking import MlflowClient

def record_mlflow_span(kind: str, attributes: dict):
"""Helper in #81 — marks span boundaries and records latency."""
# ...implementation owned by #81...
pass
```

Once a consumer (e.g. #77) implements against this contract rather than an ad-hoc schema, the acceptance criteria for this section are met.
18 changes: 18 additions & 0 deletions docs/AUTHORING.md
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,24 @@ generated paths (for example two Docker overlays that both ship `Dockerfile` /
are isolated by `type`; when a type gains a second packaging strategy, declare
mutual incompatibility like cna-templates does for Redux saga/thunk.

**Authoring rules:**

1. **Declare on both sides.** If extension `A` is incompatible with `B`, then `A` must list `B` in its `incompatibleWith` array **and** `B` must list `A` in its `incompatibleWith` array. CPA validates this symmetry at registry load time.
2. **Use slugs, not names.** Reference entries by their `slug` string — not the human-readable `name` — so renames to display names don't silently break validation.
3. **Scope to the narrowest conflict surface.** Only declare incompatibility when the overlay truly overwrites shared paths (e.g. `Dockerfile`, `compose.yml`, `app/core/providers.py`). For softer constraints — version ranges, optional features, shared optional deps — prefer dependency versioning or optional `cpa.config.json` toggles rather than hard incompatibility.
4. **Same `type` first.** Most `incompatibleWith` declarations are within a single template `type` (e.g. two FastAPI Docker strategies). Cross-type incompatibility is rare and should be explicitly justified in the PR description.
5. **Document the rationale.** Record the colliding paths in the PR description, this document (`AUTHORING.md`), or `AI_ML_AUTHORING.md` so future maintainers know whether the constraint can be relaxed. (`templates.json` is strict JSON and does not support inline comments.)

**Checklist for new `incompatibleWith` entries:**

- [ ] Both entries list each other by `slug`
- [ ] Slugs referenced are valid entries in `templates.json`
- [ ] The collision path(s) are documented in the PR, `AUTHORING.md`, or `AI_ML_AUTHORING.md`
- [ ] An existing `incompatibleWith` wasn't already covering the pair
- [ ] If a new packaging strategy was introduced, it was discussed in the issue or Discord first

See [Registering in `templates.json`](#registering-in-templatesjson) for the JSON schema and the `templates.schema.json` validation.

### Template quality bar (every catalog template)

Every template registered in `templates.json` must ship at least:
Expand Down
87 changes: 87 additions & 0 deletions docs/CONTRIBUTING.es.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
# Contribución al proyecto

## Guía de contribución

Este documento describe cómo contribuir al proyecto Create-Python-App.

### Estructura del proyecto

El proyecto está organizado en las siguientes secciones:

- `cpa-templates/`: Plantillas para generar aplicaciones
- `fastapi-starter/`: Plantilla base para aplicaciones FastAPI
- `mlops-sklearn-starter/`: Plantilla base para MLOps con soporte de scikit-learn
- `cli-starter/`: Plantilla para el CLI
- `uv-workspace-starter/`: Plantilla para configuración de entorno

### Requisitos previos

- Python 3.10+
- Node.js 18+ (para compilar extensiones)
- `uv` instalado (versión 0.4.0 o superior)
- `curl` y `git` instalados

### Cómo crear una nueva aplicación

Para crear una nueva aplicación, utiliza el comando:

```bash
uvx create-awesome-python-app mi-app \
--template fastapi-starter \
--addons github-setup fastapi-docker \
--yes
```

Esto generará un proyecto con la plantilla FastAPI y las extensiones recomendadas.

### Contribuciones

#### Tipos de contribuciones

1. **Corrección de bugs** - Arreglar errores en el código existente
2. **Nueva función** - Añadir nuevas características solicitadas por los usuarios
3. **Mejora de documentación** - Actualizar guías, READMEs y archivos de configuración
4. **Extensión** - Crear una nueva extensión para el sistema de plantillas

#### Flujo de trabajo

1. Crea una rama nueva desde `main`:
```bash
git checkout -b mi-nueva-contribucion
```

2. Haz tus cambios y asegúrate de que el código compile correctamente:
```bash
uv sync
uv run ruff check .
```

3. Sube tu cambio al repositorio:
```bash
git push origin mi-nueva-contribucion
```

4. Abre una solicitud de pull request (PR) contra `main`.

### Reglas de estilo

- Usa **Jinja2** para variables en los archivos `.template`
- Los archivos sin sufijo se copian tal cual
- Las extensiones deben usar el patrón `.append` o `.append.template`
- El archivo `pyproject.toml` debe declarar todas las dependencias necesarias

### Calidad del código

- **Tipado**: Documenta el tipado de Python en `docs/TYPING.md`
- **Tests**: Cada nuevo cambio debe incluir pruebas unitarias
- **Linting**: Ejecuta `ruff` y `ruff format` antes de hacer commit

### Recursos

- [Guía completa de plantillas](docs/ARCHITECTURE.md)
- [Calidad del código](docs/QUALITY.md)
- [Lista de tareas](docs/TASKS.md)

### Contacto

Para preguntas sobre contribución, contacta a los mantenedores del proyecto.
59 changes: 59 additions & 0 deletions docs/incompatible-fix.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
# IncompatibleWith Rule Fix

## Original Rule 5 Issue

The current rule 5 for `incompatibleWith` documentation is infeasible because:
- JSON files (`templates.json`) do not support inline comments
- The rule instructs to "add a short comment in `templates.json` next to the `incompatibleWith` entry"

## Revised Rule 5

**Document the rationale.** Use alternative documentation methods since JSON cannot contain inline comments:

### Valid Options:

1. **PR Description**: Document collision paths and rationale in the pull request description where the `incompatibleWith` entries are added.

2. **Extension README**: Include the collision rationale in the extension's `README.md` file.

3. **AI_ML_AUTHORING.md**: For AI/ML extensions, add the rationale to the `incompatibleWith` matrix section.

4. **Issue Reference**: Document in the original issue that triggered this incompatibility.

5. **CHANGELOG/NOTES**: Add documentation to a changelog or notes file in the repository.

### Preferred Order:

1. **Primary**: PR description (most discoverable and immediate)
2. **Secondary**: Extension README (useful for users reading about extensions)
3. **Tertiary**: AI_ML_AUTHORING.md (for AI/ML catalog entries)

## Example Documentation

```json
{
"name": "fastapi-docker",
"slug": "fastapi-docker",
"incompatibleWith": ["fastapi-container", "fastapi-k8s"],
"url": "fastapi-docker/"
}
```

### Documented Rationale (in PR description):

> **Collision paths**: Both `fastapi-docker` and `fastapi-container` ship `Dockerfile` and `compose.yml` overlays for the same FastAPI template type. Selecting both would overwrite the same generated files.
>
> **Resolution**: Users choose one deployment strategy: containerization via Docker Compose or via container runtime integrations.
>
> **Related**: #91, #119

## Checklist Updates

Update the checklist for new `incompatibleWith` entries:

- [ ] Both entries list each other by `slug`
- [ ] Slugs referenced are valid entries in `templates.json`
- [ ] The collision path(s) are documented (see alternatives above)
- [ ] An existing `incompatibleWith` wasn't already covering the pair
- [ ] If a new packaging strategy was introduced, it was discussed in the issue or Discord first
- [ ] Rationale documented using one of the valid methods (PR description, README, etc.)
17 changes: 12 additions & 5 deletions templates.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
"slug": "backend-applications",
"name": "Backend Applications",
"description": "API and service starters for FastAPI and similar Python backends.",
"details": "Use when the deliverable is an HTTP API or background worker FastAPI for async APIs with OpenAPI docs.",
"details": "Use when the deliverable is an HTTP API or background worker \u2014 FastAPI for async APIs with OpenAPI docs.",
"labels": [
"Backend",
"API",
Expand Down Expand Up @@ -92,7 +92,7 @@
"slug": "monorepo",
"name": "Monorepo",
"description": "Multi-package workspaces with shared tooling and one lockfile.",
"details": "Use when you need multiple libraries and apps in one repo uv workspaces link members locally with a single virtual environment.",
"details": "Use when you need multiple libraries and apps in one repo \u2014 uv workspaces link members locally with a single virtual environment.",
"labels": [
"Monorepo",
"uv",
Expand Down Expand Up @@ -230,6 +230,7 @@
"cli-app",
"django-backend",
"fastapi-backend",
"mlops-sklearn",
"uv-workspace"
],
"category": "ci",
Expand Down Expand Up @@ -263,7 +264,7 @@
{
"name": "Postgres",
"slug": "postgres",
"description": "PostgreSQL 16 Compose service under docker/postgres/ plus env examples. Infra-only does not write application ORM code.",
"description": "PostgreSQL 16 Compose service under docker/postgres/ plus env examples. Infra-only \u2014 does not write application ORM code.",
"url": "https://github.com/Create-Python-App/cpa-templates?subdir=extensions/all-postgres",
"type": [
"celery-worker",
Expand Down Expand Up @@ -432,6 +433,9 @@
"OpenTelemetry",
"Tracing",
"Observability"
],
"incompatibleWith": [
"fastapi-mlflow-tracing"
]
},
{
Expand Down Expand Up @@ -463,6 +467,9 @@
"Tracing",
"Observability",
"FastAPI"
],
"incompatibleWith": [
"fastapi-opentelemetry"
]
},
{
Expand All @@ -485,7 +492,7 @@
{
"name": "FastAPI AI Chat",
"slug": "fastapi-ai-chat",
"description": "Minimal /chat endpoint backed by LangChain's BaseChatModel. Mock provider by default drop in a real provider via init_chat_model() with no interface change.",
"description": "Minimal /chat endpoint backed by LangChain's BaseChatModel. Mock provider by default \u2014 drop in a real provider via init_chat_model() with no interface change.",
"url": "https://github.com/Create-Python-App/cpa-templates?subdir=extensions/fastapi-ai-chat",
"type": [
"fastapi-backend"
Expand Down Expand Up @@ -515,4 +522,4 @@
]
}
]
}
}