Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions skills/.firetiger-skills-source
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Generated by scripts/sync-skills.sh from firetiger-oss/skills. Do not edit by hand.
# Update by merging the sync PR from firetiger-oss/skills, not by editing skills here.
source=firetiger-oss/skills
tag=v1.1.2
hash=a8fcc6d3c62edff443d4bc050e29dffa6219b9cf99ee0d2a484eb1d7c7bdbd56
156 changes: 156 additions & 0 deletions skills/firetiger-create-agent/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,156 @@
---
name: firetiger-create-agent
description: >
Use when creating a Firetiger monitoring agent, installing a prebuilt catalog
agent, or configuring an agent's triggers and schedules — monitoring something
automatically, scheduling recurring or one-off analysis, or wiring event-driven
automation. Always use this skill to create or automate an agent —
it carries the critical gotchas (prefer create_agent_with_goal and answer the
planner, agents must be AGENT_STATE_ON to fire, triggers are separate resources
with six config types, scheduled-agent-runs are one-shot not cron).
license: Apache-2.0
user_invocable: true
user_invocable_description: "Create or automate a monitoring agent in Firetiger"
metadata:
author: firetiger
version: "1.1.0"
homepage: https://firetiger.com
source: https://github.com/firetiger-oss/skills
---

# Firetiger Create Agent

Firetiger agents autonomously analyze telemetry, detect anomalies, investigate issues, and take actions through
connections (Slack, GitHub, Linear, databases, …). Three ways to get one — **prefer a goal or a catalog
template** over hand-building.

## Quick Start

```
create_agent_with_goal with:
goal: "Monitor Next.js API routes for errors and slow responses; alert on error-rate spikes and p95 latency increases"
title: "Checkout Health Monitor" # optional
```

The agent-planner (a system agent, `agents/agent-planner`) configures the prompt, connections, and triggers
from the goal, then returns the planner conversation. **If the planner asked a question, you must answer it**
or the agent won't finish — reply with `send_agent_message` on the plan session.

## Path A (recommended): create from a goal

1. Call `create_agent_with_goal` with a clear `goal` (optional `title`, `agent_id`, `wait_for_plan` default true).
2. **Handle planner questions.** The planner asks sparingly ("act first, refine later"), via a
`request_user_input`. Answer on the user's behalf from the codebase/context — or ask the user if genuinely
unknown — by writing to the plan session:
```
send_agent_message with session: "<plan_session from the tool output>"
message: "Monitor the Postgres connection at DATABASE_URL; alert threshold p99 > 500ms."
```
3. **Confirm.** When the planner completes, the agent flips to `AGENT_STATE_ON`. Share its name and focus.

### Example goals
- "Monitor Next.js API routes for errors and slow responses"
- "Track database query latency and alert if p99 exceeds 500ms"
- "Watch for deployment failures and notify on Slack"
- "Track error rates across all services and open issues for spikes"

## Path B: install a prebuilt catalog agent

Firetiger ships tuned templates for common stacks. Browse and install them:
```
list with resource: "catalog-agents"
create with resource: "catalog-agents/{catalog-agent}/..." # or the InstallCatalogAgent flow
```
Templates include `postgres-performance`, `postgres-schema`, `mysql-performance`, `mysql-schema`,
`clickhouse-performance`, `temporal-performance`, `aws-cost-optimizer`, and `datadog-cost-expert`.

**Key gotcha:** an installed catalog agent starts in **`AGENT_STATE_RECOMMENDED`**, not ON — it won't run
until a human turns it on. Send it a direct message (or set `state: AGENT_STATE_ON`) to activate.

## Path C: configure manually

Use the generic CRUD tools when you need explicit control. **Always run `schema` first** — fields drift.

```
schema with collection: "agents"
```

Agent fields worth knowing: `title`, `description`, `prompt` (guides every session), `state`, `connections[]`
(`{name: "connections/{id}", enabled_tools: [...]}` — what external tools the agent may use),
`mcp_connections[]` (external MCP servers), `tags[]`, `expire_time` (auto-archive after), `cache_ttl`
(`CACHE_TTL_5M` / `CACHE_TTL_1H`), `network_profile`. `skills` and `plan_session` are output-only.

**`state` (`AgentState`) values:** `AGENT_STATE_ON`, `AGENT_STATE_OFF`, `AGENT_STATE_RECOMMENDED`,
`AGENT_STATE_REVIEW_REQUIRED`, `AGENT_STATE_UNSPECIFIED`. Only an **ON** agent fires from triggers or schedules.

```
create with resource: "agents"
body: { "title": "Checkout Health Monitor",
"description": "Daily health analysis of checkout",
"prompt": "You are a daily health check agent...",
"connections": [{"name": "connections/slack-prod", "enabled_tools": ["post_message"]}],
"state": "AGENT_STATE_ON" }
```

## Triggers — how agents fire (six kinds)

Triggers are **separate resources** (`triggers/{id}`) referencing an agent. `InvokeTrigger` and automatic
firing require the target agent be `AGENT_STATE_ON`. Common fields: `display_name`, `agent` (required,
`agents/{id}`), `enabled`, `associated_resources`, and one `configuration`:

| Config | Fires when | Key fields |
|--------|-----------|------------|
| `cron` | On a recurring schedule | `schedule` (5-field cron), `timezone` (IANA, default UTC) |
| `manual` | On `InvokeTrigger` or a POST to its webhook | `webhook_token`, `webhook_address` (output-only) |
| `post_deploy` | Once when a SHA (or descendant) deploys | `repository`, `environment`, `sha`, `delay` |
| `row` | When a matching row lands in a table | `table_name`, `predicate` (SQL), `cooldown` (default 15m) |
| `slack_message_posted` | On messages in Slack channels | `slack_connection`, `channels[]`, `include_thread_replies` |
| `slack_agent_mentioned` | When the agent's Slack handle is mentioned | `slack_handle`, `channels[]` |

Cron example:
```
create with resource: "triggers"
body: { "display_name": "Daily Checkout Health Check",
"agent": "agents/{agent-id}", "enabled": true,
"configuration": { "cron": { "schedule": "0 9 * * *", "timezone": "America/Los_Angeles" } } }
```

## Scheduled agent runs — one-shot, not cron

`scheduled-agent-runs` are **single deferred invocations** ("run this prompt on agent X in 1 hour"), not
recurring schedules. Fields: `agent_id`, `prompt`, `time_delta` (delay from now), plus `use_latest_session` /
`incognito`; `scheduled_time` and `has_run` are output-only. For anything recurring, use a **cron trigger**.

## Writing good agent prompts

Specific, scoped, actionable — and say what the agent should NOT do.

```
You are a daily health check agent for the checkout service.

Your job:
1. Query the last 24 hours of traces for checkout.
2. Calculate error rates and p99 latency; compare to the previous 24h.
3. If error rate rose >10% or p99 >20%, open an issue.
4. Summarize findings briefly.

Focus only on checkout. Do not investigate other services.
```

## Common Mistakes

| # | Mistake | Fix |
|---|---------|-----|
| 1 | **Ignoring the planner's question** | `create_agent_with_goal` isn't done until you answer via `send_agent_message` on the plan session. |
| 2 | **Expecting a catalog agent to run immediately** | It installs as `AGENT_STATE_RECOMMENDED` — turn it ON. |
| 3 | **Trigger created but agent is OFF** | Firing requires `AGENT_STATE_ON`. |
| 4 | **Using `scheduled-agent-runs` for recurring work** | Those are one-shot — use a `cron` trigger for recurrence. |
| 5 | **Manual `create` without `schema`** | Field names drift — call `schema` for `agents`/`triggers` first. |
| 6 | **Embedding the trigger in the agent** | Triggers are separate resources referencing `agents/{id}`. |
| 7 | **Unbounded prompt** | State what the agent should NOT do; scope it to specific services. |

## Related

- Run and steer an existing agent's session: `firetiger-monitor-deploy`.
- Manual investigations and the issues agents open: `firetiger-investigate`.
- Connect the integrations an agent acts through: `firetiger-setup`.
102 changes: 102 additions & 0 deletions skills/firetiger-instrument/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
---
name: firetiger-instrument
description: >
Use when adding OpenTelemetry instrumentation to an application so it sends
telemetry to Firetiger — setting up tracing, configuring the OTLP exporter, or
connecting a Node.js, Next.js, Python, Go, or Rust app to Firetiger. Always use
this skill when instrumenting for Firetiger — it carries the critical gotchas
(init must load before any other import, Firetiger ingest uses HTTP Basic auth
not Bearer, credentials come from the get_ingest_credentials MCP tool) that make
telemetry actually show up.
license: Apache-2.0
user_invocable: true
user_invocable_description: "Instrument your application with OpenTelemetry for Firetiger"
metadata:
author: firetiger
version: "1.0.0"
homepage: https://firetiger.com
source: https://github.com/firetiger-oss/skills
references:
- references/nodejs.md
- references/python.md
- references/go.md
- references/rust.md
- references/custom-spans.md
---

# Firetiger Instrument

Instrument an application with OpenTelemetry so it exports traces (and logs/metrics) to Firetiger. For full
onboarding that also connects integrations and creates a monitoring agent, use `firetiger-setup` — this skill
is instrumentation only.

## Workflow

1. **Get credentials.** Call the Firetiger MCP tool **`get_ingest_credentials`** → OTLP endpoint + username +
password. If the MCP tools aren't available, tell the user to connect the Firetiger MCP server
(`https://api.cloud.firetiger.com/mcp/v1`) and sign in, then retry.
2. **Detect the framework** (table below) and open the matching reference.
3. **Check for existing instrumentation** — search for `@opentelemetry` (Node), `opentelemetry` (Python), or
`go.opentelemetry.io` (Go) imports. If present, just repoint the exporter at Firetiger instead of adding a
second SDK.
4. **Add the SDK init + env vars**, run, and verify data lands.

| Detected file | Stack | Reference |
|---------------|-------|-----------|
| `next.config.js` / `next.config.ts` | Next.js (uses `@vercel/otel`) | [references/nodejs.md](references/nodejs.md) |
| `package.json` | Node.js / TypeScript | [references/nodejs.md](references/nodejs.md) |
| `requirements.txt` / `pyproject.toml` / `setup.py` | Python | [references/python.md](references/python.md) |
| `go.mod` | Go | [references/go.md](references/go.md) |
| `Cargo.toml` | Rust | [references/rust.md](references/rust.md) |

For business-critical spans, log/trace correlation, and attribute best practices, see
[references/custom-spans.md](references/custom-spans.md).

## Critical: initialization order

**OpenTelemetry MUST initialize before any other import.** If instrumented libraries (Express, HTTP clients,
database drivers) load before the SDK starts, auto-instrumentation never attaches and you get no spans. Load
the instrumentation file first (`node --import ./instrumentation.js …`, `opentelemetry-instrument python …`,
or an init call at the very top of `main`).

## Authentication & environment variables

Firetiger authenticates ingest with **HTTP Basic auth** — the OTLP header is
`Authorization=Basic <base64(username:password)>`, **not** a Bearer token. Configure these wherever the app
runs (`.env.local` for dev, the deploy platform for prod), using values from `get_ingest_credentials`:

```bash
OTEL_EXPORTER_OTLP_ENDPOINT="https://ingest.cloud.firetiger.com" # use the value from get_ingest_credentials
OTEL_EXPORTER_OTLP_HEADERS="Authorization=Basic <base64(username:password)>"
OTEL_SERVICE_NAME="your-service-name"
```

Add credential files to `.gitignore` (`.env.local`, `.env.*.local`). Never commit credentials.

## Verify

1. Generate some traffic.
2. Query with the `firetiger-query` skill:
`SELECT name, start_time FROM "opentelemetry/traces/your-service-name" WHERE start_time >= NOW() - INTERVAL '15 minutes' LIMIT 10`
(run `SHOW TABLES;` first to confirm your service's table appeared).

## Common Mistakes

| # | Mistake | Fix |
|---|---------|-----|
| 1 | **Init imported after app code** | Load the instrumentation file first — otherwise auto-instrumentation can't patch already-imported libraries. |
| 2 | **Using `Authorization=Bearer …`** | Firetiger ingest is HTTP Basic auth — `Authorization=Basic <base64(user:pass)>`. |
| 3 | **Hardcoding the endpoint** | Use the endpoint from `get_ingest_credentials`; it can differ per org/region. |
| 4 | **Adding a second SDK on top of existing OTEL** | Detect existing instrumentation and repoint the exporter instead. |
| 5 | **`SimpleSpanProcessor` in production** | Use `BatchSpanProcessor` with `gzip` compression for throughput. |
| 6 | **Committing `.env.local`** | Add credential files to `.gitignore`. |
| 7 | **Service name mismatch** | The `OTEL_SERVICE_NAME` you set is the `{service_name}` in the query table `"opentelemetry/traces/{service_name}"`. |
| 8 | **No graceful shutdown** | Call `sdk.shutdown()` on `SIGTERM` so buffered spans flush before exit. |

## Related

- **Verify the data landed and start querying it:** `firetiger-query` (`SHOW TABLES;`, then the per-service tables this instrumentation creates).
- **Add business-critical spans, log/trace correlation, and attribute hygiene:** [references/custom-spans.md](references/custom-spans.md).
- **Full onboarding beyond instrumentation** (connect integrations, register deploys, create a monitoring agent): `firetiger-setup`.
- **Not writing app code?** If you only have platform logs/traces (Vercel, AWS, GCP, Cloudflare) or a Datadog Agent, wire a drain via `firetiger-setup` instead of an SDK.
- **Once your change ships:** have Firetiger watch the deploy with `firetiger-monitor-deploy`.
114 changes: 114 additions & 0 deletions skills/firetiger-instrument/references/custom-spans.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
# Custom spans, log correlation & attribute hygiene

Auto-instrumentation covers common libraries. Add manual spans for business-critical operations, and correlate
logs with traces so Firetiger can join them.

## Custom spans

### Node.js
```typescript
import { trace, SpanStatusCode } from '@opentelemetry/api';

const tracer = trace.getTracer('your-service-name');

async function processOrder(orderId: string, customerId: string) {
return await tracer.startActiveSpan('processOrder', async (span) => {
span.setAttribute('order.id', orderId);
span.setAttribute('customer.id', customerId);
try {
const result = await executeOrder(orderId);
span.addEvent('order.completed', { 'order.total': result.total });
return result;
} catch (error) {
span.setStatus({ code: SpanStatusCode.ERROR, message: error.message });
span.recordException(error);
throw error;
} finally {
span.end();
}
});
}
```

### Python
```python
from opentelemetry import trace

tracer = trace.get_tracer("your-service-name")

def process_order(order_id: str, customer_id: str):
with tracer.start_as_current_span("process_order") as span:
span.set_attribute("order.id", order_id)
span.set_attribute("customer.id", customer_id)
try:
result = execute_order(order_id)
span.add_event("order.completed", {"order.total": result.total})
return result
except Exception as e:
span.set_status(trace.StatusCode.ERROR, str(e))
span.record_exception(e)
raise
```

## Log correlation

Inject `trace_id` / `span_id` into logs so `"opentelemetry/logs/{service}"` rows join to their traces.

### Node.js (Winston)
```typescript
import winston from 'winston';
import { trace } from '@opentelemetry/api';

const logger = winston.createLogger({
format: winston.format.combine(
winston.format((info) => {
const span = trace.getActiveSpan();
if (span) {
const ctx = span.spanContext();
info.trace_id = ctx.traceId;
info.span_id = ctx.spanId;
}
return info;
})(),
winston.format.json()
),
transports: [new winston.transports.Console()],
});
```

### Python (structlog)
```python
import structlog
from opentelemetry import trace

def add_trace_context(logger, method_name, event_dict):
span = trace.get_current_span()
if span.is_recording():
ctx = span.get_span_context()
event_dict["trace_id"] = format(ctx.trace_id, "032x")
event_dict["span_id"] = format(ctx.span_id, "016x")
return event_dict

structlog.configure(processors=[add_trace_context, structlog.processors.JSONRenderer()])
```

## Attribute hygiene

- **Semantic conventions:** `http.method`, `http.status_code`, `http.route`; `db.system`, `db.statement`,
`db.name`; `rpc.service`, `rpc.method`. Prefix custom business attributes with `app.`.
- **Cardinality:** high-cardinality *values* (user IDs) are fine; high-cardinality attribute *names* are not.
- **Never** put secrets, tokens, passwords, or PII in attributes.
- These attributes become queryable columns in Firetiger (`attributes.app.order_id`), inferred to depth 2 —
see the `firetiger-query` schema reference.

## Performance

- Use `BatchSpanProcessor` in production (not `SimpleSpanProcessor`).
- Enable `gzip` compression on exporters.
- Size batches to your traffic; set queue limits to bound memory.
- Add manual spans for checkout/payment and anything measured for SLOs.

## Where to go next

- **Query your custom spans and correlated logs** — the attributes you set become columns; see the `firetiger-query` skill (and its schema reference for the depth-2 inference rule).
- **Investigate with them** — once you have rich spans, `firetiger-investigate` can reason over them to diagnose incidents.
Loading
Loading