From e6f6a53dc160399b3adefeac7e4e31d2f193dded Mon Sep 17 00:00:00 2001 From: Daniel Green Date: Wed, 2 Sep 2026 13:16:57 -0400 Subject: [PATCH 01/10] Add guide for getting the most out of Bits Investigations Adds a new page covering how to tune Bits Investigation accuracy for your environment: bits.md, monitor selection and runbooks, Skills, external integrations, and feedback/memories, plus how to verify changes land using Bits Chat. Also documents Skills as a knowledge source and trims the Configure page title. --- .../en/bits_ai/bits_investigation/_index.md | 3 +- .../bits_ai/bits_investigation/configure.md | 2 +- .../getting_the_most_out.md | 112 ++++++++++++++++++ .../bits_investigation/knowledge_sources.md | 12 +- 4 files changed, 126 insertions(+), 3 deletions(-) create mode 100644 hugo/content/en/bits_ai/bits_investigation/getting_the_most_out.md diff --git a/hugo/content/en/bits_ai/bits_investigation/_index.md b/hugo/content/en/bits_ai/bits_investigation/_index.md index 10a5f96f3d6..63a9f95807d 100644 --- a/hugo/content/en/bits_ai/bits_investigation/_index.md +++ b/hugo/content/en/bits_ai/bits_investigation/_index.md @@ -30,9 +30,10 @@ Bits Investigation is an autonomous AI agent that investigates production issues {{< whatsnext desc="Learn about how you can use Bits Investigation:" >}} {{< nextlink href="bits_ai/bits_investigation/investigate_issues" >}}Investigate issues{{< /nextlink >}} - {{< nextlink href="bits_ai/bits_investigation/configure" >}}Bits Investigation integrations and settings{{< /nextlink >}} + {{< nextlink href="bits_ai/bits_investigation/configure" >}}Integrations and settings{{< /nextlink >}} {{< nextlink href="bits_ai/bits_investigation/knowledge_sources" >}}Knowledge sources{{< /nextlink >}} {{< nextlink href="bits_ai/bits_investigation/chat_bits_investigation" >}}Chat with Bits Investigation{{< /nextlink >}} + {{< nextlink href="bits_ai/bits_investigation/getting_the_most_out" >}}Getting the most out of Bits Investigations{{< /nextlink >}} {{< /whatsnext >}} ## Further reading diff --git a/hugo/content/en/bits_ai/bits_investigation/configure.md b/hugo/content/en/bits_ai/bits_investigation/configure.md index 3c2ea88fc15..39d1b8cf362 100644 --- a/hugo/content/en/bits_ai/bits_investigation/configure.md +++ b/hugo/content/en/bits_ai/bits_investigation/configure.md @@ -1,5 +1,5 @@ --- -title: Configure Integrations and Settings +title: Integrations and Settings aliases: - /bits_ai/bits_ai_sre/configure/ --- diff --git a/hugo/content/en/bits_ai/bits_investigation/getting_the_most_out.md b/hugo/content/en/bits_ai/bits_investigation/getting_the_most_out.md new file mode 100644 index 00000000000..37acccf5750 --- /dev/null +++ b/hugo/content/en/bits_ai/bits_investigation/getting_the_most_out.md @@ -0,0 +1,112 @@ +--- +title: Getting the Most Out of Bits Investigations +description: "Learn how to teach Bits Investigation about your environment, systems, and best practices to improve investigation accuracy over time." +further_reading: +- link: "/bits_ai/bits_investigation/knowledge_sources/" + tag: "Documentation" + text: "Knowledge sources" +- link: "/bits_ai/bits_investigation/configure/" + tag: "Documentation" + text: "Integrations and settings" +- link: "/bits_ai/bits_investigation/chat_bits_investigation/" + tag: "Documentation" + text: "Chat with Bits Investigation" +--- + +## Overview + +Bits Investigation is built to handle uncertainty. It forms hypotheses, chases down evidence across your telemetry, and reasons through incomplete signals rather than following a fixed script. That reasoning is only as good as what it knows about your organization. Two orgs running the same products, but with different tagging schemes, different escalation channels, and different tribal knowledge, will get noticeably different results out of the same agent until someone teaches it the difference. + +Datadog gives you four places to do that teaching: [`bits.md`][1], your monitors and runbooks, [Skills][11], and the feedback you leave on completed investigations. This guide covers how to use each one well, plus how to confirm the changes you make actually land. + +## Write bits.md as rules, not background reading + +[`bits.md`][1] is read on every investigation, so it works best as a short list of specific rules Bits should follow, not general background about your systems. The difference matters: + +- **Good**: "The billing team's alerts tag the service as `billing-svc`, but our APM and log pipelines emit `billing_service`; treat them as the same service when correlating." +- **Not useful**: "Checkout is our payments service." (True, but not something Bits needs corrected or can act on differently.) + +Entries that tend to move accuracy the most: +- **Cross-system name mapping.** The same service, environment, or team often has a different spelling in monitors, APM, logs, and any connected ticketing system. Write the mapping down once instead of relying on Bits to guess it every time. +- **Known noise.** Recurring patterns that resemble an incident but are actually routine for your systems, such as a weekly reindex job, a maintenance window, or a load test, along with the conditions that would make one worth flagging as a real problem. +- **Standing scope rules.** How to handle an alert that doesn't specify environment or region explicitly. Without a rule, Bits has to infer scope from the alert alone. + +Keep it in rule form ("when X, do Y") rather than prose. See [Knowledge sources][1] for a full sample file. + +## Start with the monitors that matter most + +Tuning takes effort per monitor: a good runbook link, correct tags, a `bits.md` entry that covers its quirks. Rather than spreading that effort across every monitor in your org, start with the ones where an investigation actually saves someone time. On the [Supported Monitors][8] page, two filters are useful for finding them: + +- **High-priority monitors.** Filter by [`priority:p1`][9] (or `p2`) to see the monitors most likely to represent a real incident. +- **Monitors with notifications configured.** Filter by [`notification:*`][10] to see monitors that already alert a person or a channel. If a human is watching it, an investigation on it is worth the setup. + +Enable {{< ui >}}Auto-Investigate{{< /ui >}} on this narrower list first, and put your tuning effort into their runbooks and `bits.md` entries. Once investigations on these are reliably accurate, expand to the rest of your monitors. + +## Make your monitors self-sufficient + +Bits reads whatever is in the monitor message at investigation time, which means the monitor itself is one of your highest-leverage knowledge sources, and one that's easy to leave thin. + +Link the dashboard, log query, or notebook you'd personally check first. Plain URLs are enough; no special formatting is needed. For anything longer than a link or two, use a notebook instead: notebooks mix markdown and live Datadog queries, so they double as both a runbook and a data source Bits can query directly. + +It also helps to name the blast radius. State which downstream services or dependencies are typically affected so Bits doesn't have to rediscover your architecture on every alert. + +Finally, make sure the monitor's query is scoped or grouped by `service`. That's the tag Bits leans on most to jump from the alert into APM, logs, RUM, and [Catalog][2] for the right service. Leave it out, and Bits has to fall back on weaker signals, like the monitor name, to figure out what's affected. + +A monitor with a stale runbook link is worse than one with none: it actively points Bits at the wrong dashboard or a decommissioned service. Treat monitor messages as something that gets reviewed, not written once. + +## Package repeatable procedures as Skills + +Some knowledge doesn't fit neatly into a single monitor's runbook or a line in `bits.md`. A multi-step diagnostic procedure for a particular subsystem, or the specific way your team queries a third-party tool, is better captured as a [Skill][11], created at [{{< ui >}}Actions{{< /ui >}} > {{< ui >}}Skills{{< /ui >}}][11]. + +A skill is a named, reusable procedure that Bits Investigation invokes automatically when its name and description match the situation, the same way Bits Code discovers custom skills in your repository. Reach for a skill instead of repeating the same instructions across multiple monitors, or when you want a procedure to stay consistent across Bits Investigation, Bits Chat, and other Bits products that share it. + +## Connect the systems where the answer already lives + +If root causes are documented somewhere Bits can't see, no amount of tuning inside Datadog closes that gap. + +- **Confluence.** [Connect your Confluence account][3] and link relevant pages in monitor messages. Bits extracts telemetry links and troubleshooting steps from the page during an investigation. Turning on account crawling also lets [Bits Chat][4] search your Confluence space directly, not just the pages you've linked. +- **Source code.** Connect [GitHub][5] and [tag your APM telemetry with Git information][6] so Bits can tie a regression to the commit or deploy that introduced it, rather than stopping at "this service's error rate changed." This is also what lets Bits Code pick up the investigation and propose a fix. +- **Other observability tools.** If telemetry or history lives in Grafana, Dynatrace, Splunk, Sentry, or ServiceNow, connect those too. See [Integrate with third-party observability and SCM platforms][7]. An investigation into a service that's only half-instrumented in Datadog will stay incomplete until both sides are visible. + +When you write documentation meant for Bits to read (a Confluence page, a notebook), write it the way you'd want a new hire to read it: name the actual service and system, not "the usual suspect," and make the remediation steps explicit rather than assumed. + +## Correct it, and let the correction stick + +At the end of an investigation, tell Bits whether the conclusion was right. + +Confirm what's correct, too. Positive feedback still becomes a memory Bits can reuse; it's not just a formality. When the conclusion is wrong, be specific: name the actual root cause, the services or metrics involved, and link the telemetry that shows it. "That's wrong" gives Bits nothing to change, while naming the failing dependency and linking the query that confirms it does. + +Both kinds of feedback become **memories**, which Bits selectively applies to similar future investigations by reusing effective queries, applying past corrections, and adjusting how it prioritizes steps. You can review or delete individual memories from the {{< ui >}}Memories{{< /ui >}} column on the [Monitor Management][8] page, which is worth checking periodically to make sure a correction still reflects reality (services get renamed, causes get fixed). + +## Confirm Bits actually picked it up + +Don't assume an update to `bits.md`, a runbook, or an integration took effect. Check it. [Bits Chat][4] gives you a direct answer without waiting on a full investigation: + +| Goal | Example prompt | +|------|-----------------| +| Confirm a naming rule in `bits.md` is applied | `If I ask about billing-svc, what service does that map to in APM and logs?` | +| Confirm a noise pattern is recognized | `Is a spike in reindex job duration on Sundays something I should worry about for ?` | +| Confirm a runbook or Confluence page is being read | `What does our documentation say about diagnosing issues?` | +| Confirm a skill is being invoked | Ask a question that should trigger the skill and check whether the response follows the procedure you defined | +| Confirm a past correction is being reused | Ask a question tied to the earlier feedback (e.g. `What's your read on the current memory pressure on ?`) and check whether the response references your correction | +| Surface any remaining gaps | `What information would have made this investigation faster or more accurate?` | + +If the answer doesn't reflect what you documented, the most common causes are: the `bits.md` entry is descriptive rather than rule-like, the relevant integration isn't connected or is missing permissions, or the monitor/runbook link is stale. Fix the specific gap and re-test with the same prompt rather than rewriting broadly. + +Once Bits reflects the change in chat, re-run a known investigation to confirm it also changes the actual conclusion. Chat and investigations don't always draw on knowledge the same way, so a good chat answer isn't a guarantee the investigation itself improved. + +[1]: /bits_ai/bits_investigation/knowledge_sources/ +[2]: /internal_developer_portal/catalog/ +[3]: https://app.datadoghq.com/integrations/confluence +[4]: /bits_ai/bits_investigation/chat_bits_investigation/ +[5]: /integrations/github/ +[6]: /source_code/service-mapping +[7]: /bits_ai/bits_investigation/configure/#integrate-with-third-party-observability-and-scm-platforms +[8]: https://app.datadoghq.com/bits-ai/monitors/supported +[9]: https://app.datadoghq.com/bits-ai/monitors/supported?q=priority%3Ap1&auto_only=false +[10]: https://app.datadoghq.com/bits-ai/monitors/supported?q=notification%3A%2A&auto_only=false +[11]: https://app.datadoghq.com/actions/skills + +## Further reading + +{{< partial name="whats-next/whats-next.html" >}} diff --git a/hugo/content/en/bits_ai/bits_investigation/knowledge_sources.md b/hugo/content/en/bits_ai/bits_investigation/knowledge_sources.md index a5c10ae9882..7b44ed346cc 100644 --- a/hugo/content/en/bits_ai/bits_investigation/knowledge_sources.md +++ b/hugo/content/en/bits_ai/bits_investigation/knowledge_sources.md @@ -7,9 +7,10 @@ aliases: --- -Bits Investigation improves over time by combining three distinct sources of knowledge: +Bits Investigation improves over time by combining four distinct sources of knowledge: - [**Runbooks:**](#runbooks) Step-by-step troubleshooting guidance - [**bits.md:**](#bitsmd) Context about your environment +- [**Skills:**](#skills) Reusable procedures Bits can invoke during an investigation - [**Feedback and memories:**](#feedback-and-memories) Learnings from investigations ## Runbooks @@ -103,6 +104,14 @@ Rule: {{< /code-block >}} +## Skills + +[Skills][3] are named, reusable procedures that you create and manage centrally, then make available to Bits products, including Bits Investigation. A skill packages instructions (and, where relevant, the tools or queries needed to carry them out) for a specific task, such as how to diagnose a particular subsystem, or how to query a specific external tool your team relies on. + +Bits Investigation automatically invokes a skill when its name and description match the situation at hand, the same way Bits Code discovers and invokes custom skills in your repository. This makes skills a good place to put a procedure you'd otherwise have to repeat across multiple monitors' runbooks or restate in `bits.md`, and to keep it consistent across investigations, chat, and other Bits products that draw on the same skill. + +To create or manage skills, go to [{{< ui >}}Actions{{< /ui >}} > {{< ui >}}Skills{{< /ui >}}][3]. + ## Feedback and memories At the end of an investigation, let Bits know whether the conclusion it made was correct. @@ -122,3 +131,4 @@ To manage memories, including viewing and deleting them, go to the {{< ui >}}Mem [1]: https://app.datadoghq.com/bits-ai/monitors/supported [2]: https://app.datadoghq.com/bits-ai/settings/bits-md +[3]: https://app.datadoghq.com/actions/skills From e4075c3a1f6b6ea58d1ae8284eea485bd1af7880 Mon Sep 17 00:00:00 2001 From: Daniel Green Date: Wed, 2 Sep 2026 13:27:07 -0400 Subject: [PATCH 02/10] Rephrase Bits Chat verification intro --- .../en/bits_ai/bits_investigation/getting_the_most_out.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hugo/content/en/bits_ai/bits_investigation/getting_the_most_out.md b/hugo/content/en/bits_ai/bits_investigation/getting_the_most_out.md index 37acccf5750..5a59743db51 100644 --- a/hugo/content/en/bits_ai/bits_investigation/getting_the_most_out.md +++ b/hugo/content/en/bits_ai/bits_investigation/getting_the_most_out.md @@ -80,7 +80,7 @@ Both kinds of feedback become **memories**, which Bits selectively applies to si ## Confirm Bits actually picked it up -Don't assume an update to `bits.md`, a runbook, or an integration took effect. Check it. [Bits Chat][4] gives you a direct answer without waiting on a full investigation: +Use [Bits Chat][4] to verify changes by asking it questions about what you wrote, rather than waiting on a full investigation to find out. You can also ask it directly for suggestions on how to fine-tune `bits.md`, a runbook, or a skill further: | Goal | Example prompt | |------|-----------------| From 8d116fa5d43c6c108142c9b35eb10c91a6c79e90 Mon Sep 17 00:00:00 2001 From: Daniel Green Date: Wed, 2 Sep 2026 13:29:12 -0400 Subject: [PATCH 03/10] Tighten language throughout the page --- .../getting_the_most_out.md | 79 +++++++++---------- 1 file changed, 38 insertions(+), 41 deletions(-) diff --git a/hugo/content/en/bits_ai/bits_investigation/getting_the_most_out.md b/hugo/content/en/bits_ai/bits_investigation/getting_the_most_out.md index 5a59743db51..d0a071e01bd 100644 --- a/hugo/content/en/bits_ai/bits_investigation/getting_the_most_out.md +++ b/hugo/content/en/bits_ai/bits_investigation/getting_the_most_out.md @@ -15,85 +15,82 @@ further_reading: ## Overview -Bits Investigation is built to handle uncertainty. It forms hypotheses, chases down evidence across your telemetry, and reasons through incomplete signals rather than following a fixed script. That reasoning is only as good as what it knows about your organization. Two orgs running the same products, but with different tagging schemes, different escalation channels, and different tribal knowledge, will get noticeably different results out of the same agent until someone teaches it the difference. +Bits Investigation reasons through incomplete signals rather than following a fixed script, so its accuracy depends on how well it knows your environment. Two orgs with the same products but different tagging, escalation paths, and tribal knowledge get different results from the same agent, until someone teaches it the difference. -Datadog gives you four places to do that teaching: [`bits.md`][1], your monitors and runbooks, [Skills][11], and the feedback you leave on completed investigations. This guide covers how to use each one well, plus how to confirm the changes you make actually land. +You can teach it through four places: [`bits.md`][1], your monitors and runbooks, [Skills][11], and feedback on completed investigations. -## Write bits.md as rules, not background reading +## Write bits.md as rules, not background -[`bits.md`][1] is read on every investigation, so it works best as a short list of specific rules Bits should follow, not general background about your systems. The difference matters: +[`bits.md`][1] is read on every investigation. Write specific rules, not general descriptions of your systems. -- **Good**: "The billing team's alerts tag the service as `billing-svc`, but our APM and log pipelines emit `billing_service`; treat them as the same service when correlating." -- **Not useful**: "Checkout is our payments service." (True, but not something Bits needs corrected or can act on differently.) +- **Good**: "The billing team's alerts tag the service as `billing-svc`, but APM and logs use `billing_service`. Treat them as the same service." +- **Not useful**: "Checkout is our payments service." True, but gives Bits nothing to act on. -Entries that tend to move accuracy the most: -- **Cross-system name mapping.** The same service, environment, or team often has a different spelling in monitors, APM, logs, and any connected ticketing system. Write the mapping down once instead of relying on Bits to guess it every time. -- **Known noise.** Recurring patterns that resemble an incident but are actually routine for your systems, such as a weekly reindex job, a maintenance window, or a load test, along with the conditions that would make one worth flagging as a real problem. -- **Standing scope rules.** How to handle an alert that doesn't specify environment or region explicitly. Without a rule, Bits has to infer scope from the alert alone. +What to include: +- **Cross-system name mapping.** The same service, environment, or team often has different names in monitors, APM, logs, and any connected ticketing system. Write the mapping down once. +- **Known noise.** Patterns that look like incidents but are routine, such as a weekly reindex job or a load test, and when they'd actually count as a real problem. +- **Standing scope rules.** How to handle an alert that doesn't specify environment or region. Without a rule, Bits has to guess. -Keep it in rule form ("when X, do Y") rather than prose. See [Knowledge sources][1] for a full sample file. +See [Knowledge sources][1] for a full sample file. ## Start with the monitors that matter most -Tuning takes effort per monitor: a good runbook link, correct tags, a `bits.md` entry that covers its quirks. Rather than spreading that effort across every monitor in your org, start with the ones where an investigation actually saves someone time. On the [Supported Monitors][8] page, two filters are useful for finding them: +Tuning a monitor takes work: a runbook link, correct tags, a `bits.md` entry for its quirks. Start with monitors where an investigation actually saves time. On the [Supported Monitors][8] page: -- **High-priority monitors.** Filter by [`priority:p1`][9] (or `p2`) to see the monitors most likely to represent a real incident. -- **Monitors with notifications configured.** Filter by [`notification:*`][10] to see monitors that already alert a person or a channel. If a human is watching it, an investigation on it is worth the setup. +- Filter by [`priority:p1`][9] (or `p2`) for monitors most likely to represent a real incident. +- Filter by [`notification:*`][10] for monitors that already page a person or channel. -Enable {{< ui >}}Auto-Investigate{{< /ui >}} on this narrower list first, and put your tuning effort into their runbooks and `bits.md` entries. Once investigations on these are reliably accurate, expand to the rest of your monitors. +Enable {{< ui >}}Auto-Investigate{{< /ui >}} on this list first and tune it before expanding to the rest of your monitors. ## Make your monitors self-sufficient -Bits reads whatever is in the monitor message at investigation time, which means the monitor itself is one of your highest-leverage knowledge sources, and one that's easy to leave thin. +Bits reads the monitor message at investigation time, so the monitor itself is a knowledge source. -Link the dashboard, log query, or notebook you'd personally check first. Plain URLs are enough; no special formatting is needed. For anything longer than a link or two, use a notebook instead: notebooks mix markdown and live Datadog queries, so they double as both a runbook and a data source Bits can query directly. +- Link the dashboard, log query, or notebook you'd check first. Plain URLs work, no formatting needed. +- Use a notebook for anything longer than a link or two. Notebooks mix markdown with live Datadog queries. +- State the blast radius: which downstream services or dependencies are typically affected. +- Scope or group the monitor query by `service`. This is what lets Bits pivot into APM, logs, RUM, and [Catalog][2] for the right service. Without it, Bits falls back on weaker signals like the monitor name. -It also helps to name the blast radius. State which downstream services or dependencies are typically affected so Bits doesn't have to rediscover your architecture on every alert. - -Finally, make sure the monitor's query is scoped or grouped by `service`. That's the tag Bits leans on most to jump from the alert into APM, logs, RUM, and [Catalog][2] for the right service. Leave it out, and Bits has to fall back on weaker signals, like the monitor name, to figure out what's affected. - -A monitor with a stale runbook link is worse than one with none: it actively points Bits at the wrong dashboard or a decommissioned service. Treat monitor messages as something that gets reviewed, not written once. +Review monitor messages periodically. A stale runbook link is worse than no link, since it points Bits at the wrong dashboard or a decommissioned service. ## Package repeatable procedures as Skills -Some knowledge doesn't fit neatly into a single monitor's runbook or a line in `bits.md`. A multi-step diagnostic procedure for a particular subsystem, or the specific way your team queries a third-party tool, is better captured as a [Skill][11], created at [{{< ui >}}Actions{{< /ui >}} > {{< ui >}}Skills{{< /ui >}}][11]. +A multi-step diagnostic procedure, or the specific way your team queries a third-party tool, doesn't fit cleanly in one monitor's runbook or a `bits.md` line. Capture it as a [Skill][11] instead, created at [{{< ui >}}Actions{{< /ui >}} > {{< ui >}}Skills{{< /ui >}}][11]. -A skill is a named, reusable procedure that Bits Investigation invokes automatically when its name and description match the situation, the same way Bits Code discovers custom skills in your repository. Reach for a skill instead of repeating the same instructions across multiple monitors, or when you want a procedure to stay consistent across Bits Investigation, Bits Chat, and other Bits products that share it. +Bits Investigation invokes a skill automatically when its name and description match the situation, the same way Bits Code discovers custom skills in your repository. Use a skill when you'd otherwise repeat the same instructions across monitors, or when a procedure needs to stay consistent across Bits Investigation, Bits Chat, and other Bits products. ## Connect the systems where the answer already lives -If root causes are documented somewhere Bits can't see, no amount of tuning inside Datadog closes that gap. - -- **Confluence.** [Connect your Confluence account][3] and link relevant pages in monitor messages. Bits extracts telemetry links and troubleshooting steps from the page during an investigation. Turning on account crawling also lets [Bits Chat][4] search your Confluence space directly, not just the pages you've linked. -- **Source code.** Connect [GitHub][5] and [tag your APM telemetry with Git information][6] so Bits can tie a regression to the commit or deploy that introduced it, rather than stopping at "this service's error rate changed." This is also what lets Bits Code pick up the investigation and propose a fix. -- **Other observability tools.** If telemetry or history lives in Grafana, Dynatrace, Splunk, Sentry, or ServiceNow, connect those too. See [Integrate with third-party observability and SCM platforms][7]. An investigation into a service that's only half-instrumented in Datadog will stay incomplete until both sides are visible. +- **Confluence.** [Connect your Confluence account][3] and link relevant pages in monitor messages. Bits extracts telemetry links and troubleshooting steps from the page. Enable account crawling to let [Bits Chat][4] search your Confluence space directly, not just linked pages. +- **Source code.** Connect [GitHub][5] and [tag your APM telemetry with Git information][6] so Bits can tie a regression to the commit or deploy that caused it. This also lets Bits Code pick up the investigation and propose a fix. +- **Other observability tools.** Connect Grafana, Dynatrace, Splunk, Sentry, or ServiceNow if telemetry lives there. See [Integrate with third-party observability and SCM platforms][7]. -When you write documentation meant for Bits to read (a Confluence page, a notebook), write it the way you'd want a new hire to read it: name the actual service and system, not "the usual suspect," and make the remediation steps explicit rather than assumed. +Write documentation for Bits the way you'd write it for a new hire: name the actual service and system, and spell out remediation steps instead of assuming context. ## Correct it, and let the correction stick At the end of an investigation, tell Bits whether the conclusion was right. -Confirm what's correct, too. Positive feedback still becomes a memory Bits can reuse; it's not just a formality. When the conclusion is wrong, be specific: name the actual root cause, the services or metrics involved, and link the telemetry that shows it. "That's wrong" gives Bits nothing to change, while naming the failing dependency and linking the query that confirms it does. +Confirm what's correct, not just what's wrong; positive feedback still becomes a memory Bits reuses. When Bits gets it wrong, name the actual root cause, the services or metrics involved, and link the telemetry that proves it. "That's wrong" gives Bits nothing to change. -Both kinds of feedback become **memories**, which Bits selectively applies to similar future investigations by reusing effective queries, applying past corrections, and adjusting how it prioritizes steps. You can review or delete individual memories from the {{< ui >}}Memories{{< /ui >}} column on the [Monitor Management][8] page, which is worth checking periodically to make sure a correction still reflects reality (services get renamed, causes get fixed). +Both become **memories**, which Bits selectively reuses in similar future investigations. Review or delete them from the {{< ui >}}Memories{{< /ui >}} column on the [Monitor Management][8] page, and check periodically that older corrections still hold (services get renamed, causes get fixed). ## Confirm Bits actually picked it up -Use [Bits Chat][4] to verify changes by asking it questions about what you wrote, rather than waiting on a full investigation to find out. You can also ask it directly for suggestions on how to fine-tune `bits.md`, a runbook, or a skill further: +Use [Bits Chat][4] to check your changes by asking about what you wrote, instead of waiting on a full investigation. You can also ask Bits Chat directly for suggestions on how to improve `bits.md`, a runbook, or a skill. | Goal | Example prompt | |------|-----------------| -| Confirm a naming rule in `bits.md` is applied | `If I ask about billing-svc, what service does that map to in APM and logs?` | -| Confirm a noise pattern is recognized | `Is a spike in reindex job duration on Sundays something I should worry about for ?` | -| Confirm a runbook or Confluence page is being read | `What does our documentation say about diagnosing issues?` | -| Confirm a skill is being invoked | Ask a question that should trigger the skill and check whether the response follows the procedure you defined | -| Confirm a past correction is being reused | Ask a question tied to the earlier feedback (e.g. `What's your read on the current memory pressure on ?`) and check whether the response references your correction | -| Surface any remaining gaps | `What information would have made this investigation faster or more accurate?` | +| Check a `bits.md` naming rule | `If I ask about billing-svc, what service does that map to in APM and logs?` | +| Check a noise pattern | `Is a spike in reindex job duration on Sundays something I should worry about for ?` | +| Check a runbook or Confluence page | `What does our documentation say about diagnosing issues?` | +| Check a skill | Ask a question that should trigger it, and see if the response follows the procedure | +| Check a past correction | Ask a related question (e.g. `What's your read on memory pressure on ?`) and see if it references your correction | +| Find remaining gaps | `What information would have made this investigation faster or more accurate?` | -If the answer doesn't reflect what you documented, the most common causes are: the `bits.md` entry is descriptive rather than rule-like, the relevant integration isn't connected or is missing permissions, or the monitor/runbook link is stale. Fix the specific gap and re-test with the same prompt rather than rewriting broadly. +If the answer doesn't reflect what you wrote, check whether the `bits.md` entry is a rule or just description, whether the integration is connected with the right permissions, or whether a link is stale. Fix the specific gap and re-test with the same prompt. -Once Bits reflects the change in chat, re-run a known investigation to confirm it also changes the actual conclusion. Chat and investigations don't always draw on knowledge the same way, so a good chat answer isn't a guarantee the investigation itself improved. +Once chat reflects the change, re-run a known investigation to confirm the conclusion itself improves. Chat and investigations don't always draw on knowledge the same way. [1]: /bits_ai/bits_investigation/knowledge_sources/ [2]: /internal_developer_portal/catalog/ From 3b5a96a387abf12f9e37cd1a6fb210671317ece9 Mon Sep 17 00:00:00 2001 From: Daniel Green Date: Wed, 2 Sep 2026 13:30:59 -0400 Subject: [PATCH 04/10] Remove generic listicle-style intros --- .../en/bits_ai/bits_investigation/getting_the_most_out.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/hugo/content/en/bits_ai/bits_investigation/getting_the_most_out.md b/hugo/content/en/bits_ai/bits_investigation/getting_the_most_out.md index d0a071e01bd..7b9440052b4 100644 --- a/hugo/content/en/bits_ai/bits_investigation/getting_the_most_out.md +++ b/hugo/content/en/bits_ai/bits_investigation/getting_the_most_out.md @@ -26,7 +26,7 @@ You can teach it through four places: [`bits.md`][1], your monitors and runbooks - **Good**: "The billing team's alerts tag the service as `billing-svc`, but APM and logs use `billing_service`. Treat them as the same service." - **Not useful**: "Checkout is our payments service." True, but gives Bits nothing to act on. -What to include: +The entries that matter most: - **Cross-system name mapping.** The same service, environment, or team often has different names in monitors, APM, logs, and any connected ticketing system. Write the mapping down once. - **Known noise.** Patterns that look like incidents but are routine, such as a weekly reindex job or a load test, and when they'd actually count as a real problem. - **Standing scope rules.** How to handle an alert that doesn't specify environment or region. Without a rule, Bits has to guess. @@ -35,7 +35,7 @@ See [Knowledge sources][1] for a full sample file. ## Start with the monitors that matter most -Tuning a monitor takes work: a runbook link, correct tags, a `bits.md` entry for its quirks. Start with monitors where an investigation actually saves time. On the [Supported Monitors][8] page: +A good runbook link, correct tags, and a `bits.md` entry for a monitor's quirks all take time to set up. Start with monitors where an investigation actually saves time. On the [Supported Monitors][8] page: - Filter by [`priority:p1`][9] (or `p2`) for monitors most likely to represent a real incident. - Filter by [`notification:*`][10] for monitors that already page a person or channel. @@ -48,7 +48,7 @@ Bits reads the monitor message at investigation time, so the monitor itself is a - Link the dashboard, log query, or notebook you'd check first. Plain URLs work, no formatting needed. - Use a notebook for anything longer than a link or two. Notebooks mix markdown with live Datadog queries. -- State the blast radius: which downstream services or dependencies are typically affected. +- Note which downstream services or dependencies are typically affected. - Scope or group the monitor query by `service`. This is what lets Bits pivot into APM, logs, RUM, and [Catalog][2] for the right service. Without it, Bits falls back on weaker signals like the monitor name. Review monitor messages periodically. A stale runbook link is worse than no link, since it points Bits at the wrong dashboard or a decommissioned service. From a5e227dddea3b347eefce7f723a7a619b032e520 Mon Sep 17 00:00:00 2001 From: Daniel Green Date: Wed, 2 Sep 2026 13:31:53 -0400 Subject: [PATCH 05/10] Simplify feedback section heading --- .../en/bits_ai/bits_investigation/getting_the_most_out.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hugo/content/en/bits_ai/bits_investigation/getting_the_most_out.md b/hugo/content/en/bits_ai/bits_investigation/getting_the_most_out.md index 7b9440052b4..4a697ee2539 100644 --- a/hugo/content/en/bits_ai/bits_investigation/getting_the_most_out.md +++ b/hugo/content/en/bits_ai/bits_investigation/getting_the_most_out.md @@ -67,7 +67,7 @@ Bits Investigation invokes a skill automatically when its name and description m Write documentation for Bits the way you'd write it for a new hire: name the actual service and system, and spell out remediation steps instead of assuming context. -## Correct it, and let the correction stick +## Give feedback on investigations At the end of an investigation, tell Bits whether the conclusion was right. From 10434ae6820b6d5eb31b59e51dce835ec0290e6c Mon Sep 17 00:00:00 2001 From: Daniel Green Date: Wed, 2 Sep 2026 13:32:17 -0400 Subject: [PATCH 06/10] Simplify remaining section headings --- .../en/bits_ai/bits_investigation/getting_the_most_out.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/hugo/content/en/bits_ai/bits_investigation/getting_the_most_out.md b/hugo/content/en/bits_ai/bits_investigation/getting_the_most_out.md index 4a697ee2539..ed857fd9b00 100644 --- a/hugo/content/en/bits_ai/bits_investigation/getting_the_most_out.md +++ b/hugo/content/en/bits_ai/bits_investigation/getting_the_most_out.md @@ -59,7 +59,7 @@ A multi-step diagnostic procedure, or the specific way your team queries a third Bits Investigation invokes a skill automatically when its name and description match the situation, the same way Bits Code discovers custom skills in your repository. Use a skill when you'd otherwise repeat the same instructions across monitors, or when a procedure needs to stay consistent across Bits Investigation, Bits Chat, and other Bits products. -## Connect the systems where the answer already lives +## Connect external tools and documentation - **Confluence.** [Connect your Confluence account][3] and link relevant pages in monitor messages. Bits extracts telemetry links and troubleshooting steps from the page. Enable account crawling to let [Bits Chat][4] search your Confluence space directly, not just linked pages. - **Source code.** Connect [GitHub][5] and [tag your APM telemetry with Git information][6] so Bits can tie a regression to the commit or deploy that caused it. This also lets Bits Code pick up the investigation and propose a fix. @@ -75,7 +75,7 @@ Confirm what's correct, not just what's wrong; positive feedback still becomes a Both become **memories**, which Bits selectively reuses in similar future investigations. Review or delete them from the {{< ui >}}Memories{{< /ui >}} column on the [Monitor Management][8] page, and check periodically that older corrections still hold (services get renamed, causes get fixed). -## Confirm Bits actually picked it up +## Test your changes with Bits Chat Use [Bits Chat][4] to check your changes by asking about what you wrote, instead of waiting on a full investigation. You can also ask Bits Chat directly for suggestions on how to improve `bits.md`, a runbook, or a skill. From 2b796191bad48b5dc635599978206453b0b62c46 Mon Sep 17 00:00:00 2001 From: Daniel Green Date: Wed, 2 Sep 2026 13:32:32 -0400 Subject: [PATCH 07/10] Rename monitors section heading --- .../en/bits_ai/bits_investigation/getting_the_most_out.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hugo/content/en/bits_ai/bits_investigation/getting_the_most_out.md b/hugo/content/en/bits_ai/bits_investigation/getting_the_most_out.md index ed857fd9b00..628494303e1 100644 --- a/hugo/content/en/bits_ai/bits_investigation/getting_the_most_out.md +++ b/hugo/content/en/bits_ai/bits_investigation/getting_the_most_out.md @@ -33,7 +33,7 @@ The entries that matter most: See [Knowledge sources][1] for a full sample file. -## Start with the monitors that matter most +## Start with your most critical monitors A good runbook link, correct tags, and a `bits.md` entry for a monitor's quirks all take time to set up. Start with monitors where an investigation actually saves time. On the [Supported Monitors][8] page: From c122e705f186eab0e5359c7bed76251d4060809e Mon Sep 17 00:00:00 2001 From: Daniel Green Date: Wed, 2 Sep 2026 13:33:28 -0400 Subject: [PATCH 08/10] Rename bits.md section heading --- .../en/bits_ai/bits_investigation/getting_the_most_out.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hugo/content/en/bits_ai/bits_investigation/getting_the_most_out.md b/hugo/content/en/bits_ai/bits_investigation/getting_the_most_out.md index 628494303e1..9f3cc27985b 100644 --- a/hugo/content/en/bits_ai/bits_investigation/getting_the_most_out.md +++ b/hugo/content/en/bits_ai/bits_investigation/getting_the_most_out.md @@ -19,7 +19,7 @@ Bits Investigation reasons through incomplete signals rather than following a fi You can teach it through four places: [`bits.md`][1], your monitors and runbooks, [Skills][11], and feedback on completed investigations. -## Write bits.md as rules, not background +## Write bits.md as rules, not generic descriptions [`bits.md`][1] is read on every investigation. Write specific rules, not general descriptions of your systems. From 008939d237becd1fddd664dccdfd67d2d29b35b0 Mon Sep 17 00:00:00 2001 From: dgreen15 Date: Wed, 2 Sep 2026 13:39:23 -0400 Subject: [PATCH 09/10] Update getting_the_most_out.md --- .../en/bits_ai/bits_investigation/getting_the_most_out.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/hugo/content/en/bits_ai/bits_investigation/getting_the_most_out.md b/hugo/content/en/bits_ai/bits_investigation/getting_the_most_out.md index 9f3cc27985b..4c485372a43 100644 --- a/hugo/content/en/bits_ai/bits_investigation/getting_the_most_out.md +++ b/hugo/content/en/bits_ai/bits_investigation/getting_the_most_out.md @@ -24,18 +24,18 @@ You can teach it through four places: [`bits.md`][1], your monitors and runbooks [`bits.md`][1] is read on every investigation. Write specific rules, not general descriptions of your systems. - **Good**: "The billing team's alerts tag the service as `billing-svc`, but APM and logs use `billing_service`. Treat them as the same service." -- **Not useful**: "Checkout is our payments service." True, but gives Bits nothing to act on. +- **Not useful**: "Checkout is our payments service." The entries that matter most: - **Cross-system name mapping.** The same service, environment, or team often has different names in monitors, APM, logs, and any connected ticketing system. Write the mapping down once. - **Known noise.** Patterns that look like incidents but are routine, such as a weekly reindex job or a load test, and when they'd actually count as a real problem. -- **Standing scope rules.** How to handle an alert that doesn't specify environment or region. Without a rule, Bits has to guess. +- **Standing scope rules.** How to handle an alert that doesn't specify environment or region. See [Knowledge sources][1] for a full sample file. ## Start with your most critical monitors -A good runbook link, correct tags, and a `bits.md` entry for a monitor's quirks all take time to set up. Start with monitors where an investigation actually saves time. On the [Supported Monitors][8] page: +Start with monitors where an investigation actually saves time by focusing on your most critical alerts. On the [Supported Monitors][8] page: - Filter by [`priority:p1`][9] (or `p2`) for monitors most likely to represent a real incident. - Filter by [`notification:*`][10] for monitors that already page a person or channel. From 082bee58287183e1e616819e4f30e18a5508d7f3 Mon Sep 17 00:00:00 2001 From: dgreen15 Date: Wed, 2 Sep 2026 13:41:28 -0400 Subject: [PATCH 10/10] Update getting_the_most_out.md --- .../en/bits_ai/bits_investigation/getting_the_most_out.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/hugo/content/en/bits_ai/bits_investigation/getting_the_most_out.md b/hugo/content/en/bits_ai/bits_investigation/getting_the_most_out.md index 4c485372a43..e0d2498906a 100644 --- a/hugo/content/en/bits_ai/bits_investigation/getting_the_most_out.md +++ b/hugo/content/en/bits_ai/bits_investigation/getting_the_most_out.md @@ -57,16 +57,15 @@ Review monitor messages periodically. A stale runbook link is worse than no link A multi-step diagnostic procedure, or the specific way your team queries a third-party tool, doesn't fit cleanly in one monitor's runbook or a `bits.md` line. Capture it as a [Skill][11] instead, created at [{{< ui >}}Actions{{< /ui >}} > {{< ui >}}Skills{{< /ui >}}][11]. -Bits Investigation invokes a skill automatically when its name and description match the situation, the same way Bits Code discovers custom skills in your repository. Use a skill when you'd otherwise repeat the same instructions across monitors, or when a procedure needs to stay consistent across Bits Investigation, Bits Chat, and other Bits products. +Bits Investigation invokes a skill automatically when its name and description match the situation. Use a skill when you'd otherwise repeat the same instructions across monitors, or when a procedure needs to stay consistent across Bits Investigation, Bits Chat, and other Bits products. ## Connect external tools and documentation - **Confluence.** [Connect your Confluence account][3] and link relevant pages in monitor messages. Bits extracts telemetry links and troubleshooting steps from the page. Enable account crawling to let [Bits Chat][4] search your Confluence space directly, not just linked pages. - **Source code.** Connect [GitHub][5] and [tag your APM telemetry with Git information][6] so Bits can tie a regression to the commit or deploy that caused it. This also lets Bits Code pick up the investigation and propose a fix. +- **Notifications.** Connect Slack or MS Teams to get Bits' findings posted into your existing ops channels. You can also @Datadog for follow up questions without having to open the web. - **Other observability tools.** Connect Grafana, Dynatrace, Splunk, Sentry, or ServiceNow if telemetry lives there. See [Integrate with third-party observability and SCM platforms][7]. -Write documentation for Bits the way you'd write it for a new hire: name the actual service and system, and spell out remediation steps instead of assuming context. - ## Give feedback on investigations At the end of an investigation, tell Bits whether the conclusion was right.