docs: L7 logs restructure and eBPF collector#2724
Open
lucastigera wants to merge 5 commits into
Open
Conversation
New L7 observability pages for the eBPF-based collector (tigera/calico-private#11749): - configure-bpf.mdx: enable via L7ObservabilityEnabled on FelixConfiguration; comparison with the Istio Waypoint collector - configure-istio-waypoint.mdx: placeholder - datatypes-wip.mdx: redraft of the L7 schema with a Populated by column and new protocol / collector_name / tls_* fields - aggregation.mdx: aggregation key fields with an interactive playground component - sidebar entries for all four pages Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
✅ Deploy Preview for calico-docs-preview-next ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
❌ Deploy Preview for tigera failed. Why did it fail? →Built without sensitive environment variables
|
Contributor
There was a problem hiding this comment.
Pull request overview
This PR (marked WIP) introduces draft Calico Enterprise documentation for a new eBPF-based L7 observability collector, including a new L7 log aggregation page with an interactive sandbox component, and wires the new pages into the Enterprise sidebar navigation.
Changes:
- Adds new draft L7 documentation pages for eBPF configuration, Istio Waypoint (placeholder), aggregation, and an expanded (WIP) L7 schema reference.
- Introduces a new React component (
L7AggregationSandbox) used to provide an interactive aggregation-key playground in docs. - Updates the Calico Enterprise sidebar to surface the new L7 pages.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| sidebars-calico-enterprise.js | Adds the new L7 doc pages to the Enterprise “L7 logs” sidebar category. |
| calico-enterprise/observability/elastic/l7/configure-bpf.mdx | Draft guide for enabling/viewing eBPF-based L7 logs, including a comparison table and limitations. |
| calico-enterprise/observability/elastic/l7/configure-istio-waypoint.mdx | Placeholder page scaffold for Istio Ambient/Waypoint-based L7 collection. |
| calico-enterprise/observability/elastic/l7/aggregation.mdx | Draft documentation describing FelixConfiguration aggregation fields and embedding an interactive sandbox. |
| calico-enterprise/observability/elastic/l7/datatypes-wip.mdx | Draft schema reference adding eBPF/TLS fields and a “Populated by” column. |
| calico-enterprise/_includes/components/L7AggregationSandbox/index.js | New interactive React component that demonstrates how aggregation settings merge L7 log rows and update count. |
Comments suppressed due to low confidence (4)
calico-enterprise/observability/elastic/l7/configure-bpf.mdx:70
- Same issue as above:
L7LogsFileDirectoryis a FelixConfiguration (resource) field and should link toreference/resources/felixconfig.mdx#l7LogsFileDirectory(note the lowercaselin the anchor) rather than the Felix config file/env-var page.
The eBPF collector writes L7 events as JSON to a log file on each node. By default the file is at `/var/log/calico/l7logs/l7.log`. The directory is controlled by the [L7LogsFileDirectory](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileDirectory) field on `FelixConfiguration`.
calico-enterprise/observability/elastic/l7/aggregation.mdx:25
- The per-field links here target
.../component-resources/node/felix/configuration.mdx#L7Logs..., but the generated anchors for FelixConfiguration fields use the YAML field names (for example#l7LogsFileAggregationSourceInfo) and live onreference/resources/felixconfig.mdx. Please update the doc path and anchor casing for these aggregation field links so they resolve.
| [L7LogsFileAggregationSourceInfo](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationSourceInfo) | Source aggregated name, namespace, and type appear on each row. | Requests are merged across sources. |
| [L7LogsFileAggregationDestinationInfo](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationDestinationInfo) | Destination aggregated name, namespace, and type appear on each row. | Requests are merged across destinations. |
| [L7LogsFileAggregationServiceInfo](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationServiceInfo) | Destination service name, namespace, and port appear on each row. | Requests are merged across services. |
| [L7LogsFileAggregationHTTPMethod](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationHTTPMethod) | HTTP method appears on each row. | Requests are merged across methods. |
| [L7LogsFileAggregationTrimURL](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationTrimURL) | URL (or a trimmed prefix) appears on each row. Intermediate values trim the query string, then the path, before excluding the URL entirely. | Requests are merged across URLs. |
calico-enterprise/observability/elastic/l7/aggregation.mdx:28
- These links have the same issue as above (wrong target doc + uppercase anchors). Also,
L7LogsFileAggregationTLSSNIdoes not appear in the currentconfig-params.jsondata set, so the anchor will remain broken until the FelixConfig JSON is regenerated to include it.
| [L7LogsFileAggregationResponseCode](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationResponseCode) | Response code appears on each row. | Requests are merged across response codes. |
| [L7LogsFileAggregationHTTPHeaderInfo](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationHTTPHeaderInfo) | User agent and request type appear on each row. | Requests are merged across user agents and types. |
| [L7LogsFileAggregationTLSSNI](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationTLSSNI) | TLS Server Name Indication appears on each TLS row. | TLS flows to the same destination are merged regardless of SNI. |
calico-enterprise/observability/elastic/l7/aggregation.mdx:35
- The second table here also starts rows with
||, which will render as an extra empty column. Use a single leading|for Markdown tables.
| Field | Description |
| --- | --- |
| [L7LogsFileAggregationNumURLPath](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationNumURLPath) | Maximum number of URL path components retained. Default `5`. A negative value disables this truncation. |
| [L7LogsFileAggregationURLCharLimit](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationURLCharLimit) | Maximum URL length in characters; longer URLs are sliced. Default `250`. |
| | Additional infrastructure | Waypoint Envoy pods per namespace/service | None — runs inside Felix | | ||
| | L7 coverage | Only traffic routed through a waypoint | All TCP traffic on the node | | ||
| | Network path overhead | Two extra proxy hops | None | | ||
| | Plaintext access | Decrypted at the waypoint (mTLS terminated) | Captured at the kernel TCP layer, before TLS encryption | |
|
|
||
| ## Configure L7 logs | ||
|
|
||
| Enable the eBPF L7 collector by setting the [L7ObservabilityEnabled](../../../reference/component-resources/node/felix/configuration.mdx#L7ObservabilityEnabled) field to `true` on the default `FelixConfiguration`. The setting is independent of `bpfEnabled` — the collector works with any dataplane. |
Comment on lines
+17
to
+35
| The following `FelixConfiguration` fields control the L7 log aggregation key. For accepted values and defaults, follow the link to the [Felix Configuration reference](../../../reference/component-resources/node/felix/configuration.mdx#l7-logs). | ||
|
|
||
| | Field | Effect when included | Effect when excluded | | ||
| | --- | --- | --- | | ||
| | [L7LogsFileAggregationSourceInfo](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationSourceInfo) | Source aggregated name, namespace, and type appear on each row. | Requests are merged across sources. | | ||
| | [L7LogsFileAggregationDestinationInfo](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationDestinationInfo) | Destination aggregated name, namespace, and type appear on each row. | Requests are merged across destinations. | | ||
| | [L7LogsFileAggregationServiceInfo](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationServiceInfo) | Destination service name, namespace, and port appear on each row. | Requests are merged across services. | | ||
| | [L7LogsFileAggregationHTTPMethod](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationHTTPMethod) | HTTP method appears on each row. | Requests are merged across methods. | | ||
| | [L7LogsFileAggregationTrimURL](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationTrimURL) | URL (or a trimmed prefix) appears on each row. Intermediate values trim the query string, then the path, before excluding the URL entirely. | Requests are merged across URLs. | | ||
| | [L7LogsFileAggregationResponseCode](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationResponseCode) | Response code appears on each row. | Requests are merged across response codes. | | ||
| | [L7LogsFileAggregationHTTPHeaderInfo](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationHTTPHeaderInfo) | User agent and request type appear on each row. | Requests are merged across user agents and types. | | ||
| | [L7LogsFileAggregationTLSSNI](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationTLSSNI) | TLS Server Name Indication appears on each TLS row. | TLS flows to the same destination are merged regardless of SNI. | | ||
|
|
||
| Two additional fields control how the URL is truncated before it enters the aggregation key, rather than whether the URL participates at all: | ||
|
|
||
| | Field | Description | | ||
| | --- | --- | | ||
| | [L7LogsFileAggregationNumURLPath](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationNumURLPath) | Maximum number of URL path components retained. Default `5`. A negative value disables this truncation. | | ||
| | [L7LogsFileAggregationURLCharLimit](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationURLCharLimit) | Maximum URL length in characters; longer URLs are sliced. Default `250`. | |
Comment on lines
+408
to
+411
| 'observability/elastic/l7/configure-istio-waypoint', | ||
| 'observability/elastic/l7/aggregation', | ||
| 'observability/elastic/l7/datatypes', | ||
| 'observability/elastic/l7/datatypes-wip', |
Comment on lines
+29
to
+35
| | Aspect | Istio Waypoint collector | eBPF L7 collector | | ||
| | --- | --- | --- | | ||
| | Additional infrastructure | Waypoint Envoy pods per namespace/service | None — runs inside Felix | | ||
| | L7 coverage | Only traffic routed through a waypoint | All TCP traffic on the node | | ||
| | Network path overhead | Two extra proxy hops | None | | ||
| | Plaintext access | Decrypted at the waypoint (mTLS terminated) | Captured at the kernel TCP layer, before TLS encryption | | ||
| | Protocol coverage | HTTP/1, HTTP/2, gRPC | HTTP/1.0, HTTP/1.1, TLS handshake metadata | |
Comment on lines
+19
to
+25
| | Field | Effect when included | Effect when excluded | | ||
| | --- | --- | --- | | ||
| | [L7LogsFileAggregationSourceInfo](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationSourceInfo) | Source aggregated name, namespace, and type appear on each row. | Requests are merged across sources. | | ||
| | [L7LogsFileAggregationDestinationInfo](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationDestinationInfo) | Destination aggregated name, namespace, and type appear on each row. | Requests are merged across destinations. | | ||
| | [L7LogsFileAggregationServiceInfo](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationServiceInfo) | Destination service name, namespace, and port appear on each row. | Requests are merged across services. | | ||
| | [L7LogsFileAggregationHTTPMethod](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationHTTPMethod) | HTTP method appears on each row. | Requests are merged across methods. | | ||
| | [L7LogsFileAggregationTrimURL](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationTrimURL) | URL (or a trimmed prefix) appears on each row. Intermediate values trim the query string, then the path, before excluding the URL entirely. | Requests are merged across URLs. | |
Comment on lines
+29
to
+33
| | Name | Datatype | Populated by | Description | | ||
| | ------------------------ | -------- | ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | ||
| | `host` | keyword | All | Name of the node that collected the L7 log entry. | | ||
| | `start_time` | date | All | Start time of log collection in Unix timestamp format. | | ||
| | `end_time` | date | All | End time of log collection in Unix timestamp format. | |
Aggregation sandbox redesign: - Group rows by aggregation bucket; expand/collapse to reveal merged events - Two-tone highlight (green for added/count up, red for removed/count down) - Color legend above the table; intro/footer condensed - More diverse sample data spanning cart, payments, and TLS scenarios Pages: - Promote the WIP data types page to datatypes.mdx; move the original to datatypes-legacy.mdx - Add collector_type and protocol_version fields - Sidebar reordered with "+ " prefix on the new pages and shorter labels (Configure with eBPF, Configure with Istio Waypoint, Aggregation, Data types) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- dataplane / dataplanes → data plane / data planes - waypoint → Waypoint (Calico term rule) - Plaintext / plaintext → Plain text / plain text - Rephrase HTTP pipelining bullet to drop dictionary-rejected words Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Add Overview page (overview.mdx) with big picture, value, collectors, and a comparison table — moves the shared framing out of per-collector pages - Rename configure-bpf.mdx → enable-ebpf.mdx, refocus on enablement steps - Rename configure-istio-waypoint.mdx → enable-waypoint.mdx (placeholder) - Drop datatypes-legacy.mdx (no inbound references) - Sidebar labels: Overview, Enable eBPF/Waypoint/Envoy collector, Data types, Aggregation - Datatypes page: HTTP-only / TLS-only fields tagged inline in the description - Repoint external "L7 logs" links (dashboards, get-started-cem, kibana) to overview.mdx Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Istio Waypoint collector will land in a separate PR. For now: - enable-waypoint.mdx: add draft: true; Docusaurus excludes it from production builds and auto-filters the sidebar entry - overview.mdx, datatypes.mdx: remove inline Waypoint references; leave TODO notes in single-line JSX comments so the next-PR author can find the spots - overview.mdx: drop the Istio Waypoint column from the comparison table for this PR - datatypes.mdx: change "Populated by" from "eBPF, Waypoint" to "eBPF" Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
lucastigera
changed the title
lucastigera
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Restructures the L7 logs docs section to introduce the new eBPF-based collector alongside the existing Envoy collector, with a shared overview and per-collector enable pages.
The Istio Waypoint collector will land in a follow-up PR — its placeholder page is set to
draft: trueso it's visible during local development but excluded from production builds.L7 logs section, after this PR
overview.mdxenable-ebpf.mdxL7ObservabilityEnabledonFelixConfiguration. Limitations: kernel 5.17+, HTTP/1.0–1.1 only, TLS metadata only.enable-waypoint.mdxdraft: trueso it's hidden in production builds. Content lands in the Waypoint PR.configure.mdxdatatypes.mdxprotocol,protocol_version,collector_type,collector_name, andtls_*fields. HTTP-only / TLS-only fields tagged inline (e.g._HTTP only._ URL that the request was made against.).aggregation.mdxL7LogsFileAggregation*fields (including the newL7LogsFileAggregationTLSSNI) and includes an interactive aggregation playground — a React component (_includes/components/L7AggregationSandbox) where readers toggle aggregator checkboxes and watch sample events regroup with two-tone highlight (green = added / count up, red = removed / count down) and per-group expand/collapse.Other changes
dashboards.mdx,get-started-cem.mdx,kibana.mdx: external "L7 logs" links repointed fromconfigure.mdx(Envoy-only) to the newoverview.mdx.alp.mdx: left pointing atconfigure.mdx#kibanaper maintainer note (page will be deprecated).data plane,Waypoint,plain text).Open items / known gaps
L7ObservabilityEnabledandL7LogsFileAggregationTLSSNIwon't render in the Felix Configuration tables until_includes/components/FelixConfig/config-params.jsonis regenerated from upstream calico-private. Field-level anchors (#L7ObservabilityEnabled,#L7LogsFileAggregationTLSSNI) will activate at that time.nexttree (calico-enterprise/) only.Test plan
yarn start-next) compiles and renders all six pages..docusaurusroute data.configure-bpf.mdx,configure-istio-waypoint.mdx,datatypes-wip.mdx,datatypes-legacy.mdx).yarn build-next) excludesenable-waypoint.mdx— to be verified by reviewers.🤖 Generated with Claude Code