Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/helm-e2e.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,7 @@ jobs:
--namespace ate-system \
--create-namespace \
-f internal/e2e/suites/credentials/values.yaml \
-f internal/e2e/suites/parking/values.yaml \
--set postgres.clientCertificates.enabled=true \
--set image.registry=localhost:5001 \
--set image.tag=helm-e2e \
Expand Down
27 changes: 26 additions & 1 deletion charts/substrate/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,8 @@ See `values.yaml` for the full set; the important keys:
| `postgres.schema` | `substrate` | Store the Substrate tables in this PostgreSQL schema |
| `postgres.clientCertificates.enabled` | `false` | Project separate owner and runtime login certificates into the API server |
| `<component>.resources` | see `values.yaml` | Requests and limits for each container, e.g. `ateApi.resources` and `atenetRouter.agentgateway.resources` |
| `atenetRouter.requestParking.budget` | `5s` | Maximum wait for a suspended actor to become routable, e.g. `30s` for slower cold starts |
| `atenetRouter.requestParking.budget` | `60s` | Total budget for actor resolution and resume retries |
| `atenetRouter.requestTimeout` | `60s` | Timeout for each Substrate ingress API call, including actor resolution |
| `atenetEgress.agentgateway.logLevel` | `info` | Egress proxy `RUST_LOG` filter, e.g. `debug` or `info,agentgateway=debug` when troubleshooting |
| `rustfs.enabled` | `true` | Deploy an in-cluster S3-compatible RustFS bucket for snapshots |
| `ateApi.snapshotLocation` | `""` | Cluster-wide snapshot URI; defaults to `gs://<rustfs.bucket>/` with bundled RustFS |
Expand All @@ -85,6 +86,30 @@ See `values.yaml` for the full set; the important keys:
| `otel.logs.enabled` | `true` | Enable OTLP actor events from ateapi and the ateworkers, plus the router access log. Actor events go to stdout when OTLP logs are disabled |
| `otel.logs.endpoint` | `""` | OTLP endpoint for logs, overriding `otel.endpoint` |

## Router timeouts

The router uses `ResumeActor` to resolve actor assignments, including actors
that already run. `atenetRouter.requestTimeout` sets the HTTP backend request
timeout for those API calls. `atenetRouter.requestParking.budget` bounds the
resume attempts and retry delays. Both apply to direct and CONNECT-tunneled
HTTP/gRPC routes. Both default to `60s`, allowing each attempt to use the
remaining parking budget.

A resume must finish within both the per-call timeout and the remaining parking
budget. A shorter client deadline can still cancel it. Set both values as
duration strings when slow restores need more time, for example:

```yaml
atenetRouter:
requestParking:
budget: "120s"
requestTimeout: "120s"
```

Choose limits using measured restore latency. A parking budget of `0s` uses
agentgateway's default budget; it does not disable the limit. These settings
govern API calls during routing, not the actor's application request duration.

## Upgrading sandbox and snapshot configuration

Public APIs now use versioned packages under `api/`. Upgrade clients, control
Expand Down
2 changes: 2 additions & 0 deletions charts/substrate/templates/atenet-router.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,8 @@ data:
# AgentGateway only uses atunnel's CONNECT listener.
connectTargetPort: 8443
policies:
http:
requestTimeout: {{ .Values.atenetRouter.requestTimeout | quote }}
backendTLS:
cert: /run/podidentity.podcert.ate.dev/credential-bundle.pem
key: /run/podidentity.podcert.ate.dev/credential-bundle.pem
Expand Down
53 changes: 51 additions & 2 deletions charts/substrate/tests/control_plane_test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -398,15 +398,18 @@ tests:
- matchRegex:
path: data["config.yaml"]
pattern: 'source.spiffeId in \["spiffe://cluster.local/ns/client/sa/router-client"\]'
- it: keeps the router request parking budget at five seconds by default
- it: defaults router parking and API request timeouts to sixty seconds
template: atenet-router.yaml
documentSelector:
path: kind
value: ConfigMap
asserts:
- matchRegex:
path: data["config.yaml"]
pattern: 'requestParking:\n\s+budget: "5s"'
pattern: 'requestParking:\n\s+budget: "60s"'
- matchRegex:
path: data["config.yaml"]
pattern: 'policies:\n\s+http:\n\s+requestTimeout: "60s"'
- it: configures the router request parking budget
template: atenet-router.yaml
set:
Expand All @@ -418,6 +421,52 @@ tests:
- matchRegex:
path: data["config.yaml"]
pattern: 'requestParking:\n\s+budget: "30s"'
- matchRegex:
path: data["config.yaml"]
pattern: 'policies:\n\s+http:\n\s+requestTimeout: "60s"'
- it: configures the router API request timeout independently
template: atenet-router.yaml
set:
atenetRouter.requestTimeout: 2m
documentSelector:
path: kind
value: ConfigMap
asserts:
- matchRegex:
path: data["config.yaml"]
pattern: 'requestParking:\n\s+budget: "60s"'
- matchRegex:
path: data["config.yaml"]
pattern: 'policies:\n\s+http:\n\s+requestTimeout: "2m"'
- it: configures both router timeouts
template: atenet-router.yaml
set:
atenetRouter.requestParking.budget: 90s
atenetRouter.requestTimeout: 45s
documentSelector:
path: kind
value: ConfigMap
asserts:
- matchRegex:
path: data["config.yaml"]
pattern: 'requestParking:\n\s+budget: "90s"'
- matchRegex:
path: data["config.yaml"]
pattern: 'policies:\n\s+http:\n\s+requestTimeout: "45s"'
- it: keeps a five-second parking budget in the E2E fixture
template: atenet-router.yaml
values:
- ../../../internal/e2e/suites/parking/values.yaml
documentSelector:
path: kind
value: ConfigMap
asserts:
- matchRegex:
path: data["config.yaml"]
pattern: 'requestParking:\n\s+budget: "5s"'
- matchRegex:
path: data["config.yaml"]
pattern: 'policies:\n\s+http:\n\s+requestTimeout: "60s"'
- it: defaults the egress proxy log filter to info
template: atenet-egress.yaml
documentSelector:
Expand Down
4 changes: 3 additions & 1 deletion charts/substrate/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -148,7 +148,9 @@ ateController:
atenetRouter:
requestParking:
# Maximum time to wait for a suspended actor to become routable.
budget: 5s
budget: "60s"
# Timeout for each Substrate ingress API call, including actor resolution.
requestTimeout: "60s"
# mTLS client identities allowed to reach actors. Empty denies all ingress.
ingressAllowedSPIFFEIDs: []
agentgateway:
Expand Down
6 changes: 3 additions & 3 deletions internal/e2e/suites/parking/parking_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,8 @@
// Envoy → ext_proc → ateapi → worker path: a deliberately 1-worker pool is
// oversubscribed by two actors, so a request for the suspended actor parks
// until the worker frees (ParkThenServed) or the park budget elapses
// (BudgetExhaustion). It runs with the router's default parking configuration
// (budget 5s); flag-dependent behavior (lot-full shed, parking disabled,
// (BudgetExhaustion). It requires a 5s parking budget. Flag-dependent behavior
// (lot-full shed, parking disabled,
// custom budgets) is covered by unit tests instead, because the shared router
// cannot be reconfigured per test.
package parking
Expand All @@ -39,7 +39,7 @@ import (
// k8s namespace holding its pool.
var parkingAtespace = e2e.FixtureName("ate-e2e") + "-parking"

// The park budget the deployed router runs with (its flag default). The
// The park budget the deployed test router runs with. The
// timing assertions below are windows around it, wide enough for scheduling
// jitter but narrow enough to prove parking happened and that the router —
// not an Envoy timeout — produced the verdict.
Expand Down
18 changes: 18 additions & 0 deletions internal/e2e/suites/parking/values.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# Copyright 2026 The Agent Substrate Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# BudgetExhaustion checks a five-second parking window.
atenetRouter:
requestParking:
budget: "5s"
Loading