-
Notifications
You must be signed in to change notification settings - Fork 330
docs: Serverless Workers on AgentCore Runtime #5289
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
84 commits
Select commit
Hold shift + click to select a range
323f016
Document Serverless Workers on AgentCore Runtime
lennessyy c5f26a7
Clarify AgentCore Runtime session lifecycle
lennessyy e8933a8
Explain AgentCore Worker idle shutdown
lennessyy ae4df10
Lead AgentCore lifecycle with Worker shutdown policy
lennessyy 159c66f
Clarify AgentCore idle policy guidance
lennessyy 8ab5cf9
Document AgentCore Worker shutdown policy
lennessyy 573b08d
Share Serverless Worker autoscaling guidance
lennessyy 0c051e2
Add Python AgentCore Serverless Worker guide
lennessyy 3a4455c
Add AgentCore Worker idle policy example
lennessyy 13fcfd4
Use periods in AgentCore prose
lennessyy 0817f16
Make AgentCore idle policy self-contained
lennessyy 9f1d16e
Give an AgentCore Worker idle signal example
lennessyy 702f801
Remove unsupported Workflow Task idle guidance
lennessyy b0e0e5e
Add fixed lease idle policy example
lennessyy a47de75
Remove unsupported AgentCore lease guidance
lennessyy ad43d78
Clarify Python Worker idle detection
lennessyy 9f07493
Explain AgentCore sample ActivityTracker
lennessyy 60d919a
Explain AgentCore Worker retirement policies
lennessyy 162b237
Explain ActivityTracker idle behavior
lennessyy a3752a6
State exact AgentCore idle condition
lennessyy b5d85dc
Focus AgentCore Worker lifecycle examples
lennessyy 167adcb
Sync AgentCore Worker examples from sample
lennessyy ce63708
Tighten AgentCore idle policy context
lennessyy b5c80b8
Lead AgentCore lifecycle guidance with impact
lennessyy be60a1e
Add AgentCore Serverless Worker deployment guide
lennessyy cb56f06
Document durable agents on AgentCore
lennessyy 76f64fe
Clarify durable AgentCore tutorial architecture
lennessyy 3fed519
Condense AgentCore tutorial architecture
lennessyy 3c02beb
Remove AgentCore Memory from tutorial architecture
lennessyy 1540c97
Document background AgentCore Worker startup
lennessyy 747c287
Use Nova Lite in AgentCore tutorial
lennessyy 047880c
Make AgentCore tutorial executable end to end
lennessyy b523327
Base AgentCore tutorial on Python sample
lennessyy 496b52c
Sync AgentCore examples from Python sample
lennessyy 6e85117
Explain AgentCore tutorial design choices
lennessyy dc1fdad
Condense multi-turn AgentCore guidance
lennessyy ee97ffe
Clarify AgentCore deployment guide scope and versioning
lennessyy d5f0265
Clarify AgentCore Runtime deployment configuration
lennessyy 63ef0f9
Use AgentCore reference architecture diagram
lennessyy 3774dfd
Document AgentCore version and IAM limits
lennessyy 404df2d
Merge branch 'main' into docs/agentcore-serverless-workers
lennessyy 856f635
Document Nova Lite for AgentCore sample
lennessyy 33db1e4
Resync AgentCore docs with reference sample
lennessyy 9f41d94
Compare AgentCore Runtime with Lambda
lennessyy d7ec443
Explain Temporal and AgentCore responsibilities
lennessyy cc72ee5
Tighten AgentCore service guidance
lennessyy 2963a6f
Clarify where durable agent progress belongs
lennessyy 543aa2c
Merge branch 'main' into docs/agentcore-serverless-workers
lennessyy 30c510c
Document self-hosted AgentCore setup
lennessyy 5fc479f
Lead AgentCore readers to complete tutorial
lennessyy d9da785
Clarify AgentCore deployment guide scope
lennessyy 9ee2c0d
Lead AgentCore tutorial with agent capability
lennessyy 9a545da
Avoid assuming AgentCore client location
lennessyy 406383f
Explain AgentCore sample reliability concretely
lennessyy 1e673f2
Clarify AgentCore multi-turn guidance
lennessyy 3d3e225
Move AgentCore CLI installation to prerequisites
lennessyy 0afdca5
Align AgentCore CLI prerequisite format
lennessyy dcd128e
Explain AgentCore and client configuration scopes
lennessyy d5fbfaa
Reuse AgentCore runtime environment configuration
lennessyy b283d64
Place environment export within deployment step
lennessyy e32f524
Remove unnecessary AgentCore invocation contrast
lennessyy 6034ed0
Clarify AgentCore log search window
lennessyy dc86779
Use a repeatable tutorial External ID
lennessyy 0f2a071
Clarify AgentCore deployment target
lennessyy 1b2dfce
Merge branch 'main' into docs/agentcore-serverless-workers
lennessyy e4bfba2
Rewrite retired AI Cookbook docs links
lennessyy 9180351
Explain AgentCore Runtime name
lennessyy f073e3c
Clarify AgentCore Worker guide prerequisites
lennessyy c5400a1
Clarify AgentCore tutorial scope
lennessyy 16bc278
Reorder AgentCore Runtime configuration
lennessyy f2c6c7a
Document AgentCore Worker package structure
lennessyy d3bd288
Trim AgentCore Worker implementation note
lennessyy b547b66
Clarify AgentCore endpoint version check
lennessyy 4f8ea81
Clarify AgentCore ARNs and update sample links
lennessyy f089ef5
Clarify AgentCore and Lambda tradeoffs
lennessyy 3af9723
Make AgentCore and Lambda comparison actionable
lennessyy 1afd38e
Link TemporalAgent implementation
lennessyy 707ab4a
Restore provider comparison details
lennessyy fe36b5b
Clarify shared provider durability
lennessyy 29fe6ff
Document AgentCore in Serverless Workers evaluation
lennessyy eacbbf3
Merge branch 'main' into docs/agentcore-serverless-workers
lennessyy 00a0790
Merge remote-tracking branch 'origin/main' into docs/agentcore-server…
lennessyy 9daea16
Merge remote-tracking branch 'origin/docs/agentcore-serverless-worker…
lennessyy 09ecac2
Merge branch 'main' into docs/agentcore-serverless-workers
lennessyy File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
280 changes: 280 additions & 0 deletions
280
docs/develop/python/workers/serverless-workers/agentcore.mdx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,280 @@ | ||
| --- | ||
| id: agentcore | ||
| title: Serverless Workers on Amazon Bedrock AgentCore Runtime - Python SDK | ||
| sidebar_label: Amazon Bedrock AgentCore | ||
| description: Run a Temporal Worker on Amazon Bedrock AgentCore Runtime using the Python SDK. | ||
| slug: /develop/python/workers/serverless-workers/agentcore | ||
| toc_max_heading_level: 4 | ||
| tags: | ||
| - Workers | ||
| - Python SDK | ||
| - Serverless | ||
| - Amazon Bedrock AgentCore | ||
| --- | ||
|
|
||
| import { ReleaseNoteHeader } from '@site/src/components' | ||
|
|
||
| <ReleaseNoteHeader type="prerelease"> | ||
| Amazon Bedrock AgentCore Runtime support is in Pre-release, and its APIs may change in backwards-incompatible ways. | ||
| </ReleaseNoteHeader> | ||
|
|
||
| On Amazon Bedrock AgentCore Runtime, you run a standard long-lived Python Worker inside an AgentCore Runtime handler. | ||
| Temporal starts the handler when the Worker Controller Instance needs capacity. The handler starts a Worker that polls | ||
| the Task Queue, then stops it when your idle policy decides to release capacity. | ||
|
|
||
| The Worker uses the normal Python SDK. The handler uses the `bedrock-agentcore` package to receive AgentCore Runtime | ||
| invocations. | ||
|
|
||
| For the provider behavior, including autoscaling, Worker Versioning, and the Runtime session lifecycle, see | ||
| [Serverless Workers on Amazon Bedrock AgentCore Runtime](/serverless-workers/agentcore). | ||
| For the infrastructure procedure, see | ||
| [Deploy a Serverless Worker on Amazon Bedrock AgentCore Runtime](/production-deployment/worker-deployments/serverless-workers/agentcore). | ||
|
|
||
| ## Install the AgentCore Runtime SDK {/* #install-agentcore-runtime-sdk */} | ||
|
|
||
| Install the AgentCore Runtime SDK alongside the Temporal Python SDK: | ||
|
|
||
| ```bash | ||
| pip install bedrock-agentcore | ||
| ``` | ||
|
|
||
| ## Create a versioned Worker {/* #versioned-worker */} | ||
|
|
||
| Serverless Workers require [Worker Versioning](/worker-versioning). Create the Worker as you would any long-lived | ||
| Python Worker, then set `deployment_config` to declare its Worker Deployment Version and enable versioning: | ||
|
|
||
| ```python | ||
| worker = Worker( | ||
| # ... | ||
| deployment_config=WorkerDeploymentConfig( | ||
| version=WorkerDeploymentVersion( | ||
| deployment_name=DEPLOYMENT_NAME, | ||
| build_id=BUILD_ID, | ||
| ), | ||
| use_worker_versioning=True, | ||
| default_versioning_behavior=VersioningBehavior.PINNED, | ||
| ), | ||
| ) | ||
| ``` | ||
|
|
||
| `TEMPORAL_DEPLOYMENT_NAME` and `TEMPORAL_BUILD_ID` must match the Worker Deployment Version that you create with | ||
| `temporal worker deployment create-version`. Configure that Worker Deployment Version with the AgentCore Runtime | ||
| endpoint that Temporal invokes. For the endpoint configuration, see | ||
| [Worker Versioning](/serverless-workers/agentcore#worker-versioning). | ||
|
|
||
| Every Workflow needs a [versioning behavior](/worker-versioning#versioning-behaviors), either `PINNED` or | ||
| `AUTO_UPGRADE`. Setting `default_versioning_behavior` as shown applies `PINNED` behavior to every Workflow on the | ||
| Worker. To set the behavior per Workflow instead, pass `versioning_behavior` to the `@workflow.defn` decorator. | ||
|
|
||
| ## Start the Worker from the Runtime handler {/* #runtime-handler */} | ||
|
|
||
| AgentCore Runtime invokes an HTTP handler. Use `BedrockAgentCoreApp` to provide that handler. Register the Worker as an | ||
| asynchronous AgentCore task, then return an acknowledgment while the Worker continues polling in the background. The | ||
| sample stores the background task in `_worker` and uses it to prevent another invocation from starting a duplicate | ||
| Worker in the same Runtime session: | ||
|
|
||
| <!--SNIPSTART python-agentcore-runtime-handler--> | ||
| [bedrock_agentcore/strands_agent/agentcore_worker.py](https://github.com/temporalio/samples-python/blob/main/bedrock_agentcore/strands_agent/agentcore_worker.py) | ||
| ```py | ||
| async def run_worker() -> None: | ||
| """Poll until idle, then drain.""" | ||
| api_key = os.environ.get("TEMPORAL_API_KEY") or None | ||
| client = await Client.connect( | ||
| os.environ.get("TEMPORAL_ADDRESS", "localhost:7233"), | ||
| namespace=os.environ.get("TEMPORAL_NAMESPACE", "default"), | ||
| api_key=api_key, | ||
| tls=bool(api_key), | ||
| plugins=[StrandsPlugin()], | ||
| ) | ||
|
|
||
| tracker = ActivityTracker() | ||
| log.info("polling %s as %s/%s", TASK_QUEUE, DEPLOYMENT_NAME, BUILD_ID) | ||
| # execute_code is a sync Activity, so it needs an executor to block on. | ||
| with ThreadPoolExecutor(max_workers=4) as activity_executor: | ||
| worker = Worker( | ||
| client, | ||
| task_queue=TASK_QUEUE, | ||
| workflows=[workflows.StrandsAgentWorkflow], | ||
| activities=[execute_code], | ||
| activity_executor=activity_executor, | ||
| interceptors=[tracker], | ||
| deployment_config=WorkerDeploymentConfig( | ||
| version=WorkerDeploymentVersion( | ||
| deployment_name=DEPLOYMENT_NAME, build_id=BUILD_ID | ||
| ), | ||
| use_worker_versioning=True, | ||
| default_versioning_behavior=VersioningBehavior.PINNED, | ||
| ), | ||
| graceful_shutdown_timeout=DRAIN, | ||
| ) | ||
| async with worker: | ||
| await tracker.wait_until_idle(DEBOUNCE) | ||
| log.info("worker idle for %ss; drained", DEBOUNCE) | ||
|
|
||
|
|
||
| async def _run_until_idle(task_id: int) -> None: | ||
| """Own the Worker's whole life, and always release the async task.""" | ||
| try: | ||
| await run_worker() | ||
| except Exception: | ||
| # Nothing awaits this task, so an error would otherwise be swallowed. | ||
| log.exception("worker failed in async task") | ||
| finally: | ||
| # Without this the session stays HealthyBusy until MaxLifetime. | ||
| app.complete_async_task(task_id) | ||
|
|
||
|
|
||
| @app.entrypoint | ||
| async def invoke(payload: dict) -> dict: | ||
| """Start the Worker and acknowledge. The payload is unused.""" | ||
| # Prevent duplicate workers since we exit early | ||
| global _worker | ||
| if _worker is not None and not _worker.done(): | ||
| log.info("worker already polling %s", TASK_QUEUE) | ||
| return {"message": "worker already polling", "task_queue": TASK_QUEUE} | ||
|
|
||
| task_id = app.add_async_task("temporal-worker") | ||
| _worker = asyncio.create_task(_run_until_idle(task_id)) | ||
|
|
||
| return {"message": "worker starting", "task_queue": TASK_QUEUE} | ||
|
|
||
|
|
||
| ``` | ||
| <!--SNIPEND--> | ||
|
|
||
| The payload does not represent a Workflow input. The Worker Controller Instance invokes the endpoint to add Worker | ||
| capacity. Applications start Workflows through the Temporal Client, as usual. `add_async_task` causes AgentCore to | ||
| report the Runtime as busy while the Worker polls. `complete_async_task` releases that status after the Worker drains | ||
| or fails. | ||
|
|
||
| ## Configure the Temporal connection {/* #configure-connection */} | ||
|
|
||
| The `temporalio.envconfig` package loads [Temporal Client](/develop/python/client/temporal-client) configuration from | ||
| environment variables and an optional TOML configuration file. Set the Temporal address, Namespace, Task Queue, and | ||
| Worker Deployment Version values as Runtime environment variables. Store a Temporal Cloud API key or TLS material in a | ||
| secret store rather than in the Runtime definition. | ||
|
|
||
| For the supported connection variables, config-file format, and profiles, see | ||
| [Environment configuration](/develop/environment-configuration). | ||
|
|
||
| ## Stop and drain the Worker {/* #stop-and-drain-the-worker */} | ||
|
|
||
| AgentCore cannot tell when a Worker that is still polling has no Temporal work. The Runtime remains busy while the | ||
| asynchronous task is registered, so it can remain active until its eight-hour maximum lifetime. To release capacity | ||
| sooner, have the handler detect when the Worker has no useful work and complete the asynchronous task. | ||
|
|
||
| When the condition remains true for an idle period, leave the `async with worker` block. The Worker stops polling for | ||
| new Tasks and gives in-flight Activities time to complete before the Runtime handler returns. | ||
|
|
||
| The following example from the | ||
| [AgentCore sample Worker](https://github.com/temporalio/samples-python/blob/9d5c46bed0f0f6f8a726fa91f371c4c83f232ba2/bedrock_agentcore/strands_agent/agentcore_worker.py) | ||
| defines an `ActivityTracker`. It uses an [Activity inbound Interceptor](/develop/python/workers/interceptors) to count | ||
| running Activities. | ||
|
|
||
| <!--SNIPSTART python-agentcore-activity-tracker--> | ||
| [bedrock_agentcore/strands_agent/agentcore_worker.py](https://github.com/temporalio/samples-python/blob/main/bedrock_agentcore/strands_agent/agentcore_worker.py) | ||
| ```py | ||
| # How long the Worker keeps polling after it goes idle. | ||
| DEBOUNCE = float(os.environ.get("AGENTCORE_DEBOUNCE_SECONDS", "60")) | ||
| # How long the drain waits for in-flight Activities (a model or tool call). | ||
| DRAIN = timedelta(seconds=120) | ||
|
|
||
|
|
||
| class ActivityTracker(Interceptor): | ||
| """Tracks in-flight activities and blocks until AGENTCORE_DEBOUNCE_SECONDS elapses with no events.""" | ||
|
|
||
| def __init__(self) -> None: | ||
| self.inflight = 0 | ||
| self.changed = asyncio.Event() | ||
|
|
||
| def intercept_activity( | ||
| self, next: ActivityInboundInterceptor | ||
| ) -> ActivityInboundInterceptor: | ||
| return _TrackedActivity(next, self) | ||
|
|
||
| async def wait_until_idle(self, debounce: float) -> None: | ||
| """Return once no Activity has run for ``debounce`` seconds.""" | ||
| while True: | ||
| self.changed.clear() | ||
| try: | ||
| # Wake the moment an Activity starts or finishes; a timeout | ||
| # instead means nothing has happened for the whole window. | ||
| await asyncio.wait_for(self.changed.wait(), timeout=debounce) | ||
| except asyncio.TimeoutError: | ||
| if self.inflight == 0: | ||
| return | ||
|
|
||
|
|
||
| class _TrackedActivity(ActivityInboundInterceptor): | ||
| def __init__( | ||
| self, next: ActivityInboundInterceptor, tracker: ActivityTracker | ||
| ) -> None: | ||
| super().__init__(next) | ||
| self._tracker = tracker | ||
|
|
||
| async def execute_activity(self, input: ExecuteActivityInput): | ||
| self._tracker.inflight += 1 | ||
| self._tracker.changed.set() | ||
| log.info("activity in flight: %d", self._tracker.inflight) | ||
| try: | ||
| return await self.next.execute_activity(input) | ||
| finally: | ||
| self._tracker.inflight -= 1 | ||
| self._tracker.changed.set() | ||
|
|
||
|
|
||
| ``` | ||
| <!--SNIPEND--> | ||
|
|
||
| Register the tracker as a Worker Interceptor and wait for it inside the Worker context: | ||
|
|
||
| ```python | ||
| tracker = ActivityTracker() | ||
| worker = Worker( | ||
| client, | ||
| # ... | ||
| interceptors=[tracker], | ||
| graceful_shutdown_timeout=DRAIN, | ||
| ) | ||
|
|
||
| async with worker: | ||
| await tracker.wait_until_idle(DEBOUNCE) | ||
| ``` | ||
|
|
||
| `ActivityTracker` retires the Worker only after 60 seconds without an Activity starting or completing and with no | ||
| Activity running. A long-running Activity keeps the count above zero, so the idle policy does not interrupt it. The | ||
| two-minute `graceful_shutdown_timeout` is a safety limit for any Activity still in flight when shutdown starts. | ||
|
|
||
| Memory pressure can be another retirement condition. For example, the Runtime handler can monitor process memory and | ||
| initiate the same graceful shutdown when usage crosses a threshold. Memory usage is not an idle signal. It tells you | ||
| when to recycle a Worker, not whether it has work to do. Test any memory-based policy against the Runtime's memory | ||
| limit and your Activity retry behavior. | ||
|
|
||
| `AGENTCORE_DEBOUNCE_SECONDS` controls the idle period. `graceful_shutdown_timeout` controls how long the Worker waits | ||
| for in-flight Activities after it stops polling. Choose both values for your workload, and account for AgentCore's | ||
| maximum Runtime lifetime. For the AgentCore lifecycle settings, see | ||
| [Lifecycle](/serverless-workers/agentcore#lifecycle). | ||
|
|
||
| ## Keep Activities safe across Worker termination {/* #activity-recovery */} | ||
|
|
||
| AgentCore can end the compute that runs a Worker. An Activity running at that time can be interrupted and retried. | ||
| Use [Activity Heartbeats](/develop/python/activities/timeouts#activity-heartbeats) so a retry resumes from its last | ||
| recorded progress instead of starting over: | ||
|
|
||
| ```python | ||
| from temporalio import activity | ||
|
|
||
|
|
||
| @activity.defn | ||
| async def my_activity(items: list[str]) -> str: | ||
| for i, item in enumerate(items): | ||
| activity.heartbeat(i) | ||
| # ... process item | ||
| return "done" | ||
| ``` | ||
|
|
||
| ## Add observability {/* #add-observability */} | ||
|
|
||
| An AgentCore Runtime Worker emits the same traces and metrics as a Worker on other compute. For metrics export and | ||
| OpenTelemetry tracing interceptors, see [Observability - Python SDK](/develop/python/platform/observability) and the | ||
| [SDK metrics reference](/references/sdk-metrics). | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.