Feat/gateway v1#37
Merged
Merged
Conversation
Introduces a central broker that fully isolates modules — they only
talk to the Gateway + Redis, never to each other directly.
RPCs:
- StartStream: Module B requests execution of Module A (ACK + task_id)
- ProduceStream: BiDi for Module A to emit output / receive input,
persisted to Redis Streams
- ConsumeStream: BiDi for Module B to read Module A's output and send
data back through the Gateway
- SendSignal: out-of-band control signals (cancel, pause) via Redis
pub/sub
Gateway injects ModuleStartInfo as the first entry of the Redis
output stream.
refactor(gateway)!: simplify to single Stream RPC and add cache-invalidation signals
Collapse ProduceStream and ConsumeStream into a single
Stream(StreamRequest) -> stream StreamOutput between consumer and
Gateway. Producers no longer connect back through the Gateway; outputs
land in Redis and are replayed to consumers in seq order.
- Drop ProduceStream / ConsumeStream RPCs and their message families
(GatewayResponse, StreamStatus, StreamError, ServerHeartbeat).
- Lifecycle rides as sentinels in StreamOutput.data (module_start_info
first, end_of_stream last); StreamState enum removed.
- Document stateless (from_seq=0) vs. stateful (from_seq=highest_seq)
consumer resume contract, gap detection, and Redis retention bounds.
- Errors surface only via gRPC status codes.
- Extend SignalAction with server-wide INVALIDATE_ALL / _CHANNELS /
_MODELS / _SETUP / _TOOLS / _SHARED; drop PAUSE; strip SIGNAL_ACTION_
prefix. Renumber StartStream{Request,Response} fields; rename
Module A/B to producer/consumer in docstrings.
Other proto changes bundled here: remove deprecated module_registry/v1
and task_manager/v1 packages (discovery/registration moved into
registry/v1); minor touch-ups across cost, filesystem, module
(information, monitoring, service), setup, storage, user_profile, and
buf.lock.
BREAKING CHANGE: removes ProduceStream/ConsumeStream RPCs and the
GatewayResponse/StreamStatus/StreamError/ServerHeartbeat/StreamState
messages; renames SIGNAL_ACTION_* enum values; renumbers StartStream
request/response fields; deletes module_registry.v1 and task_manager.v1.
feat(gateway): rename the stream reponse/request
feat(gateway): add task_id to the streamoutput
feat(gateway): update amp to 1.0.0.dev2 - fix stream rpc
feat(proto): add user info and secret retrieval messages and RPCs
feat(user_profile): add CheckResourceAccess RPC
Add a CheckResourceAccess RPC to UserProfileService that returns whether
the calling task (resolved from the request metadata) is authorized to
access a given resource.
Access is reported as a boolean flag with an OK status rather than a
PERMISSION_DENIED error, keeping "denied" a normal answer and reserving
non-OK statuses for actual failures. Introduces the ResourceType enum and
the CheckResourceAccess request/response messages.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
feat(gateway): add AssociateTask RPC to mint sub-task_ids
Add GatewayService.AssociateTask, which generates and registers a new
sub-task_id linked to a running parent task (and its mission) before it
is dispatched via StartStream. Introduces AssociateTaskRequest
(parent_task_id) and AssociateTaskResponse (task_id, parent_task_id).
Also apply buf format normalization to filesystem, registry, and setup
protos (single-line field options, import ordering, 2-space indent).
feat(context): allow user and organisation scoping in filesystem and storage
Extend the context validation patterns to accept `users:` and
`organizations:` prefixes (in addition to missions/setups) for filesystem
file queries (FileFilter, GetFilesRequest) and storage ListRecordsRequest.
DeleteFilesRequest stays restricted to missions/setups.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.