feat: add multi-cluster setup for ecs casting - #141
Draft
Nageshbansal wants to merge 41 commits into
Draft
Conversation
- resource ref: no fabricated default, apiVersion enum, name pattern - reject absent resource kind before the merge can default it to Installation - reject unsupported consumer kinds (self-reference) at load - drop the no-op resource molding: the kind has zero moldings - rework tests to the Subject_Outcome table idiom; add loader coverage
… terraform casting
Nageshbansal
marked this pull request as ready for review
July 22, 2026 11:07
…e molding Persistence moves onto the node groups: each group declares whether its nodes persist data, and the standalone storage block goes away. The molding now merges contributions (enricher deltas, operator overrides) at the document level, so casting-specific keys survive, and validates the shared shape of the resolved document before storing it. A contribution owns the node groups it states.
Castings deposit their outputs into a pourer instead of returning materials, matching the CollectionAgent contract: the planner builds the pourer for the kind's output directory, the casting adds content under relative paths, and the planner pours the staged entries into materials. Enrichers follow the established shape: a stateless named type with a constructor.
ListTypeSet and ListTypeOrdered both degrade to atomic on a list of maps, so nothing could merge a list of objects: an override stating one element replaced the whole list and silently dropped the rest. ListTypeMap(key) mirrors Kubernetes' listType: map with a listMapKey. Elements match on the key, a matched pair merges as a document so the override states only what it changes, and override-only elements append. It degrades to atomic when an element is not a map or lacks the key.
The requirement document invented its own words for a machine pool, and
one of them meant two different things: `disk` was the data volume on a
persistent group and the root volume on an ephemeral one, with different
minimums. There was also no way to name a machine type in the casting,
only in a terraform variable.
Adopt the vocabulary every node-pool abstraction already uses. kOps is
the closest precedent -- the one machine-pool shape that spans AWS, GCE,
Azure and DigitalOcean -- and where it had to choose it picked
machineType, minSize and maxSize, which overlap with GKE's and eksctl's
terms too. Narrowed to what foundry has to understand:
nodeGroups:
- name: persistent
storage: persistent
minSize: 3
maxSize: 3
machineType: "" # empty resolves from cpu/memory
cpu: 2
memory: 8
rootVolume: {size: 30}
dataVolume: {size: 50}
cpu and memory stay as the portable criteria so the document does not
name provider machine types, with machineType as the escape hatch. Volumes
are nested objects rather than flat scalars so volume type or encryption
can arrive without new top-level keys, and splitting them lets the AMI
snapshot floor apply to every root disk unconditionally. dataVolume is
singular, not a list as kOps has it: the claim controller assumes one
volume per node, and a list would advertise what foundry cannot honour.
StorageClass is a value object rather than a string with an enum tag, so
an unknown class fails at unmarshal instead of deep in the molding, and
each class carries what it implies -- whether it needs a data volume,
whether the group is pinned. Adding a class is one var entry rather than
another branch wherever groups are checked.
Node groups now merge by name, so a contribution or an operator override
states only the group and the fields it changes. Before this, stating one
group deleted the others. `spec.resource.spec.config.data` merges last,
so an operator's own document beats the casting's contribution.
The scaffolding provides the Kind, the casting contract, the resource molding and the registry; a concrete casting is a separate change with its own review. Carrying one here made the scaffolding PR argue for an EKS substrate at the same time as the machinery that hosts it. The registry is left empty. The first casting to register lands with the platform it serves.
Foundry generates rather than reconciles, so it can never ask a platform what it created. A consuming casting finds a producing casting's resources only by deriving the same names and filtering the same tags, with nothing between them to reconcile a mismatch. Derive both from one description of a resource, so a fact stated once cannot render two ways: a subnet's visibility reaches its name and its tag from the same field, and the tags a consumer filters on are a subset of the tags the producer stamps rather than a parallel list. Each resource type declares its own ordered qualifiers, so adding a resource is one var entry. Add domain.MetadataPrefix for the foundry key namespace, shared by labels, annotations and tags.
A node group's only identity is its storage class. A consuming casting selects nodes by class and has no way to name a group, so a second group of the same class is unreachable: nothing can be steered onto it, and the claim controller spreads stateful identities across every volume of that class by id, which reads as replication while leaving copies on one disk. Key nodeGroups by class and drop the name and storage fields. One group per class becomes structurally impossible to violate rather than a rule to check, and the class already supplies every identifier the generated Terraform needs. This also removes the reason the keyed list merge was added: a map merges under plain document semantics, so revert internal/domain to its three list types.
Doc comments carried design discussion rather than documentation: why a decision was reached, what a reviewer should notice, how the code is structured. Cut to what a reader cannot infer -- formats, invariants, and the reason a check exists. Field comments in the api package restated their own description tag, which is the published documentation and strictly more informative. Drop them and keep the tag, matching the tag-only style in meta.go. Fix Selection's doc, which still explained itself in terms of node group names after the storage class became a group's only identity.
Covers what an Infrastructure casting declares, the requirement document it produces, the order the two castings apply in, and the naming and tagging convention that joins them. The requirement document and every derived name in it are the molding's and the convention's real output rather than hand-written examples. States plainly that no platform is registered yet, so forging reports an unsupported deployment until the castings land, and records the limits that would otherwise be found the hard way: one node group per storage class, and a substrate that does not grow when SigNoz is scaled.
… deployment The requirement document now has two halves. The top is a declaration in kOps' vocabulary: networking with keyed subnets, instanceGroups, cloudLabels. The bottom is derived from it once the declaration settles, and holds every name and tag the substrate stamps. A casting interpolates the derived half rather than assembling names of its own, which is what keeps a producing and a consuming casting from spelling the same string two ways. Derivation walks a provider's topology, so the molding takes a Deriver the registry supplies per deployment instead of reaching for one cloud's. The convention package splits to match: Substrate, Key, NodeGroup, Selection, Identity and the tag facts stay neutral, and the AWS noun table and topology walk land beside the casting that uses them. TagKey names a fact and no longer carries a prefix. A GCP label key rejects the dot and the slash and an Azure tag name rejects the slash, so the spelling is rendered where the resource is created. StorageClass and SubnetType move to api/v1alpha1. An installation casting selects on both and should not import the Infrastructure kind's API to do it. Drops resource.kind. Once workload identity moved to the workload castings it selected only a node-group baseline, and one of its two values had no consumer: a collection agent lands on capacity that already exists and asks a substrate for nothing. One baseline now covers every substrate, and a substrate that keeps nothing drops the persistent group with a null. Drops the edge addresses, which the molding recorded and no template read.
The neutral core states what a substrate is; the aws package states how AWS spells it. Both the infrastructure casting's deriver and the installation casting's tag filters read from here, so it sits in the base rather than in either consumer.
Provisions the substrate an ECS/EC2 installation lands on: a VPC with public and private subnets, an ECS cluster, node groups per storage class, and the IAM the nodes need. The enricher contributes the declaration; the molding derives every name and tag through the aws convention, so the templates interpolate the resources map rather than assembling identifiers inline. Registers the casting with terraform as its tooler and awsconvention.Resources as its deriver.
Emits one ECS service per ClickHouse Keeper and per shard replica, each pinned to the substrate node holding its data, so ClickHouse and Keeper can run as clusters rather than single tasks. Every identifier the stack needs arrives as a variable defaulted to what the casting resolved. Left alone, the stack finds the cluster, subnets, VPC, security group and roles by their foundry.signoz.io tags, which is what the infrastructure casting stamps. Stating the matching annotation replaces that lookup with the value itself and emits no data source, so an existing cluster still works. Flattens the templates out of module/ so the deployment forges as one stack.
Drops ecs-config-bucket, which no longer exists, fixes the task execution role to map to execution_role_arn, and records that every annotation but the region is an optional override of a tag lookup.
Nageshbansal
force-pushed
the
fix/casting/ecscasting
branch
from
August 6, 2026 13:09
dca1c3e to
0bd60a6
Compare
Nageshbansal
marked this pull request as draft
August 6, 2026 13:12
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Features
Fixes
docs/reference/casting-file.md. Dropsecs-config-bucket, which does not exist, and maps the task execution role toexecution_role_arn.Refactors
module/so the deployment forges as one stack.Example
Related: https://github.com/SigNoz/platform-pod/issues/2743