feat(infrastructure): add scaffolding for the infrastructure kind - #154
Open
Nageshbansal wants to merge 39 commits into
Open
feat(infrastructure): add scaffolding for the infrastructure kind#154Nageshbansal wants to merge 39 commits into
Nageshbansal wants to merge 39 commits into
Conversation
Nageshbansal
marked this pull request as draft
July 14, 2026 13:27
- resource ref: no fabricated default, apiVersion enum, name pattern - reject absent resource kind before the merge can default it to Installation - reject unsupported consumer kinds (self-reference) at load - drop the no-op resource molding: the kind has zero moldings - rework tests to the Subject_Outcome table idiom; add loader coverage
Nageshbansal
marked this pull request as ready for review
July 15, 2026 13:28
… terraform casting
…e molding Persistence moves onto the node groups: each group declares whether its nodes persist data, and the standalone storage block goes away. The molding now merges contributions (enricher deltas, operator overrides) at the document level, so casting-specific keys survive, and validates the shared shape of the resolved document before storing it. A contribution owns the node groups it states.
Castings deposit their outputs into a pourer instead of returning materials, matching the CollectionAgent contract: the planner builds the pourer for the kind's output directory, the casting adds content under relative paths, and the planner pours the staged entries into materials. Enrichers follow the established shape: a stateless named type with a constructor.
ListTypeSet and ListTypeOrdered both degrade to atomic on a list of maps, so nothing could merge a list of objects: an override stating one element replaced the whole list and silently dropped the rest. ListTypeMap(key) mirrors Kubernetes' listType: map with a listMapKey. Elements match on the key, a matched pair merges as a document so the override states only what it changes, and override-only elements append. It degrades to atomic when an element is not a map or lacks the key.
The requirement document invented its own words for a machine pool, and
one of them meant two different things: `disk` was the data volume on a
persistent group and the root volume on an ephemeral one, with different
minimums. There was also no way to name a machine type in the casting,
only in a terraform variable.
Adopt the vocabulary every node-pool abstraction already uses. kOps is
the closest precedent -- the one machine-pool shape that spans AWS, GCE,
Azure and DigitalOcean -- and where it had to choose it picked
machineType, minSize and maxSize, which overlap with GKE's and eksctl's
terms too. Narrowed to what foundry has to understand:
nodeGroups:
- name: persistent
storage: persistent
minSize: 3
maxSize: 3
machineType: "" # empty resolves from cpu/memory
cpu: 2
memory: 8
rootVolume: {size: 30}
dataVolume: {size: 50}
cpu and memory stay as the portable criteria so the document does not
name provider machine types, with machineType as the escape hatch. Volumes
are nested objects rather than flat scalars so volume type or encryption
can arrive without new top-level keys, and splitting them lets the AMI
snapshot floor apply to every root disk unconditionally. dataVolume is
singular, not a list as kOps has it: the claim controller assumes one
volume per node, and a list would advertise what foundry cannot honour.
StorageClass is a value object rather than a string with an enum tag, so
an unknown class fails at unmarshal instead of deep in the molding, and
each class carries what it implies -- whether it needs a data volume,
whether the group is pinned. Adding a class is one var entry rather than
another branch wherever groups are checked.
Node groups now merge by name, so a contribution or an operator override
states only the group and the fields it changes. Before this, stating one
group deleted the others. `spec.resource.spec.config.data` merges last,
so an operator's own document beats the casting's contribution.
The scaffolding provides the Kind, the casting contract, the resource molding and the registry; a concrete casting is a separate change with its own review. Carrying one here made the scaffolding PR argue for an EKS substrate at the same time as the machinery that hosts it. The registry is left empty. The first casting to register lands with the platform it serves.
Foundry generates rather than reconciles, so it can never ask a platform what it created. A consuming casting finds a producing casting's resources only by deriving the same names and filtering the same tags, with nothing between them to reconcile a mismatch. Derive both from one description of a resource, so a fact stated once cannot render two ways: a subnet's visibility reaches its name and its tag from the same field, and the tags a consumer filters on are a subset of the tags the producer stamps rather than a parallel list. Each resource type declares its own ordered qualifiers, so adding a resource is one var entry. Add domain.MetadataPrefix for the foundry key namespace, shared by labels, annotations and tags.
A node group's only identity is its storage class. A consuming casting selects nodes by class and has no way to name a group, so a second group of the same class is unreachable: nothing can be steered onto it, and the claim controller spreads stateful identities across every volume of that class by id, which reads as replication while leaving copies on one disk. Key nodeGroups by class and drop the name and storage fields. One group per class becomes structurally impossible to violate rather than a rule to check, and the class already supplies every identifier the generated Terraform needs. This also removes the reason the keyed list merge was added: a map merges under plain document semantics, so revert internal/domain to its three list types.
Nageshbansal
force-pushed
the
feat/infrastructure/scaffolding
branch
from
August 5, 2026 10:45
670fe55 to
267bdb2
Compare
Doc comments carried design discussion rather than documentation: why a decision was reached, what a reviewer should notice, how the code is structured. Cut to what a reader cannot infer -- formats, invariants, and the reason a check exists. Field comments in the api package restated their own description tag, which is the published documentation and strictly more informative. Drop them and keep the tag, matching the tag-only style in meta.go. Fix Selection's doc, which still explained itself in terms of node group names after the storage class became a group's only identity.
Covers what an Infrastructure casting declares, the requirement document it produces, the order the two castings apply in, and the naming and tagging convention that joins them. The requirement document and every derived name in it are the molding's and the convention's real output rather than hand-written examples. States plainly that no platform is registered yet, so forging reports an unsupported deployment until the castings land, and records the limits that would otherwise be found the hard way: one node group per storage class, and a substrate that does not grow when SigNoz is scaled.
… deployment The requirement document now has two halves. The top is a declaration in kOps' vocabulary: networking with keyed subnets, instanceGroups, cloudLabels. The bottom is derived from it once the declaration settles, and holds every name and tag the substrate stamps. A casting interpolates the derived half rather than assembling names of its own, which is what keeps a producing and a consuming casting from spelling the same string two ways. Derivation walks a provider's topology, so the molding takes a Deriver the registry supplies per deployment instead of reaching for one cloud's. The convention package splits to match: Substrate, Key, NodeGroup, Selection, Identity and the tag facts stay neutral, and the AWS noun table and topology walk land beside the casting that uses them. TagKey names a fact and no longer carries a prefix. A GCP label key rejects the dot and the slash and an Azure tag name rejects the slash, so the spelling is rendered where the resource is created. StorageClass and SubnetType move to api/v1alpha1. An installation casting selects on both and should not import the Infrastructure kind's API to do it. Drops resource.kind. Once workload identity moved to the workload castings it selected only a node-group baseline, and one of its two values had no consumer: a collection agent lands on capacity that already exists and asks a substrate for nothing. One baseline now covers every substrate, and a substrate that keeps nothing drops the persistent group with a null. Drops the edge addresses, which the molding recorded and no template read.
The neutral core states what a substrate is; the aws package states how AWS spells it. Both the infrastructure casting's deriver and the installation casting's tag filters read from here, so it sits in the base rather than in either consumer.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Features
Infrastructurekind: its own api package and schema, wired through the loader, lock file, planner and registry.resourcemolding. Every forge settles one document intocasting.yaml.lockunderspec.resource.status.internal/convention, the provider-neutral vocabulary the Infrastructure and Installation kinds find each other by: substrate, keys, node groups, identities, selections and five tag keys. Each side works the tags out from the substrate name alone, so neither reads the other's state.internal/convention/aws, the same vocabulary in AWS grammar plus the name and tag derivation. The registry hands the molding aDeriver, so the molding itself names no provider.docs/concepts/infrastructure.md.Refactors
spec.infrastructureto{name}, the infrastructure casting it runs on. Breaking for anyone who set the oldenabledandstatus.internal/infrastructure, the inline terraform path this replaces, along with the forge step that called it.Tests
Example
Related: https://github.com/SigNoz/platform-pod/issues/2495