Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,9 +34,9 @@ the evaluator that plans and interprets campaigns lives in `malt-evaluation`.

## Package Ownership

- `transport/capability` owns URL-free, route-free, trust-free semantic ports
and untrusted dataset/mutation results shared by Gateway, local, peer, and
hybrid implementations. `transport/capabilitytest` owns the reusable adapter
- `nodeapi` owns URL-free, route-free, trust-free semantic ports
and untrusted authentication/dataset results shared by Gateway, local, peer, and
hybrid implementations. `nodeapi/nodetest` owns the reusable adapter
contract, `transport/local` owns the bounded durable local CAS, and
`transport/hybrid` owns Gateway-primary/read-through-CAS policy. Hybrid `Has`
remains primary-authoritative and every cache body is CID-verified. Local
Expand Down
18 changes: 12 additions & 6 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ It owns:
- the `malt` CLI and local daemon lifecycle;
- trusted/candidate root state and explicit acceptance;
- managed-Bucket base/remote/stash state and stash-before-fetch synchronization;
- semantic remote capabilities with a current Gateway HTTP implementation;
- public `nodeapi` service capabilities shared with Gateway and its HTTP adapter;
- UnixFS path, manifest, fixed-list payload, import, and range-body semantics;
- IPFS-compatible Merkle DAG UnixFS import as an alternative runtime target;
- local replay verification for gateway Merkle DAG compatibility reads;
Expand All @@ -36,6 +36,12 @@ or a permanent prerequisite. The local and hybrid CAS implementations already
use the same semantic boundary; future peer and local MALT executors must do so
without changing application, trust, sync, or filesystem layers.

`nodeapi` owns the transport-independent contract above Core. Gateway implements
it in process and routes HTTP through its managed node services; local/hybrid
CAS and `transport.Client` implement the same interfaces. Authentication
semantics remain Core-owned. See [Public Node API](docs/node-api.md) for the
capability matrix, managed-policy boundary and shared conformance suites.

## Data flow

```text
Expand All @@ -44,9 +50,9 @@ UnixFS path / local files
v
MALT local runtime application adapter
|
| canonical segments, generic resolve/read/mutation/CAS requests
| Core query/candidate/batch values and node CAS/dataset requests
v
semantic transport capability
public nodeapi capability
|
v
Gateway HTTP / Local CAS / Hybrid / future Peer
Expand Down Expand Up @@ -117,7 +123,7 @@ unixfs -> MALT core verifier + narrow transport ports
merkledag -> CID/link replay + fixed profile transport
trust -> accepted/candidate root persistence
transport -> Gateway/local/hybrid adapters; never imports unixfs, merkledag, or trust
bucketsync -> transport + independent durable synchronization metadata
bucketsync -> nodeapi.DatasetBranch + independent durable synchronization metadata
tools/evaluation/cmd -> internal/evaluation + public runtime capabilities
```

Expand All @@ -138,7 +144,7 @@ read-through cache, and primary bytes are independently CID-verified before
return. Cache presence can never satisfy primary `Has`. The default runtime
configuration remains `gateway`; `local` currently supports local-only
Merkle-DAG import, and managed native MALT operations reject it until a local
authentication materializer exists. `transport/capabilitytest` runs the same CAS
authentication materializer exists. `nodeapi/nodetest` runs the same CAS
contract against mock, Gateway HTTP, local, hybrid, and a peer-loopback adapter;
the loopback proves the port boundary without declaring a P2P wire profile.

Expand Down Expand Up @@ -297,7 +303,7 @@ profile; it does not change MALT Core or the existing plaintext semantics of
concrete transport.
- `transport`: untrusted Gateway HTTP adapter and narrow capability interfaces;
`transport/local` provides durable local CAS, `transport/hybrid` owns
Gateway-primary/read-through policy, and `transport/capabilitytest` provides
Gateway-primary/read-through policy, and `nodeapi/nodetest` provides
adapter conformance.
- `bucketsync`: durable Bucket base/remote/stash state and push orchestration.
- `trust`: observed, candidate, and accepted root policy plus durable local
Expand Down
7 changes: 6 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,11 @@ not update an accepted root automatically: remote heads are observations,
while locally computed or strictly verified writes are candidates. Each has a
separate explicit acceptance path.

The [public Node API](docs/node-api.md) defines transport-independent CAS,
authentication and dataset-branch capabilities. Gateway services and HTTP,
local and hybrid adapters implement the same contracts; synchronization consumes
them without acquiring root-acceptance authority.

## Status

This is an experimental, pre-v1 local runtime. It currently provides the
Expand Down Expand Up @@ -468,7 +473,7 @@ persists base, observed remote head, and local stashes under a cross-process
lock and implements stash-before-fetch push ordering. It deliberately does not
import or mutate package `trust`.

`transport/capability` also defines public single and ordered-batch CAS
`nodeapi` also defines public single and ordered-batch CAS
contracts plus stable not-found/corruption classification. `transport/local`
stores bounded immutable blocks atomically in an owner-private directory and
pins the opened store boundary with no-follow platform handles, rejects
Expand Down
30 changes: 15 additions & 15 deletions bucketsync/remote_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,37 +5,37 @@ import (
"testing"
"time"

transportcap "github.com/dewebprotocol/malt-client/transport/capability"
"github.com/dewebprotocol/malt-client/nodeapi"
cid "github.com/ipfs/go-cid"
)

type capabilityRemote struct {
binding transportcap.DatasetBinding
head transportcap.ObservedHead
binding nodeapi.DatasetBinding
head nodeapi.ObservedHead
calls int
apply func(transportcap.ApplyRequest) transportcap.ApplyResult
last transportcap.ApplyRequest
apply func(nodeapi.ApplyRequest) nodeapi.ApplyResult
last nodeapi.ApplyRequest
}

func (r *capabilityRemote) DatasetBinding() transportcap.DatasetBinding {
func (r *capabilityRemote) DatasetBinding() nodeapi.DatasetBinding {
return r.binding
}

func (r *capabilityRemote) ObserveHead(context.Context) (*transportcap.ObservedHead, error) {
func (r *capabilityRemote) ObserveHead(context.Context) (*nodeapi.ObservedHead, error) {
r.calls++
value := r.head
return &value, nil
}

func (r *capabilityRemote) ApplyCandidate(_ context.Context, request transportcap.ApplyRequest) (*transportcap.ApplyResult, error) {
func (r *capabilityRemote) ApplyCandidate(_ context.Context, request nodeapi.ApplyRequest) (*nodeapi.ApplyResult, error) {
r.calls++
r.last = request
value := r.apply(request)
return &value, nil
}

func TestOpenRemoteBranchRejectsMisbindingBeforeIO(t *testing.T) {
remote := &capabilityRemote{binding: transportcap.DatasetBinding{DatasetID: "another", Branch: "main"}}
remote := &capabilityRemote{binding: nodeapi.DatasetBinding{DatasetID: "another", Branch: "main"}}
if _, err := OpenRemote(t.TempDir()+"/workspace.json", remote, "dataset-one"); err == nil {
t.Fatal("OpenRemote accepted a capability bound to another dataset")
}
Expand All @@ -49,21 +49,21 @@ func TestRemoteCapabilityRunsPullAndVerifiedApplyWithoutGatewayDTOs(t *testing.T
candidate := testCID(t, "capability-candidate")
now := time.Date(2026, time.August, 17, 1, 2, 3, 0, time.UTC)
remote := &capabilityRemote{
binding: transportcap.DatasetBinding{DatasetID: "dataset-one", Branch: "main"},
head: transportcap.ObservedHead{
binding: nodeapi.DatasetBinding{DatasetID: "dataset-one", Branch: "main"},
head: nodeapi.ObservedHead{
DatasetID: "dataset-one", Name: "main", Kind: "main", State: "open",
CommitID: "commit-base", Root: base.String(), Revision: 1, CreatedAt: now, UpdatedAt: now,
},
}
remote.apply = func(request transportcap.ApplyRequest) transportcap.ApplyResult {
commit := transportcap.Commit{
remote.apply = func(request nodeapi.ApplyRequest) nodeapi.ApplyResult {
commit := nodeapi.Commit{
ID: "commit-candidate", DatasetID: "dataset-one", Root: request.CandidateRoot,
Parents: []string{request.BaseCommit}, BaseRoot: request.BaseRoot,
Author: "device-one", Message: request.Message, CreatedAt: now,
}
return transportcap.ApplyResult{
return nodeapi.ApplyResult{
Status: "fast_forward",
Head: transportcap.ObservedHead{
Head: nodeapi.ObservedHead{
DatasetID: "dataset-one", Name: "main", Kind: "main", State: "open",
CommitID: commit.ID, Root: commit.Root, Revision: 2, CreatedAt: now, UpdatedAt: now,
},
Expand Down
22 changes: 11 additions & 11 deletions bucketsync/service.go
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ import (
"github.com/dewebprotocol/malt-client/internal/filelock"
"github.com/dewebprotocol/malt-client/internal/securefile"
"github.com/dewebprotocol/malt-client/internal/strictjson"
transportcap "github.com/dewebprotocol/malt-client/transport/capability"
"github.com/dewebprotocol/malt-client/nodeapi"
cid "github.com/ipfs/go-cid"
)

Expand All @@ -32,7 +32,7 @@ var (

const bucketWorkspaceVersion = 3

type Remote = transportcap.DatasetBranch
type Remote = nodeapi.DatasetBranch

type Head struct {
CommitID string `json:"commit_id,omitempty"`
Expand Down Expand Up @@ -73,8 +73,8 @@ type Workspace struct {
}

type PushOutcome struct {
Result transportcap.ApplyResult `json:"result"`
Workspace Workspace `json:"workspace"`
Result nodeapi.ApplyResult `json:"result"`
Workspace Workspace `json:"workspace"`
}

type persistedState struct {
Expand Down Expand Up @@ -108,7 +108,7 @@ func OpenRemoteBranch(path string, remote Remote, bucketID, branch string) (*Ser
if path == "" || remote == nil || bucketID == "" {
return nil, fmt.Errorf("Bucket sync path, remote dataset capability, Bucket ID, and branch are required")
}
if err := transportcap.ValidateBinding(bucketID, branch, remote.DatasetBinding()); err != nil {
if err := nodeapi.ValidateBinding(bucketID, branch, remote.DatasetBinding()); err != nil {
return nil, err
}
service := &Service{
Expand All @@ -131,7 +131,7 @@ func (s *Service) Pull(ctx context.Context) (Workspace, error) {
if head == nil {
return Workspace{}, fmt.Errorf("remote returned an empty dataset head response")
}
if err := transportcap.ValidateObservedHead(s.bucketID, s.branch, *head); err != nil {
if err := nodeapi.ValidateObservedHead(s.bucketID, s.branch, *head); err != nil {
return Workspace{}, err
}
remote, err := headFromRef(*head)
Expand Down Expand Up @@ -351,12 +351,12 @@ func (s *Service) Push(ctx context.Context, candidateRoot cid.Cid, changeSet cid
if _, err := s.Pull(ctx); err != nil {
return PushOutcome{}, err
}
request := transportcap.ApplyRequest{
request := nodeapi.ApplyRequest{
OperationID: stash.PushID, Branch: s.branch, BaseCommit: stash.Base.CommitID, BaseRoot: stash.Base.Root,
CandidateRoot: stash.CandidateRoot, BaseRevision: stash.Base.Revision,
ChangeSetCID: stash.ChangeSetCID, Message: stash.Message,
}
request, err := transportcap.NormalizeApplyRequest(s.branch, request)
request, err := nodeapi.NormalizeApplyRequest(s.branch, request)
if err != nil {
return PushOutcome{}, err
}
Expand All @@ -367,7 +367,7 @@ func (s *Service) Push(ctx context.Context, candidateRoot cid.Cid, changeSet cid
if result == nil {
return PushOutcome{}, fmt.Errorf("remote returned an empty dataset apply response")
}
if err := transportcap.ValidateApplyResult(s.bucketID, request, *result); err != nil {
if err := nodeapi.ValidateApplyResult(s.bucketID, request, *result); err != nil {
return PushOutcome{}, err
}
var workspace Workspace
Expand Down Expand Up @@ -628,7 +628,7 @@ func (s *Service) write() error {
return nil
}

func headFromRef(value transportcap.ObservedHead) (Head, error) {
func headFromRef(value nodeapi.ObservedHead) (Head, error) {
head := Head{CommitID: value.CommitID, Root: value.Root, Revision: value.Revision}
return head, validateHead(head)
}
Expand Down Expand Up @@ -671,7 +671,7 @@ func hasPending(values []Stash) bool {
return false
}

func conflictsFromRemote(values []transportcap.Conflict) []Conflict {
func conflictsFromRemote(values []nodeapi.Conflict) []Conflict {
result := make([]Conflict, len(values))
for i, value := range values {
result[i] = Conflict{Coordinate: value.Coordinate, Base: value.Base, Local: value.Local, Remote: value.Remote}
Expand Down
2 changes: 1 addition & 1 deletion bucketsync/service_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ import (
"time"

clientcas "github.com/dewebprotocol/malt-client/internal/cas"
transport "github.com/dewebprotocol/malt-client/transport/capability"
transport "github.com/dewebprotocol/malt-client/nodeapi"
cid "github.com/ipfs/go-cid"
)

Expand Down
7 changes: 5 additions & 2 deletions cmd/malt/backup_plans.go
Original file line number Diff line number Diff line change
Expand Up @@ -209,15 +209,18 @@ func resolveBucket(ctx context.Context, client *gatewayclient.Client, selector s
}

func ensureBucketBranch(ctx context.Context, client *gatewayclient.Client, branch string, create bool) error {
if branch == "main" {
want, err := bucketbranch.RefName(branch)
if err != nil {
return err
}
if want == "main" {
_, err := client.BucketHead(ctx)
return err
}
refs, err := client.BucketBranches(ctx)
if err != nil {
return err
}
want := "heads/" + branch
for _, ref := range refs {
if ref.Name == want {
return nil
Expand Down
28 changes: 28 additions & 0 deletions cmd/malt/backup_plans_test.go
Original file line number Diff line number Diff line change
@@ -1,16 +1,44 @@
package main

import (
"encoding/json"
"fmt"
"net/http"
"net/http/httptest"
"path/filepath"
"strings"
"testing"

clientbackup "github.com/dewebprotocol/malt-client/application/backup"
clientconfig "github.com/dewebprotocol/malt-client/internal/config"
gatewayclient "github.com/dewebprotocol/malt-client/transport"
truststore "github.com/dewebprotocol/malt-client/trust"
)

func TestEnsureBucketBranchRecognizesNestedHeadsName(t *testing.T) {
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodGet || r.URL.Path != "/v1/buckets/bkt_one/branches" {
t.Errorf("unexpected branch request: %s %s", r.Method, r.URL.Path)
http.NotFound(w, r)
return
}
_ = json.NewEncoder(w).Encode(map[string]any{"branches": []gatewayclient.BucketRef{
{BucketID: "bkt_one", Name: "heads/heads/topic", Kind: "explicit", State: "open"},
}})
}))
defer server.Close()
client, err := gatewayclient.New(gatewayclient.Options{BaseURL: server.URL, BucketID: "bkt_one", TenantBearerToken: "test"})
if err != nil {
t.Fatal(err)
}
if err := ensureBucketBranch(t.Context(), client, "heads/heads/topic", true); err != nil {
t.Fatalf("existing nested branch was not recognized: %v", err)
}
if err := ensureBucketBranch(t.Context(), client, "topic", false); err == nil {
t.Fatal("nested branch was mistaken for its sibling")
}
}

func TestResolveBackupBindingSourcePreservesWhitespace(t *testing.T) {
parent := t.TempDir()
spaced := filepath.Join(parent, "notes ")
Expand Down
Loading
Loading