Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 33 additions & 10 deletions docs/private-network-migration.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

This procedure restores an existing prototype into an unused checkout. It does
not migrate repository working copies, GitHub credentials or model weights.
Use [clean installation](controller.md) first; keep project migration and GitHub
Use [clean installation](installation.md) first; keep project migration and GitHub
access as separate acceptance checks. No new runtime dependency is introduced.

## Before restoring
Expand Down Expand Up @@ -56,17 +56,31 @@ Continue only if creation succeeds; do not overwrite an existing TLS directory.
-out "$HOME/forge-tls/server-cert.pem")
```

Restore from the checkout, mapping only repository IDs whose approved working
copies exist on the destination:
Preview from the destination checkout, mapping only repository IDs whose approved
working copies exist there:

```sh
.venv/bin/python scripts/restore.py /private/path/to/snapshot \
.venv/bin/python scripts/restore.py /private/path/to/snapshot --plan \
--repository "forge=$PWD" \
--host 192.168.1.20 \
--certfile "$HOME/forge-tls/server-cert.pem" \
--keyfile "$HOME/forge-tls/server-key.pem"
```

The preview runs the same snapshot, configuration, TLS and staged database checks
as restore, using temporary owner-only staging files that are removed on return.
It publishes no `.secrets/` or `state/` directories. Its summary includes counts,
mapped roots and disabled repository IDs, never credentials or handoff contents.
It is a validation preview, not a reservation or a running-service acceptance
result. It does not exercise the selected-file read policy; use grant/source
checks below after the project files and controller are ready.

Review the summary, then repeat the same command without `--plan` to restore.
Restore revalidates all inputs and refuses newly introduced destination conflicts.
The historical command without `--plan` still performs a restore; the default was
preserved for compatibility. Never remove an existing private directory just to
make a preview or restore pass.

The restore keeps principal identities/tokens, the MCP token, database history,
reviewed examples and all handoff records. Repository mappings retain their
existing selected-file manifests; omitted repositories and their principal
Expand Down Expand Up @@ -170,9 +184,9 @@ To repeat from another host, transfer the public certificate and checker through
SSH and use that client's existing scoped token-only file. Never transfer the
server private key or share the full principals database with another client.

The existing convenience client, MCP bridge and handoff CLI currently target
local HTTP. This server change does not make those clients HTTPS/network-capable.
Use an HTTPS API client with explicit certificate trust for network acceptance.
The [MCP bridge](mcp.md) supports verified HTTPS with an explicit CA file. The
convenience client and handoff CLI still target local HTTP; do not infer that
every client became network-capable. Use the HTTPS checker for network acceptance.
Saved handoff records retain their old endpoints: do not rewrite their endpoint
strings or resubmit their work to bypass reconciliation. Verify known task IDs
against the recovered database using authenticated GET requests. Retire source
Expand All @@ -194,6 +208,15 @@ matched that snapshot at implementation time, with zero active tasks.

Automated validation covers real local HTTPS, certificate trust, authentication,
restore integrity/overwrite refusal, preserved receipts and service selection.
Foundry restore/startup, remote authenticated inference, firewall reachability,
login recovery, GitHub connectivity and complete project/history migration still
require target-host evidence. Do not mark those complete from CI alone.
2026-11-15 follow-up: the operator reported Foundry restore/startup, authenticated
HTTPS inference locally and from Macbox, GitHub reads/project access and 20
projects with 48 historical checkouts imported. Native MCP tasks were independently
exercised through Foundry after Macbox services were disabled (see #67). These
reports do not establish reboot/login recovery or a supported general-purpose
project importer; [#64](https://github.com/kudama/forge/issues/64) retains that work.

The preview increment is tested with synthetic private snapshots: unchanged
source/target bytes and permissions, temporary-file cleanup, shared validation
failures, successful restore after preview, and refusal of a destination conflict
introduced after preview. No preview or restore was run against the live Foundry
deployment for this increment. Do not mark target-host acceptance complete from CI.
11 changes: 9 additions & 2 deletions scripts/restore.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
from forge_controller.config import Settings


def restore(snapshot, root, repositories, listener=None):
def restore(snapshot, root, repositories, listener=None, *, apply=True):
snapshot = snapshot.expanduser().absolute()
root = root.expanduser().absolute()
if root.resolve() != root or not root.is_dir():
Expand Down Expand Up @@ -88,6 +88,11 @@ def copy(source, target):
raise ValueError('Restored task count mismatch')
# Recheck the unchanged source before publishing any destination files.
backup.verify(snapshot)
if not apply:
return {**evidence, 'restored': False, 'validation_only': True,
'mapped_repositories': {name: repo['root'] for name, repo in mapped.items()},
'disabled_repositories': disabled, 'services_started': False,
'handoff_endpoints_unchanged': True}
created = []
try:
for name in ('.secrets', 'state'):
Expand All @@ -113,6 +118,8 @@ def main():
parser.add_argument('--host', default='127.0.0.1')
parser.add_argument('--certfile')
parser.add_argument('--keyfile')
parser.add_argument('--plan', action='store_true',
help='validate using temporary private staging; do not publish restored state')
args = parser.parse_args()
repositories = {}
for item in args.repository:
Expand All @@ -126,7 +133,7 @@ def main():
if args.keyfile:
listener['FORGE_TLS_KEYFILE'] = args.keyfile
os.umask(0o077)
print(json.dumps(restore(args.snapshot, args.checkout, repositories, listener)))
print(json.dumps(restore(args.snapshot, args.checkout, repositories, listener, apply=not args.plan)))


if __name__ == '__main__':
Expand Down
78 changes: 74 additions & 4 deletions tests/test_restore.py
Original file line number Diff line number Diff line change
Expand Up @@ -72,19 +72,21 @@ def test_restore_preserves_tokens_database_receipts_and_original_snapshot(migrat
restore(snapshot, target, {'forge': target})


@pytest.mark.parametrize('apply', [True, False])
@pytest.mark.parametrize('existing', ['.secrets', 'state', 'symlink'])
def test_existing_state_is_never_merged(migration, existing):
def test_existing_state_is_never_merged(migration, existing, apply):
snapshot, target = migration
if existing == 'symlink':
(target/'.secrets').symlink_to(target/'nonexistent')
else:
(target/existing).mkdir()
with pytest.raises(ValueError, match='preserving'):
restore(snapshot, target, {'forge': target})
restore(snapshot, target, {'forge': target}, apply=apply)


@pytest.mark.parametrize('apply', [True, False])
@pytest.mark.parametrize('failure', ['checksum', 'new_grant', 'missing_root', 'invalid_principal', 'tls'])
def test_invalid_restore_never_publishes_partial_state(migration, failure):
def test_invalid_restore_never_publishes_partial_state(migration, failure, apply):
snapshot, target = migration
mapping = {'forge': target}; listener = None
if failure == 'checksum':
Expand All @@ -101,6 +103,74 @@ def test_invalid_restore_never_publishes_partial_state(migration, failure):
else:
listener = {'FORGE_LISTEN_HOST': '10.0.0.106'}
with pytest.raises(ValueError):
restore(snapshot, target, mapping, listener)
restore(snapshot, target, mapping, listener, apply=apply)
assert not (target/'.secrets').exists() and not (target/'state').exists()
assert not list(target.glob('.forge-restore-*'))


def tree_evidence(root):
return {str(path.relative_to(root)):(path.stat().st_mode,
path.read_bytes() if path.is_file() else None)
for path in [root, *root.rglob('*')]}


def test_preview_preserves_inputs_and_target_then_apply_revalidates(migration):
snapshot, target = migration
before = tree_evidence(snapshot), tree_evidence(target)
result = restore(snapshot, target, {'forge': target}, apply=False)
assert result == {'verified':True, 'task_count':1, 'git_commit':'a'*40,
'manifest_version':2, 'handoff_count':3, 'restored':False,
'validation_only':True, 'mapped_repositories':{'forge':str(target)},
'disabled_repositories':['missing'], 'services_started':False,
'handoff_endpoints_unchanged':True}
assert before == (tree_evidence(snapshot), tree_evidence(target))
assert restore(snapshot, target, {'forge': target})['restored']


def test_conflict_created_after_preview_blocks_apply(migration):
snapshot, target = migration
restore(snapshot, target, {'forge': target}, apply=False)
(target/'state').mkdir()
(target/'state/keep').write_text('preserve newer state')
before = tree_evidence(target)
with pytest.raises(ValueError, match='preserving'):
restore(snapshot, target, {'forge': target})
assert tree_evidence(target) == before


@pytest.mark.parametrize('apply', [True, False])
def test_changed_snapshot_during_validation_cleans_staging(migration, monkeypatch, apply):
snapshot, target = migration
original = backup.verify
calls = []
def verify(path):
calls.append(path)
if len(calls) == 2:
raise ValueError('Snapshot changed during validation')
return original(path)
monkeypatch.setattr(backup, 'verify', verify)
before = tree_evidence(target)
with pytest.raises(ValueError, match='Snapshot changed'):
restore(snapshot, target, {'forge':target}, apply=apply)
assert len(calls) == 2
assert tree_evidence(target) == before


@pytest.mark.parametrize('plan', [True, False])
def test_cli_preview_flag_preserves_default_apply(migration, monkeypatch, capsys, plan):
import importlib
module = importlib.import_module('scripts.restore')
snapshot, target = migration
calls = []
def fake_restore(*args, **kwargs):
calls.append((args, kwargs))
return {'restored':kwargs['apply']}
monkeypatch.setattr(module, 'restore', fake_restore)
monkeypatch.setattr(module.os, 'umask', lambda value: None)
monkeypatch.setattr(module.sys, 'argv', ['restore.py', str(snapshot),
'--checkout', str(target), '--repository', 'forge='+str(target),
*(['--plan'] if plan else [])])
module.main()
assert calls[0][0][:3] == (snapshot, target, {'forge':target})
assert calls[0][1] == {'apply':not plan}
assert json.loads(capsys.readouterr().out) == {'restored':not plan}
Loading