Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -157,7 +157,6 @@ Browse pre-built agents, tools, filesystems, and software packages at the [agent
| `@agentos-software/git` | git | git version control | rust | - | - |
| `@agentos-software/grep` | grep | GNU grep pattern matching (grep, egrep, fgrep) | rust | - | - |
| `@agentos-software/gzip` | gzip | GNU gzip compression (gzip, gunzip, zcat) | rust | - | - |
| `@agentos-software/http-get` | http-get | Minimal HTTP GET fetch helper | c | - | - |
| `@agentos-software/jq` | jq | jq JSON processor | rust | - | - |
| `@agentos-software/ripgrep` | ripgrep | ripgrep fast recursive search | rust | - | - |
| `@agentos-software/sed` | sed | GNU sed stream editor | rust | - | - |
Expand Down
2 changes: 1 addition & 1 deletion docs-internal/registry-flatten-colocation-spec.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ repo-root/
│ │ ├── sh/ cat/ ls/ cp/ mv/ sort/ … # ~80 command crates
│ │ └── (du/ expr/ column/ rev/ strings/ — 1:1 libs that belong to coreutils)
│ │
│ ├── sqlite3/ duckdb/ wget/ zip/ unzip/ http-get/ # C-based, native/c/<name>.c
│ ├── sqlite3/ duckdb/ wget/ zip/ unzip/ # C-based, native/c/<name>.c
│ ├── grep/ sed/ gawk/ jq/ yq/ fd/ ripgrep/ tree/ file/ tar/ gzip/ diffutils/ findutils/ # Rust, native/crates/
│ │
│ ├── claude/ codex/ opencode/ pi/ pi-cli/ # agent adapters — @agentos-software/*, JS only, no native/
Expand Down
28 changes: 22 additions & 6 deletions docs-internal/registry-parity-worklist.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ actual backing:
|---|---|---|---|
| **curl** | DONE | our custom driver over a libcurl fork | real `curl` CLI (upstream `src/tool_*.c`) |
| **wget** | DONE | our 174-line `wget.c` (dropped) | real GNU Wget vs our sysroot — stub `getrlimit`/`getgroups`, then build |
| **http-get** | TODO | our 95-line `http_get.c` | drop, or a real tool |
| **http-get** | DONE | our 95-line `http_get.c` | dropped; real curl covers HTTP fetches |
| **git** | TODO | our hand-rolled git from `sha1`+`flate2` | **real git** (upstream C), patched for WASI — **NOT gitoxide** |
| **fd** | TODO | our `secureexec-fd` on raw `regex` (not sharkdp/fd) | real **fd** (sharkdp) |
| **findutils** (`find`,`xargs`) | TODO | our hand-rolled on `regex`/shims | real GNU findutils, or `uutils/findutils` |
Expand Down Expand Up @@ -184,10 +184,26 @@ works (`wasi-spawn` broker), so `xargs` is not a blocker.
aggregate C `programs` builds 57 commands in
`2026-07-08T05-09-53-0700-sqlite3-make-programs-final.log`.
Rev: `typytnkk` — `fix(sqlite3): build official SQLite shell`.
- **http-get — drop, don't port.** `http_get.c` is a 95-line raw-socket **loopback
test client**, not an HTTP fetcher; real curl covers GET. But it's imported by
`packages/shell` and is the client in cross-runtime network tests — migrate those
dependents first, then drop the command.
- **http-get — DONE.** Dropped the 95-line raw-socket loopback client instead of
porting it: real curl now covers HTTP fetch behavior in DuckDB remote CSV tests,
runtime cross-network loopback tests, and the C conformance loopback row.
Removed the `@agentos-software/http-get` package, fallback command, shell/core
dependencies, registry listing, C command install entries, and lockfile package
edges. The low-level `http_get_test` fixture remains because it is a socket
diagnostic test program, not a shipped registry command. Proof: pnpm lockfile
refresh succeeds in
`2026-07-08T05-39-44-0700-http-get-pnpm-lock-refresh.log`; core and shell
type checks pass in
`2026-07-08T05-40-57-0700-http-get-core-check-types-after-install.log` and
`2026-07-08T05-41-34-0700-http-get-shell-check-types-final.log`; DuckDB test
file typecheck passes in
`2026-07-08T05-40-57-0700-http-get-duckdb-test-typecheck-after-install.log`;
aggregate C `programs` builds 57 commands, with `http_get` absent after stale
generated cleanup/install, in
`2026-07-08T05-41-34-0700-http-get-make-c-programs-without-command.log` and
`2026-07-08T05-43-24-0700-http-get-clear-stale-generated-and-install.log`;
cross-runtime network tests pass 11/11 with the WASM curl rows in
`2026-07-08T05-47-43-0700-http-get-runtime-cross-network-test-pass.log`.
- **tree — DONE.** Replaced the custom Rust `secureexec-tree`/`cmd-tree` crates
with upstream Steve Baker `tree` 2.3.2 from `OldManProgrammer/unix-tree`.
It builds as a C toolchain command from pinned source, stages into
Expand Down Expand Up @@ -421,7 +437,7 @@ real e2e tests that prove Linux-parity behavior — not smoke tests.

### 12. No tests at all — 12 software + 5 agents
- **Broken:** zero e2e coverage: `gawk, sed, grep, tar, gzip, jq, ripgrep, yq,
diffutils, file, http-get, vim`; agents `claude, codex, opencode, pi, pi-cli`.
diffutils, file, vim`; agents `claude, codex, opencode, pi, pi-cli`.
- **Objective:** write real e2e tests proving each behaves like its Linux
counterpart (jq processes real JSON, sed edits streams, tar round-trips archives,
grep/rg search real trees, gzip round-trips, etc.); agents exercise the real ACP
Expand Down
2 changes: 1 addition & 1 deletion packages/core/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,7 @@ An agent's launch config lives entirely in its `/opt/agentos` package `agentos-p
- Sandbox toolkit quickstarts/tests that depend on external Docker should use an explicit `SKIP_DOCKER=1` gate instead of `skipIf`, and the truthful host-tool path is to read `AGENTOS_TOOLS_PORT` inside the VM and `POST` `{ toolkit, tool, input }` to `http://127.0.0.1:$AGENTOS_TOOLS_PORT/call` from a guest Node script.
- Shared Vitest helpers under `src/test/` should register optional capability coverage conditionally in code instead of with `describe.skipIf` / `test.skipIf`; `US-088` treats those markers as product-debt skips even when they only guard backend capability differences.
- Pi bash-tool E2E coverage depends on registry WASM commands being built locally. Gate those tests with `tests/helpers/registry-commands.ts` `hasRegistryCommands` and include the `@agentos-software/common` software package only when the command artifacts exist.
- Software package tests for C-built commands such as `duckdb` and `http_get` should go through `tests/helpers/registry-commands.ts`: prefer copied `../secure-exec/software/*/wasm` artifacts, fall back to `../secure-exec/toolchain/c/build` when available, and let the helper build missing C-source artifacts on demand before declaring the command unavailable. When bootstrapping from secure-exec `toolchain/c`, build `make sysroot` first and then run a second `make` for the concrete `build/...` targets so `SYSROOT` resolves to the patched tree instead of the vanilla SDK sysroot chosen at parse time; in that second pass, treat `sysroot/lib/wasm32-wasi/libc.a` as already built so `make` does not loop back through the patch pipeline because of preserved sysroot timestamps.
- Software package tests for C-built commands such as `duckdb` and `curl` should go through `tests/helpers/registry-commands.ts`: prefer copied `../secure-exec/software/*/wasm` artifacts, fall back to `../secure-exec/toolchain/c/build` when available, and let the helper build missing C-source artifacts on demand before declaring the command unavailable. When bootstrapping from secure-exec `toolchain/c`, build `make sysroot` first and then run a second `make` for the concrete `build/...` targets so `SYSROOT` resolves to the patched tree instead of the vanilla SDK sysroot chosen at parse time; in that second pass, treat `sysroot/lib/wasm32-wasi/libc.a` as already built so `make` does not loop back through the patch pipeline because of preserved sysroot timestamps.
- `tests/claude-session.test.ts` is the Claude SDK truth suite. It runs the real `@anthropic-ai/claude-agent-sdk` session path through llmock and covers PATH-backed `xu`, text-only replies, nested `node` `execSync` and `spawn`, metadata, lifecycle, and mode updates. Run it with `pnpm --dir packages/core exec vitest run tests/claude-session.test.ts --reporter=verbose` when verifying Claude regressions.
- **Kernel permissions are declarative pass-through config.** `AgentOsOptions.permissions` should stay JSON-serializable and be forwarded to the native sidecar without host-side probing or callback evaluation; Rust owns glob matching and policy decisions.
- ACP session events are live-only over `onSessionEvent()`. Do not reintroduce sequence numbers, local replay buffers, or event cursor recovery.
Expand Down
2 changes: 1 addition & 1 deletion packages/core/CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,7 @@ An agent's launch config lives entirely in its `/opt/agentos` package `agentos-p
- Sandbox toolkit quickstarts/tests that depend on external Docker should use an explicit `SKIP_DOCKER=1` gate instead of `skipIf`, and the truthful host-tool path is to read `AGENTOS_TOOLS_PORT` inside the VM and `POST` `{ toolkit, tool, input }` to `http://127.0.0.1:$AGENTOS_TOOLS_PORT/call` from a guest Node script.
- Shared Vitest helpers under `src/test/` should register optional capability coverage conditionally in code instead of with `describe.skipIf` / `test.skipIf`; `US-088` treats those markers as product-debt skips even when they only guard backend capability differences.
- Pi bash-tool E2E coverage depends on registry WASM commands being built locally. Gate those tests with `tests/helpers/registry-commands.ts` `hasRegistryCommands` and include the `@agentos-software/common` software package only when the command artifacts exist.
- Software package tests for C-built commands such as `duckdb` and `http_get` should go through `tests/helpers/registry-commands.ts`: prefer copied `../secure-exec/software/*/wasm` artifacts, fall back to `../secure-exec/toolchain/c/build` when available, and let the helper build missing C-source artifacts on demand before declaring the command unavailable. When bootstrapping from secure-exec `toolchain/c`, build `make sysroot` first and then run a second `make` for the concrete `build/...` targets so `SYSROOT` resolves to the patched tree instead of the vanilla SDK sysroot chosen at parse time; in that second pass, treat `sysroot/lib/wasm32-wasi/libc.a` as already built so `make` does not loop back through the patch pipeline because of preserved sysroot timestamps.
- Software package tests for C-built commands such as `duckdb` and `curl` should go through `tests/helpers/registry-commands.ts`: prefer copied `../secure-exec/software/*/wasm` artifacts, fall back to `../secure-exec/toolchain/c/build` when available, and let the helper build missing C-source artifacts on demand before declaring the command unavailable. When bootstrapping from secure-exec `toolchain/c`, build `make sysroot` first and then run a second `make` for the concrete `build/...` targets so `SYSROOT` resolves to the patched tree instead of the vanilla SDK sysroot chosen at parse time; in that second pass, treat `sysroot/lib/wasm32-wasi/libc.a` as already built so `make` does not loop back through the patch pipeline because of preserved sysroot timestamps.
- `tests/claude-session.test.ts` is the Claude SDK truth suite. It runs the real `@anthropic-ai/claude-agent-sdk` session path through llmock and covers PATH-backed `xu`, text-only replies, nested `node` `execSync` and `spawn`, metadata, lifecycle, and mode updates. Run it with `pnpm --dir packages/core exec vitest run tests/claude-session.test.ts --reporter=verbose` when verifying Claude regressions.
- **Kernel permissions are declarative pass-through config.** `AgentOsOptions.permissions` should stay JSON-serializable and be forwarded to the native sidecar without host-side probing or callback evaluation; Rust owns glob matching and policy decisions.
- ACP session events are live-only over `onSessionEvent()`. Do not reintroduce sequence numbers, local replay buffers, or event cursor recovery.
Expand Down
1 change: 0 additions & 1 deletion packages/core/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,6 @@
"@agentos-software/git": "workspace:*",
"@agentos-software/grep": "workspace:*",
"@agentos-software/gzip": "workspace:*",
"@agentos-software/http-get": "workspace:*",
"@agentos-software/jq": "workspace:*",
"@agentos-software/opencode": "workspace:*",
"@agentos-software/pi": "workspace:*",
Expand Down
24 changes: 13 additions & 11 deletions packages/core/tests/duckdb-package.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,19 +5,19 @@ import {
type ServerResponse,
} from "node:http";
import coreutils from "@agentos-software/coreutils";
import curl from "@agentos-software/curl";
import { afterEach, beforeEach, describe, expect, test } from "vitest";
import duckdb from "@agentos-software/duckdb";
import httpGet from "@agentos-software/http-get";
import { AgentOs } from "../dist/index.js";
import { cSysrootPackageSkipReason } from "./helpers/registry-commands.js";

const DUCKDB_PACKAGE = duckdb;
const HTTP_GET_PACKAGE = httpGet;
const CURL_PACKAGE = curl;
// C-sysroot packages are the ONE sanctioned skip: they need the patched wasi C
// sysroot most checkouts don't build (see helpers/registry-commands.ts).
const duckdbPackageSkipReason = cSysrootPackageSkipReason(
{ pkg: DUCKDB_PACKAGE, name: "duckdb" },
{ pkg: HTTP_GET_PACKAGE, name: "http-get" },
{ pkg: CURL_PACKAGE, name: "curl" },
);

function closeServer(server: Server) {
Expand Down Expand Up @@ -47,7 +47,7 @@ describe("duckdb registry package", () => {
await vm.dispose();
}
vm = await AgentOs.create({
software: options?.software ?? [coreutils, HTTP_GET_PACKAGE, DUCKDB_PACKAGE],
software: options?.software ?? [coreutils, CURL_PACKAGE, DUCKDB_PACKAGE],
...(options?.loopbackExemptPorts
? { loopbackExemptPorts: options.loopbackExemptPorts }
: {}),
Expand Down Expand Up @@ -106,7 +106,7 @@ describe("duckdb registry package", () => {
await recreateVm({ loopbackExemptPorts: [address.port] });

let result = await vm.exec(
`http_get ${address.port} /remote.csv /tmp/remote.csv`,
`curl -fsS -o /tmp/remote.csv http://127.0.0.1:${address.port}/remote.csv`,
);
expect(result.exitCode, result.stderr || result.stdout).toBe(0);

Expand Down Expand Up @@ -166,11 +166,11 @@ describe("duckdb registry package", () => {
async () => {
await vm.dispose();

const httpGetReadOnly = {
...HTTP_GET_PACKAGE,
commands: [{ name: "http_get", permissionTier: "read-only" as const }],
const curlReadOnly = {
...CURL_PACKAGE,
commands: [{ name: "curl", permissionTier: "read-only" as const }],
};
vm = await AgentOs.create({ software: [coreutils, httpGetReadOnly] });
vm = await AgentOs.create({ software: [coreutils, curlReadOnly] });

const server = createServer((req: IncomingMessage, res: ServerResponse) => {
res.writeHead(200, { "Content-Type": "text/plain" });
Expand All @@ -187,11 +187,13 @@ describe("duckdb registry package", () => {
throw new Error("failed to bind test HTTP server");
}
await recreateVm({
software: [coreutils, httpGetReadOnly],
software: [coreutils, curlReadOnly],
loopbackExemptPorts: [address.port],
});

const result = await vm.exec(`http_get ${address.port} /blocked`);
const result = await vm.exec(
`curl -fsS http://127.0.0.1:${address.port}/blocked`,
);
expect(result.exitCode).not.toBe(0);
} finally {
await closeServer(server);
Expand Down
6 changes: 3 additions & 3 deletions packages/core/tests/helpers/registry-commands.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,8 @@
* when it exists (local registry builds produce both), so a stale or empty
* command set still fails loudly.
*
* The only sanctioned exception is the C-sysroot package set (duckdb,
* http-get, sqlite3, zip, unzip): those need the patched wasi C sysroot
* The only sanctioned exception is the C-sysroot package set (curl, duckdb,
* sqlite3, zip, unzip): those need the patched wasi C sysroot
* that most checkouts don't have, so `cSysrootPackageSkipReason` reports a
* skip reason instead of throwing. Everything else is load-or-throw.
*/
Expand Down Expand Up @@ -160,7 +160,7 @@ export function requireBuilt<T extends RegistryPackageRef>(
}

/**
* Skip reason for the C-sysroot package set ONLY (duckdb, http-get, sqlite3,
* Skip reason for the C-sysroot package set ONLY (curl, duckdb, sqlite3,
* zip, unzip). These need the patched wasi C sysroot
* (`make -C toolchain/c`), which most checkouts don't build — a missing
* artifact is an environment limitation, not a forgotten build, so suites may
Expand Down
Binary file removed packages/runtime-core/commands/http_get
Binary file not shown.
Loading