diff --git a/CLAUDE.md b/CLAUDE.md index 3977e8b1..3502d842 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -256,6 +256,7 @@ include/naab/ All headers - **Subprocess containment** (`src/runtime/subprocess_helpers.h/cpp`): `SubprocessContainment` struct applied to all polyglot child processes via `execute_subprocess_with_pipes()`. The memory budget is `RLIMIT_DATA` (private writable memory), with `RLIMIT_AS` only as a backstop ceiling (4x the budget, at least 2 GB) for shared anonymous mappings, which `RLIMIT_DATA` does not count. It used to be `RLIMIT_AS` at the budget, which counts untouched reservations: node 22 needs 512-768 MB of address space just to start, so `process.run("node", ...)` died with "Failed to reserve virtual memory" under `elevated`. Test: `tests/security/test_child_memory_limit.sh` (M-02/M-03 are why the fix is not "drop the limit"). 5-layer defense: RLIMIT_NPROC=0 (blocks fork/subprocess), PATH restriction, env scrubbing, timeout (SIGKILL), allow_exec/allow_fork flags. `SubprocessContainment::fromCurrentSandbox()` reads current sandbox level: `standard` → fork+exec blocked, `elevated` → fork allowed, `unrestricted` → no restrictions. Pre-execution source scanning (governance checks) catches static patterns; containment catches runtime-constructed commands that evade static analysis. ### Stdlib Notable +- **String `slice` / `substring` / `replace` have ONE implementation**, `include/naab/string_ops.h`, used by the VM's method dispatch, both tree-walker method paths and the `string` module. There were four copies and they disagreed: `s.replace` replaced only the FIRST match on the VM (all on the tree-walker and in `string.replace`), the tree-walker aliased `slice` to `substring` (so `s.slice(-3)` was `hello` there and `llo` on the VM), and its reversed-range `substring` returned the rest of the string through a wrapped `size_t`. Worst, the tree-walker's `replace` had no empty-pattern guard: `"ab".replace("", "+")` inserted forever with growing memory, which `--timeout` cannot interrupt, and REST runs the tree-walker. `string.slice(s, start[, end])` now exists with the method's JavaScript semantics. The differential corpus had only module-form string probes; `tests/differential/corpus/string_methods.naab` covers the method forms, and `tests/parser/test_dogfood_hints.sh` E-01 guards the hang. - `array.sort()` mutates in place, `array.sorted()` returns a new sorted array (non-mutating) - Both support optional comparator: `arr.sorted(fn(a, b) { return a - b })` - `sorted()` is registered in `hasFunction()` but NOT in `isMutatingFunction()` @@ -399,6 +400,7 @@ Always run `bash tests/security/test_error_msg_leaks.sh` after changing any erro - **Agent responses auto-strip markdown fences** — `agent.send()` strips ` ```json ... ``` ` wrapping when the entire response is a single fenced block. Only strips when fence is at start/end (with optional whitespace). Does not strip partial fences or multiple fence blocks. - **Blocked turns never enter history (split commit)** — CDD/OA-blocked `agent.send()` turns (including catchable DETECT-level blocks) do NOT append to the handle's message history; the tracker/exposure accounting still commits. Quarantine/attest disposition is controlled by `output_admissibility.inadmissible_history`. - **Scanner regexes must be bounded — an unbounded one crashes instead of deciding.** `SECRET_PATTERNS` in `governance_engine.cpp` carried `-----BEGIN[\s\S]*PRIVATE KEY-----`, whose greedy `[\s\S]*` runs to end of input then backtracks one position at a time; libstdc++ executes that recursively (`_Executor::_M_dfs`), so ~130k frames exhausted the stack and any input with `-----BEGIN` plus roughly 30KB of following text died with SIGSEGV. A crash is worse than a false negative: `code_quality.no_secrets` is HARD, and a crash renders NO verdict — it neither blocks nor passes, and exits with a signal rather than the documented exit 3. Scope, because the call-site list is easy to over-read: `checkSecrets()` returns at its first line unless `code_quality.no_secrets` is enabled, and that key defaults **false** — with it unset the regex never runs and the same 30KB input exits 0. Where it IS enabled the exposure is real and untrusted, since the call sites cover the agent prompt, response content, tool arguments and tool results, so model output could terminate the interpreter. The surrounding `catch (const std::regex_error&)` cannot help — stack exhaustion is not an exception. Fixed by bounding to `[A-Z0-9 ]{0,40}`, which still matches every real PEM header. When adding to `SECRET_PATTERNS`, `DANGEROUS_PATTERNS_DB` or `PLACEHOLDER_PATTERNS_DB`, never use an unbounded `.*` / `[\s\S]*` between two required literals. Test: `tests/security/test_secret_scan_redos.sh` (Group B is the positive control — deleting the pattern also stops the crash). +- **Inline interpreter code in `process.run` gets the polyglot-block checks.** `process.run("python3", ["-c", code])` is a `<>` block by another name, and it used to skip every code check the block and `codegen.run()` get — measured building repo-sentinel: `except Exception: return 0` was a HARD `no_incomplete_logic` block in both and ran silently here, reporting every C++ file as 0 bytes. `inlineInterpreterCode()` in `process_impl.cpp` recognises python/pypy (`-c`), node (`-e`/`-p`/`--eval`/`--print`, also `--eval=`), ruby (`-e`), perl (`-e`/`-E`), php (`-r`) and sh/bash/dash/zsh/ksh (`-c`) — through a directory prefix, `.exe` and a version suffix (`python3.12`), with clustered short flags when the code flag is last (`-Ic`, `-ec`) — and passes the code to `checkPolyglotBlock()`, so `languages.allowed`/`blocked` now apply to it too. **Scope**: a script FILE (`python3 s.py`) is not inline code and is not scanned, and only the OUTER interpreter is recognised (`sh -c "python3 -c ..."` is checked as shell). Test: `tests/security/test_process_run_inline_gate.sh` (PI-00 is the reference that the config exercises the check at all; PI-02/04/05 are the controls a refuse-everything gate would fail). - **A `process.run` pipeline is governed by BSD and taint, NOT by the agent layer.** Scripts that reach a model through `process.run` instead of `agent.send()` get zero agent governance: no CDD (all 23 signals are fed from `AGENT_RESPONSE`, emitted only by `agent_impl.cpp`), no output admissibility, no step-up challenges, no pulse, no transcript (all 22 hook points are inside `agentSend()`), and no response secret/PII scan. What DOES still apply is taint tracking and behavioural sequence detection — `vm.cpp` emits `PROCESS_EXEC`/`FILE_READ`/`FILE_WRITE`/`ENV_READ`/`ENV_WRITE`/`ENCODE`/`DECODE` from ordinary stdlib calls, gated only on `behavioral_sequences.enabled` (default **false**). Setting `context_drift.enabled` on such a config governs nothing and prints a warning; setting both silences the warning while still scoring nothing. Worked example, with the trap of narrowing taint sinks to unblock a script documented rather than repeated: `examples/hivemind_governed/` (test: `tests/governance_v4/test_hivemind_governed.sh`). - **A test's OUTPUT CHANNEL is part of the instrument and needs its own control.** A test that runs a tool, captures its stdout and compares against a baseline has two things that can be wrong: the tool's answer, and the path from that answer to the comparison. The second breaks only on platforms nobody runs locally, and it has bitten three times in two days across two sessions (`test_hivemind_governed.sh`, `test_coverage_visibility.sh`, `test_state_field_screen.sh` twice) — never retained from the previous fix. Two shapes. **Encoding**: Python encodes stdout with the LOCALE's encoding, which is cp1252 under MSYS2 — it writes U+2014 as the single byte `0x97` and a UTF-8 reader dies; under a bare C locale the writer dies instead. A traceback announces itself. **Newline**: `print()` translates `\n` to `\r\n` on Windows, and command substitution strips the trailing newline but not the embedded CRs — so the captured string compares unequal to a baseline it renders IDENTICALLY to. Nothing announces itself: `test_state_field_screen.sh` reported `flagged set drifted from the pinned baseline` above two character-for-character identical lists. The fix for that half is making the difference VISIBLE (`sed -n l`), more than any comparison change — an assertion that cannot show why it failed gets believed over the code. **Reproducing the encoding half on Linux, the detail that costs the most time: `LC_ALL=C` does NOT reproduce it** — PEP 538 coerces the C locale back to UTF-8; use `PYTHONUTF8=0 PYTHONCOERCECLOCALE=0 LC_ALL=C`. The newline half reproduces by injecting `sys.stdout.reconfigure(newline="\r\n")`. Prevention in the tool: print ASCII only, write bytes (`sys.stdout.buffer.write`) for anything a test parses, and give every `open()` an explicit `encoding=` AND `errors=`. Controls: `tests/helpers/encoding_controls.sh` (`enc_has_non_ascii`, `enc_strip_cr`, `enc_escaped_diff`, `enc_self_test`) — both failure shapes reproduce on Linux, so this class is testable without a Windows runner. **The renderer itself must not use an external text tool.** `enc_escaped_diff` originally piped through `sed -n l` and reddened build-windows a THIRD time — the assertion that it surfaces a CR passed on Linux and failed under MSYS2. Two hypotheses fit that evidence and they demand different fixes (sed rendering CR as octal `\015`, or sed stripping a trailing CR as part of a CRLF terminator, in which case nothing renders and widening the grep fixes nothing), so the renderer stopped asking: it now escapes via bash parameter expansion, where no line-ending convention gets a vote. Note POSIX *does* list `\r` among `l`'s named escapes, so the octal hypothesis needs a non-conforming sed — which is why `enc_platform_probe` records what the platform actually does on every self-test failure, rather than leaving the next person to re-derive it from a runner they do not have. **A third shape, and it needs no exotic platform: `set -o pipefail` plus `cmd | grep -q`.** `grep -q` exits the instant it matches and closes the pipe; the producer takes SIGPIPE; `pipefail` then makes the pipeline status non-zero. So the pipeline reports FAILURE precisely when the pattern IS present. In a security suite that inverts every verdict in the safe-looking direction — a successful read reads as "refused", and every assertion expecting a refusal passes for free. Capture into a variable and match with `case`, which also leaves you holding the output to print when an assertion fails. **A fourth shape, and it caught the guard rather than the code: handing a PATH to a helper that does not share the shell's path vocabulary.** `test_path_precedence.sh` validated each generated `govern.json` with `python3 -c "...json.load(open('$W/govern.json'))"`. Under MSYS2 `python3` is a NATIVE Windows build and cannot open an MSYS `/tmp/...` path, so the validator meant to catch a broken fixture became a broken probe — it reported `fixture is broken` eleven times while all eleven real assertions passed in the same run. Feed the helper BYTES, not a path (`python3 -c "...json.load(sys.stdin)" < "$file"`): the shell does the open, and no path crosses the boundary. The general rule is the same one that produced the relative-path fixtures two commits earlier — whenever a shell hands a native program a filename, ask which vocabulary that name is in. - **A polyglot block that cannot run ABORTS the program, so one unusable executor fails every arm after it.** `test_function_capability_gates.sh` put six one-action functions in a single program, one of them a `<>` block. On the Windows runner the shell executor is unusable, the program died there, and `flushGroupedAdvisories()` (clean-exit only, at the time) never emitted the summary every arm greps — so 11 of 12 arms failed, including the success-expecting controls. The one that passed, FG-11, is the only arm reading the per-occurrence message instead of the summary, and that asymmetry is what identified the cause: a uniform failure with a single odd survivor points at the shared output path, not at the subject. Two fixes, and both are the general lesson: put anything platform-dependent in its OWN program so it cannot take unrelated assertions down with it, and give it a **viability probe asked with the action GRANTED** — so the only thing that can fail is the executor — reporting UNMEASURABLE rather than a governance FAILURE. Verified by forcing the probe to fail: 10 pass, 2 skip, 0 fail. Note the first attempt to simulate that was itself a broken probe: copying the suite to `/tmp` made `SCRIPT_DIR/../../build/naab-lang` resolve nowhere and every arm returned `rc=127`; a suite must be simulated from its own directory. diff --git a/include/naab/string_ops.h b/include/naab/string_ops.h new file mode 100644 index 00000000..a8ae5143 --- /dev/null +++ b/include/naab/string_ops.h @@ -0,0 +1,66 @@ +// String operations shared by BOTH engines' method forms (`s.slice(...)`) and +// the string module (`string.slice(s, ...)`). +// +// Each existed as three hand-written copies -- VM callBuiltinMethod, two +// tree-walker method paths in call_dispatch.cpp, and the module -- and the +// copies disagreed. Measured on "hello" / "a-b-c": +// +// s.replace("-", "+") VM "a+b-c" (first only) tree-walker "a+b+c" +// s.substring(3, 1) VM "" tree-walker "lo" +// s.slice(-3) VM "llo" tree-walker "hello" +// +// The module functions agreed with each other in every case, so they are the +// reference: replace replaces every occurrence, substring clamps and yields "" +// for an empty or reversed range, slice follows JavaScript (negative indices +// count from the end, reversed range -> ""). Positions are byte offsets, as +// they always were. + +#pragma once + +#include + +namespace naab { +namespace strops { + +// substring(start[, end]): start < 0 -> 0; end clamped to the length; +// end <= start (including a negative end) -> "". +inline std::string substring(const std::string& s, int start, bool has_end, int end) { + int len = static_cast(s.size()); + if (start < 0) start = 0; + if (start >= len) return ""; + if (!has_end) return s.substr(static_cast(start)); + if (end > len) end = len; + if (end <= start) return ""; + return s.substr(static_cast(start), static_cast(end - start)); +} + +// slice(start[, end]), JavaScript semantics: a negative index counts from the +// end; out-of-range indices clamp; an empty or reversed range yields "". +inline std::string slice(const std::string& s, int start, bool has_end, int end) { + int len = static_cast(s.size()); + if (start < 0) start += len; + if (start < 0) start = 0; + if (start > len) start = len; + if (!has_end) end = len; + if (end < 0) end += len; + if (end < 0) end = 0; + if (end > len) end = len; + if (start >= end) return ""; + return s.substr(static_cast(start), static_cast(end - start)); +} + +// replace(from, to): every occurrence. An empty `from` returns s unchanged +// (replacing "" everywhere would never terminate). +inline std::string replaceAll(const std::string& s, const std::string& from, const std::string& to) { + if (from.empty()) return s; + std::string out = s; + size_t pos = 0; + while ((pos = out.find(from, pos)) != std::string::npos) { + out.replace(pos, from.size(), to); + pos += to.size(); + } + return out; +} + +} // namespace strops +} // namespace naab diff --git a/run-all-tests.sh b/run-all-tests.sh index f65b6d66..546a950c 100755 --- a/run-all-tests.sh +++ b/run-all-tests.sh @@ -1649,6 +1649,23 @@ else echo " test_path_policy_reach.sh: not found, skipping" fi +echo "" +echo "═══════════════════════════════════════════════════════════" +echo " process.run Inline Code Gate (python -c, sh -c, ...)" +echo "═══════════════════════════════════════════════════════════" +echo "" +PROC_INLINE_SCRIPT="tests/security/test_process_run_inline_gate.sh" +if [ -f "$PROC_INLINE_SCRIPT" ]; then + if run_shell_test "$PROC_INLINE_SCRIPT" 2>&1; then + echo " test_process_run_inline_gate.sh: ALL PASSED" + else + FAILED=$((FAILED + 1)) + FAILED_TESTS+=("test_process_run_inline_gate.sh") + fi +else + echo " test_process_run_inline_gate.sh: not found, skipping" +fi + # --- Signed govern.json vs Package Operations (F39) --- echo "" echo "═══════════════════════════════════════════════════════════" diff --git a/src/interpreter/call_dispatch.cpp b/src/interpreter/call_dispatch.cpp index 367e574f..12642d2b 100644 --- a/src/interpreter/call_dispatch.cpp +++ b/src/interpreter/call_dispatch.cpp @@ -27,6 +27,8 @@ #include #include +#include "naab/string_ops.h" + namespace naab { namespace interpreter { @@ -1020,33 +1022,29 @@ void Interpreter::visit(ast::CallExpr& node) { result_ = NaabVal::makeInt(pos != std::string::npos ? static_cast(pos) : -1); return; } - if (method_name == "substring" || method_name == "substr" || method_name == "slice") { + if (method_name == "substring" || method_name == "substr") { if (args.empty()) throw std::runtime_error("string.substring() requires at least 1 argument"); - int start = args[0].asInt(); - if (start < 0) start = 0; - if (start >= static_cast(str.size())) { - result_ = NaabVal::makeString(""); - return; - } - if (args.size() >= 2) { - int end = args[1].asInt(); - if (end > static_cast(str.size())) end = static_cast(str.size()); - result_ = NaabVal::makeString(str.substr(static_cast(start), static_cast(end - start))); - } else { - result_ = NaabVal::makeString(str.substr(static_cast(start))); - } + result_ = NaabVal::makeString(strops::substring( + str, args[0].asInt(), args.size() >= 2, args.size() >= 2 ? args[1].asInt() : 0)); + return; + } + // slice is NOT substring: JavaScript semantics, as on the VM. This path + // aliased it to substring, so s.slice(-3) gave "hello" here and "llo" on + // the VM; substring's reversed-range case also returned the rest of the + // string ("lo") via a wrapped size_t. + if (method_name == "slice") { + if (args.empty()) throw std::runtime_error("string.slice() requires at least 1 argument (start)"); + result_ = NaabVal::makeString(strops::slice( + str, args[0].asInt(), args.size() >= 2, args.size() >= 2 ? args[1].asInt() : 0)); return; } if (method_name == "replace") { + // Through strops::replaceAll: this loop had no empty-pattern guard, and + // find("") matches at every position, so "ab".replace("", "+") inserted + // forever -- an unbounded hang with growing memory that --timeout cannot + // stop (it never returns to the interpreter). REST runs the tree-walker. if (args.size() < 2) throw std::runtime_error("string.replace() requires 2 arguments (old, new)"); - std::string old_s = args[0].toString(), new_s = args[1].toString(); - std::string result = str; - size_t pos = 0; - while ((pos = result.find(old_s, pos)) != std::string::npos) { - result.replace(pos, old_s.length(), new_s); - pos += new_s.length(); - } - result_ = NaabVal::makeString(result); + result_ = NaabVal::makeString(strops::replaceAll(str, args[0].toString(), args[1].toString())); return; } if (method_name == "toUpperCase" || method_name == "upper") { @@ -2010,34 +2008,29 @@ void Interpreter::visit(ast::CallExpr& node) { result_ = NaabVal::makeInt(pos != std::string::npos ? static_cast(pos) : -1); return; } - if (method_name == "substring" || method_name == "substr" || method_name == "slice") { + if (method_name == "substring" || method_name == "substr") { if (args.empty()) throw std::runtime_error("string.substring() requires at least 1 argument (start)"); - int start = args[0].asInt(); - if (start < 0) start = 0; - if (start >= static_cast(str.size())) { - result_ = NaabVal::makeString(""); - return; - } - if (args.size() >= 2) { - int end = args[1].asInt(); - if (end > static_cast(str.size())) end = static_cast(str.size()); - result_ = NaabVal::makeString(str.substr(static_cast(start), static_cast(end - start))); - } else { - result_ = NaabVal::makeString(str.substr(static_cast(start))); - } + result_ = NaabVal::makeString(strops::substring( + str, args[0].asInt(), args.size() >= 2, args.size() >= 2 ? args[1].asInt() : 0)); + return; + } + // slice is NOT substring: JavaScript semantics, as on the VM. This path + // aliased it to substring, so s.slice(-3) gave "hello" here and "llo" on + // the VM; substring's reversed-range case also returned the rest of the + // string ("lo") via a wrapped size_t. + if (method_name == "slice") { + if (args.empty()) throw std::runtime_error("string.slice() requires at least 1 argument (start)"); + result_ = NaabVal::makeString(strops::slice( + str, args[0].asInt(), args.size() >= 2, args.size() >= 2 ? args[1].asInt() : 0)); return; } if (method_name == "replace") { + // Through strops::replaceAll: this loop had no empty-pattern guard, and + // find("") matches at every position, so "ab".replace("", "+") inserted + // forever -- an unbounded hang with growing memory that --timeout cannot + // stop (it never returns to the interpreter). REST runs the tree-walker. if (args.size() < 2) throw std::runtime_error("string.replace() requires 2 arguments (old, new)"); - std::string old_str = args[0].toString(); - std::string new_str = args[1].toString(); - std::string result = str; - size_t pos = 0; - while ((pos = result.find(old_str, pos)) != std::string::npos) { - result.replace(pos, old_str.length(), new_str); - pos += new_str.length(); - } - result_ = NaabVal::makeString(result); + result_ = NaabVal::makeString(strops::replaceAll(str, args[0].toString(), args[1].toString())); return; } if (method_name == "toUpperCase" || method_name == "upper") { diff --git a/src/stdlib/process_impl.cpp b/src/stdlib/process_impl.cpp index 449b04d8..ef9cda4d 100644 --- a/src/stdlib/process_impl.cpp +++ b/src/stdlib/process_impl.cpp @@ -15,6 +15,7 @@ #include #include #include +#include #ifndef _WIN32 # include @@ -28,6 +29,52 @@ namespace naab { namespace stdlib { +namespace { + +// Recognise ` ` and return the code with +// the governance language name. The command may carry a directory, a Windows +// .exe suffix or a version suffix (python3.12, /usr/bin/node). Short flags may +// be clustered (`bash -ec`, `python3 -Ic`) when the code flag comes last. +bool inlineInterpreterCode(const std::string& cmd, + const std::vector& argv, + std::string& lang, std::string& code) { + std::string base = cmd; + auto slash = base.find_last_of("/\\"); + if (slash != std::string::npos) base = base.substr(slash + 1); + for (auto& c : base) c = static_cast(std::tolower(static_cast(c))); + if (base.size() > 4 && base.compare(base.size() - 4, 4, ".exe") == 0) + base.resize(base.size() - 4); + // strip a trailing version: python3.12 -> python, ruby3.2 -> ruby + while (!base.empty() && (std::isdigit(static_cast(base.back())) || base.back() == '.')) + base.pop_back(); + + std::string short_flags; // single letters that take the code as the next arg + std::vector long_flags; + if (base == "python" || base == "pypy") { lang = "python"; short_flags = "c"; } + else if (base == "node" || base == "nodejs") { lang = "javascript"; short_flags = "ep"; long_flags = {"--eval", "--print"}; } + else if (base == "ruby") { lang = "ruby"; short_flags = "e"; } + else if (base == "perl") { lang = "perl"; short_flags = "eE"; } + else if (base == "php") { lang = "php"; short_flags = "r"; } + else if (base == "bash" || base == "sh" || base == "dash" || base == "zsh" || base == "ksh") { lang = "shell"; short_flags = "c"; } + else return false; + + for (size_t i = 0; i + 1 < argv.size(); ++i) { + const std::string& a = argv[i]; + bool hit = false; + for (const auto& lf : long_flags) if (a == lf) hit = true; + if (!hit && a.size() >= 2 && a[0] == '-' && a[1] != '-' && + short_flags.find(a.back()) != std::string::npos) hit = true; + if (hit) { code = argv[i + 1]; return true; } + } + // `node --eval=CODE` + for (const auto& a : argv) + for (const auto& lf : long_flags) + if (a.rfind(lf + "=", 0) == 0) { code = a.substr(lf.size() + 1); return true; } + return false; +} + +} // namespace + bool ProcessModule::hasFunction(const std::string& name) const { static const std::unordered_set functions = { "run", "exit", "kill", "getpid" @@ -87,6 +134,20 @@ interpreter::NaabVal ProcessModule::call( for (const auto& a : argv_vec) full_cmd += " " + a; std::string gerr = gov->checkShellCommandAllowed(full_cmd); if (!gerr.empty()) throw std::runtime_error(gerr); + + // Inline interpreter code (python3 -c, node -e, sh -c, ...) is a + // polyglot block by another name. It used to skip every code check + // a <> block or codegen.run() gets: measured, the same + // `except Exception: return 0` was a HARD block in both of those + // and ran silently here. Route it through the same check. + if (gov->isActive()) { + std::string lang, code; + if (inlineInterpreterCode(cmd, argv_vec, lang, code)) { + std::string berr = gov->checkPolyglotBlock( + lang, code, "", 0); + if (!berr.empty()) throw std::runtime_error(berr); + } + } } std::string stdout_str, stderr_str; diff --git a/src/stdlib/string_impl.cpp b/src/stdlib/string_impl.cpp index adf5a336..f23d1240 100644 --- a/src/stdlib/string_impl.cpp +++ b/src/stdlib/string_impl.cpp @@ -6,6 +6,7 @@ #include "naab/stdlib_new_modules.h" #include "naab/interpreter.h" #include "naab/utils/string_utils.h" +#include "naab/string_ops.h" #include #include #include @@ -30,7 +31,7 @@ static interpreter::NaabVal makeStringArray(const std::vector& arr) bool StringModule::hasFunction(const std::string& name) const { static const std::unordered_set functions = { - "length", "substring", "concat", "split", "join", + "length", "substring", "slice", "concat", "split", "join", "trim", "upper", "lower", "replace", "contains", "starts_with", "ends_with", "index_of", "repeat", "char_at", "reverse", "format", "fmt", @@ -57,16 +58,23 @@ interpreter::NaabVal StringModule::call( if (args.size() != 3) { throw std::runtime_error("substring() takes exactly 3 arguments"); } - std::string s = getString(args[0]); - int start = getInt(args[1]); - int end = getInt(args[2]); - - // Bounds checking - if (start < 0) start = 0; - if (end > static_cast(s.length())) end = s.length(); - if (start >= end) return makeString(""); + return makeString(strops::substring(getString(args[0]), getInt(args[1]), true, getInt(args[2]))); + } - return makeString(s.substr(start, end - start)); + // slice(s, start[, end]): JavaScript semantics, the same as the method + // form s.slice(...) -- negative indices count from the end. It existed only + // as a method, so string.slice(s, ...) was an unknown function while + // s.slice(...) worked (repo-sentinel F-007). + if (function_name == "slice") { + if (args.size() != 2 && args.size() != 3) { + throw std::runtime_error( + "slice() takes 2 or 3 arguments\n\n" + " Expected: string.slice(s, start[, end])\n" + " Example: string.slice(\"hello\", -3) // \"llo\"\n"); + } + bool has_end = args.size() == 3; + return makeString(strops::slice(getString(args[0]), getInt(args[1]), has_end, + has_end ? getInt(args[2]) : 0)); } // Function 3: concat @@ -167,14 +175,7 @@ interpreter::NaabVal StringModule::call( std::string old_str = getString(args[1]); std::string new_str = getString(args[2]); - if (old_str.empty()) return makeString(s); - - size_t pos = 0; - while ((pos = s.find(old_str, pos)) != std::string::npos) { - s.replace(pos, old_str.length(), new_str); - pos += new_str.length(); - } - return makeString(s); + return makeString(strops::replaceAll(s, old_str, new_str)); } // Function 10: contains @@ -443,10 +444,10 @@ interpreter::NaabVal StringModule::call( // Names from JavaScript and Python that an LLM (or a person) reaches for. // The generic "did you mean" picks by edit distance, which sent `slice` // to split(); these say the equivalent directly. - if (function_name == "slice" || function_name == "substr") { + if (function_name == "substr") { throw std::runtime_error( - "Unknown string function: " + function_name + "\n\n" - " Did you mean: string.substring(s, start, end)? The end index is exclusive.\n" + "Unknown string function: substr\n\n" + " Did you mean: string.substring(s, start, end)? It takes an END index, not a length.\n" " Example: string.substring(\"hello\", 1, 3) // \"el\"\n" ); } @@ -542,7 +543,7 @@ interpreter::NaabVal StringModule::call( // Generic unknown function with suggestions static const std::vector FUNCTIONS = { - "length", "substring", "upper", "lower", "trim", "split", + "length", "substring", "slice", "upper", "lower", "trim", "split", "contains", "starts_with", "ends_with", "replace", "index_of", "char_at", "repeat", "reverse", "format", "fmt", "pad_left", "pad_right" diff --git a/src/vm/vm.cpp b/src/vm/vm.cpp index 3d17a381..cb4b6c86 100644 --- a/src/vm/vm.cpp +++ b/src/vm/vm.cpp @@ -10,6 +10,7 @@ #include "naab/language_registry.h" #include "naab/module_resolver.h" #include "naab/stdlib.h" +#include "naab/string_ops.h" #include "naab/stdlib_new_modules.h" #include "naab/sandbox.h" #include "naab/error_helpers.h" @@ -4288,36 +4289,21 @@ interpreter::NaabVal VM::callBuiltinMethod(interpreter::NaabVal& obj, const std: } if (method == "substring" || method == "substr") { if (argc < 1) runtimeError("substring() requires at least 1 argument"); - int start = args[0].toInt(); - if (start < 0) start = 0; - if (start >= static_cast(s.size())) return interpreter::NaabVal::makeString(""); - if (argc >= 2) { - int end_val = args[1].toInt(); - if (end_val < start) return interpreter::NaabVal::makeString(""); - return interpreter::NaabVal::makeString(s.substr(start, end_val - start)); - } - return interpreter::NaabVal::makeString(s.substr(start)); + return interpreter::NaabVal::makeString( + strops::substring(s, args[0].toInt(), argc >= 2, argc >= 2 ? args[1].toInt() : 0)); } if (method == "slice") { if (argc < 1) runtimeError("slice() requires at least 1 argument"); - int len = static_cast(s.size()); - int start = args[0].toInt(); - if (start < 0) start += len; - if (start < 0) start = 0; - int end_val = argc >= 2 ? args[1].toInt() : len; - if (end_val < 0) end_val += len; - if (end_val > len) end_val = len; - if (start >= end_val) return interpreter::NaabVal::makeString(""); - return interpreter::NaabVal::makeString(s.substr(start, end_val - start)); + return interpreter::NaabVal::makeString( + strops::slice(s, args[0].toInt(), argc >= 2, argc >= 2 ? args[1].toInt() : 0)); } if (method == "replace") { + // Every occurrence, like string.replace() and the tree-walker. This + // replaced only the FIRST, so s.replace("-", "+") gave "a+b-c" on + // the VM and "a+b+c" on the tree-walker. if (argc < 2) runtimeError("replace() requires 2 arguments"); - std::string result = s; - std::string from = args[0].toString(); - std::string to = args[1].toString(); - size_t pos = result.find(from); - if (pos != std::string::npos) result.replace(pos, from.size(), to); - return interpreter::NaabVal::makeString(std::move(result)); + return interpreter::NaabVal::makeString( + strops::replaceAll(s, args[0].toString(), args[1].toString())); } if (method == "startsWith" || method == "starts_with") { if (argc < 1) runtimeError("startsWith() requires 1 argument"); diff --git a/tests/differential/corpus.list b/tests/differential/corpus.list index 5c174e50..8998da11 100644 --- a/tests/differential/corpus.list +++ b/tests/differential/corpus.list @@ -19,6 +19,7 @@ tests/differential/corpus/null_coalesce.naab probe tests/differential/corpus/plus_overload_order.naab probe tests/differential/corpus/precedence.naab probe tests/differential/corpus/string_ops.naab probe +tests/differential/corpus/string_methods.naab probe # Reused engine test corpus (governance/taint/typed-scoring robustness tests excluded — # they legitimately differ across engines or need the repo govern.json): tests/vm/test_enum_module.naab vm diff --git a/tests/differential/corpus/string_methods.naab b/tests/differential/corpus/string_methods.naab new file mode 100644 index 00000000..696ae925 --- /dev/null +++ b/tests/differential/corpus/string_methods.naab @@ -0,0 +1,28 @@ +// Differential probe: string METHOD forms (s.slice / s.substring / s.replace) +// against each other and against the string module. The method forms had one +// copy per engine and they disagreed -- s.replace replaced only the first match +// on the VM, and the tree-walker aliased slice to substring and returned the +// rest of the string for a reversed substring range. string_ops.naab only +// exercised the module functions, so none of it showed up here. +use string +main { + let s = "a-b-c" + let h = "hello" + print(s.replace("-", "+")) + print(string.replace(s, "-", "+")) + print(s.replace("", "+")) + print(h.substring(1, 3)) + print("[" + h.substring(3, 1) + "]") + print("[" + h.substring(1, -1) + "]") + print(h.substring(-3, 5)) + print(h.substring(2)) + print(h.slice(1)) + print(h.slice(-3)) + print(h.slice(1, -1)) + print("[" + h.slice(3, 1) + "]") + print(h.slice(-99, 99)) + print(string.slice(h, -3)) + print(string.slice(h, 1, -1)) + print("[" + string.slice(h, 3, 1) + "]") + print("END") +} diff --git a/tests/parser/test_dogfood_hints.sh b/tests/parser/test_dogfood_hints.sh index 9dec7f16..1ddd0022 100755 --- a/tests/parser/test_dogfood_hints.sh +++ b/tests/parser/test_dogfood_hints.sh @@ -127,7 +127,9 @@ echo "=== S: string names from other languages point at the NAAb one ===" # Match the suggestion itself ("Did you mean: string.X"): the old message # listed every function after "Available:", so a loose match on the name # passed S-01 on the build that suggested split(). -for case_ in "S-01:slice:string.substring" "S-02:includes:string.contains" \ +# (S-01 was slice -> substring; string.slice now exists, see S-06. substr keeps +# a hint because JavaScript's substr takes a LENGTH, not an end index.) +for case_ in "S-01:substr:string.substring" "S-02:includes:string.contains" \ "S-03:padStart:string.pad_left" "S-04:replaceAll:string.replace" \ "S-05:trimStart:string.trim"; do id="${case_%%:*}"; rest="${case_#*:}"; fn="${rest%%:*}"; want="${rest#*:}" @@ -143,6 +145,40 @@ out="$(run "" s_ok.naab)" if grep -qx 'el' <<<"$out"; then pass S-00 "control: string.substring still works" else fail S-00 "control: string.substring broke" "$(head -2 <<<"$out")"; fi +# S-06 (round 3): `s.slice(...)` worked as a METHOD while `string.slice(...)` +# was an unknown function. It is now a module function with the method's +# JavaScript semantics, in both engines. +printf 'use string\nmain {\n print(string.slice("hello", -3))\n print(string.slice("hello", 1, -1))\n}\n' > "$WORK/s_slice.naab" +for eng in "" "--tree-walk"; do + tag="${eng:-vm}"; tag="${tag#--}" + out="$(run "$eng" s_slice.naab)" + if [ "$(grep -xE 'llo|ell' <<<"$out" | tr '\n' ' ')" = "llo ell " ]; then + pass "S-06/$tag" "string.slice works, negative indices count from the end" + else + fail "S-06/$tag" "string.slice missing or wrong" "$(head -3 <<<"$out")" + fi +done + +echo "=== E: an empty pattern cannot hang replace ===" +# The tree-walker's s.replace had no empty-pattern guard: find("") matches at +# every position, so "ab".replace("", "+") inserted forever with growing +# memory, and --timeout could not stop it (the loop never returns to the +# interpreter). The REST API runs the tree-walker. The expected output is the +# string unchanged, as string.replace already did. +printf 'main {\n print("ab".replace("", "+"))\n print("E_DONE")\n}\n' > "$WORK/e_empty.naab" +for eng in "" "--tree-walk"; do + tag="${eng:-vm}"; tag="${tag#--}" + out="$( (cd "$WORK" && timeout 10 "$NAAB" $eng "$WORK/e_empty.naab" 2>&1) )" + rc=$? + if [ "$rc" -eq 124 ]; then + fail "E-01/$tag" "s.replace(\"\", ...) hung (killed after 10s)" + elif grep -qx 'ab' <<<"$out" && grep -qx 'E_DONE' <<<"$out"; then + pass "E-01/$tag" "empty-pattern replace returns the string unchanged" + else + fail "E-01/$tag" "empty-pattern replace gave the wrong result" "$(head -3 <<<"$out")" + fi +done + echo "" echo "Results: $PASS passed, $FAIL failed" [ "$FAIL" -eq 0 ] diff --git a/tests/security/test_process_run_inline_gate.sh b/tests/security/test_process_run_inline_gate.sh new file mode 100755 index 00000000..dba5eb80 --- /dev/null +++ b/tests/security/test_process_run_inline_gate.sh @@ -0,0 +1,153 @@ +#!/usr/bin/env bash +# test_process_run_inline_gate.sh -- inline interpreter code passed through +# process.run gets the same governance checks as a <> block and +# codegen.run(). +# +# Found building repo-sentinel in NAAb: a helper that swallowed every error +# (`except Exception: return 0`) and reported every C++ file as 0 bytes. The +# project's own `code_quality.no_incomplete_logic` (HARD) blocks that code in a +# <> block and in codegen.run(), but the program ran it as +# process.run("python3", ["-c", code]) and it went through silently. +# +# Every blocking arm asserts on a SIDE EFFECT (the code writes a file), not only +# on the exit code -- an exit code is not a block. The controls matter as much: +# PI-00 the <> block is blocked under this config (the reference) +# PI-02 benign inline code still runs (a gate that refuses every +# process.run("python3", ["-c", ...]) passes PI-01 for free) +# PI-05 a non-interpreter command carrying the same text as an argument +# is not treated as code +# PI-06 a script FILE is not inline code; it is outside this gate (stated +# scope, not an endorsement) + +set -uo pipefail +PASS=0 +FAIL=0 +SKIP=0 +REPO="$(cd "$(dirname "$0")/../.." && pwd)" +NAAB="${NAAB:-$REPO/build/naab-lang}" +WORK="$(mktemp -d "${TMPDIR:-/tmp}/naab_procgate.XXXXXX")" +[ -n "$WORK" ] && [ -d "$WORK" ] || { echo "FATAL: could not create work dir" >&2; exit 1; } +source "$REPO/tests/helpers/trust_setup.sh" +setup_isolated_trust +trap 'rm -rf "$WORK"; teardown_isolated_trust' EXIT + +pass() { echo " PASS [$1] $2"; PASS=$((PASS+1)); } +fail() { echo " FAIL [$1] $2"; [ -n "${3:-}" ] && echo " $3"; FAIL=$((FAIL+1)); } +skip() { echo " SKIP [$1] $2 (UNMEASURABLE)"; SKIP=$((SKIP+1)); } + +if [ ! -x "$NAAB" ]; then + echo "FAIL: naab-lang not built at $NAAB (UNMEASURABLE, not a pass)" + exit 1 +fi +PY="$(command -v python3 || true)" +if [ -z "$PY" ]; then + echo "SKIP: python3 not available -- every arm needs a real interpreter (UNMEASURABLE)" + exit 0 +fi + +cat > "$WORK/govern.json" <<'EOF' +{ + "version": "4.0", + "mode": "enforce", + "security": { "sandbox_level": "elevated" }, + "code_quality": { "no_incomplete_logic": { "enabled": true, "level": "hard" } } +} +EOF + +# The swallowed-error helper, plus a marker write so "it ran" is observable. +BAD='import os\ndef size_of(p):\n try:\n return os.path.getsize(p)\n except Exception:\n return 0\nopen(\"MARK\", \"w\").write(\"x\")\nprint(size_of(\"missing.cpp\"))\n' +GOOD='import os\nopen(\"MARK\", \"w\").write(\"x\")\nprint(os.path.exists(\"MARK\"))\n' + +# write_prog NAME CMD ARGS_EXPR MARK -- ARGS_EXPR is a NAAb list literal +write_prog() { + local name="$1" cmd="$2" args="$3" mark="$4" + printf 'use process\nmain {\n let r = process.run("%s", %s)\n print("EXIT:" + string(r["exit_code"]))\n}\n' \ + "$cmd" "${args//MARK/$mark}" > "$WORK/$name.naab" +} + +run() { # run FLAG NAME -> sets OUT, RC + OUT="$(cd "$WORK" && timeout 30 "$NAAB" $1 "$WORK/$2.naab" 2>&1)" + RC=$? +} + +for eng in "" "--tree-walk"; do + tag="${eng:-vm}"; tag="${tag#--}" + echo "=== engine: $tag ===" + + # PI-00: reference -- the same code as a <> block + rm -f "$WORK"/m00 + printf 'main {\n let x = <>\n print(x)\n}\n' > "$WORK/p00.naab" + run "$eng" p00 + if [ "$RC" -eq 3 ] && [ ! -e "$WORK/m00" ]; then + pass "PI-00/$tag" "reference: the <> block is HARD-blocked and did not run" + else + fail "PI-00/$tag" "reference block not blocked (rc=$RC) -- the config does not exercise the check, every other arm is void" "$(head -3 <<<"$OUT")" + fi + + # PI-01: python3 -c + rm -f "$WORK"/m01 + write_prog p01 python3 "[\"-c\", \"$BAD\"]" m01 + run "$eng" p01 + if [ "$RC" -eq 3 ] && [ ! -e "$WORK/m01" ]; then + pass "PI-01/$tag" "process.run(\"python3\", [\"-c\", code]) gets the block's check" + else + fail "PI-01/$tag" "inline python ran past the check (rc=$RC, marker $( [ -e "$WORK/m01" ] && echo written || echo absent ))" "$(head -3 <<<"$OUT")" + fi + + # PI-02: control -- benign inline code still runs + rm -f "$WORK"/m02 + write_prog p02 python3 "[\"-c\", \"$GOOD\"]" m02 + run "$eng" p02 + if [ "$RC" -eq 0 ] && [ -e "$WORK/m02" ] && [[ "$OUT" == *"EXIT:0"* ]]; then + pass "PI-02/$tag" "control: benign inline python still runs" + else + fail "PI-02/$tag" "benign inline python refused (rc=$RC)" "$(head -3 <<<"$OUT")" + fi + + # PI-03: absolute interpreter path + clustered short flags (-Ic) + rm -f "$WORK"/m03 + write_prog p03 "$PY" "[\"-Ic\", \"$BAD\"]" m03 + run "$eng" p03 + if [ "$RC" -eq 3 ] && [ ! -e "$WORK/m03" ]; then + pass "PI-03/$tag" "absolute path and a clustered flag (-Ic) do not evade the check" + else + fail "PI-03/$tag" "evaded via path/flag spelling (rc=$RC)" "$(head -3 <<<"$OUT")" + fi + + # PI-04: the same code reached through sh -c is shell, not python -- but a + # python3 -c nested inside sh -c is still one hop away. Stated scope: only + # the outer interpreter is recognised. Measure that a plain sh -c benign + # command still runs (the shell mapping must not refuse ordinary use). + rm -f "$WORK"/m04 + write_prog p04 sh '["-c", "echo x > MARK"]' m04 + run "$eng" p04 + if [ "$RC" -eq 0 ] && [ -e "$WORK/m04" ]; then + pass "PI-04/$tag" "control: benign sh -c still runs" + else + fail "PI-04/$tag" "benign sh -c refused (rc=$RC)" "$(head -3 <<<"$OUT")" + fi + + # PI-05: a non-interpreter command carrying the text is not code + write_prog p05 echo "[\"-c\", \"$BAD\"]" m05 + run "$eng" p05 + if [ "$RC" -eq 0 ] && [[ "$OUT" == *"EXIT:0"* ]]; then + pass "PI-05/$tag" "control: echo -c is not treated as inline code" + else + fail "PI-05/$tag" "non-interpreter command refused (rc=$RC)" "$(head -3 <<<"$OUT")" + fi + + # PI-06: a script file is outside this gate (scope, not endorsement) + rm -f "$WORK"/m06 + printf "$BAD" | sed 's/MARK/m06/; s/\\"/"/g' > "$WORK/s06.py" + write_prog p06 python3 '["s06.py"]' m06 + run "$eng" p06 + if [ "$RC" -eq 0 ] && [ -e "$WORK/m06" ]; then + pass "PI-06/$tag" "scope: a script file is not inline code and is not scanned here" + else + fail "PI-06/$tag" "script-file behaviour changed (rc=$RC) -- update the stated scope" "$(head -3 <<<"$OUT")" + fi +done + +echo "" +echo "Results: $PASS passed, $FAIL failed, $SKIP skipped" +[ "$FAIL" -eq 0 ]