diff --git a/changelog.d/warnet-lab.md b/changelog.d/warnet-lab.md new file mode 100644 index 000000000..d2e89dee0 --- /dev/null +++ b/changelog.d/warnet-lab.md @@ -0,0 +1,12 @@ +Fixed + +- The Warnet kind image tag is the release `28.0.0` (not an rbitcoin + version). Helm `semverCompare ">=0.17.0"` is true for that tag, so the + chart writes `[regtest]` before the tank's RPC and `addnode` lines. + `0.7.99` and any prerelease of it compare below `0.17.0` and omit the + section. The compose example stays `rbitcoin-warnet:local`. +- Lab tanks copy node log lines to stdout when `RBITCOIN_LOG_STDOUT=1`, + and the RPC proxy accepts Warnet's fork-observer `rpcauth` login for + its whitelisted chain methods while the tank `rpcpassword` user stays + unrestricted. A tank whose conf omits `head_scale` uses tiny heads + inside the lab image; `head_scale=mainnet` still selects mainnet heads. diff --git a/docs/core-functional.md b/docs/core-functional.md index 6094d324c..143b50ae3 100644 --- a/docs/core-functional.md +++ b/docs/core-functional.md @@ -293,12 +293,30 @@ python3 scripts/core-functional/create_cache.py --ensure ## Warnet lab (all-rbitcoin tanks) -Not an operator musl Release. Image tag `rbitcoin-warnet:local`. Warnet -stays Core Helm (`bitcoin.conf`, `rpcuser`/`rpcpassword`, `addnode=tank-N`, -`pidof bitcoind`). The node is not taught Core conf; the Python shim -translates `addnode=` to `--connect host:18444`, seeds `{datadir}/rpc.token` -from `rpcpassword`, and binds the test proxy on `rpcbind=0.0.0.0`. Basic -auth accepts any username whose password matches that token. +Not an operator musl Release. The compose example keeps image tag +`rbitcoin-warnet:local`. Kind and Helm use `rbitcoin-warnet:28.0.0`. +That tag is the chart's Core-version gate, not the rbitcoin version. +Warnet passes it to `semverCompare ">=0.17.0"` and, when the compare +succeeds, writes `[regtest]` before `rpcport`, `rpcpassword`, `rpcauth`, +and `addnode`. A prerelease does not count: Warnet main strips `-…` and +then compares, and `0.7.99` is below `0.17.0`, so the section is omitted +while `helm template` still exits 0. `local` is not a semantic version +(Warnet 1.1.20 compares the raw tag; current Warnet main normalizes a few +Core shapes and still rejects that fallback). A release tag `>= 0.17.0` +with no prerelease, such as `28.0.0`, emits the section on both. +Warnet stays Core Helm (`bitcoin.conf`, `rpcuser`/`rpcpassword`, +`addnode=tank-N`, `pidof bitcoind`). The node is not taught Core conf; the +Python shim translates `addnode=` to `--connect host:18444`, seeds +`{datadir}/rpc.token` from `rpcpassword`, and binds the test proxy on +`rpcbind=0.0.0.0`. Basic auth accepts the cookie, any username whose +password matches that token, and Core `rpcauth=` (HMAC-SHA256, salt as +the key). `rpcwhitelistdefault=0` limits only users named in +`rpcwhitelist=`, so fork-observer's published login can call +`getchaintips`, `getblockheader`, `getblockhash`, `getblock`, and +`getnetworkinfo`, and the tank `rpcuser` is not locked out. The image +sets `RBITCOIN_LOG_STDOUT=1` (node lines on container stdout and in +`debug.log`) and `RBITCOIN_LAB_HEAD_SCALE=tiny` when conf omits +`head_scale`. Conf `head_scale=mainnet` still selects mainnet heads. Hostname `--connect` / `addnode add` resolve at each dial on a blocking thread and retry every 2 seconds until a live session exists. `--connect` @@ -322,8 +340,9 @@ cargo build -p rbitcoin-node ./scripts/core-functional/warnet/example.sh ``` -The example conf sets `head_scale=tiny` so a tank does not fallocate mainnet -heads. `example.sh` copies a **bookworm-linked** `rbitcoin-node` into the image +The example conf sets `head_scale=tiny`. The lab image does the same when +conf omits it, so a Helm tank does not fallocate mainnet heads. `example.sh` +copies a **bookworm-linked** `rbitcoin-node` into the image (a nix devshell binary will not exec: its dynamic loader is not in Debian). It starts `tank1` first (`addnode=tank0`), waits until that RPC is up at height 0 with no peer, then starts `tank0`, mines one regtest block, and @@ -339,17 +358,19 @@ Needs Docker + kind on an operator host. Do not open mainnet. Wallet keys are RAM-only. ```bash -docker build -t rbitcoin-warnet:local \ +docker build -t rbitcoin-warnet:28.0.0 \ --build-arg NODE_BIN=target/dev/debug/rbitcoin-node \ -f scripts/core-functional/warnet/Dockerfile . -kind load docker-image rbitcoin-warnet:local +kind load docker-image rbitcoin-warnet:28.0.0 python3 -m venv .venv && source .venv/bin/activate pip install warnet warnet setup warnet new /tmp/rbtc-warnet ``` -Three tanks, ring `addnode`, unique `rpcpassword`, `pullPolicy: Never`. +Three tanks, ring `addnode`, unique `rpcpassword`. Set +`image.repository` to `rbitcoin-warnet`, `image.tag` to `28.0.0`, +and `pullPolicy: Never`. Pass when `miner_std.py --interval=10 --mature` leaves all three `getblockcount` values equal and greater than 0. Fail classes: CrashLoop `pidof`; 401 on RPC; height only on the miner (DNS or bind); ImagePullBackOff diff --git a/scripts/core-functional/bitcoind b/scripts/core-functional/bitcoind index 4438904f5..a3d82f26d 100755 --- a/scripts/core-functional/bitcoind +++ b/scripts/core-functional/bitcoind @@ -678,15 +678,24 @@ def translate( if log_level: cmd += ["--log-level", log_level] cmd += extra_node_args - for scale in read_conf_keys(datadir / "bitcoin.conf", "head_scale"): - if scale == "tiny" or scale == "mainnet": - cmd += ["--head-scale", scale] for host in read_conf_keys(datadir / "bitcoin.conf", "addnode") + addnodes: if not host: continue if ":" not in host: host = f"{host}:{p2pport}" cmd += ["--connect", host] + scales = [ + s for s in read_conf_keys(datadir / "bitcoin.conf", "head_scale") if s + ] + # Lab entrypoint sets RBITCOIN_LAB_HEAD_SCALE=tiny. An explicit conf + # value wins, including mainnet. Unset env leaves the node default. + if not scales: + lab_scale = os.environ.get("RBITCOIN_LAB_HEAD_SCALE", "").strip() + if lab_scale == "tiny": + scales = ["tiny"] + for scale in scales: + if scale == "tiny" or scale == "mainnet": + cmd += ["--head-scale", scale] pw = (conf.get("rpcpassword") or "").strip() if pw: @@ -751,6 +760,14 @@ def main(argv: list[str] | None = None) -> int: # TestNode tails this process's stderr and requires it empty on a clean # stop. Core writes logs to debug.log (printtoconsole=0). Keep that shape: # node stdio → {datadir}/debug.log; only `Error:` lines on shim stderr. + # The lab image sets RBITCOIN_LOG_STDOUT=1 so kubectl / warnet logs see + # the same bytes. Functional tests leave the switch off. + log_stdout = os.environ.get("RBITCOIN_LOG_STDOUT", "").strip().lower() in ( + "1", + "true", + "yes", + "on", + ) log_path = node_dir / "debug.log" prior = log_path.stat().st_size if log_path.is_file() else 0 log_f = open(log_path, "ab", buffering=0) @@ -782,6 +799,12 @@ def main(argv: list[str] | None = None) -> int: for mapped in apply_line(text, rules): log_f.write((mapped + "\n").encode()) log_f.flush() + if log_stdout and raw: + try: + sys.stdout.buffer.write(raw) + sys.stdout.buffer.flush() + except OSError: + pass pump = threading.Thread(target=_pump, daemon=True) pump.start() @@ -856,8 +879,17 @@ def main(argv: list[str] | None = None) -> int: try: core_datadir = node_dir.parent - rpcbind = read_conf(core_datadir / "bitcoin.conf").get("rpcbind") or "" - proxy = RpcProxy((proxy_bind_host(rpcbind), public_rpc), node_url, _cookie) + conf_path = core_datadir / "bitcoin.conf" + conf = read_conf(conf_path) + rpcbind = conf.get("rpcbind") or "" + proxy = RpcProxy( + (proxy_bind_host(rpcbind), public_rpc), + node_url, + _cookie, + rpcauth_lines=read_conf_keys(conf_path, "rpcauth"), + whitelist_lines=read_conf_keys(conf_path, "rpcwhitelist"), + whitelist_default=conf.get("rpcwhitelistdefault"), + ) except OSError: if child.poll() is None: child.terminate() diff --git a/scripts/core-functional/bitcoind.test.sh b/scripts/core-functional/bitcoind.test.sh index ea2fab3a1..c16daf7f8 100755 --- a/scripts/core-functional/bitcoind.test.sh +++ b/scripts/core-functional/bitcoind.test.sh @@ -345,6 +345,74 @@ else FAIL=$((FAIL + 1)) fi +# Lab image: no head_scale in conf, entrypoint default selects tiny. +# Explicit head_scale=mainnet still wins. Unset lab env leaves the node default. +HS_DD="$WORKDIR/warnet-no-scale" +mkdir -p "$HS_DD" +printf 'regtest=1\nrpcpassword=secret0\nrpcbind=0.0.0.0\naddnode=tank-0001\nport=18444\nrpcport=18443\n' >"$HS_DD/bitcoin.conf" +OUT_HS="$(RBITCOIN_LAB_HEAD_SCALE=tiny "$SHIM" --print-cmd -datadir="$HS_DD" -regtest 2>/dev/null)" || OUT_HS="" +if printf '%s' "$OUT_HS" | grep -q -- "--head-scale tiny" \ + && printf '%s' "$OUT_HS" | grep -q -- "--connect tank-0001:18444" \ + && [[ "$(cat "$HS_DD/regtest/rpc.token")" == "secret0" ]]; then + echo "ok - lab default head scale is tiny when conf omits it" + PASS=$((PASS + 1)) +else + echo "not ok - lab default head scale is tiny (got: $OUT_HS)" + FAIL=$((FAIL + 1)) +fi +printf 'regtest=1\nhead_scale=mainnet\nport=18444\nrpcport=18443\n' >"$HS_DD/bitcoin.conf" +OUT_HM="$(RBITCOIN_LAB_HEAD_SCALE=tiny "$SHIM" --print-cmd -datadir="$HS_DD" -regtest 2>/dev/null)" || OUT_HM="" +if printf '%s' "$OUT_HM" | grep -q -- "--head-scale mainnet" \ + && ! printf '%s' "$OUT_HM" | grep -q -- "--head-scale tiny"; then + echo "ok - conf head_scale=mainnet overrides the lab default" + PASS=$((PASS + 1)) +else + echo "not ok - conf head_scale=mainnet overrides the lab default (got: $OUT_HM)" + FAIL=$((FAIL + 1)) +fi +# Conf still says mainnet; drop it and leave the lab env unset. +printf 'regtest=1\nport=18444\nrpcport=18443\n' >"$HS_DD/bitcoin.conf" +OUT_HU="$(env -u RBITCOIN_LAB_HEAD_SCALE "$SHIM" --print-cmd -datadir="$HS_DD" -regtest 2>/dev/null)" || OUT_HU="" +if ! printf '%s' "$OUT_HU" | grep -q -- "--head-scale"; then + echo "ok - omitted head_scale without the lab env stays the node default" + PASS=$((PASS + 1)) +else + echo "not ok - omitted head_scale without the lab env (got: $OUT_HU)" + FAIL=$((FAIL + 1)) +fi + +# Lab log switch: node stdout is tee'd. Off, the shim stays quiet. +MARK="warnet-log-marker-$RANDOM-$$" +LOG_FAKE="$WORKDIR/rbitcoin-node-logmark" +printf '%s\n' '#!/bin/sh' "printf '%s\n' '$MARK'" 'exit 0' >"$LOG_FAKE" +chmod +x "$LOG_FAKE" +LOG_DD="$WORKDIR/log-on" +mkdir -p "$LOG_DD" +printf 'regtest=1\nport=18444\nrpcport=18443\n' >"$LOG_DD/bitcoin.conf" +LOG_OUT="$(RBITCOIN_NODE="$LOG_FAKE" RBITCOIN_LOG_STDOUT=1 "$SHIM" -datadir="$LOG_DD" -regtest 2>"$WORKDIR/log-on.err")" || true +if printf '%s\n' "$LOG_OUT" | grep -qx -- "$MARK" \ + && ! grep -q -- "$MARK" "$WORKDIR/log-on.err" \ + && grep -qx -- "$MARK" "$LOG_DD/regtest/debug.log"; then + echo "ok - RBITCOIN_LOG_STDOUT=1 tees the node line to stdout and debug.log" + PASS=$((PASS + 1)) +else + echo "not ok - log stdout on (out=$(printf %q "$LOG_OUT") err=$(cat "$WORKDIR/log-on.err") log=$(cat "$LOG_DD/regtest/debug.log" 2>/dev/null))" + FAIL=$((FAIL + 1)) +fi +LOG_DD2="$WORKDIR/log-off" +mkdir -p "$LOG_DD2" +printf 'regtest=1\nport=18444\nrpcport=18443\n' >"$LOG_DD2/bitcoin.conf" +LOG_OUT2="$(RBITCOIN_NODE="$LOG_FAKE" env -u RBITCOIN_LOG_STDOUT "$SHIM" -datadir="$LOG_DD2" -regtest 2>"$WORKDIR/log-off.err")" || true +if ! printf '%s\n' "$LOG_OUT2" | grep -q -- "$MARK" \ + && ! grep -q -- "$MARK" "$WORKDIR/log-off.err" \ + && grep -qx -- "$MARK" "$LOG_DD2/regtest/debug.log"; then + echo "ok - log switch off keeps the node line in debug.log only" + PASS=$((PASS + 1)) +else + echo "not ok - log stdout off (out=$(printf %q "$LOG_OUT2") err=$(cat "$WORKDIR/log-off.err") log=$(cat "$LOG_DD2/regtest/debug.log" 2>/dev/null))" + FAIL=$((FAIL + 1)) +fi + assert_fail_msg "port 65536 invalid" "Error: Invalid port specified in -port: '65536'" \ env RBITCOIN_NODE="$FAKE" "$SHIM" --print-cmd -datadir="$DATADIR" -regtest -listen -port=65536 assert_fail_msg "port 0 invalid" "Error: Invalid port specified in -port: '0'" \ diff --git a/scripts/core-functional/rpc_proxy.py b/scripts/core-functional/rpc_proxy.py index 911e9a9b6..6dbd7db74 100644 --- a/scripts/core-functional/rpc_proxy.py +++ b/scripts/core-functional/rpc_proxy.py @@ -9,6 +9,7 @@ from __future__ import annotations import base64 +import hmac import json import threading import urllib.error @@ -16,6 +17,8 @@ from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer from typing import Any, Callable +from rpcauth import password_to_hmac + # GBT longpoll can sit ~80s; stay under Core's client-side patience. FORWARD_TIMEOUT_S = 180.0 @@ -75,6 +78,149 @@ def authorization_ok(authorization: str, cookie_line: str | None) -> bool: return password == token_from_cookie_line(cookie_line) +def parse_rpcauth_line(line: str) -> tuple[str, str, str] | None: + """Core `rpcauth=user:salt$hash` (HMAC-SHA256, salt is the key).""" + raw = line.strip() + if ":" not in raw or "$" not in raw: + return None + user, rest = raw.split(":", 1) + salt, mac = rest.split("$", 1) + if not user or not salt or not mac: + return None + return user, salt, mac + + +def parse_whitelist_line(line: str) -> tuple[str, set[str]] | None: + """Core `rpcwhitelist=user:method,method`.""" + raw = line.strip() + if ":" not in raw: + return None + user, methods = raw.split(":", 1) + user = user.strip() + if not user: + return None + return user, {m.strip() for m in methods.split(",") if m.strip()} + + +def whitelist_map(lines: list[str]) -> dict[str, set[str]]: + out: dict[str, set[str]] = {} + for line in lines: + parsed = parse_whitelist_line(line) + if parsed is None: + continue + user, methods = parsed + # Core set-intersects a later rpcwhitelist line for the same user. + if user in out: + out[user] &= methods + else: + out[user] = methods + return out + + +def _hmac_ok(salt: str, password: str, expected: str) -> bool: + got = password_to_hmac(salt, password) + if len(got) != len(expected): + return False + return hmac.compare_digest(got, expected) + + +def _others_restricted(whitelist_default: str | None, has_entries: bool) -> bool: + """Core: default 0 limits only users who have a whitelist entry. + + Unset, or any other value, subjects every other user to an empty list + once a whitelist exists. Explicit non-zero with no entries denies all. + """ + if whitelist_default is None: + return has_entries + if whitelist_default.strip().lower() in ("0", "false", "no", "off"): + return False + return True + + +def method_permitted( + user: str, + method: str | None, + entries: dict[str, set[str]], + whitelist_default: str | None, +) -> bool: + if user in entries: + return method is not None and method in entries[user] + if _others_restricted(whitelist_default, bool(entries)): + return False + return True + + +def authenticated_user( + authorization: str, + cookie_line: str | None, + rpcauth_lines: list[str], +) -> str | None: + """Username that passed cookie, tank-token, or `rpcauth` HMAC. + + No cookie and no `rpcauth` lines stays open (the functional harness + before `.cookie` exists). A cookie line refuses a password that matches + neither the token nor an `rpcauth` row. + """ + parsed = parse_basic_userpass(authorization) + if not cookie_line and not rpcauth_lines: + return parsed[0] if parsed else "" + if cookie_line: + want = "Basic " + base64.b64encode(cookie_line.encode()).decode() + if authorization == want: + return "__cookie__" + if parsed is not None and parsed[1] == token_from_cookie_line(cookie_line): + return parsed[0] + if parsed is None: + return "" if not cookie_line else None + user, password = parsed + # Core keeps scanning rpcauth rows for this user (password rotation). + saw_user = False + for line in rpcauth_lines: + rec = parse_rpcauth_line(line) + if rec is None or rec[0] != user: + continue + saw_user = True + if _hmac_ok(rec[1], password, rec[2]): + return user + if saw_user or cookie_line: + return None + return user + + +def rpc_methods(payload: Any) -> list[str | None]: + if isinstance(payload, dict): + method = payload.get("method") + return [method if isinstance(method, str) else None] + if isinstance(payload, list): + out: list[str | None] = [] + for item in payload: + if isinstance(item, dict) and isinstance(item.get("method"), str): + out.append(item["method"]) + else: + out.append(None) + return out or [None] + return [None] + + +def request_authorized( + authorization: str, + cookie_line: str | None, + rpcauth_lines: list[str], + whitelist_lines: list[str], + whitelist_default: str | None, + payload: Any, +) -> str: + """`ok`, `unauthorized`, or `forbidden` for one HTTP RPC body.""" + user = authenticated_user(authorization, cookie_line, rpcauth_lines) + if user is None: + return "unauthorized" + entries = whitelist_map(whitelist_lines) + for method in rpc_methods(payload): + if not method_permitted(user, method, entries, whitelist_default): + return "forbidden" + return "ok" + + def core_btc_kvb_to_sat_vb(value: Any) -> int: """Core `maxfeerate` BTC/kvB → node sat/vB. `>= 1` is Core `-8`.""" if value is None: @@ -219,9 +365,15 @@ def __init__( listen: tuple[str, int], node_url: str, cookie_line: Callable[[], str | None], + rpcauth_lines: list[str] | None = None, + whitelist_lines: list[str] | None = None, + whitelist_default: str | None = None, ) -> None: self.node_url = node_url.rstrip("/") + "/" self.cookie_line = cookie_line + self.rpcauth_lines = list(rpcauth_lines or []) + self.whitelist_lines = list(whitelist_lines or []) + self.whitelist_default = whitelist_default self._handlers: dict[str, Callable[[Any], dict[str, Any]]] = {} proxy = self @@ -257,11 +409,25 @@ def shutdown(self) -> None: def handle_http(self, raw: bytes, authorization: str) -> tuple[int, bytes]: cookie = self.cookie_line() - if not authorization_ok(authorization, cookie): - return 401, b'{"error":"unauthorized"}\n' try: payload = json.loads(raw.decode() or "null") + parsed = True except (UnicodeDecodeError, json.JSONDecodeError): + payload = None + parsed = False + decision = request_authorized( + authorization, + cookie, + self.rpcauth_lines, + self.whitelist_lines, + self.whitelist_default, + payload, + ) + if decision == "unauthorized": + return 401, b'{"error":"unauthorized"}\n' + if decision == "forbidden": + return 403, b'{"error":"forbidden"}\n' + if not parsed: return self.forward_raw(raw) try: if isinstance(payload, list): diff --git a/scripts/core-functional/rpc_proxy.test.sh b/scripts/core-functional/rpc_proxy.test.sh index 8047d4bd8..6418813c2 100755 --- a/scripts/core-functional/rpc_proxy.test.sh +++ b/scripts/core-functional/rpc_proxy.test.sh @@ -11,9 +11,11 @@ import urllib.error import urllib.request from http.server import BaseHTTPRequestHandler, HTTPServer +from rpcauth import password_to_hmac from rpc_proxy import ( RpcError, RpcProxy, + authenticated_user, authorization_ok, core_btc_kvb_to_sat_vb, esplora_port, @@ -23,6 +25,7 @@ from rpc_proxy import ( rewrite_core_maxfeerate, rewrite_testmempoolaccept_abort, shim_gettxoutsetinfo, + whitelist_map, ) seq = { @@ -275,4 +278,161 @@ assert body["result"][1] == {"txid": "bb", "wtxid": "wb"}, body node.shutdown() proxy.shutdown() print("ok - rpc_proxy forward + local handler") + +# Warnet fork-observer: published rpcauth, whitelist, tank password, cookie. +# Drove through RpcProxy.handle_http (the shim's listen path). +FORK_AUTH = ( + "forkobserver:1418183465eecbd407010cf60811c6a0$" + "d4e5f0647a63429c218da1302d7f19fe627302aeb0a71a74de55346a25d8057c" +) +FORK_PASSWORD = "tabconf2024" +FORK_WHITELIST = ( + "forkobserver:getchaintips,getblockheader,getblockhash,getblock,getnetworkinfo" +) +TANK_PASSWORD = "secret0" +TANK_COOKIE = f"__cookie__:{TANK_PASSWORD}" + + +class TankNode(BaseHTTPRequestHandler): + def log_message(self, *args): + return + + def do_POST(self): + auth = self.headers.get("Authorization", "") + n = int(self.headers.get("Content-Length", "0")) + raw = self.rfile.read(n) + item = json.loads(raw.decode()) + if auth != "Bearer " + TANK_PASSWORD: + body = json.dumps( + { + "result": None, + "error": {"code": -32600, "message": "auth"}, + "id": item.get("id"), + } + ).encode() + self.send_response(401) + else: + body = json.dumps( + { + "result": item.get("method"), + "error": None, + "id": item.get("id"), + } + ).encode() + self.send_response(200) + self.send_header("Content-Type", "application/json") + self.send_header("Content-Length", str(len(body))) + self.end_headers() + self.wfile.write(body) + + +node2 = HTTPServer(("127.0.0.1", 0), TankNode) +threading.Thread(target=node2.serve_forever, daemon=True).start() +proxy2 = RpcProxy( + ("127.0.0.1", 0), + f"http://127.0.0.1:{node2.server_address[1]}", + lambda: TANK_COOKIE, + rpcauth_lines=[FORK_AUTH], + whitelist_lines=[FORK_WHITELIST], + whitelist_default="0", +) +proxy2.start() +port2 = proxy2._httpd.server_address[1] + + +def call2(method, user, password): + token = base64.b64encode(f"{user}:{password}".encode()).decode() + req = urllib.request.Request( + f"http://127.0.0.1:{port2}/", + data=json.dumps( + {"jsonrpc": "1.0", "id": 1, "method": method, "params": []} + ).encode(), + headers={ + "Content-Type": "application/json", + "Authorization": "Basic " + token, + }, + method="POST", + ) + try: + with urllib.request.urlopen(req, timeout=5) as resp: + return resp.status, json.loads(resp.read().decode()) + except urllib.error.HTTPError as e: + raw = e.read().decode() + try: + body = json.loads(raw) if raw else {} + except json.JSONDecodeError: + body = {"raw": raw} + return e.code, body + + +def call2_cookie(method): + token = base64.b64encode(TANK_COOKIE.encode()).decode() + req = urllib.request.Request( + f"http://127.0.0.1:{port2}/", + data=json.dumps( + {"jsonrpc": "1.0", "id": 1, "method": method, "params": []} + ).encode(), + headers={ + "Content-Type": "application/json", + "Authorization": "Basic " + token, + }, + method="POST", + ) + with urllib.request.urlopen(req, timeout=5) as resp: + return resp.status, json.loads(resp.read().decode()) + + +for method in ( + "getchaintips", + "getblockheader", + "getblockhash", + "getblock", + "getnetworkinfo", +): + st, body = call2(method, "forkobserver", FORK_PASSWORD) + assert st == 200, (method, st, body) + assert body["result"] == method, (method, body) + +st, body = call2("generatetoaddress", "forkobserver", FORK_PASSWORD) +assert st == 403, (st, body) +st, body = call2("generatetoaddress", "user", TANK_PASSWORD) +assert st == 200, (st, body) +assert body["result"] == "generatetoaddress", body +st, body = call2("getblock", "forkobserver", "not-the-password") +assert st == 401, (st, body) +st, body = call2_cookie("generatetoaddress") +assert st == 200, (st, body) +assert body["result"] == "generatetoaddress", body + +node2.shutdown() +proxy2.shutdown() +print("ok - fork-observer rpcauth whitelist and tank password") + +# Core keeps scanning rpcauth rows for one user. A failed HMAC on the +# first row must not hide a later row that matches (password rotation). +_salt_old = "11" * 16 +_salt_new = "22" * 16 +_old_line = f"rot:{_salt_old}${password_to_hmac(_salt_old, 'old-pass')}" +_new_line = f"rot:{_salt_new}${password_to_hmac(_salt_new, 'new-pass')}" + + +def _basic(user, password): + return "Basic " + base64.b64encode(f"{user}:{password}".encode()).decode() + + +assert authenticated_user(_basic("rot", "new-pass"), TANK_COOKIE, [_old_line, _new_line]) == "rot" +assert authenticated_user(_basic("rot", "old-pass"), TANK_COOKIE, [_old_line, _new_line]) == "rot" +assert authenticated_user(_basic("rot", "new-pass"), None, [_old_line, _new_line]) == "rot" +assert authenticated_user(_basic("rot", "nope"), TANK_COOKIE, [_old_line, _new_line]) is None +assert authenticated_user(_basic("rot", "nope"), None, [_old_line, _new_line]) is None +print("ok - later rpcauth row for the same user") + +# Core set-intersects repeated rpcwhitelist lines for one user. +assert whitelist_map( + [ + "alice:getblock,getnetworkinfo,getbestblockhash", + "alice:getblock,getchaintips", + ] +) == {"alice": {"getblock"}} +print("ok - repeated rpcwhitelist lines intersect") PY diff --git a/scripts/core-functional/warnet/Dockerfile b/scripts/core-functional/warnet/Dockerfile index 94dd443bf..dce2defc6 100644 --- a/scripts/core-functional/warnet/Dockerfile +++ b/scripts/core-functional/warnet/Dockerfile @@ -5,7 +5,11 @@ # docker run --rm -v "$PWD":/src -w /src -e CARGO_TARGET_DIR=/src/target/docker \ # rust:1.95.0-bookworm cargo build -p rbitcoin-node # ./scripts/core-functional/init-submodule.sh -# docker build -t rbitcoin-warnet:local \ +# Kind/Helm tag is a release >= 0.17.0 with no prerelease. That is the +# chart's semverCompare gate for writing [regtest], not the rbitcoin +# version. `local` does not render. Compose uses rbitcoin-warnet:local +# via example.sh. +# docker build -t rbitcoin-warnet:28.0.0 \ # --build-arg NODE_BIN=target/docker/debug/rbitcoin-node \ # -f scripts/core-functional/warnet/Dockerfile . @@ -33,6 +37,7 @@ RUN chmod +x /usr/local/bin/rbitcoin-node /usr/local/bin/bitcoin-cli \ ENV BITCOIN_DATA=/root/.bitcoin ENV RBITCOIN_NODE=/usr/local/bin/rbitcoin-node ENV RBITCOIN_LOG_STDOUT=1 +ENV RBITCOIN_LAB_HEAD_SCALE=tiny ENV PYTHONPATH=/opt/rbitcoin/shim:/opt/rbitcoin/functional ENTRYPOINT ["/entrypoint.sh"] diff --git a/scripts/core-functional/warnet/Dockerfile.test.sh b/scripts/core-functional/warnet/Dockerfile.test.sh index b18ae4f3d..679a70f7b 100755 --- a/scripts/core-functional/warnet/Dockerfile.test.sh +++ b/scripts/core-functional/warnet/Dockerfile.test.sh @@ -2,6 +2,7 @@ # Text contract for the Warnet lab image (no docker required). set -euo pipefail HERE="$(cd "$(dirname "$0")" && pwd)" +ROOT="$(cd "$HERE/../../.." && pwd)" PASS=0 FAIL=0 @@ -23,10 +24,69 @@ if [[ -f "$DF" ]] \ && grep -q 'bitcoind' "$DF" \ && grep -q 'bitcoin-cli' "$DF" \ && grep -q 'test_framework' "$DF" \ - && grep -q 'RBITCOIN_LOG_STDOUT' "$DF"; then - ok "Dockerfile copies node, shims, test_framework, log tee" + && grep -q 'RBITCOIN_LOG_STDOUT' "$DF" \ + && grep -q 'RBITCOIN_LAB_HEAD_SCALE' "$DF"; then + ok "Dockerfile copies node, shims, test_framework, lab env" else - bad "Dockerfile copies node, shims, test_framework, log tee" + bad "Dockerfile copies node, shims, test_framework, lab env" +fi + +SHIM="$ROOT/scripts/core-functional/bitcoind" +if [[ -f "$SHIM" ]] \ + && grep -q 'RBITCOIN_LOG_STDOUT' "$SHIM" \ + && grep -q 'sys.stdout.buffer.write' "$SHIM"; then + ok "shim tees node output when RBITCOIN_LOG_STDOUT is set" +else + bad "shim tees node output when RBITCOIN_LOG_STDOUT is set" +fi + +DOC="$ROOT/docs/core-functional.md" +KIND_OUT="" +if KIND_OUT="$(python3 - "$DOC" <<'PY' +import re +import sys +from pathlib import Path + +doc = Path(sys.argv[1]).read_text() +start = doc.find("### Operator kind") +if start < 0: + sys.exit("missing operator kind section") +section = doc[start:] +def release_ge_017(tag: str) -> bool: + """Warnet emits [regtest] only when the tag is a release >= 0.17.0. + + A prerelease is stripped on current Warnet main before semverCompare, + and 0.7.99 is below 0.17.0, so the section is omitted. helm template + still exits 0 in that case. Warnet 1.1.20's raw compare is also false + for that prerelease; the hyphen only happens to keep the section. + """ + if re.fullmatch(r"\d+\.\d+\.\d+", tag) is None: + return False + parts = tuple(int(x) for x in tag.split(".")) + return parts >= (0, 17, 0) + +if not release_ge_017("28.0.0") or not release_ge_017("0.17.0"): + sys.exit("release gate accepts a known good tag") +for bad in ("0.7.99", "0.7.99-warnet0", "0.16.9", "local", "28.0.0-warnet0"): + if release_ge_017(bad): + sys.exit(f"release gate accepted {bad}") +found = False +for line in section.splitlines(): + if "docker build -t" not in line and "kind load docker-image" not in line: + continue + found = True + if ":local" in line or "rbitcoin-warnet:local" in line: + sys.exit(f"kind instructions still use local: {line}") + match = re.search(r"rbitcoin-warnet:([^\s\\]+)", line) + if match is None or not release_ge_017(match.group(1)): + sys.exit(f"kind tag is below 0.17.0 or not a release: {line}") +if not found: + sys.exit("kind section has no docker build -t or kind load docker-image") +PY +)"; then + ok "kind instructions use a release tag >= 0.17.0" +else + bad "kind instructions use a release tag >= 0.17.0 (${KIND_OUT})" fi echo diff --git a/scripts/core-functional/warnet/entrypoint.sh b/scripts/core-functional/warnet/entrypoint.sh index 5199b03b9..846dc2d41 100755 --- a/scripts/core-functional/warnet/entrypoint.sh +++ b/scripts/core-functional/warnet/entrypoint.sh @@ -4,6 +4,8 @@ DATADIR="${BITCOIN_DATA:-/root/.bitcoin}" export BITCOIN_DATA="$DATADIR" export RBITCOIN_NODE="${RBITCOIN_NODE:-/usr/local/bin/rbitcoin-node}" export RBITCOIN_LOG_STDOUT="${RBITCOIN_LOG_STDOUT:-1}" +# Regtest tanks. Conf head_scale= wins, including mainnet. +export RBITCOIN_LAB_HEAD_SCALE="${RBITCOIN_LAB_HEAD_SCALE:-tiny}" export PYTHONPATH="${PYTHONPATH:-/opt/rbitcoin/shim:/opt/rbitcoin/functional}" mkdir -p "$DATADIR" if [ "$#" -eq 0 ]; then