Skip to content

feat(api): add pooled RCON client and game event bus - #22

Merged
and3rn3t merged 3 commits into
mainfrom
feature/rcon-client-and-event-bus
Sep 19, 2026
Merged

and3rn3t merged 3 commits into
mainfrom
feature/rcon-client-and-event-bus

Conversation

@and3rn3t

Copy link
Copy Markdown
Owner

Description

Phase 0 of the new family server roadmap: two
pieces of plumbing that most of the planned gameplay features depend on, plus
the roadmap itself.

api/rcon.py — an in-process RCON client

Every command previously shelled out to scripts/rcon-client.sh, which opened a
new TCP connection and re-authenticated for each one. That is roughly 50-100ms
of overhead per command and makes command batches impractical.

The script's Python fallback also had two defects:

  • A single unframed sock.recv(4096), so responses longer than that were
    truncated or split across reads.
  • Any reply of four or more bytes was treated as a successful login, rather than
    checking for the -1 request id the protocol uses to signal auth failure.

The new client holds one authenticated connection open behind a lock,
reassembles the multi-packet responses Minecraft sends for output over 4096
bytes, reconnects transparently when the container restarts underneath it, and
rejects commands long enough that the server would silently truncate them.

api/server.py reaches it through a new run_rcon_command() helper that falls
back to the shell script when RCON is unconfigured or unreachable, since that
script can also reach rcon-cli inside the container when the port is not
published to the host. scripts/rcon-client.sh is unchanged for CLI use.

api/events.py — a game event bus

See docs/EVENT_BUS.md.

The server log is the only real-time signal Minecraft produces. It was followed
purely to stream raw text to the browser, so anything reactive had to re-parse
raw lines for itself, and nothing was captured at all unless someone had the
dashboard open.

Lines are now parsed into typed events, appended to
data/events/YYYY-MM-DD.jsonl, and dispatched to registered handlers. Writes
are batched and files older than 30 days are pruned, to limit SD-card wear on
the Pi. A handler that raises is logged and skipped rather than stopping the bus.

The follower now starts with the API rather than on the first browser
connection, and no longer stops when the last client disconnects. Events that
happened with the dashboard closed were previously lost. stop_log_reader()
brings it down deliberately.

Adds GET /api/events and GET /api/events/types (both logs.view), a
game_event WebSocket message alongside the existing raw logs stream,
getEvents() and getEventTypes() in web/src/services/api.js, and the
matching OpenAPI paths and schema.

Security fix included

The WebSocket execute_command handler reached RCON without sanitising its
input
, bypassing the command allowlist that POST /api/server/command
enforces. Anyone holding an API key could run arbitrary console commands through
it. It now runs the same validation and writes the same audit entries.

Notes for review

Two decisions worth a look:

  1. connect and join are separate event types. A single player session
    logs two lines, one carrying the network address and one saying they joined.
    Mapping both to join double-counted every session, which a smoke test on a
    realistic log caught.
  2. tests/api/test_log_streaming.py was updated, not worked around. It
    asserted that the follower stops when the last viewer disconnects, which is
    exactly the behaviour this PR removes on purpose. Those tests now cover the
    new contract and the stop flag that replaced it.

Death detection matches the opening phrases of Minecraft's death.attack.*
strings, since vanilla death messages have no marker of their own. Chat is
matched first so a player typing a fake death message cannot fake a death event.
Anything the phrase list does not cover is simply not treated as a death, which
is the safe direction to fail.

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Refactor (code change that neither fixes a bug nor adds a feature)
  • Documentation (changes to docs only)
  • Other (please describe):

Checklist

  • I have performed a self-review of my code
  • I have added tests that prove my fix or feature works
  • I have updated documentation as needed
  • Lint and tests pass locally
  • I have not introduced any security vulnerabilities

Verification run locally:

Check Result
make test-api 336 pass, up from 251
make test-web 144 pass
Coverage 50.7%, threshold 40%
make lint passes
docker compose config valid

The RCON tests run against a real socket server speaking the protocol, so
framing, authentication, multi-packet reassembly and reconnection are exercised
end to end rather than mocked.

Related Issues

🤖 Generated with Claude Code

and3rn3t and others added 2 commits September 19, 2026 08:26
The existing roadmaps (ROADMAP.md, TASKS.md, MINECRAFT_GAMEPLAY_ENHANCEMENTS.md)
all plan the server management product: more endpoints, more admin pages, more
ops polish. None of them plan the experience the people actually playing on the
server see.

This adds a roadmap for that, organised by feasibility on the Pi 5 hardware
rather than by release. It covers four foundations that most other ideas depend
on, then tiers of gameplay and integration features, with a suggested build
order and the Pi-specific gotchas that would otherwise be discovered the hard
way (the RP1 GPIO change, NeoPixel drivers, the 1.20.5 item component
migration, SD-card write endurance).

It also lists the known issues sitting in the path of that work, each verified
against the code rather than assumed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Phase 0 of the family server roadmap. Two pieces of plumbing that most of the
planned gameplay features depend on.

api/rcon.py — an in-process RCON client

Every command previously shelled out to scripts/rcon-client.sh, which opened a
new TCP connection and re-authenticated for each one. That is roughly 50-100ms
of overhead per command and makes command batches impractical.

The script's Python fallback also had two defects. It did a single unframed
sock.recv(4096), so responses longer than that were truncated or split across
reads, and it treated any reply of four or more bytes as a successful login
rather than checking for the -1 request id the protocol uses to signal auth
failure.

The new client holds one authenticated connection open behind a lock,
reassembles the multi-packet responses Minecraft sends for output over 4096
bytes, reconnects transparently when the container restarts underneath it, and
rejects commands long enough that the server would silently truncate them.

api/server.py reaches it through run_rcon_command(), which falls back to the
shell script when RCON is unconfigured or unreachable, since that script can
also reach rcon-cli inside the container when the port is not published to the
host. scripts/rcon-client.sh is unchanged for CLI use.

api/events.py — a game event bus

The server log is the only real-time signal Minecraft produces. It was followed
purely to stream raw text to the browser, so anything reactive had to re-parse
raw lines for itself, and nothing was captured at all unless someone had the
dashboard open.

Lines are now parsed into typed events (chat, connect, join, leave, death,
advancement, command, server_ready, server_stopping), appended to
data/events/YYYY-MM-DD.jsonl, and dispatched to registered handlers. Writes are
batched and files older than 30 days are pruned, to limit SD-card wear on the
Pi. A handler that raises is logged and skipped rather than stopping the bus.

The follower now starts with the API rather than on the first browser
connection, and no longer stops when the last client disconnects. Events that
happened with the dashboard closed were previously lost. stop_log_reader()
brings it down deliberately; the log streaming tests were updated from the old
contract to this one.

Connect and join are separate event types on purpose. A single player session
logs two lines, one carrying the network address and one saying they joined, so
treating both as a join double-counts every session.

Also fixes: the WebSocket execute_command handler reached RCON without
sanitising its input, bypassing the command allowlist that
POST /api/server/command enforces. It now runs the same validation and writes
the same audit entries.

Adds GET /api/events and GET /api/events/types (both logs.view), a game_event
WebSocket message alongside the existing raw logs stream, getEvents() and
getEventTypes() in web/src/services/api.js, the OpenAPI paths and schema, and
docs/EVENT_BUS.md.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings September 19, 2026 13:27
@github-actions

Copy link
Copy Markdown

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

None

@github-actions github-actions Bot added documentation Documentation additions or updates tests labels Sep 19, 2026
Comment thread api/rcon.py Fixed
Comment thread tests/api/test_events.py Fixed
Comment thread tests/api/test_rcon.py Fixed
Comment thread tests/api/test_rcon.py Fixed

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Critical command-retry and event-persistence issues, along with moderate correctness gaps, remain unresolved.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 4 High severity · 1 Medium severity · 1 Low severity

Open (6)
What changed in this PR

Adds pooled RCON support and an always-on, typed Minecraft event bus with REST/WebSocket exposure, persistence, and frontend integration.

Changes:

  • Adds framed pooled RCON execution, reconnection, fallback, and validation.
  • Parses, persists, and dispatches game events.
  • Updates APIs, tests, OpenAPI, documentation, roadmap, and changelog.
File Reviewed changes and final review notes
web/​src/​services/​api.js Adds event API helpers. Nit: add coverage for parameters and cached event types.
tests/​api/​test_rcon.py Adds RCON protocol, authentication, framing, and reconnection tests.
tests/​api/​test_log_streaming.py Updates tests for the always-on follower contract.
tests/​api/​test_events.py Adds event bus and endpoint tests. Moderate: the documented 500-item limit is not actually asserted.
docs/​INDEX.md Updates documentation navigation.
docs/​FAMILY_SERVER_ROADMAP.md Adds the family server roadmap. Nit: listed event names differ from shipped names.
docs/​EVENT_BUS.md Documents the event bus and event delivery.
CHANGELOG.md Adds release notes for the new infrastructure.
api/​server.py Integrates RCON, event endpoints, log capture, and WebSocket validation. Critical: replayed tails duplicate events, 502 fallback can repeat sent commands, and non-string commands can escape error handling. Moderate: stopping may not terminate a blocked follower.
api/​rcon.py Adds the pooled RCON client. Critical: post-send failures can retry side-effecting commands. Moderate: cached credentials do not reflect password rotation.
api/​openapi.yaml Adds event API paths and schemas.
api/​events.py Adds parsing, persistence, dispatch, batching, and retention. Moderate: quiet periods may leave events unflushed, and concurrent flushes can write out of order.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread api/rcon.py Outdated
Comment thread api/server.py
Comment thread api/server.py
Comment thread api/server.py
Comment thread api/server.py
Comment thread web/src/services/api.js
Every finding from the Copilot review, plus the CodeQL notes. Each one was
verified against the code rather than taken on trust; all six were real.

Duplicate events on every restart

The follower attached with --tail 200 and persisted every line it read, so the
backlog was replayed into the bus on each API restart and each re-attach. The
same chat, join and death lines were recorded again and counts climbed with
every restart, which is exactly the flaw this work set out to remove from
scripts/player-stats-tracker.sh.

It now attaches with --tail 0. Connecting clients still get scrollback:
handle_connect sends get_log_tail() separately and that path does not touch the
bus. The trade is that events occurring while the API is down are not captured,
which is far better than recording some of them repeatedly.

Unsafe retries of non-idempotent commands

RconConnectionError could be raised after the command had already been written
to the socket, and both the client's retry and run_rcon_command's shell fallback
would then run it a second time. A lost response to a give, summon, kill or stop
would apply the effect twice.

Transmission is now separated from response reading. A failure on the first
write raises RconNotSentError, which is retried because nothing reached the
server. Any failure after that raises RconUnknownOutcomeError, which is never
retried and is reported by execute() as 500 so run_rcon_command will not re-run
it through the script either. Fallback remains for 502 and 503, both of which
mean nothing was sent.

Non-string commands escaped error handling

A truthy non-string value such as {"command": 1} reached
sanitize_minecraft_command() and raised on .lstrip(). On the REST path that
surfaced as a 500 instead of a 400; on the WebSocket path it raised before the
handler's try block, so the client received no command_error at all. Both paths
now reject the value explicitly, as does RconClient.command().

stop_log_reader() could not stop a blocked reader

It only set a flag. On a quiet server the reader sits blocked in
`for line in proc.stdout` and never reaches the check, so it kept running. The
active process is now tracked and killed, which breaks the read.

Event bus flush correctness

The buffer swap was locked but the file append was not, so two concurrent
flushes could interleave their lines; a dedicated write lock now serialises the
append without holding up publishes for the duration of the I/O. The size and
age thresholds were also only evaluated on publish, leaving the last few events
of a quiet period buffered indefinitely, so a daemon timer now flushes them.

Also

- The RCON config is re-read when its mtime changes, so rotating the password
  with scripts/rcon-setup.sh no longer needs an API restart.
- Tests for both new event methods in web/src/services/api.js.
- The 500-item cap on GET /api/events is now actually asserted.
- The roadmap listed server_start/server_stop and omitted connect; it now names
  the types that shipped.
- CodeQL: explained three empty except clauses, and removed the duplicate
  module import in tests/api/test_events.py.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@and3rn3t

Copy link
Copy Markdown
Owner Author

All six review findings are fixed in e9a9a33, along with the four CodeQL notes and the three issues raised in the summary table. Each was verified against the code before acting; all six were real, and two of them had a second instance the review had not reached.

Beyond the inline threads:

  • api/events.py flush correctness. The buffer swap was locked but the file append was not, so two concurrent flushes could interleave their lines. A dedicated write lock now serialises the append, without holding up publishes for the duration of the I/O. Separately, the size and age thresholds were only evaluated on publish, so the last few events of a quiet period sat buffered indefinitely and would be lost if the process stopped; a daemon timer now flushes them.
  • Cached RCON credentials. The config is re-read when its mtime changes, so rotating the password with scripts/rcon-setup.sh no longer needs an API restart.
  • The 500-item cap on GET /api/events was documented but not asserted. The test now stores 600 events and checks that 500 come back.
  • The roadmap listed server_start/server_stop and omitted connect. It now names the types that shipped.
  • CodeQL: explained three empty except clauses and removed the duplicate module import in tests/api/test_events.py.

Two things I want to flag rather than leave implied:

  1. The backlog replay was the worst of these. It reproduced exactly the flaw this PR set out to remove from scripts/player-stats-tracker.sh, which re-reads the whole log and inflates its counts. I went with --tail 0 rather than a durable cursor, because deduplicating identical lines is unreliable: two identical chat messages in the same second are legitimately distinct events. The cost is that events occurring while the API is down are not captured.
  2. The retry fix has a caveat I would rather state than paper over. A partial write cannot be distinguished from a failed one in principle. The retry relies on a command packet being well under one TCP segment, so a stale socket fails with nothing delivered. That is the case the retry exists for, and it is noted in the docstring.
Check Before After
API tests 336 368
Web tests 144 150
Coverage 50.7% 52.4%

make lint passes, docker compose config validates, and Black is clean on the new files.

🤖 Generated with Claude Code

@and3rn3t and3rn3t self-assigned this Sep 19, 2026
Comment thread api/rcon.py Dismissed
@and3rn3t
and3rn3t merged commit d9f66d4 into main Sep 19, 2026
17 checks passed
@and3rn3t
and3rn3t deleted the feature/rcon-client-and-event-bus branch September 19, 2026 13:56
@and3rn3t and3rn3t mentioned this pull request Sep 19, 2026
11 tasks
@and3rn3t and3rn3t added the feature New feature request label Sep 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Documentation additions or updates feature New feature request tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants