Skip to content

TLS was documented, not implemented - #42

Merged
d3vhex merged 1 commit into
mainfrom
feat/tls-ingest-integrity-and-retention
Sep 5, 2026
Merged

d3vhex merged 1 commit into
mainfrom
feat/tls-ingest-integrity-and-retention

Conversation

@d3vhex

@d3vhex d3vhex commented Sep 5, 2026

Copy link
Copy Markdown
Owner

TLS_ENABLED, TLS_CERT and TLS_KEY appeared in .env.example, in the
production guide, and in the closing output of certs/generate_certs.py — and
were read by no line of code. Setting TLS_ENABLED=1 produced a console that
came up cleanly on plain HTTP while its operator had every reason to believe
it was HTTPS. Assume any console that ran with that setting was never
encrypted, and rotate credentials typed into it.

core/tls.py makes it real on one rule: asking for TLS and not getting it
is a startup failure, never a fallback.
Telemetry moves to port 5011 inside
TLS, the control channel to wss://, and SESSION_COOKIE_SECURE follows
TLS_ENABLED when unset — the two had to agree and nothing made them.

Six things only turning it on could have found

Every one of these passed the unit tests and needed a real container, a real
certificate and a real handshake to surface:

  1. The certificate could not be written. certs/ ships in the image owned
    by root; the server runs unprivileged. Generation died with EACCES and,
    correctly refusing to downgrade, restart-looped. Material now lives in the
    data volume beside data/fernet.key.
  2. /api/agent/ca read a path of its own while the generator wrote to
    TLS_DIR, so the endpoint installers depend on for trust answered 404 on
    every containerised deployment.
  3. The control channel never received the CA. Telemetry flowed over TLS
    and the channel could not open — reporting, and unreachable.
  4. Neither did the requests client. Three fixes for one defect, so trust
    is now installed once at startup as the system store plus this CA
    (REQUESTS_CA_BUNDLE, CURL_CA_BUNDLE, SSL_CERT_FILE). Trusting the
    private CA instead of the system roots would have been the easier change
    and would have made the agent distrust every other endpoint on the machine.
  5. Enrolment broke entirely. Both installers fetched the CA last, after
    registration and the binary download, so on a self-signed server every
    earlier call failed and the fix arrived two steps too late. Trust is now
    established before the first network call, and the Windows installer adds
    the CA to the machine store.
  6. ServerCertificateValidationCallback = {$true} does not work on
    PowerShell 5.1.
    .NET invokes the script block on an I/O thread with no
    runspace; the callback throws and surfaces as "an unexpected error occurred
    on a send", which reads as a TLS fault. An hour went into protocol
    versions, cipher suites, ALPN and IPv6 before the inner exception was
    unwrapped. Both the console snippet and the installer use curl.exe now.

Three of the six were the same defect wearing different clothes: trust wired
per connection rather than per host.

The ingest protocol had no reply

The agent marked rows sent when sendall returned — which means the bytes
reached a socket buffer and nothing more. There is a receipt frame now, and
three cases stay distinguishable: rows stored, an error (keep them, retry, say
why), and no receipt at all from a server older than the frame. Once a server
has acknowledged once, later silence is treated as a fault — there is a real
configuration that accepts a batch and discards it.

Four live data-loss bugs, found by making failure visible

Moving the insertion failure out of if debug turned up four paths dropping
rows while every layer reported success:

  • docker_containers — one duplicate row rolled back the whole batch.
    Rows insert individually now; snapshots take the last write.
  • registry_logs, process_events, security_audit — accepted by
    ALLOWED_TABLES with no table in db/init.sql. 7,788 rows held on one
    endpoint against a table that did not exist.
  • siem_events — the console's Clear button truncated rows and left every
    ingest fingerprint, so each future row was skipped as a duplicate of a row
    that no longer existed. One click permanently blinded the table.
  • critical_files — a delta stream classified as a server snapshot, so
    every batch deleted the 1,200 rows held and kept the handful that changed.

A rejected row now releases its dedup fingerprint; without that it could never
be stored again even after the cause was fixed.

A server reset stranded every quiet table

sent lives on the agent and is its only record of what the server holds.
Nothing told it when the server's database was recreated, and a sent row is
never re-offered — so fim_data refilled within minutes and
portscan_result sat at 93 rows locally against 0 on the server, for ever.

The server now stamps each database incarnation with an identity and returns
it on every receipt. An agent that sees it change re-offers its rows, bounded
to 1,000 per table and deduplicated server-side. It re-offers on first sight
too — the agents that most need this are the ones running since before the
epoch existed.

The telemetry health view can finally tell the two apart, and says server
reset
rather than "accepted and discarded" when the rows predate the
database.

The agent's local database grew for ever

Nothing pruned it: 530,428 rows of hardware_inventory on one host, all
shipped, never read again. Hourly retention keeps the newest 5,000 per table —
above the recovery limit, so pruning cannot quietly cap what a reset can
rescue — and only ever deletes rows the server has confirmed.

security_audit was shipped by the agent with no collector anywhere. Removed
from the shipping list; the server keeps the schema.

The channel could starve itself

GET /config/rules timed out at 5s against a healthy agent. Stream payloads
and control frames shared one send lock, an orphaned screen pump never
stopped, and PING_INTERVAL_S was doubling as the socket timeout, so one slow
send could hold the lock for 30s. Stream frames are droppable and yield to a
queued reply; the socket deadline is its own constant; the server tells the
agent when a stream is gone.

Charts across the console

Eleven pages now carry the dashboard's visual language, each derived from rows
the page already holds — a chart with its own request can disagree with the
table beside it. Two places deliberately have none and say why in the code:
OSV returns no CVSS score, and resource_usage is a snapshot table.

Tests

2,317 passing. New structural invariants: a table the agent deduplicates
locally can never be a server snapshot; every ALLOWED_TABLES entry must have
somewhere to land; every table the agent ships must have a writer; clearing a
deduplicated table must clear its fingerprints; no certificate callback may be
a PowerShell script block; and every setting the TLS guide documents must be
one the code reads.

The nine skips in test_encrypted_column_widths are gone — they were skipping
because three tables had no schema at all, so the suite reported "nothing to
check here" about tables the agent was filling.

TLS_ENABLED was documented in three places and read by nothing; the ingest protocol had no reply frame; and the agent's sent flag was the only record of what the server held, so a server-side reset stranded every quiet table for ever.
Comment thread Sentora/modules/link.py
context = _ssl.SSLContext(_ssl.PROTOCOL_TLS_CLIENT)
context.check_hostname = False
context.verify_mode = _ssl.CERT_NONE
with context.wrap_socket(raw, server_hostname=host):
@d3vhex
d3vhex merged commit 64a0a17 into main Sep 5, 2026
9 of 10 checks passed
@d3vhex
d3vhex deleted the feat/tls-ingest-integrity-and-retention branch September 7, 2026 18:16
d3vhex added a commit that referenced this pull request Sep 8, 2026
…ntion

TLS was documented, not implemented
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants