Improve RFC 7489 compliance of aggregate reports (issue #52)#317
Open
thegushi wants to merge 20 commits into
Open
Improve RFC 7489 compliance of aggregate reports (issue #52)#317thegushi wants to merge 20 commits into
thegushi wants to merge 20 commits into
Conversation
added 20 commits
May 16, 2026 10:27
…steddomainproject#52) RFC 7489 requires a <version> element as the first child of <feedback> and an <envelope_from> element in <identifiers>. Both pieces of data were already available; they just weren't being emitted. envelope_from is omitted for null reverse-path messages (empty env_domain).
…domainproject#52) RFC 7489 requires <scope> in SPF auth_results and <fo> in policy_published. Neither was being captured or reported. - Add mctx_spfmode to dmarcf_msgctx and capture SPF origin (MAILFROM vs HELO) in all three SPF code paths; write spf_scope to the history file alongside the existing spf line - Call opendmarc_policy_fetch_fo() after the DMARC policy query and write fo bitmap to the history file - Add spf_scope TINYINT to messages table and fo TINYINT to requests table in schema.mysql - Update opendmarc-import.in to parse spf_scope and fo and store in DB; update the requests UPDATE to persist fo - Update opendmarc-reports.in to SELECT and emit <scope> (mfrom/helo) in SPF auth_results and convert the fo bitmap to a colon-separated RFC 7489 string for <fo> in policy_published - Add startup warnings with ALTER TABLE commands for existing installs missing the new columns
…ssue trusteddomainproject#230) Use LEFT JOIN for selectors in the signatures query so that signatures stored with selector_id=0 (from import of history files where the selector was empty) are still returned. Treat a NULL selector name as an empty string rather than dropping the row.
…sue trusteddomainproject#217) Initialize arc, arc_policy, align_dkim, align_spf to safe sentinel values in opendmarc-import so old history files without these fields can be imported cleanly. Also add DEFAULT clauses to schema.mysql for these columns so strict mode accepts the CREATE TABLE.
…omparison (issue trusteddomainproject#210) DATE(messages.date) >= DATE(FROM_UNIXTIME(?)) truncates to calendar day in the MySQL server's local timezone, causing messages near day boundaries to be included in the wrong report when the server timezone differs from the reporting timezone. Replace with direct timestamp comparisons, which are timezone-agnostic and consistent with the non-daybound query path.
…riaDB hang (issue trusteddomainproject#196) DBI passes untyped parameters as strings, which can trigger MDEV-27242 in MariaDB causing queries to hang when comparing integer columns to string values. Add bind_param with SQL_INTEGER for the two domain selection queries that were still using execute() with untyped args.
…main (issue trusteddomainproject#270) RFC 7489 appendix C requires policy_published.domain to be the domain at which the DMARC record was found. For subdomains inheriting policy from the organizational domain, opendmarc-reports was emitting the from domain (e.g. subdomain.example.com) rather than the domain where the record was actually located (e.g. example.com). The milter already captures the correct value via opendmarc_policy_fetch_utilized_domain() and stores it in messages.policy_domain. This change adds a query to look up that value per reporting window and use it in the XML output, falling back to the from domain if no rows are found.
…rusteddomainproject#269) Adds --smtp-username, --smtp-password, and --smtp-ssl options to opendmarc-reports for sites that relay through an SMTP server requiring authentication. Unauthenticated submission to a local MTA remains the default. opendmarc-run is updated with commented-out SMTPUSER, SMTPPASSWD, and SMTPSSL variables that are passed through when set.
…rusteddomainproject#269) Allows specifying a CA bundle for SMTP connections to servers using a private or self-signed certificate. Without this, Net::SMTP verifies against the system CA bundle, which fails for internal relay hosts.
…ddomainproject#202) When RequiredHeaders rejects a message, the reason was logged but the SMTP response was a generic 550 5.7.1. Call dmarcf_setreply() with the specific error string so the client sees e.g. "not exactly one Date field" rather than a Postfix default message.
…sue trusteddomainproject#25) Adds --report-bcc to opendmarc-reports, which adds the address as an SMTP envelope recipient and a Bcc: header. opendmarc-run gains a commented-out REPORTBCC variable passed through when set.
…s7.1) Adds support for rua= URIs with http:// or https:// schemes. Reports are submitted via HTTP POST with Content-Type application/gzip using LWP::UserAgent. The existing mailto: path is unchanged. Unsupported schemes are still logged and skipped.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Addresses the RFC 7489 compliance gaps identified in issue #52. All missing elements that can be added without breaking changes are included here.
Pure report-side fixes (data already in DB):
<version>1</version>as first child of<feedback><envelope_from>to<identifiers>(omitted for null reverse-path)<selector>and SPF<domain>were already presentNew data capture through the full pipeline (C → history file → DB → report):
<scope>in SPF<auth_results>: captures whether SPF was evaluated against MAIL FROM or HELO in all three code paths (Authentication-Results header, Received-SPF header, internal libspf2). Stored asspf_scopein themessagestable.<fo>in<policy_published>: callsopendmarc_policy_fetch_fo()after the DMARC policy query and converts the bitmap to a colon-separated RFC 7489 string (0,1,d,s). Stored asfoin therequeststable.Schema changes:
opendmarc-reportswill warn at startup with these commands if the columns are missing.Fixes #52.
Test plan
opendmarc-reportswarns on startup against old schema missingspf_scopeorfo<version>,<envelope_from>,<scope>, and<fo>elements<scope>correctly reflects mfrom vs helo based on how SPF was evaluated<fo>correctly reflects the fo= tag from the sender's DMARC record<envelope_from>rather than emitting an empty element