Skip to content

perf: non-blocking serial output of the datalog - #172

Merged
FredM67 merged 1 commit into
devfrom
perf/non-blocking-serial
Sep 30, 2026
Merged

FredM67 merged 1 commit into
devfrom
perf/non-blocking-serial

Conversation

@FredM67

@FredM67 FredM67 commented Sep 30, 2026

Copy link
Copy Markdown
Owner

Problem

At 9600 baud a datalog line takes ~130 ms to send, and the UART's transmit buffer holds 64 characters. Printing it with Serial.print() kept the main loop waiting for most of that time, so everything else the main loop does was delayed.

The RF mock (#171) showed the effect with two remote units (#160):

  • the refresh frames were up to 0.164 s apart;
  • a command to unit 1 could wait for the print to finish, and meanwhile the energy bucket kept filling;
  • as a result, unit 2 switched on for two mains cycles (2000 W) about every 10 s when it shouldn't have.

Change

  • serial_output.h (new): a step printer. Each step writes at most 32 characters. SerialOutput::poll() is called on every pass of the main loop and only runs a step when the transmit buffer has room for it, so printing never waits.
  • IoT: the TeleInfo frame is drained from its existing buffer (TeleInfo::writeNext()). No extra RAM is needed.
  • HumanReadable: the line is printed one field per step.
  • JSON: also one field per step, written directly without ArduinoJson.
  • Debug messages: DBUG/DBUGLN first complete any output in progress, so they never land inside a line or frame. In that rare case they block, like before.
  • Skipped datalogs: if the previous output were still in progress at the next datalog event, that datalog is skipped rather than waited for (~150 ms of output every 5 s).

Output format

The format is unchanged for HumanReadable, IoT and JSON. This was checked in grid_sim against dev:

  • configurations: the default config, all options (temperatures, relays, dual tariff), and automatic rotation;
  • digits masked for the comparison, since the simulated values differ at the last digit between builds;
  • every IoT line checksum is valid.

One deliberate difference: in JSON, temperatures no longer show float noise. For example −9.97 is now written -9.97 instead of ArduinoJson's -9.969999, which affected about 2 % of values.

Results

On #160's firmware with two remote units:

before after
Longest gap between RF frames 0.164 s 0.103 s
Unit 2 first switched on at 26.1 s (too early) 40.06 s
ADC ISR max / overruns 1418 / 0 1412 / 0

The RF scenarios on dev now require frames at most 0.15 s apart (was 0.25 s). The old firmware fails this limit (0.164 s) and the new one passes (0.101 s).

Size

Build Flash RAM
HumanReadable +766 B +3 B
IoT +446 B +3 B
JSON −1720 B (−3.9 KB with all options) +3 B

About 230 bytes come from the debug hook at each of the 57 DBUG call sites.

Tests

  • New simavr test test_serial_output (6 cases):
    • poll() waits for room and never overfills the buffer;
    • output split into steps equals the whole line;
    • starting while busy is refused;
    • JSON number format against ArduinoJson for every value from −55.00 to 125.00;
    • a TeleInfo frame written in chunks is identical to the whole frame.
  • All embedded tests (simavr) and native tests (122/122) pass.

ArduinoJson is removed from the firmware's dependencies. It stays in the uno_sim environment only, as the reference for the format test. The READMEs and CLAUDE.md are updated accordingly.

🤖 Generated with Claude Code

At 9600 baud a datalog line takes ~130 ms to send and the UART buffer holds
64 characters, so printing it kept the main loop waiting: RF commands to
remote loads were delayed, which made a second remote unit switch on for two
mains cycles about every 10 s (seen in grid_sim with two remote units).

The output is now cut into steps of at most 32 characters (serial_output.h),
run from the main loop only when the transmit buffer has room: printing never
waits. The IoT frame is drained from its existing buffer, the text and JSON
lines are printed field by field. Debug messages first complete a pending
output, so they never land inside a line or frame.

The JSON line is written directly, without ArduinoJson (-1.7 KB of flash, up
to -3.9 KB with all options), in the same format; temperatures no longer show
float noise (-9.97 instead of -9.969999). ArduinoJson remains only as the
reference of the new embedded test.

The RF scenarios now require refresh frames at most 0.15 s apart (was 0.25 s).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@FredM67
FredM67 merged commit de6dfd0 into dev Sep 30, 2026
10 checks passed
@FredM67
FredM67 deleted the perf/non-blocking-serial branch September 30, 2026 20:33
FredM67 added a commit that referenced this pull request Sep 30, 2026
…nits

Brings in the non-blocking serial output (#172). The two-unit RF scenarios
now require refresh frames at most 0.15 s apart for both nodes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant