Repository navigation
perf(cloudini_lib): faster single-field and lossless encoding/decoding - #167
Merged
Merged
Conversation
RelWithDebInfo (the default of ROS/colcon builds) and distribution packages build with -O2, where lossy float fields decode about 1.5x slower.
Lossless fields (and lossy float fields without a resolution) were decoded through the generic per-point loop: a virtual decode() call and a libc memcpy() call per value. A single-field cloud now takes one memcpy, and interleaved ones a strided copy with a compile-time size: 37x faster without compression, and 2.4x with ZSTD (now bound by ZSTD itself).
New FieldEncoder::encodePoints(), by default one encode() call per point, used by the V4 codec when a cloud has a single field. FieldEncoderCopy overrides it with one (or a strided) copy, FieldEncoderFloat_Lossy with a loop without a virtual call per point. Streams are byte-identical. With ZSTD, a lossless float field encodes 2x faster, and a lossy one 1.37x.
jlblancoc
marked this pull request as ready for review
October 24, 2026 12:22
Owner
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Summary
Faster encoding and decoding of single-field clouds and lossless fields. The streams are byte-identical and the API is unchanged:
decodePoints()forFieldDecoderCopy. Lossless fields, and float fields without a resolution, were decoded through the generic per-point loop, with a virtualdecode()call and a libcmemcpy()call per value;perfshowed about 90% of the time there. A single-field cloud now takes onememcpy. Interleaved points take a strided copy with a compile-time size.encodePoints()for single-field clouds. I added a new virtualFieldEncoder::encodePoints(), whose default callsencode()once per point. The V4 codec uses it when a cloud has a single field.FieldEncoderCopyoverrides it with a copy, andFieldEncoderFloat_Lossywith a loop that has no virtual call per point.-O3in non-Debug builds (GCC and Clang). RelWithDebInfo, the default of ROS/colcon builds, and distribution packages compile with-O2. There, the lossy float decoder runs about 1.5x slower.Motivation: in MRPT, each point field is a separate array, so each one is encoded as its own single-field Cloudini stream (in 64k-point chunks, in parallel). That is about 2.8-3.5x faster than raw serialization for large maps. Maps with many lossless fields (normals, timestamps...) spend most of their time in these two paths.
Measurements
Times per value (or per point, for x/y/z) on a 262k-point LiDAR-like cloud,
use_threads=false. Both builds are RelWithDebInfo (the default in ROS/colcon), GCC 13, AMD Ryzen 7 2700X. "1.4.1" is currentmain.Decoding
Encoding
Testing
FieldEncoders.SingleFieldBulkMatchesPerPoint. For contiguous and padded points, lossless and lossy, it checks that the bulk encoder writes the same bytes asencode()once per point. It also checks a whole-cloud round trip.