pyxsdata-core

Ultra-fast, zero-allocation native Rust data binding and XML deserialization engine for pyxsdata.

---

Overview

pyxsdata-core is a high-performance compiled extension built with PyO3 and quick-xml. It replaces the standard Python XML event loop, stack frame allocation, and type conversion pipeline with streaming native Rust code that constructs target Python @dataclass and Pydantic v2 BaseModel instances directly via the CPython C-API.

Key Highlights

• Up to 15x Faster than Legacy xsdata: Reaches ~300,000+ objects/sec on standard benchmarks.
• 11.5x Faster on Complex Defense/Aerospace Schemas: Tested against production UCI (Universal Command and Control Interface) message definitions at ~60,360 messages/sec.
• Zero Intermediate DOM Allocation: Completely bypasses xml.etree.ElementTree, lxml, and SAX event objects.
• Native Support for Complex Types: Built-in, panic-free conversion for str, int, float, bool, bytes, decimal.Decimal, enum.Enum, XmlTime, XmlDuration, and xsi:nil="true" nullability.
• Thread & Poison-Resilient: Concurrency-safe design using lock poisoning recovery.
• 100.00% Test Coverage: Fully verified statement and branch coverage on Python bindings and strict zero-panic Rust invariants.

---

Performance & Deserializer Benchmarks

1. High-Volume Dataclass Deserialization (10,000 Complex Items, 3.36 MB)

Deserializer Engine	Technology	Throughput	Latency (10k items)	Speedup vs Legacy
pyxsdata-core	Rust + PyO3 (quick-xml)	~290,700 objs/s	34.4 ms	~15.0x faster (1,490%)
pyxsdata (Pure Python)	Python 3.12 xml.etree	~30,075 objs/s	332.5 ms	+54.4% (2.2x faster)
pyxsdata (lxml)	C libxml2	~26,650 objs/s	375.2 ms	+50.2% (2.0x faster)
Legacy xsdata	Python xml.etree	~19,490 objs/s	513.0 ms	1.0x (Baseline)

(CPython 3.12.14, Linux x86_64, lowest of 5 runs)

2. Pydantic v2 BaseModel Deserialization (1,000 Complex Items)

Deserializer Engine	Technology	Throughput	Latency (1k items)	Speedup
pyxsdata-core (pyxsdata.pydantic)	Rust + PyO3 (quick-xml)	~311,245 objs/s	3.2 ms	~7.67x faster (767%)
pyxsdata.pydantic (Pure Python)	Python 3.12 xml.etree	~40,580 objs/s	24.6 ms	1.0x (Baseline)
Legacy xsdata-pydantic	Python xml.etree	~26,170 objs/s	38.2 ms	0.64x (~11.9x slower vs Core)

3. Real-World Enterprise Benchmark: UCI Entity Message

Parsing production-grade, deeply nested Universal Command and Control Interface (UCI v2.5) Entity telemetry messages (with security markings, timestamps, headers, metadata, and enums):

Deserializer Engine	Latency / Message	Throughput	Speedup vs Pure Python
pyxsdata-core	16.5 µs	~60,360 msgs/s	~11.47x (1,047% faster)
XmlParser (Pure Python)	190.0 µs	~5,262 msgs/s	1.0x (Baseline)

---

Installation

Install as a standalone native module:

pip install pyxsdata-core

Or enable it as an accelerated backend in pyxsdata:

pip install "pyxsdata[core]"

---

Usage

1. Seamless Integration with pyxsdata (Recommended)

When using pyxsdata, switch to CoreXmlParser to enable pyxsdata-core acceleration with zero code changes:

from my_models import PurchaseOrder
from pyxsdata.formats.dataclass.parsers import CoreXmlParser

parser = CoreXmlParser()
order = parser.from_string(xml_text, PurchaseOrder)

For Pydantic v2 models:

from my_pydantic_models import TelemetryEvent
from pyxsdata.pydantic.bindings import CoreXmlParser

parser = CoreXmlParser()
event = parser.from_string(xml_text, TelemetryEvent)

2. Standalone Direct Deserialization

You can also use pyxsdata_core directly without any parser wrappers:

from dataclasses import dataclass
import pyxsdata_core

@dataclass
class Item:
    id: int
    name: str

xml_bytes = b"<Item><id>42</id><name>Telemetry Probe</name></Item>"
item = pyxsdata_core.deserialize(xml_bytes, Item)
assert item.id == 42
assert item.name == "Telemetry Probe"

---

Architecture & How It Works

Traditional Python XML parsers incur heavy interpreter overhead:

1. XML bytes are read by C/Python parsers into temporary DOM Element or event tuples.
2. Python event dispatchers loop over millions of items, calling Python methods for each tag and attribute.
3. Intermediate string parsing and dictionary building happens in the bytecode interpreter.

pyxsdata-core collapses this into a single zero-allocation pipeline:

• Rust Streaming: quick-xml scans slices in-memory without allocating intermediate string objects.
• Cached Model Schema: On first use, ModelSchema is built once and cached in an internal RwLock, mapping XML tag names directly to target constructor positions and scalar converters.
• Native Allocator: Instantiates Python dataclasses directly via the C-API, passing pre-allocated Rust-converted values with zero interpreter bounce.

---

Development

Prerequisites: Rust 1.80+ and Python 3.12+.

# Install dependencies
pip install maturin pytest pytest-cov pytest-benchmark

# Build native extension in debug/editable mode
maturin develop

# Run tests and verify 100% coverage
pytest --cov

# Run clippy and format checks
cargo fmt --check
cargo clippy -- -D warnings

# Build documentation
zensical build

---

License

MIT License. See LICENSE for details.