Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions docs/src/python_config.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,3 +78,41 @@ proving validation. Choose `strict` to reject unmatched inputs.

Dictionary/additional-property model preservation and multi-member `allOf`
composition are separate concerns and are not redesigned by this option.

## Additional properties

An object with explicit `additionalProperties` gets typed mutable
`additional_properties` storage, including a pure dictionary with no named fields.
The storage is a keyword-only dataclass field with an independent empty-dict
default. `from_dict` separates known wire keys from extension keys; `to_dict`
flattens the entries back into the same JSON object, without a storage wrapper.

```python
value = MixedMap(
known_field="fixed",
additional_properties={"keep_key": NestedValue(wire_value="nested")},
)
value.to_dict() # {"known_field": "fixed", "keep_key": {"wire_value": "nested"}}
```

Extension keys remain verbatim, even when they resemble renamed Python fields.
Opaque values (`true` or `{}`) keep arbitrary JSON, including null entries and
large integers. Typed values use the declared model/container/alias converters;
recursive references and union policies use the same boundaries as ordinary
values. The storage does not make a pure dictionary a `dict` subtype: use its
`additional_properties` field and normal `from_dict` / `to_dict` entrypoints.

If an authored field already uses the Python name `additional_properties`, the
storage name gains trailing underscores until it is distinct. Authored fields
are not renamed. Supplying an extension key equal to a declared wire key raises
`ValueError` on serialization, even when that named field is currently omitted;
extensions never overwrite or impersonate named fields.

Objects without explicit additional properties and closed objects keep their
existing behavior. Ordinary named-field codecs and optional-null omission are
unchanged. This is not new JSON Schema validation or an `allOf`/tagged-wrapper
representation redesign.

Value conversion follows the retained schema facts. General inline composed
additional-property schemas currently lowered as opaque values remain opaque;
this storage repair does not change that shared lowering behavior.
5 changes: 3 additions & 2 deletions just/golden-python-clients.just
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,11 @@ set working-directory := '..'
strategy="$policy"
[ "$policy" != default ] || strategy=firstMatch
cargo run --quiet --bin openapi-nexus -- generate --config tests/generated-clients/config.toml --input tests/fixtures/valid/python-untagged-union.yaml --output "tests/generated-clients/python/generated/{{backend}}/$policy" --generators 'python-{{backend}}' --generator-config "python-{{backend}}.package_name=union_$policy" --generator-config "python-{{backend}}.untagged_union_strategy=$strategy"
cargo run --quiet --bin openapi-nexus -- generate --config tests/generated-clients/config.toml --input tests/fixtures/valid/python-additional-properties.yaml --output "tests/generated-clients/python/generated/{{backend}}/additional/$policy" --generators 'python-{{backend}}' --generator-config "python-{{backend}}.package_name=additional_$policy" --generator-config "python-{{backend}}.untagged_union_strategy=$strategy"
done

@build backend: (prepare backend)
"${PYRIGHT:-pyright}" --pythonpath "$(command -v "${PYTHON:-python3}")" --project tests/generated-clients/python/pyrightconfig.{{backend}}.json tests/generated-clients/python/generated/{{backend}}/default/union_default tests/generated-clients/python/generated/{{backend}}/strict/union_strict tests/generated-clients/python/generated/{{backend}}/raw/union_raw tests/generated-clients/python/typing_contracts.py
"${PYRIGHT:-pyright}" --pythonpath "$(command -v "${PYTHON:-python3}")" --project tests/generated-clients/python/pyrightconfig.{{backend}}.json tests/generated-clients/python/generated/{{backend}}/default/union_default tests/generated-clients/python/generated/{{backend}}/strict/union_strict tests/generated-clients/python/generated/{{backend}}/raw/union_raw tests/generated-clients/python/generated/{{backend}}/additional/default/additional_default tests/generated-clients/python/generated/{{backend}}/additional/strict/additional_strict tests/generated-clients/python/generated/{{backend}}/additional/raw/additional_raw tests/generated-clients/python/typing_contracts.py tests/generated-clients/python/additional_typing_contracts.py

@test backend: (build backend)
NEXUS_PYTHON_BACKEND={{backend}} PYTHONPATH="tests/generated-clients/python/generated/{{backend}}/default:tests/generated-clients/python/generated/{{backend}}/strict:tests/generated-clients/python/generated/{{backend}}/raw" "${PYTHON:-python3}" -m unittest discover -v -s tests/generated-clients/python/tests
NEXUS_PYTHON_BACKEND={{backend}} PYTHONPATH="tests/generated-clients/python/generated/{{backend}}/default:tests/generated-clients/python/generated/{{backend}}/strict:tests/generated-clients/python/generated/{{backend}}/raw:tests/generated-clients/python/generated/{{backend}}/additional/default:tests/generated-clients/python/generated/{{backend}}/additional/strict:tests/generated-clients/python/generated/{{backend}}/additional/raw" "${PYTHON:-python3}" -m unittest discover -v -s tests/generated-clients/python/tests
124 changes: 124 additions & 0 deletions src/generators/python/additional_properties.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
//! Explicit extension storage and flat object conversion for Python models.

use std::collections::BTreeSet;

use sigil_stitch::prelude::*;

use super::emit_models::python_field_name;
use super::value::{Direction, ValuePlan};
use crate::generators::type_application;
use crate::ir::types::{IrObject, IrTypeExpr};

pub(super) struct AdditionalProperties<'a> {
name: String,
values: &'a IrTypeExpr,
known: Vec<&'a str>,
}

impl<'a> AdditionalProperties<'a> {
pub fn new(object: &'a IrObject) -> Option<Self> {
let values = object.additional_properties.as_ref()?;
let used = object
.properties
.values()
.map(|property| python_field_name(&property.name))
.collect::<BTreeSet<_>>();
let mut name = "additional_properties".to_string();
while used.contains(&name) {
name.push('_');
}
Some(Self {
name,
values,
known: object.properties.keys().map(String::as_str).collect(),
})
}

pub fn field(&self, schema: &str, plan: &ValuePlan<'_>) -> FieldSpec {
let factory = TypeName::importable("dataclasses", "field").with_alias("_dataclass_field");
let dictionary = TypeName::importable("builtins", "dict").with_alias("_dict");
let key = TypeName::importable("builtins", "str").with_alias("_str");
let container =
type_application(dictionary, vec![key, plan.model_type(self.values, schema)]);
// Specialize the factory for strict typing, but defer recursive model
// references until construction, after their modules finish importing.
FieldSpec::builder(&self.name, container.clone())
.initializer(
sigil_quote!(Python {
$T(factory)(default_factory=lambda: $T(container)(), kw_only=True, metadata={"openapi_additional_properties": True, "openapi_dictionary": $if(self.known.is_empty()) { True } $else { False }})
})
.expect("Python extension storage initializer builds")
.__sigil_trim_trailing_newline(),
)
.build()
.expect("Python extension storage field builds")
}

pub fn write_entries(&self, plan: &ValuePlan<'_>) -> CodeBlock {
let value = plan.call(
self.values,
sigil_quote!(Python { item }).expect("Python extension value access builds"),
Direction::To,
);
sigil_quote!(Python {
for key, item in self.$N(self.name.as_str()).items() {
$if(!self.known.is_empty()) {
if key in [$for(key in &self.known; separator = ", ") { $S(*key) }] {
raise ValueError("Additional property collides with declared wire field: " + key)
}
}
result[key] = $C(value)
}
})
.expect("Python flat extension serialization builds")
}

pub fn read_entries(&self, schema: &str, plan: &ValuePlan<'_>) -> CodeBlock {
let value = plan.call(
self.values,
sigil_quote!(Python { item }).expect("Python extension wire value builds"),
Direction::From,
);
let value = plan.coerce(self.values, plan.model_type(self.values, schema), value);
sigil_quote!(Python {
$N(self.name.as_str()) = {key: $C(value) for key, item in data.items() $if(!self.known.is_empty()) { if key not in [$for(key in &self.known; separator = ", ") { $S(*key) }] }},
})
.expect("Python typed extension deserialization builds")
}
}

#[cfg(test)]
mod tests {
use super::*;
use crate::ir::types::IrSchemaKind;
use sigil_stitch::{assert_rendered, lang::python::Python};

#[test]
fn storage_factory_aliases_and_collision_name_render_exactly() {
let source = "openapi: 3.1.0\ninfo: {title: Storage, version: '1'}\npaths: {}\ncomponents:\n schemas:\n Model:\n type: object\n properties:\n additionalProperties: {type: string}\n additionalProperties: {type: string}\n";
let ir =
crate::ir::lower::lower(crate::parser::parse_content_yaml(source).unwrap()).unwrap();
let IrSchemaKind::Object(object) = &ir.schemas["Model"].kind else {
panic!("extension model retains object facts");
};
let plan = ValuePlan::new(&ir, super::super::value::UntaggedUnionStrategy::FirstMatch);
let storage = AdditionalProperties::new(object).unwrap();
let declaration = TypeSpec::builder("Model", TypeKind::Class)
.annotate(AnnotationSpec::importable(TypeName::importable(
"dataclasses",
"dataclass",
)))
.add_field(storage.field("Model", &plan))
.build()
.unwrap()
.emit(&Python::new())
.unwrap();
let body = sigil_quote!(Python { $C_each(declaration) }).unwrap();
assert_rendered!(
Python::new(),
width = 300,
body,
"from builtins import dict as _dict, str as _str\nfrom dataclasses import dataclass, field as _dataclass_field\n\n@dataclass\nclass Model:\n additional_properties_: _dict[_str, str] = _dataclass_field(default_factory = lambda: _dict[_str, str](), kw_only = True, metadata = {\"openapi_additional_properties\": True, \"openapi_dictionary\": False})\n\n",
);
}
}
7 changes: 5 additions & 2 deletions src/generators/python/emit_api_values.rs
Original file line number Diff line number Diff line change
Expand Up @@ -25,10 +25,13 @@ pub(super) fn multipart_data(
ir: &IrSpec,
values: &ValuePlan<'_>,
) -> CodeBlock {
if parts.iter().any(|part| values.has_union(&part.type_expr)) {
if parts
.iter()
.any(|part| values.needs_conversion(&part.type_expr))
{
let assignments = parts.iter().map(|part| {
let source = sigil_quote!(Python { $N(body_name).$N(part.field_name.as_str()) }).expect("Python multipart input access builds");
let assignment = if part.value_encoding == MultipartValueEncoding::Json && values.has_union(&part.type_expr) {
let assignment = if part.value_encoding == MultipartValueEncoding::Json && values.needs_conversion(&part.type_expr) {
let converted = values.call(&part.type_expr, source.clone(), Direction::To);
sigil_quote!(Python {
files[$S(part.wire_name.as_str())] = (None, json.dumps($C(converted)), $S(part.content_type.as_str()))
Expand Down
Loading
Loading