Repository navigation
fix(typescript): infer Long from protobuf resource bindings - #345
tconley1428 wants to merge 1 commit into
Conversation
| } => { | ||
| // Construct names are either authored record names or descriptor message names. | ||
| // The latter need not have a native declaration (e.g. external replacements). | ||
| let proto = api.record(message_name).map_or(true, |record| { |
There was a problem hiding this comment.
Each nested record sets this flag again, from that record alone. So a proto-backed record inside a JSON request is treated as proto. For example, take a JSON touch-request with a field at: timestamp, where timestamp is bound to google.protobuf.Timestamp. The resource method then takes seconds: Long. But the request goes through the JSON converter, which writes the Long as {"low":...,"high":...,"unsigned":false}, and a Python handler can't read that. #296 refuses this case. Should a value stay JSON when any record around it is JSON? A fixture for this case would help too.
There was a problem hiding this comment.
Oh lord. Mixed records like that are... not something I really want to consider yet. I would either leave that undefined behavior or straight up disallow it.
There was a problem hiding this comment.
Thanks. The record check covers this case now. It can still happen through a resource that a JSON result returns. For example, resource counter { constructor(name: string, at: option<timestamp>) } with timestamp bound to Timestamp, and get-counter returning type counter-result = own<counter> with no proto. A TS handler sends at.seconds as {"low":5,"high":0,"unsigned":false}, while Python sends 5. Should the check also cover constructor fields of a resource that a JSON result returns?
| match value { | ||
| PlannedType::Int(_) => "number".to_string(), | ||
| PlannedType::Float => "number".to_string(), | ||
| PlannedType::Int(int) if proto => typescript_proto::integer_annotation(*int).to_string(), |
There was a problem hiding this comment.
The type comes from the resource value's own WIT type, not from the field it binds to. So nanos: option<s64> bound to Timestamp.nanos, which is int32, gives Long, and tsc fails. main compiles this. It starts from a WIT mistake, so it's minor. Should nexgen refuse the width mismatch instead?
There was a problem hiding this comment.
Probably, yeah
There was a problem hiding this comment.
Thanks. It's refused now for method parameters and for constructor fields that feed a request. A resource that a result returns skips the check. A clock with only constructor(seconds: option<s64>, nanos: option<s64>) and get-clock: func(request: timestamp) -> clock-result is accepted, and tsc fails in get-clock.ts with number | undefined not assignable to Long | undefined. Main fails here too, on seconds. Should bind_resource_return_field check widths as well?
| ).toBe("stream-1"); | ||
| }); | ||
|
|
||
| test("preserves offsets above 2^53 through resource calls and proto conversion", async () => { |
There was a problem hiding this comment.
Does the Long survive a real payload? toTransferType returns a plain object, not a protobufjs message. So the default payload converter writes it as JSON, and the Long becomes {"low":...,"high":...}. This test checks only the in-memory conversion. It was already like this before this PR, but should we test the payload too?
There was a problem hiding this comment.
Probably yeah.
There was a problem hiding this comment.
Thanks for the Int64Value payload test. This stream test still mocks the response model, so the response converter never runs, and maxBytes isn't covered. A real round trip keeps fromOffset and maxBytes as Long, but the response loses every item, see the toTransferType thread. Could we add toPayloadWithTypeInfo and fromPayloadWithTypeInfo checks for the read request and response converters here?
| output.push_str("): "); | ||
| output.push_str(&proto_type); | ||
| output.push_str(" {\n return "); | ||
| output.push_str(&format!(" {{\n return {message_type}.create(")); |
There was a problem hiding this comment.
Now that toTransferType returns create(...), a TS-to-TS call goes through json/protobuf. On decode, protojson.fromJson builds only the top-level message, and nested messages stay plain objects. The generated fromProto switches on the virtual oneof field, for example proto.item, and a plain object doesn't have it. So after a json/protobuf round trip, every records[].item in a ReadRecordsResponse is undefined, data records too. Top-level oneofs still work. Could we pick the case from the member that is set, for example proto.data != null ? "data" : ...? A json/protobuf round trip test with a nested oneof would cover it.
|
|
||
| function timestampFromProto(proto: google.protobuf.ITimestamp): Timestamp { | ||
| return { | ||
| seconds: proto.seconds ?? undefined, |
There was a problem hiding this comment.
seconds is typed Long, but this copies the value as it comes. Through json/protobuf, protojson decodes google.protobuf.Timestamp and Duration with seconds as a JS number. So request.seconds?.add(1) in a touch handler throws add is not a function. The plain JSON converter gives a string here. The Int64Value test passes because that type happens to decode to Long. Should fromProto normalize 64-bit reads with Long.fromValue(...), with true for u64?
|
|
||
| function unsignedValueToProto(model: UnsignedValue): google.protobuf.IUInt64Value { | ||
| return { | ||
| value: model.value, |
There was a problem hiding this comment.
For a u64, this sends the Long as is. If the caller builds a signed Long, for example Long.fromString("18446744073709551615"), it's -1. Through json/protobuf, the wire then gets "-1", and the decode fails with out of range for uint64. 2^63 fails the same way. Should the u64 toProto path call .toUnsigned()?
| panic!("planned model should exist for {}", model.full_name) | ||
| }); | ||
| if let Some(proto) = &planned_record.data.proto { | ||
| rendered.model_requirements.value_namespace_imports.insert( |
There was a problem hiding this comment.
This adds a value import for every proto namespace, also for a model with no converter. Could we skip it there?
| impl ModelBackend { | ||
| /// Render the complete model/resource graph together. All protobuf binding state | ||
| /// and per-model conversion orchestration stays within this invocation. | ||
| pub(in crate::generator) fn render_types( |
There was a problem hiding this comment.
render_types in the proto backend now renders every enum, flag, variant and JSON-only model, and the planner types are now pub(in crate::generator). So a spec with no proto still goes through the proto backend. Is it worth keeping the shared rendering in typescript.rs, and leaving only the resource annotations and converters here?
| }); | ||
| const clock = await getClock({ seconds, nanos: 42 }); | ||
| expect(clock).toBeInstanceOf(Clock); | ||
| expect(clock.seconds?.toString()).toBe("3600"); |
There was a problem hiding this comment.
Could the getClock test also check nanos?
| } => { | ||
| // Construct names are either authored record names or descriptor message names. | ||
| // The latter need not have a native declaration (e.g. external replacements). | ||
| let proto = api.record(message_name).map_or(true, |record| { |
There was a problem hiding this comment.
Thanks. The record check covers this case now. It can still happen through a resource that a JSON result returns. For example, resource counter { constructor(name: string, at: option<timestamp>) } with timestamp bound to Timestamp, and get-counter returning type counter-result = own<counter> with no proto. A TS handler sends at.seconds as {"low":5,"high":0,"unsigned":false}, while Python sends 5. Should the check also cover constructor fields of a resource that a JSON result returns?
| match value { | ||
| PlannedType::Int(_) => "number".to_string(), | ||
| PlannedType::Float => "number".to_string(), | ||
| PlannedType::Int(int) if proto => typescript_proto::integer_annotation(*int).to_string(), |
There was a problem hiding this comment.
Thanks. It's refused now for method parameters and for constructor fields that feed a request. A resource that a result returns skips the check. A clock with only constructor(seconds: option<s64>, nanos: option<s64>) and get-clock: func(request: timestamp) -> clock-result is accepted, and tsc fails in get-clock.ts with number | undefined not assignable to Long | undefined. Main fails here too, on seconds. Should bind_resource_return_field check widths as well?
| ).toBe("stream-1"); | ||
| }); | ||
|
|
||
| test("preserves offsets above 2^53 through resource calls and proto conversion", async () => { |
There was a problem hiding this comment.
Thanks for the Int64Value payload test. This stream test still mocks the response model, so the response converter never runs, and maxBytes isn't covered. A real round trip keeps fromOffset and maxBytes as Long, but the response loses every item, see the toTransferType thread. Could we add toPayloadWithTypeInfo and fromPayloadWithTypeInfo checks for the read request and response converters here?
344597f to
4fece57
Compare
4fece57 to
39eaf22
Compare
39eaf22 to
bd16ba7
Compare
bd16ba7 to
1ea71f6
Compare
Alternative to #296, stacked on #348 for the shared mixed-backing validation rule.
Long; JSON-only and unbound values remainnumber. Reject conflicting constructor bindings and mismatched integer widths, including fields populated in returned resources.Longvalues on protobuf reads and normalize signedness on writes. Leave message construction to the transport rather than requiring generated protobufjs instances or a custom user payload converter.The payload boundary tests use real SDK serializers; a live System Nexus integration test remains unavailable in the installed SDK version.