Skip to content

fix(typescript): infer Long from protobuf resource bindings - #345

Open
tconley1428 wants to merge 1 commit into
mainfrom
tconley/ts-binding-aware-long
Open

tconley1428 wants to merge 1 commit into
mainfrom
tconley/ts-binding-aware-long

Conversation

@tconley1428

@tconley1428 tconley1428 commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

Alternative to #296, stacked on #348 for the shared mixed-backing validation rule.

  • Infer resource signatures from their bindings: protobuf-backed 64-bit values use Long; JSON-only and unbound values remain number. Reject conflicting constructor bindings and mismatched integer widths, including fields populated in returned resources.
  • Reconstruct Long values on protobuf reads and normalize signedness on writes. Leave message construction to the transport rather than requiring generated protobufjs instances or a custom user payload converter.
  • Render models, conversions, and resources through one backend call, with shared conversion-expression handling and per-method backing analysis.

The payload boundary tests use real SDK serializers; a live System Nexus integration test remains unavailable in the installed SDK version.

} => {
// Construct names are either authored record names or descriptor message names.
// The latter need not have a native declaration (e.g. external replacements).
let proto = api.record(message_name).map_or(true, |record| {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Each nested record sets this flag again, from that record alone. So a proto-backed record inside a JSON request is treated as proto. For example, take a JSON touch-request with a field at: timestamp, where timestamp is bound to google.protobuf.Timestamp. The resource method then takes seconds: Long. But the request goes through the JSON converter, which writes the Long as {"low":...,"high":...,"unsigned":false}, and a Python handler can't read that. #296 refuses this case. Should a value stay JSON when any record around it is JSON? A fixture for this case would help too.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh lord. Mixed records like that are... not something I really want to consider yet. I would either leave that undefined behavior or straight up disallow it.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. The record check covers this case now. It can still happen through a resource that a JSON result returns. For example, resource counter { constructor(name: string, at: option<timestamp>) } with timestamp bound to Timestamp, and get-counter returning type counter-result = own<counter> with no proto. A TS handler sends at.seconds as {"low":5,"high":0,"unsigned":false}, while Python sends 5. Should the check also cover constructor fields of a resource that a JSON result returns?

Comment thread src/generator/typescript.rs Outdated
match value {
PlannedType::Int(_) => "number".to_string(),
PlannedType::Float => "number".to_string(),
PlannedType::Int(int) if proto => typescript_proto::integer_annotation(*int).to_string(),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The type comes from the resource value's own WIT type, not from the field it binds to. So nanos: option<s64> bound to Timestamp.nanos, which is int32, gives Long, and tsc fails. main compiles this. It starts from a WIT mistake, so it's minor. Should nexgen refuse the width mismatch instead?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably, yeah

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. It's refused now for method parameters and for constructor fields that feed a request. A resource that a result returns skips the check. A clock with only constructor(seconds: option<s64>, nanos: option<s64>) and get-clock: func(request: timestamp) -> clock-result is accepted, and tsc fails in get-clock.ts with number | undefined not assignable to Long | undefined. Main fails here too, on seconds. Should bind_resource_return_field check widths as well?

).toBe("stream-1");
});

test("preserves offsets above 2^53 through resource calls and proto conversion", async () => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the Long survive a real payload? toTransferType returns a plain object, not a protobufjs message. So the default payload converter writes it as JSON, and the Long becomes {"low":...,"high":...}. This test checks only the in-memory conversion. It was already like this before this PR, but should we test the payload too?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably yeah.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the Int64Value payload test. This stream test still mocks the response model, so the response converter never runs, and maxBytes isn't covered. A real round trip keeps fromOffset and maxBytes as Long, but the response loses every item, see the toTransferType thread. Could we add toPayloadWithTypeInfo and fromPayloadWithTypeInfo checks for the read request and response converters here?

Comment thread src/generator/typescript.rs Resolved
Comment thread GUIDE.md Resolved
Comment thread src/generator/proto/typescript.rs Outdated
output.push_str("): ");
output.push_str(&proto_type);
output.push_str(" {\n return ");
output.push_str(&format!(" {{\n return {message_type}.create("));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now that toTransferType returns create(...), a TS-to-TS call goes through json/protobuf. On decode, protojson.fromJson builds only the top-level message, and nested messages stay plain objects. The generated fromProto switches on the virtual oneof field, for example proto.item, and a plain object doesn't have it. So after a json/protobuf round trip, every records[].item in a ReadRecordsResponse is undefined, data records too. Top-level oneofs still work. Could we pick the case from the member that is set, for example proto.data != null ? "data" : ...? A json/protobuf round trip test with a nested oneof would cover it.


function timestampFromProto(proto: google.protobuf.ITimestamp): Timestamp {
return {
seconds: proto.seconds ?? undefined,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seconds is typed Long, but this copies the value as it comes. Through json/protobuf, protojson decodes google.protobuf.Timestamp and Duration with seconds as a JS number. So request.seconds?.add(1) in a touch handler throws add is not a function. The plain JSON converter gives a string here. The Int64Value test passes because that type happens to decode to Long. Should fromProto normalize 64-bit reads with Long.fromValue(...), with true for u64?


function unsignedValueToProto(model: UnsignedValue): google.protobuf.IUInt64Value {
return {
value: model.value,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For a u64, this sends the Long as is. If the caller builds a signed Long, for example Long.fromString("18446744073709551615"), it's -1. Through json/protobuf, the wire then gets "-1", and the decode fails with out of range for uint64. 2^63 fails the same way. Should the u64 toProto path call .toUnsigned()?

Comment thread src/generator/proto/typescript.rs Outdated
panic!("planned model should exist for {}", model.full_name)
});
if let Some(proto) = &planned_record.data.proto {
rendered.model_requirements.value_namespace_imports.insert(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This adds a value import for every proto namespace, also for a model with no converter. Could we skip it there?

impl ModelBackend {
/// Render the complete model/resource graph together. All protobuf binding state
/// and per-model conversion orchestration stays within this invocation.
pub(in crate::generator) fn render_types(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

render_types in the proto backend now renders every enum, flag, variant and JSON-only model, and the planner types are now pub(in crate::generator). So a spec with no proto still goes through the proto backend. Is it worth keeping the shared rendering in typescript.rs, and leaving only the resource annotations and converters here?

});
const clock = await getClock({ seconds, nanos: 42 });
expect(clock).toBeInstanceOf(Clock);
expect(clock.seconds?.toString()).toBe("3600");

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could the getClock test also check nanos?

} => {
// Construct names are either authored record names or descriptor message names.
// The latter need not have a native declaration (e.g. external replacements).
let proto = api.record(message_name).map_or(true, |record| {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. The record check covers this case now. It can still happen through a resource that a JSON result returns. For example, resource counter { constructor(name: string, at: option<timestamp>) } with timestamp bound to Timestamp, and get-counter returning type counter-result = own<counter> with no proto. A TS handler sends at.seconds as {"low":5,"high":0,"unsigned":false}, while Python sends 5. Should the check also cover constructor fields of a resource that a JSON result returns?

Comment thread src/generator/typescript.rs Outdated
match value {
PlannedType::Int(_) => "number".to_string(),
PlannedType::Float => "number".to_string(),
PlannedType::Int(int) if proto => typescript_proto::integer_annotation(*int).to_string(),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. It's refused now for method parameters and for constructor fields that feed a request. A resource that a result returns skips the check. A clock with only constructor(seconds: option<s64>, nanos: option<s64>) and get-clock: func(request: timestamp) -> clock-result is accepted, and tsc fails in get-clock.ts with number | undefined not assignable to Long | undefined. Main fails here too, on seconds. Should bind_resource_return_field check widths as well?

).toBe("stream-1");
});

test("preserves offsets above 2^53 through resource calls and proto conversion", async () => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the Int64Value payload test. This stream test still mocks the response model, so the response converter never runs, and maxBytes isn't covered. A real round trip keeps fromOffset and maxBytes as Long, but the response loses every item, see the toTransferType thread. Could we add toPayloadWithTypeInfo and fromPayloadWithTypeInfo checks for the read request and response converters here?

Comment thread src/generator/typescript.rs Resolved
@tconley1428
tconley1428 force-pushed the tconley/ts-binding-aware-long branch from 344597f to 4fece57 Compare November 2, 2026 17:39
@tconley1428
tconley1428 changed the base branch from main to tconley/reject-mixed-record-backing November 2, 2026 17:39
@tconley1428
tconley1428 added this pull request to stack #349 November 2, 2026 17:39
@tconley1428
tconley1428 force-pushed the tconley/ts-binding-aware-long branch from 4fece57 to 39eaf22 Compare November 4, 2026 19:24
@tconley1428
tconley1428 force-pushed the tconley/ts-binding-aware-long branch from 39eaf22 to bd16ba7 Compare November 5, 2026 18:51
Base automatically changed from tconley/reject-mixed-record-backing to main November 5, 2026 19:01
@tconley1428
tconley1428 force-pushed the tconley/ts-binding-aware-long branch from bd16ba7 to 1ea71f6 Compare November 5, 2026 19:01
@tconley1428
tconley1428 marked this pull request as ready for review November 5, 2026 19:01
@tconley1428
tconley1428 requested a review from a team as a code owner November 5, 2026 19:01
@moetemp
moetemp self-requested a review November 10, 2026 04:23

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants