fix(http_request): JSON-encode string variables when building a JSON body - #235
Open
Mr-Neutr0n wants to merge 1 commit into
Open
fix(http_request): JSON-encode string variables when building a JSON body#235Mr-Neutr0n wants to merge 1 commit into
Mr-Neutr0n wants to merge 1 commit into
Conversation
…body convert_template(...).text concatenates raw values into the template, so a string carrying a quote, backslash or newline breaks the surrounding JSON. repair_json then does not repair it: for the case in langgenius/dify#31927 it parses the wreckage down to an empty string, so the node silently posts "content": "" to the upstream API. The template the author wrote is valid JSON. Substitution is what breaks it, so encode each string value with json.dumps at the point of substitution. The document stays valid, no repair is needed, and the value round-trips byte for byte. A variable already inside a string literal gets the escaped body without the quotes json.dumps would add. Quote state is tracked across the literal parts rather than sniffed from the adjacent characters, so a variable embedded partway through a string behaves the same as one standing alone. repair_json is retained as a fallback for templates that were already malformed before substitution, which escaping cannot fix. Signed-off-by: Harikrishna KP <harikp2002@gmail.com>
|
All contributors on this pull request have signed the CLA. |
Author
|
I have read the CLA Document and I hereby sign the CLA |
Author
|
recheck |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ports the fix from langgenius/dify#32317, which was opened before this node moved out of
difyintographon. The defect is unchanged here._init_json_body(executor.py:275) still doesconvert_template(...).textthenrepair_json.Originally reported as langgenius/dify#31927, which a stale bot closed as
not_plannedon 2026-05-05 without a fix.The bug
The template the workflow author writes is valid JSON:
{"model": "pro", "messages": [{"role": "user", "content": {{#node.result#}}}]}convert_template(...).textconcatenates the raw value in, so a string carrying a quote, backslash or newline (ordinary markdown out of a Code node) breaks the surrounding document.repair_jsonthen does not repair it — it parses the wreckage down to an empty string:The request goes out with
"content": "". No error, no warning; the model just receives nothing.There's a second, unreported variant. With
{"content": "prefix {{#node.result#}} suffix"}and valuea"b, the repair pass injects a character rather than dropping one, yieldingprefix a"b suffix".The fix
Encode each string value with
json.dumpsat the point of substitution. The document stays valid, so no repair is needed and the value round-trips byte for byte.A variable already inside a string literal gets the escaped body without the quotes
json.dumpswould add. Quote state is tracked across the literal parts (_json_string_open_after) rather than sniffed from the adjacent characters, so a variable embedded partway through a string behaves the same as one standing alone.repair_jsonis retained as a fallback for templates that were already malformed before substitution, which escaping cannot fix. That keeps existing workflows working, and after this change the repair path is no longer load-bearing for well-formed templates._log_json_textis rendered through the same encoder withmasked=True, so the log copy stays valid JSON and secrets stay obfuscated.Tests
Six cases in
tests/nodes/http_request/test_dispatch.py. Against unpatchedmain, three fail and the other three pass, so they pin the change rather than the implementation:main..._preserves_a_string_containing_json_metacharacters(the reported bug)''..._preserves_a_string_embedded_partway_through_a_literalprefix a"b suffix"..._encodes_the_log_copy_the_same_way..._preserves_a_string_the_author_already_quoted..._keeps_non_string_values_as_json_tokens..._still_repairs_a_template_that_was_malformed_to_begin_withFull suite: 678 passed (
tests/http/test_client.pyskipped locally, it needspytest-mock).ruff checkandruff format --checkclean.Note on scope
@MezentsevIlya argued on the dify PR that the JSON repairing is an anti-pattern and should be removed or made switchable. I think that's right in direction, and this change is a step toward it rather than away: once substitution stops producing invalid JSON,
repair_jsonstops mattering for anyone writing a valid template. Whether to then drop the fallback entirely is a separate, breaking call that's yours to make — happy to follow up with that if you want it.