FIX: raise on characters AsciiSmugglerConverter cannot encode - #2540
Open
fei (feiiiiii5) wants to merge 1 commit into
Open
FIX: raise on characters AsciiSmugglerConverter cannot encode#2540fei (feiiiiii5) wants to merge 1 commit into
fei (feiiiiii5) wants to merge 1 commit into
Conversation
encode_message collected characters outside the ASCII printable range (0x20-0x7E), logged them at error level, and dropped them, while convert_async still returned a success ConverterResult. A non-ASCII objective was therefore smuggled as a corrupted or empty payload and the run reported success. Raise ValueError naming the offending characters and pointing to the lossless sibling converters instead. Adds regression tests (pre-fix: DID NOT RAISE) covering the reported non-ASCII cases, an astral-plane emoji, a control character, and the inclusive 0x20/0x7E boundaries.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Addresses the data-loss bug reported in #2539.
Root cause.
AsciiSmugglerConverter.encode_messagemaps each character in the ASCII printable range (0x20-0x7E) to a Unicode tag. Characters outside that range were collected intoinvalid_chars, logged vialogger.error, and then dropped, butSmugglerConverter.convert_asyncstill wrapped the truncated output in a successConverterResult. So a non-ASCII objective was smuggled as a corrupted payload (café->caf) or an empty one (你好世界->""), while the run reported success. In a red-team scan that means an attack against a non-English objective can be scored "did not succeed" against a target that was never sent the objective.Fix. Validate up front and fail loudly instead of silently dropping.
encode_messagenow raisesValueErrornaming the offending characters and pointing to the lossless siblings. This matches the module's existing error contract (convert_asyncalready raisesValueErrorfor an unsupported input type,__init__for an invalid action) and the siblingSneakyBitsSmugglerConverter/VariationSelectorSmugglerConverter, which round-trip arbitrary Unicode via UTF-8 bytes and never drop.Behavior change. Encoding a prompt with any character outside
0x20-0x7Enow raises instead of returning a success result with those characters removed. This only affects inputs the converter could never represent faithfully (it also covers control characters such as newline, dropped by the same path). Happy to carry a[BREAKING]tag on the title if you prefer.Scope. I left the report's second, smaller point out of this PR: the
decode_messagelength check that logs "No hidden Unicode Tag characters discovered." for an all-hidden string. That is a log-only inaccuracy with no effect on the returned value, so this stays scoped to the data-loss/false-success bug. I can send it as a follow-up or fold it in here, whichever you prefer.Tests and Documentation
tests/unit/converter/test_ascii_smuggler_converter.py:test_ascii_smuggler_encode_unrepresentable_chars_raise(parametrized overcafé,你好世界,naïve résumé, an astral-plane emoji, andline1\nline2) assertsValueError.test_ascii_smuggler_encode_printable_boundaries_do_not_raisepins the inclusive0x20/0x7Eedges so the range check cannot drift by one.Regression evidence:
main(6d5b1a9):5 failed, 7 passed- each new caseDID NOT RAISE ValueError, captured logERROR ...ascii_smuggler_converter.py:81 Invalid characters detected:.12 passed. All four token-smuggling converter test files:38 passed.Diff scope: 2 files, +38/-10 (
ascii_smuggler_converter.py+17/-10, test +21). Formatted with the pinned ruffv0.16.4(ruff format+ruff check-> all passed). No documentation change needed: the only doc sample that calls this converter (doc/code/converters/1_text_to_text_converters.py) uses the ASCII prompt"secret message".