Skip to content

Parser: do not treat '#=>' inside string literals as assertion delimiters #3

Description

@jcouball

Summary

Fix example normalization so #=> is only treated as an assertion marker when it is an inline trailing comment or a standalone comment line, not when it appears inside Ruby string literals.

Motivation

The current normalization logic performs global string substitution:

text = text.gsub('# =>', '#=>')
text = text.gsub('#=>', "\n#=>")

This rewrites any #=> sequence regardless of Ruby syntax context. If #=> appears inside a string literal, the source code is split into invalid lines and the example fails for the wrong reason.

Concrete example

# @example
#   puts '#=> not assertion'
#   sum(1, 2) #=> 3

Current normalization produces:

["puts '", "#=> not assertion'", "sum(1,2)", "#=> 3"]

This corrupts the first line and can produce syntax errors unrelated to the documented code.

Current behaviour

normalize_example_lines blindly inserts a newline before every #=> token in the raw text, including string literal content.

Proposed solution

Parse line-by-line and treat #=> as an expectation delimiter only when one of these is true:

  1. The line starts with #=> (already split assertion line), or
  2. The token appears in a trailing Ruby comment position outside quotes.

A practical implementation is to scan each line while tracking quote state (' / ") and escapes, then split only on comment #=> boundaries that are outside quoted regions.

Scope

In scope

  • Update normalize_example_lines to be syntax-aware enough to avoid splitting inside string literals
  • Keep support for both #=> and # => assertion comment styles
  • Add integration tests showing #=> inside strings does not create assertions

Out of scope

  • Full Ruby parser integration
  • Supporting assertion markers embedded in heredoc content

Implementation notes

  1. Preserve existing behavior for valid forms:
    • sum(1, 2) #=> 3
    • sum(1, 2)\n#=> 3
    • sum(1, 2) # => 3
  2. Do not split when #=> is inside '...' or "...".
  3. Keep normalized output shape compatible with extract_expectations.

Acceptance criteria

  • puts '#=> not assertion' remains a single code line after normalization
  • sum(1, 2) #=> 3 still creates one expectation
  • sum(1, 2) # => 3 still creates one expectation
  • Existing cucumber scenarios continue to pass

Related

  • lib/yard/cli/test_examples.rbnormalize_example_lines
  • lib/yard/cli/test_examples.rbextract_expectations

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions