optimize string accumulation and devirtualize hot Silver runtime paths

[theMackabu]

• cache UTF-16 lengths across flat strings, ropes, and builders to avoid repeated flattening and scanning.
• optimize self-appending strings and templates while preserving snapshot and coercion semantics.
• expand Silver JIT fast paths for globals, calls, properties, elements, iteration, typed arrays, BigInt, regex, and deletion.
• consolidate function debug metadata and add regression coverage for the new optimizations and bailout paths.

Summary by CodeRabbit

• New Features
  • Improved JavaScript string handling, including accurate UTF-16 lengths, Unicode searches, template literal conversion, and string accumulation.
  • Added support for additional runtime and JIT execution paths, including optimized method calls, global access, iteration, typed arrays, regular expressions, and BigInt operations.
• Bug Fixes
  • Improved error propagation, accessor behavior, bound methods, deoptimization, and source-location reporting.
  • Corrected edge cases involving arrays, proxies, sparse properties, numeric keys, and template coercion.
• Tests
  • Added broad regression coverage for string, iteration, JIT, accessor, collection, and coercion behavior.

[coderabbitai]

📝 Walkthrough

Walkthrough

This PR centralizes UTF-16 string length and metadata handling, relocates function debug data, adds compiler and VM support for templates and string builders, expands JIT fast paths and devirtualization guards, and adds broad regression tests.

Changes

String runtime and metadata

Layer / File(s)	Summary
Packed metadata and UTF-16 handling include/internal.h, include/pool.h, include/utf8.h, src/ant.c, src/utf8.c, src/pool.c, src/modules/io.c	Flat and large strings use packed metadata, while UTF-16 lengths are computed lazily across flat, builder, and rope strings. String operations and indexOf use the centralized helpers.
Supporting runtime cleanup src/modules/json.c	JSON object key hash cleanup is centralized in json_key_hash_free across conversion error and completion paths.

Compiler and VM metadata

Layer / File(s)	Summary
Function debug data and source locations include/silver/engine.h, src/silver/compiler.c, src/silver/engine.c, src/errors.c, src/modules/builtin.c, src/silver/ops/objects.h	Function names, filenames, source positions, and source buffers move into sv_func_debug_t, with compiler, error reporting, disassembly, and class metadata updated accordingly.
Template and self-append compilation src/silver/compiler.c	Template cooked segments are validated, and self-append matching/code generation supports normal expressions and template-rest forms.
VM opcode wiring include/silver/opcode.h, src/silver/engine.c, src/silver/ops/coercion.h, src/silver/ops/globals.h	DEBUGGER, NOP, and TO_STRING handling is added, while global lookup errors route through standard VM checks.

JIT specialization

Layer / File(s)	Summary
String-builder JIT paths include/silver/glue.h, src/silver/glue.c, src/silver/engine.c, src/silver/swarm.c	Builder-target slots are prescanned and specialized reads, snapshots, caching, numeric mirrors, and GC-safe builder value reads are emitted.
Global IC and devirtualized calls include/silver/opcode.h, src/silver/swarm.c, src/silver/ops/globals.h	Global and field IC fast paths gain epoch guards, while method devirtualization checks bound arguments and propagates inline errors.
JIT conversion and bailout paths src/silver/swarm.c, src/silver/glue.c	TO_STRING, numeric OSR guards, self-binding preservation, debug naming, and runtime error-site attribution are integrated into JIT code generation.

Regression coverage

Layer / File(s)	Summary
Dispatch and JIT behavior tests examples/spec/devirtualization.js, tests/test_jit_*.cjs, tests/test_global_accessor_read.cjs	Tests cover devirtualized method calls, global accessor and IC invalidation behavior, inline-call errors, native calls, element operations, typed arrays, iterators, and self-reference deoptimization.
String and coercion tests examples/spec/strings.js, tests/test_string_length_accumulation.cjs, tests/test_template_self_append.cjs, tests/test_jit_string_builder_snapshot.cjs, tests/test_number_simple_decimal_fast.cjs	Coverage validates UTF-16 offsets, accumulated string lengths, template coercion and exceptions, builder snapshots, and decimal conversion.
Regression manifest tests/harness/manifest.js	Five new regression scripts are included in the harness target list.

Estimated code review effort: 5 (Critical) | ~120 minutes

Sequence Diagram(s)

sequenceDiagram
  participant JavaScriptCode
  participant Compiler
  participant JIT
  participant Runtime
  JavaScriptCode->>Compiler: compile templates, self-append, and member access
  Compiler->>JIT: emit builder, global IC, and devirtualized call paths
  JIT->>Runtime: invoke guarded fast paths or fallback helpers
  Runtime-->>JIT: return values or errors
  JIT-->>JavaScriptCode: preserve results and exception propagation

Loading

Possibly related PRs

• theMackabu/ant#5: Related JSON object key hash-table cleanup changes.
• theMackabu/ant#12: Related string coercion API and template conversion changes.
• theMackabu/ant#44: Related JIT string intrinsic and helper changes.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 2.54% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main changes: string accumulation optimizations and hot Silver runtime devirtualization.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch perf/string-flatten-fix

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[theMackabu]

@macroscope-app review

[macroscopeapp]

Manual reviews triggered for commit 096abfe:

• Approvability Check

^{All prior checks · these links stay valid even if you push more commits.}

[macroscopeapp]

Review in progress. Results will be posted as check runs when complete.

[macroscopeapp]

Approvability

Verdict: Needs human review

Significant runtime optimization changes affecting string metadata caching, JIT devirtualization, template literal compilation, and global accessor handling. The scope and complexity of these core engine modifications warrant careful review despite comprehensive test coverage.

^{You can customize Macroscope's approvability policy. Learn more.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@include/silver/engine.h`:
- Around line 1072-1073: Prevent null dereferences of relocated debug metadata
by guarding every listed access with a debug-pointer check: in
include/silver/engine.h lines 1072-1073 and 1083, src/silver/engine.c lines 123,
136, 169, and 204, src/errors.c lines 376 and 418-428, and
src/silver/ops/objects.h lines 312-315. Preserve existing fallback behavior for
missing names, filenames, source data, and positions, and audit other
func->debug consumers, including js_set_error_site_from_bc calls in
src/silver/ops/property.h, applying the same guards where needed.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 36db738b-f174-4584-811b-1b783641506f

📥 Commits

Reviewing files that changed from the base of the PR and between 3628be4 and 096abfe.

📒 Files selected for processing (42)

• examples/spec/devirtualization.js
• examples/spec/strings.js
• include/internal.h
• include/pool.h
• include/silver/engine.h
• include/silver/glue.h
• include/silver/opcode.h
• include/utf8.h
• src/ant.c
• src/errors.c
• src/modules/builtin.c
• src/modules/io.c
• src/modules/json.c
• src/pool.c
• src/silver/compiler.c
• src/silver/engine.c
• src/silver/glue.c
• src/silver/ops/coercion.h
• src/silver/ops/globals.h
• src/silver/ops/objects.h
• src/silver/ops/property.h
• src/silver/swarm.c
• src/utf8.c
• tests/harness/manifest.js
• tests/test_for_in_dense_array_fast.cjs
• tests/test_for_of_dense_array_fast.cjs
• tests/test_global_accessor_read.cjs
• tests/test_jit_bigint_ops.cjs
• tests/test_jit_cfunc_call_fast.cjs
• tests/test_jit_delete_numeric_key.cjs
• tests/test_jit_for_of.cjs
• tests/test_jit_global_read_ic.cjs
• tests/test_jit_inline_call_errors.cjs
• tests/test_jit_put_elem_fast.cjs
• tests/test_jit_put_field_ic.cjs
• tests/test_jit_regexp_literal_intrinsics.cjs
• tests/test_jit_string_builder_snapshot.cjs
• tests/test_jit_typed_array_elem_fast.cjs
• tests/test_jit_upvalue_self_reference.cjs
• tests/test_number_simple_decimal_fast.cjs
• tests/test_string_length_accumulation.cjs
• tests/test_template_self_append.cjs