feat: allow backing an ArrayBuffer/SharedArrayBuffer with embedder-owned memory

[mansiverma897993]

This PR implements the feature requested in #5447: a public API to back an ArrayBuffer/SharedArrayBuffer with embedder-owned memory, enabling zero-copy sharing of byte regions (WebAssembly linear memories, mmap'd files, GPU-mapped buffers) between JavaScript and native host code.

What's added

Public API:

• JsArrayBuffer::from_external_ptr(ptr, len, context) (unsafe) — creates an ArrayBuffer whose bytes alias len bytes of embedder-owned memory at ptr. No copy, no allocation: JS writes are immediately visible to the embedder through ptr and vice versa.
• JsSharedArrayBuffer::from_external_ptr(ptr, len, context) (unsafe) — same for SharedArrayBuffer.
• ArrayBuffer::from_external_data(ptr, len) / SharedArrayBuffer::from_external_ptr(ptr, len) (unsafe) — lower-level constructors on the buffer data types themselves.
• is_external() on all four types.

Semantics of externally-backed buffers (the conservative answers to the design questions raised in the issue):

• Always fixed-length; resize/grow throw a TypeError.
• Cannot be detached nor transferred (detach, ArrayBuffer.prototype.transfer) — these throw a TypeError and leave the buffer intact. This matches the Wasm-memory use case (WebAssembly.Memory.prototype.buffer is non-detachable by user code) and keeps ownership unambiguous: Boa never allocates, grows nor frees the region.
• Everything else (typed arrays, DataView, Atomics, slice, etc.) works transparently on top of the aliased memory.

Implementation

• ArrayBuffer's internal slot changes from Option<AlignedVec<u8>> to Option<BufferData> with Owned(AlignedVec<u8>) / External(ExternalMemory) variants. All existing accessors (bytes, bytes_mut, bytes_with_len*, len) work on both variants, so typed arrays, DataView and Atomics need no changes.
• SharedArrayBuffer's Inner.buffer changes from AlignedBox<[AtomicU8]> to a SharedData enum with Owned/External variants (with unsafe impl Send/Sync justified by the atomic-only access contract).
• The lifetime/ownership question is answered with a documented unsafe pointer constructor (option 1 of the issue), with safety contracts spelled out on every constructor. An Arc-based BackingStore handle could be layered on top later without changing the internals.

Tests

• external_array_buffer_zero_copy — JS writes visible through the embedder's pointer and vice versa.
• external_array_buffer_cannot_detach_nor_resize — detach/resize throw and leave the buffer usable.
• external_shared_array_buffer_zero_copy — same aliasing checks for SharedArrayBuffer, plus growable === false.
• A doctest on JsArrayBuffer::from_external_ptr demonstrating the zero-copy roundtrip.

Closes #5447

🤖 Generated with Claude Code

[github-actions]

Test262 conformance changes

Test result	main count	PR count	difference
Total	53,125	53,125	0
Passed	51,073	51,073	0
Ignored	1,482	1,482	0
Failed	570	570	0
Panics	0	0	0
Conformance	96.14%	96.14%	0.00%

Tested main commit: 4fc75c6ae9d85f2b8065c6716f88e9b35318438c
Tested PR commit: 762e3c48c7c2ad3f76c9497cee1cc9fa69f8db1b
Compare commits: 4fc75c6...762e3c4

[codecov]

Codecov Report

❌ Patch coverage is 80.89888% with 17 lines in your changes missing coverage. Please review.
✅ Project coverage is 62.86%. Comparing base (6ddc2b4) to head (762e3c4).
⚠️ Report is 1012 commits behind head on main.

Files with missing lines	Patch %	Lines
core/engine/src/builtins/array_buffer/mod.rs	75.86%	14 Missing ⚠️
core/engine/src/builtins/array_buffer/shared.rs	84.21%	3 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##             main    #5466       +/-   ##
===========================================
+ Coverage   47.24%   62.86%   +15.61%     
===========================================
  Files         476      530       +54     
  Lines       46892    59167    +12275     
===========================================
+ Hits        22154    37195    +15041     
+ Misses      24738    21972     -2766     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[Shine-neko]

Thanks for tackling this so quickly! I tested this branch end-to-end embedding Boa, backing WebAssembly.Memory.prototype.buffer with a wasm linear memory via from_external_ptr, and it works: a napi-rs wasm addon now sees JS-side writes and runs correctly. Two things I hit while reviewing the unsafe surface:

[Shine-neko]

ArrayBuffer derives Clone, which is now unsound for the External variant: cloning copies the raw pointer, so two ArrayBuffers alias the same region and can each hand out &mut [u8] to it simultaneously. Owned buffers are fine (the Vec is deep-copied), but an external clone breaks the aliasing invariant from_external_data documents.

Would a manual Clone that deep-copies the External variant into an Owned one (or rejects cloning external buffers) work here?

[mansiverma897993]

Good catch, that derive was unsound as written. Fixed with a manual Clone for BufferData that deep-copies the External variant into an Owned one, so a cloned buffer is always independent and can never hand out an aliasing &mut [u8].

(With the rework from jedel1043's review the engine no longer creates &[u8]/&mut [u8] into external regions at all — only atomic accesses — so a bitwise clone would technically no longer be UB, but deep-copying keeps the "clone = independent buffer" semantics and avoids surprising aliasing either way.)

[Shine-neko]

For a SharedArrayBuffer, Atomics and wider views (Int32Array, Float64Array, BigInt64Array) perform aligned atomic accesses. A byte-aligned ptr isn't enough — the base address must satisfy the largest alignment those ops need. Worth asserting/documenting it here, e.g.:

Suggested change

	let ptr = NonNull::new(ptr.cast::<AtomicU8>()).expect("`ptr` must be non-null");
	let ptr = NonNull::new(ptr.cast::<AtomicU8>()).expect("`ptr` must be non-null");
	debug_assert!(ptr.as_ptr() as usize % 8 == 0, "external SharedArrayBuffer backing must be 8-byte aligned");

[mansiverma897993]

Agreed — and it actually applies to non-shared external ArrayBuffers too now, since after the redesign the engine accesses all external regions through atomics (including AtomicU64 element accesses and the batched copies). I went one step further than the debug_assert! and added a hard assert! at construction (it's a one-time cost) requiring 8-byte base alignment for any non-empty external region, in both ArrayBuffer and SharedArrayBuffer constructors, plus a # Panics note in the docs. There's also a #[should_panic] test for the misaligned case.

[Shine-neko]

One more precondition for slice::from_raw_parts: the region size must be <= isize::MAX bytes. It's a documented safety requirement of from_raw_parts, so it'd be worth either asserting it at construction (from_external_data/from_external_ptr) or adding it to the # Safety list the caller must uphold.

[mansiverma897993]

Fixed by construction on the safe path: external regions are now represented as &'static [AtomicU8], and the safe from_external_data constructors take that slice directly, so the len <= isize::MAX bound is guaranteed by the slice type itself. The unsafe from_external_ptr wrappers assert len <= isize::MAX (and non-null) before building the slice, documented under # Panics.

[Shine-neko]

The SAFETY comment covers lifetime/validity, but one case worth spelling out: if the external region can be relocated (e.g. a growable WebAssembly memory that moves its base on memory.grow), the stored ptr silently dangles. The invariant the caller must uphold is stronger than "stays valid" — it's "stays valid and unmoved". Might be worth adding that word here and in the # Safety docs, since the wasm-memory use case is exactly where it bites.

[mansiverma897993]

Good point — the wasm use case is exactly where this bites. The safety docs of both from_external_ptr constructors now require the region to stay "valid and unmoved at the same address", and explicitly call out the growable-WebAssembly-memory-relocating-on-memory.grow case as an example of a silently-dangling pointer.

This particular unsafe impl Send/Sync is actually gone entirely now: with the external region stored as &'static [AtomicU8], SharedData is automatically Send + Sync, so the invariant lives only in the constructor's safety contract where the caller can actually uphold it.

[jedel1043]

Thank you for tackling this problem!

While I do think we should offer an API similar to this for embedders, I'm worried that this might not be the ideal API for it, for a couple of reasons:

• Taking a pointer and a length then casting to slices is basically what &'static mut [u8] already does, so we should try to delegate to that if possible. However, this is mostly minor.

• Embedders passing a buffer to the engine don't have a good way to ensure the buffer is not held by the engine if they want to access the written data, which would cause UB if the buffer is still alive at the point of reading e.g. an async task that suspended while holding an arraybuffer mutable reference.

• This might not be completely thread safe? Its safe if the engine and the embedder run interchangeably in the same thread, but if the embedder is running its main loop in another thread, nothing guarantees that the embedder will read the correct values from the buffer after the engine finishes writing data to it.

[mansiverma897993]

@jedel1043 @Shine-neko Thanks for the detailed feedback! I reworked the design around your three points:

Delegating to slices. External regions are now a real slice &'static [AtomicU8]. The safe primary constructors are ArrayBuffer::from_external_data / SharedArrayBuffer::from_external_data, and the unsafe from_external_ptr versions are thin wrappers that just build the slice and delegate. I went with a shared slice of atomics instead of &'static mut [u8] because a stored &mut is exactly what makes your second point UB, and the slice type also guarantees the from_raw_parts length/validity requirements by construction.

"Buffer still held by the engine" UB. The engine now accesses external regions only through the existing atomic paths (SliceRef::AtomicSlice, the same machinery SharedArrayBuffer uses) and never materializes &[u8]/&mut [u8] into memory it doesn't own. So a suspended async task holding a buffer reference no longer makes embedder reads UB. Additionally, detach() now works on external buffers (returns a copy, drops the engine's reference), giving embedders a hard release before unmapping/freeing the region.

Thread safety. Every engine access to external memory is atomic, so cross-thread access follows the ordinary happens-before rules the same, now documented, contract as SharedArrayBuffer memory. This also let SharedData drop its hand-written Send/Sync impls.

Shine-neko's points (deep-copying Clone, 8-byte alignment assert, isize::MAX, "valid and unmoved" docs) are folded in too, with tests covering all of the above. Happy to iterate if you'd prefer a different shape!