AddressSanitizer: CHECK failed: `((ptr[0] == kCurrentStackFrameMagic)) != (0)`

[mschwager]

When fuzzing bson-ruby I'm repeatedly getting this error.

sigaltstack:

AddressSanitizer: CHECK failed: asan_thread.cpp:383 "((ptr[0] == kCurrentStackFrameMagic)) != (0)" (0x0, 0x0) (tid=40)
    #0 0xffff82d21d60 in __asan::CheckUnwind() /home/tcwg-buildslave/workspace/tcwg-llvm-release/tcwg-jade-03/final/llvm-project/compiler-rt/lib/asan/asan_rtl.cpp:69:3
    #1 0xffff82d3a658 in __sanitizer::CheckFailed(char const*, int, char const*, unsigned long long, unsigned long long) /home/tcwg-buildslave/workspace/tcwg-llvm-release/tcwg-jade-03/final/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_termination.cpp:86:5
    #2 0xffff82d252b0 in __asan::AsanThread::GetStackFrameAccessByAddr(unsigned long, __asan::AsanThread::StackFrameAccess*) /home/tcwg-buildslave/workspace/tcwg-llvm-release/tcwg-jade-03/final/llvm-project/compiler-rt/lib/asan/asan_thread.cpp
    #3 0xffff82c81e84 in GetStackAddressInformation /home/tcwg-buildslave/workspace/tcwg-llvm-release/tcwg-jade-03/final/llvm-project/compiler-rt/lib/asan/asan_descriptions.cpp:202:11
    #4 0xffff82c81e84 in __asan::AddressDescription::AddressDescription(unsigned long, unsigned long, bool) /home/tcwg-buildslave/workspace/tcwg-llvm-release/tcwg-jade-03/final/llvm-project/compiler-rt/lib/asan/asan_descriptions.cpp:455:21
    #5 0xffff82c83898 in __asan::ErrorGeneric::ErrorGeneric(unsigned int, unsigned long, unsigned long, unsigned long, unsigned long, bool, unsigned long) /home/tcwg-buildslave/workspace/tcwg-llvm-release/tcwg-jade-03/final/llvm-project/compiler-rt/lib/asan/asan_errors.cpp:408:7
    #6 0xffff82d1f91c in __asan::ReportGenericError(unsigned long, unsigned long, unsigned long, unsigned long, bool, unsigned long, unsigned int, bool) /home/tcwg-buildslave/workspace/tcwg-llvm-release/tcwg-jade-03/final/llvm-project/compiler-rt/lib/asan/asan_report.cpp:494:16
    #7 0xffff82cee3cc in sigaltstack /home/tcwg-buildslave/workspace/tcwg-llvm-release/tcwg-jade-03/final/llvm-project/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:10100:5
    #8 0xffff82d1bbc8 in __asan::PlatformUnpoisonStacks() /home/tcwg-buildslave/workspace/tcwg-llvm-release/tcwg-jade-03/final/llvm-project/compiler-rt/lib/asan/asan_posix.cpp:45:3
    #9 0xffff82d21a6c in __asan_handle_no_return /home/tcwg-buildslave/workspace/tcwg-llvm-release/tcwg-jade-03/final/llvm-project/compiler-rt/lib/asan/asan_rtl.cpp:589:8
    #10 0xffff7bfba5a4 in pvt_validate_length /var/lib/gems/3.1.0/gems/bson-4.15.0/ext/bson/read.c:56:7
...

memcpy:

AddressSanitizer: CHECK failed: asan_thread.cpp:383 "((ptr[0] == kCurrentStackFrameMagic)) != (0)" (0x0, 0x0) (tid=70)
    #0 0xffff88b71d60 in __asan::CheckUnwind() /home/tcwg-buildslave/workspace/tcwg-llvm-release/tcwg-jade-03/final/llvm-project/compiler-rt/lib/asan/asan_rtl.cpp:69:3
    #1 0xffff88b8a658 in __sanitizer::CheckFailed(char const*, int, char const*, unsigned long long, unsigned long long) /home/tcwg-buildslave/workspace/tcwg-llvm-release/tcwg-jade-03/final/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_termination.cpp:86:5
    #2 0xffff88b752b0 in __asan::AsanThread::GetStackFrameAccessByAddr(unsigned long, __asan::AsanThread::StackFrameAccess*) /home/tcwg-buildslave/workspace/tcwg-llvm-release/tcwg-jade-03/final/llvm-project/compiler-rt/lib/asan/asan_thread.cpp
    #3 0xffff88ad1e84 in GetStackAddressInformation /home/tcwg-buildslave/workspace/tcwg-llvm-release/tcwg-jade-03/final/llvm-project/compiler-rt/lib/asan/asan_descriptions.cpp:202:11
    #4 0xffff88ad1e84 in __asan::AddressDescription::AddressDescription(unsigned long, unsigned long, bool) /home/tcwg-buildslave/workspace/tcwg-llvm-release/tcwg-jade-03/final/llvm-project/compiler-rt/lib/asan/asan_descriptions.cpp:455:21
    #5 0xffff88ad3898 in __asan::ErrorGeneric::ErrorGeneric(unsigned int, unsigned long, unsigned long, unsigned long, unsigned long, bool, unsigned long) /home/tcwg-buildslave/workspace/tcwg-llvm-release/tcwg-jade-03/final/llvm-project/compiler-rt/lib/asan/asan_errors.cpp:408:7
    #6 0xffff88b6f91c in __asan::ReportGenericError(unsigned long, unsigned long, unsigned long, unsigned long, bool, unsigned long, unsigned int, bool) /home/tcwg-buildslave/workspace/tcwg-llvm-release/tcwg-jade-03/final/llvm-project/compiler-rt/lib/asan/asan_report.cpp:494:16
    #7 0xffff88b66e78 in memcpy /home/tcwg-buildslave/workspace/tcwg-llvm-release/tcwg-jade-03/final/llvm-project/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors_memintrinsics.inc:115:5
    #8 0xffff8882b7f4  (/lib/aarch64-linux-gnu/libruby-3.1.so.3.1+0x1db7f4) (BuildId: fd962b9494525f203fb3a8ad66cf5eab02b640d2)
    #9 0xffff8882d1cc in rb_usascii_str_new (/lib/aarch64-linux-gnu/libruby-3.1.so.3.1+0x1dd1cc) (BuildId: fd962b9494525f203fb3a8ad66cf5eab02b640d2)
...

clock_gettime:

AddressSanitizer: CHECK failed: asan_thread.cpp:383 "((ptr[0] == kCurrentStackFrameMagic)) != (0)" (0x0, 0x0) (tid=92)
    #0 0xffff9d041d60 in __asan::CheckUnwind() /home/tcwg-buildslave/workspace/tcwg-llvm-release/tcwg-jade-03/final/llvm-project/compiler-rt/lib/asan/asan_rtl.cpp:69:3
    #1 0xffff9d05a658 in __sanitizer::CheckFailed(char const*, int, char const*, unsigned long long, unsigned long long) /home/tcwg-buildslave/workspace/tcwg-llvm-release/tcwg-jade-03/final/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_termination.cpp:86:5
    #2 0xffff9d0452b0 in __asan::AsanThread::GetStackFrameAccessByAddr(unsigned long, __asan::AsanThread::StackFrameAccess*) /home/tcwg-buildslave/workspace/tcwg-llvm-release/tcwg-jade-03/final/llvm-project/compiler-rt/lib/asan/asan_thread.cpp
    #3 0xffff9cfa1e84 in GetStackAddressInformation /home/tcwg-buildslave/workspace/tcwg-llvm-release/tcwg-jade-03/final/llvm-project/compiler-rt/lib/asan/asan_descriptions.cpp:202:11
    #4 0xffff9cfa1e84 in __asan::AddressDescription::AddressDescription(unsigned long, unsigned long, bool) /home/tcwg-buildslave/workspace/tcwg-llvm-release/tcwg-jade-03/final/llvm-project/compiler-rt/lib/asan/asan_descriptions.cpp:455:21
    #5 0xffff9cfa3898 in __asan::ErrorGeneric::ErrorGeneric(unsigned int, unsigned long, unsigned long, unsigned long, unsigned long, bool, unsigned long) /home/tcwg-buildslave/workspace/tcwg-llvm-release/tcwg-jade-03/final/llvm-project/compiler-rt/lib/asan/asan_errors.cpp:408:7
    #6 0xffff9d03f91c in __asan::ReportGenericError(unsigned long, unsigned long, unsigned long, unsigned long, bool, unsigned long, unsigned int, bool) /home/tcwg-buildslave/workspace/tcwg-llvm-release/tcwg-jade-03/final/llvm-project/compiler-rt/lib/asan/asan_report.cpp:494:16
    #7 0xffff9cfc36b0 in clock_gettime /home/tcwg-buildslave/workspace/tcwg-llvm-release/tcwg-jade-03/final/llvm-project/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:2175:5
    #8 0xffff9cbe1188  (/lib/aarch64-linux-gnu/libruby-3.1.so.3.1+0xc1188) (BuildId: fd962b9494525f203fb3a8ad66cf5eab02b640d2)
    #9 0xffff9cbef514  (/lib/aarch64-linux-gnu/libruby-3.1.so.3.1+0xcf514) (BuildId: fd962b9494525f203fb3a8ad66cf5eab02b640d2)
    #10 0xffff9cbf2f48  (/lib/aarch64-linux-gnu/libruby-3.1.so.3.1+0xd2f48) (BuildId: fd962b9494525f203fb3a8ad66cf5eab02b640d2)
    #11 0xffff9cbf37d4  (/lib/aarch64-linux-gnu/libruby-3.1.so.3.1+0xd37d4) (BuildId: fd962b9494525f203fb3a8ad66cf5eab02b640d2)
    #12 0xffff9cbf3988 in rb_wb_protected_newobj_of (/lib/aarch64-linux-gnu/libruby-3.1.so.3.1+0xd3988) (BuildId: fd962b9494525f203fb3a8ad66cf5eab02b640d2)
...

It looks like I'm not the only one:

• ASan CHECK failed "((ptr[0] == kCurrentStackFrameMagic)) != (0)" google/sanitizers#1010
• AddressSanitizer CHECK failed google/sanitizers#1171
• Unexpected failure when using -O1: CHECK failed: asan_thread.cpp:371 "((ptr[0] == kCurrentStackFrameMagic)) != (0)" google/sanitizers#1531

[mschwager]

I noticed a few things. Ruby does in fact use sigaltstack.

And when running with ASAN, they do recommend setting the following:

https://github.com/ruby/ruby/blob/v3_3_0/doc/contributing/building_ruby.md#building-with-address-sanitizer

However, when I set those compiler flags and ASAN options the issue still persists. On one hand, this does indeed seem like some weird behavior with sigaltstack, however I'm seeing the same type of crash with memcpy and clock_gettime, so I'm not so sure.

[mschwager]

Some additional interesting links:

• https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101476
• https://bugs.ruby-lang.org/issues/11514#note-8
• https://news.ycombinator.com/item?id=25181648

[mschwager]

I forgot to fill out the repro steps from the original crash. Here are the details...

fuzz_bson.rb:

# frozen_string_literal: true

require 'ruzzy'
require 'bson'

test_one_input = lambda do |data|
  begin
    Hash.from_bson(BSON::ByteBuffer.new(data))
  rescue Exception
    # We're looking for memory corruption, not Ruby exceptions
  end
  return 0
end

Ruzzy.fuzz(test_one_input)

If I compile with -fsanitize=address,fuzzer-no-link like the following, then I get the check failure above:

MAKE="make --environment-overrides V=1" \
CC="clang" \
CXX="clang++" \
LDSHARED="clang -shared" \
LDSHAREDXX="clang++ -shared" \
CFLAGS="-fsanitize=address,fuzzer-no-link -fno-omit-frame-pointer -fno-common -fPIC -g" \
CXXFLAGS="-fsanitize=address,fuzzer-no-link -fno-omit-frame-pointer -fno-common -fPIC -g" \
    gem install --verbose bson

However, if compile with just -fsanitize=fuzzer-no-link, then fuzzing proceeds as expected:

MAKE="make --environment-overrides V=1" \
CC="clang" \
CXX="clang++" \
LDSHARED="clang -shared" \
LDSHAREDXX="clang++ -shared" \
CFLAGS="-fsanitize=fuzzer-no-link -fno-omit-frame-pointer -fno-common -fPIC -g" \
CXXFLAGS="-fsanitize=fuzzer-no-link -fno-omit-frame-pointer -fno-common -fPIC -g" \
    gem install --verbose bson

Run like so:

LD_PRELOAD=$(ruby -e 'require "ruzzy"; print Ruzzy::ASAN_PATH') \
    ruby fuzz_bson.rb