Bug Report: Mutex Lock Failed During Indexing on Intel Mac

[rothnic]

Bug Report: Mutex Lock Failed During Indexing on Intel Mac

System Information

• OS: macOS Darwin 25.2.0 (macOS Sequoia 15.2)
• Architecture: x86_64 (Intel Mac)
• Machine: 2019 MacBook Pro, Intel Core i9
• Node.js: v24.12.0
• npm/npx: 11.6.2
• codebase-context version: 1.8.2

What we tried

1. Command: npx -y codebase-context metadata
2. Command: npx -y codebase-context status
3. Both resulted in: Tool started indexing but crashed during embedding phase

Error Details

Indexing (full): /Users/nroth/workspace/resume-chatbot
[scanning] 0%
Found 238 files to index
[analyzing] 0%
[git-dates] Loaded commit dates for 269 files
[analyzing] 10%
[analyzing] 20%
...
[analyzing] 100%
[embedding] 50%
Creating embeddings for 2162 chunks...
libc++abi: terminating due to uncaught exception of type std::__1::system_error: mutex lock failed: Invalid argument

Project Details

• Repository: TypeScript/Next.js project (resume-chatbot)
• Files: 238 files
• Chunks: 2162 chunks to embed
• Crashed at: ~50% through embedding phase

Questions

1. Expected setup: Are there specific system requirements or native dependencies needed for Intel Macs?
2. @huggingface/transformers compatibility: Could this be related to the ONNX/Transformers.js native bindings on Intel architecture?
3. Resource limits: Is there a known memory/thread limit that might cause mutex failures?
4. Workaround: Is there a way to skip embeddings or use a different provider to test if it's embedding-specific?

Additional Context

• We checked existing issues and found Cannot seem to start the MCP server in multiple tools #2 (MCP initialization timeout) and Single MCP server instance should work across multiple projects #63 (multi-project routing) but nothing matching this specific mutex error
• The crash happens consistently at the same point during indexing
• No other Node.js or native module issues on this system

Thank you for the excellent tool concept - the pattern detection and memory features are exactly what we need for our team!

[rothnic]

Additional Analysis: Embedding Provider Issue

We investigated using an external embedding provider (Ollama) as a workaround, but discovered the tool doesn't currently support custom API endpoints via environment variables.

Code Analysis

Looking at the embedding provider code in dist/embeddings/:

1. OpenAI Provider (openai.js) accepts an apiEndpoint parameter:

constructor(modelName = 'text-embedding-3-small', apiKey, apiEndpoint = 'https://api.openai.com/v1')

2. Environment Variables (types.js) only support:

   • EMBEDDING_PROVIDER (transformers|openai)
   • EMBEDDING_MODEL
   • OPENAI_API_KEY

3. Missing: No OPENAI_BASE_URL or similar for custom endpoints

Workaround Attempted

We tried using Ollama (OpenAI-compatible API at http://100.79.168.98:11434/v1):

EMBEDDING_PROVIDER=openai \
OPENAI_BASE_URL=http://100.79.168.98:11434/v1 \
OPENAI_API_KEY=ollama \
npx codebase-context metadata

But the apiEndpoint parameter isn't read from environment variables, so it still tries to use https://api.openai.com/v1.

Potential Fix

The fix would be in dist/embeddings/types.js:

export const DEFAULT_EMBEDDING_CONFIG = {
    provider: parseEmbeddingProviderName(process.env.EMBEDDING_PROVIDER) ?? 'transformers',
    model: DEFAULT_MODEL,
    batchSize: 32,
    maxRetries: 3,
    apiKey: process.env.OPENAI_API_KEY,
    apiEndpoint: process.env.OPENAI_BASE_URL || 'https://api.openai.com/v1'  // Add this
};

Alternative: Native Ollama Support

We also noticed there's a placeholder for Ollama:

if (mergedConfig.provider === 'ollama') {
    console.warn('Ollama provider not yet implemented, falling back to Transformers.js');
}

Native Ollama support would be ideal since it would allow using local embeddings without the mutex/threading issues on Intel Macs.

Summary

The mutex crash appears to be in the @huggingface/transformers library's native bindings on Intel Macs. Workarounds:

1. Short-term: Add OPENAI_BASE_URL env var support to use external providers
2. Medium-term: Implement native Ollama provider
3. Long-term: Fix the underlying threading issue with Transformers.js on Intel Mac

Happy to test any patches or provide more system info!

[rothnic]

Created separate issue #70 to track the custom OpenAI-compatible API endpoint support (OPENAI_BASE_URL). This will allow using Ollama, LiteLLM, and other OpenAI-compatible providers instead of the official OpenAI API.\n\nThe mutex lock issue (#68) and the base URL support (#70) are separate concerns, so tracking them independently.

[PatrickSys]

Hey @rothnic, thanks for the detailed analysis; you basically solved it.

The root cause is onnxruntime-node@1.21.0, which has a well-documented C++ mutex
destruction order bug on macOS tracked in microsoft/onnxruntime#24579, fixed in 1.24.x. @huggingface/transformers@3.8.1 (current stable) still pins 1.21.0 exactly.

To answer your first post questions directly:

1. No special system requirements needed for Intel Macs - the issue isn't the setup here.
2. Yes, exactly this. Intel vs Apple Silicon doesn't matter - both will have the same bug.
3. Isn't a memory or thread limit. The crash is in C++ static destructor ordering at
   shutdown, not resource exhaustion. The 50% mark is coincidental.
4. Yes: once the fix for Feature Request: Support custom OpenAI-compatible API endpoints (OPENAI_BASE_URL) #70 lands, EMBEDDING_PROVIDER=openai + OPENAI_BASE_URL=http://your-ollama:11434/v1 bypasses Transformers.js entirely and sidesteps the crash.

On your three workarounds:

• Short-term (OPENAI_BASE_URL) is covered by Feature Request: Support custom OpenAI-compatible API endpoints (OPENAI_BASE_URL) #70, which you already opened and have a branch for.
• Medium-term (native Ollama provider): I've opened feat: implement native Ollama embedding provider #71 to track this properly.
• Long-term (fix the threading issue): @huggingface/transformers v4 pre-release already brings onnxruntime-node@1.24.3 which fixes this. I've opened chore: upgrade @huggingface/transformers to v4 #72 to track the upgrade once v4 is stable.