Braintrust Support Chat

A React-based chat application that simulates Braintrust support, with full instrumentation to log all interactions to Braintrust for observability and evaluation.

🚀 Quick Start

# Install dependencies
npm install

# Set up environment variables
cp .env.example .env
# Edit .env and add your API keys

# Start the development server
npm start

Visit http://localhost:3000 to start chatting!

✨ Features

• 🤖 AI-Powered Support: Chat interface powered by OpenAI GPT-4, configured to act as Braintrust support
• 📊 Braintrust Integration: All responses are automatically logged to Braintrust using their OpenAI SDK wrapper
• 🧪 Test Sessions: Run predefined test chat sessions to evaluate and log interactions
• 🎨 Modern UI: Beautiful, responsive chat interface with smooth animations
• 📝 Full Tracing: Complete observability of all API calls and responses
• 🔗 MCP Server Support: Configured to work with Braintrust's MCP server for enhanced integration

Prerequisites

• Node.js (v16 or higher)
• npm or yarn
• OpenAI API key
• Braintrust API key

📦 Installation

Step 1: Install Dependencies

npm install

Step 2: Configure Environment Variables

xCreate a.env` file in the root directory:

cp .env.example .env

Edit .env and add your API keys:

REACT_APP_OPENAI_API_KEY=sk-your-openai-api-key-here
REACT_APP_BRAINTRUST_API_KEY=your-braintrust-api-key-here
REACT_APP_BRAINTRUST_PROJECT=braintrust-support-chat

Important: The .env file is gitignored for security. Never commit your API keys!

🔑 Getting API Keys

OpenAI API Key

1. Visit OpenAI Platform
2. Sign up or log in to your account
3. Navigate to API Keys in your account settings
4. Click Create new secret key
5. Copy the key (it starts with sk-) and add it to your .env file

Note: You'll need a paid OpenAI account with access to GPT-4. Free tier accounts may not work.

Braintrust API Key

1. Visit Braintrust
2. Sign up or log in to your account
3. Navigate to Settings → API Keys
4. Click Create API Key
5. Copy the key and add it to your .env file

Note: If you don't have a Braintrust account, you can sign up for free at braintrust.dev

💻 Usage

Starting the Application

npm start

The app will automatically open in your browser at http://localhost:3000. If it doesn't, navigate to that URL manually.

Using the Chat Interface

Interactive Chat

1. Type your message in the input field at the bottom
2. Press Enter or click the Send button
3. Wait for the AI response (you'll see a typing indicator)
4. Continue the conversation naturally

All messages are automatically logged to Braintrust in real-time!

Test Sessions

The app includes three predefined test sessions in the right panel:

1. Basic Questions - General questions about Braintrust

   • "What is Braintrust?"
   • "How do I get started with Braintrust?"
   • "What are the main features?"

2. Technical Questions - Technical questions about logging and features

   • "How do I log traces in Braintrust?"
   • "What is the difference between experiments and datasets?"
   • "How do I use the OpenAI SDK wrapper?"

3. Integration Questions - Questions about integrating Braintrust

   • "How do I integrate Braintrust with my React app?"
   • "What is the MCP server?"
   • "How do I set up authentication?"

To run a test session:

1. Click "Run Test Session" on any test card
2. Watch as the session executes automatically
3. View the results in the test results panel
4. All interactions are logged to Braintrust for analysis

Clearing the Chat

Click the "Clear Chat" button to start a new conversation.

How It Works

Braintrust Integration

The app uses Braintrust's wrapOpenAI function to automatically instrument all OpenAI API calls:

import { wrapOpenAI } from 'braintrust';
import OpenAI from 'openai';

const openai = wrapOpenAI(
  new OpenAI({
    apiKey: process.env.REACT_APP_OPENAI_API_KEY,
  }),
  {
    project: 'braintrust-support-chat',
  }
);

This wrapper automatically:

• Logs all API requests and responses
• Tracks token usage
• Records metadata and timing information
• Creates traces for debugging and evaluation

MCP Server Reference

This app is designed to work with Braintrust's MCP (Model Context Protocol) server. The MCP server provides:

• Access to Braintrust experiments and datasets
• Real-time logging and monitoring
• Integration with development tools

For more information about Braintrust's MCP server, see the Braintrust MCP Documentation.

📁 Project Structure

eval_test/
├── public/
│   └── index.html              # HTML template
├── src/
│   ├── components/
│   │   ├── ChatApp.js          # Main chat component with UI
│   │   └── ChatApp.css         # Chat component styles
│   ├── services/
│   │   └── braintrustService.js # Braintrust integration & OpenAI wrapper
│   ├── App.js                  # Root React component
│   ├── App.css                 # App-level styles
│   ├── index.js                # React entry point
│   └── index.css               # Global CSS styles
├── .cursor/
│   └── mcp.json                # MCP server configuration reference
├── .env.example                # Environment variables template
├── .gitignore                  # Git ignore rules
├── package.json                # Dependencies and npm scripts
└── README.md                   # This file

Key Files

• src/components/ChatApp.js: Main chat interface component with test session functionality
• src/services/braintrustService.js: Core service that wraps OpenAI SDK with Braintrust logging
• .cursor/mcp.json: MCP server configuration for Braintrust integration

📊 Viewing Logs in Braintrust

All chat interactions are automatically logged to your Braintrust project. Here's how to view them:

1. Open Braintrust Dashboard

   • Go to https://www.braintrust.dev/
   • Log in to your account

2. Navigate to Your Project

   • Select the project: braintrust-support-chat (or your custom project name)
   • You'll see all logged interactions

3. Explore the Data

   • Logs Tab: See all chat interactions with timestamps
   • Traces Tab: View detailed request/response traces with full context
   • Metrics Tab: Analyze token usage, latency, and performance metrics
   • Experiments Tab: Compare different test sessions

4. Filter and Search

   • Filter by test session name
   • Search for specific messages or responses
   • View metadata like message count, model used, etc.

Example: Finding Test Session Logs

1. In Braintrust dashboard, go to your project
2. Use the filter/search to find logs with metadata containing testSession
3. View the complete conversation flow for each test session

🛠️ Customization

Adding Test Sessions

Edit src/components/ChatApp.js and add to the TEST_SESSIONS array:

const TEST_SESSIONS = [
  {
    name: 'Your Custom Test',
    messages: [
      'First test message',
      'Second test message',
      'Third test message',
    ],
  },
  // ... existing test sessions
];

Changing the System Prompt

Edit src/services/braintrustService.js and modify the SYSTEM_PROMPT constant:

const SYSTEM_PROMPT = `Your custom system prompt here...`;

This changes how the AI behaves and responds to users.

Changing the OpenAI Model

Edit src/services/braintrustService.js and change the model in the sendMessage method:

const response = await this.openai.chat.completions.create({
  model: 'gpt-3.5-turbo', // Change from 'gpt-4' to any supported model
  messages: fullMessages,
  temperature: 0.7,
});

Supported models: gpt-4, gpt-4-turbo, gpt-3.5-turbo, etc.

Adjusting Temperature

Change the temperature parameter to control response randomness:

temperature: 0.7, // Lower = more deterministic, Higher = more creative

Customizing the UI

• Colors: Edit src/components/ChatApp.css to change the color scheme
• Layout: Modify component structure in src/components/ChatApp.js
• Styling: Update CSS classes in ChatApp.css

🔧 Troubleshooting

API Key Issues

Problem: App shows errors about missing API keys

Solutions:

• ✅ Ensure your .env file is in the root directory (same level as package.json)
• ✅ Restart the development server after changing .env (stop with Ctrl+C and run npm start again)
• ✅ Verify API keys are correct - no extra spaces or quotes
• ✅ Check that keys start with correct prefixes:
  • OpenAI: sk-...
  • Braintrust: Usually a long alphanumeric string

Braintrust Logging Not Working

Problem: Interactions aren't appearing in Braintrust dashboard

Solutions:

• ✅ Check that your Braintrust API key is valid and active
• ✅ Verify the project name in .env matches an existing project in Braintrust
• ✅ Check the browser console (F12) for error messages
• ✅ Ensure you have internet connectivity
• ✅ Try creating a new project in Braintrust and updating the project name

OpenAI API Errors

Problem: Chat responses fail or show errors

Solutions:

• ✅ Verify your OpenAI API key is valid
• ✅ Check your OpenAI account has sufficient credits/billing set up
• ✅ Ensure you have access to GPT-4 model (some accounts only have GPT-3.5)
• ✅ Check OpenAI status page: status.openai.com
• ✅ Try switching to gpt-3.5-turbo if GPT-4 isn't available

Build Errors

Problem: npm start fails with errors

Solutions:

• ✅ Delete node_modules and package-lock.json, then run npm install again
• ✅ Ensure you're using Node.js v16 or higher: node --version
• ✅ Clear npm cache: npm cache clean --force

Port Already in Use

Problem: Port 3000 is already in use

Solutions:

• ✅ Kill the process using port 3000
• ✅ Or set a different port: PORT=3001 npm start

👨‍💻 Development

Available Scripts

Command	Description
npm start	Start development server on http://localhost:3000
npm build	Create optimized production build
npm test	Run test suite (if configured)
npm eject	Eject from Create React App (irreversible)

Building for Production

npm run build

This creates an optimized production build in the build directory. The build is minified and ready for deployment.

Deploy to:

• Vercel: vercel --prod
• Netlify: Drag and drop the build folder
• GitHub Pages: Use gh-pages package
• Any static host: Upload the build folder contents

Development Tips

• Hot Reload: Changes to code automatically refresh the browser
• Console Logging: Check browser console (F12) for debug information
• React DevTools: Install React Developer Tools browser extension
• Environment Variables: Remember to restart the dev server after changing .env

📚 Additional Resources

Documentation

• Braintrust Documentation - Complete Braintrust platform docs
• Braintrust MCP Server - MCP server integration guide
• OpenAI API Documentation - OpenAI API reference
• React Documentation - React framework docs

Related Links

• Braintrust GitHub - Braintrust SDK source code
• OpenAI Platform - OpenAI dashboard and API management
• Create React App Docs - CRA documentation

Getting Help

• Braintrust Support: support@braintrust.dev
• Braintrust Discord: Join their community Discord
• OpenAI Support: help.openai.com

📄 License

This project is provided as-is for demonstration and educational purposes.

🙏 Acknowledgments

• Built with React
• Powered by OpenAI
• Instrumented with Braintrust

---

Made with ❤️ for AI observability and evaluation