A robot cactus with personality that integrates Raspberry Pi Pico, React Native, Bluetooth, and OpenAI's Realtime API to create an interactive, voice-controlled robot with a unique personality.
Señor Cactus is a mobile-controlled robot that brings a cactus toy to life with:
- Voice interactions using OpenAI's Realtime API
- Bluetooth control from a mobile app
- Driving motors and interactive features
- A spicy Mexican personality that makes interacting with it fun!
- Natural language voice interactions with OpenAI's Realtime API
- Mexican-accented personality that responds with character
- Voice-commanded movement and control
- Bluetooth connectivity for direct robot control
- Forward, backward, left, and right movement
- Speed control (Low, Medium, High)
- LED and buzzer control
- RGB LED color control
- Toy activation through GPIO
- React Native application for iOS and Android
- Realtime voice interaction interface
- Manual control interface with buttons
- Color picker for RGB LEDs
- Autonomous mode
- Activity logging
- Raspberry Pi Pico (RP2040)
- PicoGo robot platform
- Motors, LEDs, buzzer, and sensors
- Bluetooth module
- React Native with Expo
- Bluetooth Classic integration
- OpenAI Realtime API for voice interactions
- WebRTC for voice streaming
- OpenAI GPT-4o mini with Realtime capabilities
- Function calling for hardware control
- Voice synthesis and recognition
- Node.js and npm
- Expo CLI
- Android Studio or Xcode for native development
- A paired PicoGo robot
- Clone the repository:
git clone https://github.com/KowalewskiPawel/SenorCactus.AI.git
cd SenorCactus.AI- Install dependencies:
npm install- Start the development server:
npx expo start- Run on your device or emulator:
- Press 'a' for Android
- Press 'i' for iOS
- Scan the QR code with Expo Go app
- Raspberry Pi Pico
- PicoGo robot platform
- MicroPython environment
- Connect your Raspberry Pi Pico to your computer
- Flash MicroPython to the Pico
- Copy the following files to the Pico:
main.py- Main control programMotor.py- Motor control libraryws2812.py- RGB LED controlTRSensor.py- Sensor interface
- Turn on the robot
- Pair with the robot from your mobile device's Bluetooth settings
- The robot's Bluetooth name will appear as "PicoGo" or similar
- Open the app and connect to your robot via Bluetooth
- Use the directional buttons to control movement
- Adjust speed using the speed buttons
- Control accessories (LED, buzzer) using the corresponding buttons
- Connect to your robot via Bluetooth
- Tap "Start Voice Assistant" to connect to the OpenAI Realtime API
- Press and hold the microphone button to speak
- Release to get a response from Señor Cactus
- Ask Señor Cactus to move or perform actions:
- "Move forward"
- "Turn left"
- "Change the color to blue"
- "Make some noise"
app/(tabs)/- Main app screensrobot.tsx- Robot control interfacerealtime-cactus.tsx- Voice assistant interface
components/- Reusable UI componentspico-go-code/- Robot firmwaremain.py- Main robot control programMotor.py- Motor control libraryws2812.py- RGB LED control library
You can modify Señor Cactus's personality by updating the system prompt in app/(tabs)/realtime-cactus.tsx:
const systemMessage = {
type: 'session.update',
session: {
instructions: `You are SEÑOR CACTUS, the world's first robotic motivational cactus with a strong Mexican accent...`
// Modify this to change personality
}
};To add new robot functions, update both:
- The function definitions sent to the AI in
sendFunctionDefinitions() - The
handleFunctionCall()implementation to handle the new function
This project is licensed under the MIT License - see the LICENSE file for details.
- OpenAI for the Realtime API
- Waveshare for the PicoGo platform
- React Native and Expo communities
- All contributors to this spicy project!
Created by Pawel Kowalewski
GitHub:
¡Ándale! ¡Vamos a programar un cactus inteligente! 🌵✨