Option 2: Docker (Isolated Environment)

#🐇🔌 USBunny — AI On A Stick

"Raw. Local. Unfiltered."

Note

USBunny is your locally-sourced, farm-to-USB AI experience. 🥕 Plug in your USB 3.0+ drive, right-click start-(your_OS), and hop to it with "Run as Administrator." Choose your model and enjoy private, uncensored AI... on a stick. Bonus: Windows admin repair tool included for when your warren needs maintenance. ✨ Why USBunny?

🔒 Your Data, Your Rules — Models and conversations stay on your drive. No cloud harvesting, ever.

🚫 Truly Uncensored — No corporate filters. Pure, unfiltered compute for any narrative.

🖥️ Works Everywhere — Windows, Mac, Linux. High-end rig or modest laptop—it adapts.##

🎉 Enjoy the Ride — Life's short. Have fun, don't take it all so seriously.

🚀 Quick Start (USB Method)

The easiest way to get started:

Download the USBunny zip to a USB 3.0+ drive (16GB+ free space)

Open the drive folder

***Right-click start-(`your_OS`) and Run as Administrator***

USBunny will automatically detect your hardware and select the best model for your machine.

Important

Linux users: Run the setup script instead:

 chmod +x setup_ollama.sh && ./setup_ollama.sh

🖥️ Local Installation (Manual Fallback)

If the automated USB script encounters an error on your system, you can set things up manually using one of the methods below. Option 1: Native Ollama Install (Easiest)

Windows/macOS/Linux:

curl -fsSL https://ollama.com/install.sh | sh

(Note: The install script starts Ollama automatically in the background. If you need to run it manually, use ollama serve)

Then pull and run a model:

# For a balanced, uncensored experience
ollama run everythinglm

# Or try these popular uncensored models
ollama pull UncensoredAi/inkling
ollama run UncensoredAi/inkling

# Swap in any model you like
ollama run llama3
ollama run mistral
ollama run phi3

Option 2: Docker (Isolated Environment)

Perfect for keeping your system clean:

# Install Docker (Linux example)
sudo apt update && sudo apt install -y docker.io
sudo systemctl enable --now docker
sudo usermod -aG docker $USER && newgrp docker

# Run Ollama container
docker run -d --gpus all --name usbunny-ollama -v ollama_data:/root/.ollama -p 11434:11434 ollama/ollama

Flags Explained:

--gpus all — Enable GPU acceleration (Requires NVIDIA Container Toolkit installed on host)

--name usbunny-ollama — Names the container for easy management

-v ollama_data:/root/.ollama — Persist models on your machine

To use models in the container: Bash

docker exec -it usbunny-ollama ollama run llama3

Option 3: llama.cpp (Advanced Users)

For full control over models and quantization:

# Install dependencies
sudo apt update && sudo apt install -y git cmake build-essential

# Clone and build
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
make                  # Add GGML_CUDA=1 for NVIDIA GPU acceleration

# Download a modern GGUF model and run it
wget https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGUF/resolve/main/llama-2-7b-chat.Q4_K_M.gguf -O model.gguf
./llama-cli -m model.gguf -n 512 --repeat_penalty 1.1 -p "Your prompt here"

No GPU? Check out BitNet.cpp for CPU-optimized inference.

📊 Model Selection Guide

USBunny automatically selects the best model for your hardware. Here's what you can run: RAM Max Parameters (4-bit) Recommended Models 0–4 GB < 1B Phi-2, Qwen-0.5B, TinyLlama 8 GB 3B Llama 3.2 3B, Phi-3.5 Mini 16 GB 8B Llama 3.1 8B, Mistral 7B 24 GB 14B Qwen 2.5 14B, Mistral NeMo 12B 32 GB 27B–32B Gemma 2 27B, Qwen 2.5 32B 48 GB 40B–50B Mixtral 8x7B, Command R 35B 64 GB+ 70B+ Llama 3.3 70B, Qwen 2.5 72B 🛠️ Windows Power Users

We've included a comprehensive PowerShell diagnosis and repair script for Windows users. It's been thoroughly tested and works like a charm.

Important

Right-click and Run as Administrator for full functionality.

🔧 Troubleshooting Common Issues & Fixes

💥 Out of Memory? Use a smaller model, or add swap space:

Linux:
```Bash

sudo fallocate -l 8G /swapfile && sudo chmod 600 /swapfile
sudo mkswap /swapfile && sudo swapon /swapfile


    Windows: Create a page file in System Properties > Advanced > Performance Settings > Advanced > Virtual Memory

    macOS: Use Disk Utility to create a swap partition

🖥️ GPU Not Detected?
Install the latest drivers for your GPU:

    Linux:
    ```Bash

    sudo ubuntu-drivers autoinstall && sudo reboot

Windows: Download from NVIDIA or AMD

macOS: Use Apple's built-in Metal support (M1/M2/M3 chips work natively)

💬 Community & Support

Your conversation stays on your machine. But if you need help:

Check the GitHub Issues

Join the discussion in GitHub Discussions

huggingface.co/TheBloke/Llama-2-7B-Chat-GGUF/resolve/main/llama-2-7b-chat.Q4_K_M.gguf) -O model.gguf ./llama-cli -m model.gguf -n 512 --repeat_penalty 1.1 -p "Your prompt here"

No GPU? Check out BitNet.cpp for CPU-optimized inference.