Project Gabriel Framework
The open-source framework we used to build Gabriel, our AI in VRChat. It is now maintained as a general-purpose Python framework named Project Gabriel Framework. Primary vision uses Gemini Live, while YOLOv11 is used for following players and fast detection. It supports audio streaming, OSC integration, and small optional memories about people and interactions to enable richer, real-time interactions.
Live Audio Streaming
Bidirectional audio for natural conversation inside VRChat.
Vision and Detection
Primary vision uses Gemini Live. YOLOv11 is used for following players and fast object detection.
VRChat Integration
OSC control for movement, gestures, and chatbox messages.
Setup and Extensibility
Uses a simple configuration file and includes setup scripts to get started quickly.
Overview
Project Gabriel Framework is the open-source Python framework we used to build Gabriel, our live AI assistant for VRChat. It is now maintained as a general-purpose framework providing audio routing, VRChat OSC hooks, vision features, and optional memories that let the AI remember people and interactions for better context.
Key features
- Originally built to power Gabriel, our live AI for VRChat; now maintained as a general-purpose Python framework
- Real-time audio input and output for live voice interaction
- Primary vision via Gemini Live; YOLOv11 used for following players and fast detection
- OSC integration for movement, gestures, and chat messages
- Small, optional memories and prompt pipelines for more consistent, contextual behavior
Quick install
The project includes a setup script that automates environment creation and dependency installation on Windows. For details and advanced options, see the GitHub repository.
setup.batStatus
The repository was archived and set to read-only on January 7, 2026. The code remains a useful reference and starting point for experiments.