- build interactive user-avatar experiences (speech-based, with video/multimodal support)?
- collect high-quality conversational data for research or industry projects?
- automatically analyze conversations for behavioral, linguistic, or engagement metrics?
| Session overview | Automated analysis summary |
|---|---|
 |
 |
If so, Riverst is for you.
| Video Demo | Description |
|---|---|
| Riverst Demo | Demo of the Riverst platform in action. What can I do with Riverst? |
| KIVA Activity Demo | Demonstration of a vocabulary teaching activity workflow, exploiting pipecat-ai-flows under the hood. |
| Riverst + KIVA Activity Summary | Overview and summary of configuration pages in Riverst and automatically generated summary of the KIVA activity. |
Riverst is a platform for building, running, and analyzing interactive user-avatar conversations. It enables you to:
- Create engaging, speech-driven (and optionally multimodal) avatar interactions.
- Use these interactions for real-time applications, data collection, or research studies.
- Automatically process and analyze collected conversations with built-in pipelines (leveraging senselab) for behavioral and speech analysis.
- User interacts with an avatar (primarily via speech, with optional video/multimodal input).
- Conversations are recorded and stored for later review or analysis.
- Automated pipelines process the data, extracting features and generating insights (e.g., speech metrics, behavioral markers).
- Results can be used for research, product feedback, or to power adaptive experiences.
src/
├── server/ # Bot server implementation (FastAPI)
│ ├── main.py # Server entrypoint
│ └── requirements.txt
└── flow-builder/ # Complex conversational flow/tree builder
└── client/ # Client implementations
└── react/ # React web client
└── index.html # Client main page
-
Supported OS:
- ✅ Apple Silicon (M-chip) macOS
- ✅ Ubuntu Linux
- ❌ Not supported on Windows
-
Python 3.11+
-
Node.js 16+ (for React client)
-
API keys for 3rd party services (see .env.example files in both client and server)
-
Modern web browser with WebRTC support (e.g., Chrome 134+)
git clone https://github.com/sensein/riverst.git
cd riverst- In
src/server/, rename.env.exampleto.envand fill in the required API keys and configuration - In
src/client/react/, rename.env.exampleto.envand fill in the required API keys and configuration
More detailed instructions for setting up the client and server can be found in their respective README documents.
Note: Not all API KEYS are strictly required. Only if you want to use a remote service, you need to expose the corresponding API KEY
Note: If you want to run local models, you will need to have Ollama or another method for running models locally. Please see #5 under Getting Started for the server setup.
docker compose up --build-
Start the server:
cd src/server conda create -n riverst python=3.11 conda activate riverst conda install -c conda-forge "ffmpeg=7.*" pip install -r requirements.txt python main.py
-
Start the client:
cd src/client/react npm install npm run dev
ℹ️ Note 2: For AWS EC2 deployment instructions, see here.
It's possible some issues might have arisen along the way. Feel free to post an issue asking for help as well as check out our Common Pitfalls Guide for some issues that we have seen pop up.
Riverst builds on the work of these fantastic open-source projects:
- TalkingHead — WebGL/Three.js talking-head renderer for avatar animation.
- Contextless Phonemes (CUPE) — Efficient phoneme modeling utilities.
- Pipecat — Real-time, multimodal agent framework for low-latency streaming.
- senselab — Python package for speech processing, feature extraction, and behavioral analysis.
The project is in continuous progress. Follow the project plan here.