nandi.uk
This repository has no description
1.0 kB
39 lines
1# VRM Lip Sync MVP
2
3Minimal TypeScript web app that loads a VRM avatar and drives its mouth from uploaded audio using the Web Audio API.
4
5The project now includes a default sample avatar at `public/default-avatar.vrm`, so you can run it immediately without finding your own model first.
6
7It also includes a Gemini Live text-to-speech loop: enter a prompt, send it to Gemini, receive native audio, and feed that audio back into the avatar player for lip sync.
8
9## Run
10
111. Install dependencies:
12 bun install
132. Start dev server:
14 bun run dev
153. Open the local Vite URL.
16
17## Gemini setup
18
19Set your Gemini API key before starting Vite:
20
21```bash
22export VITE_GEMINI_API_KEY=your_key_here
23bun run dev
24```
25
26## Usage
27
281. Load a `.vrm` file.
292. Load an audio file.
303. Press Play.
31
32The avatar uses amplitude-based mouth opening, not phoneme-level viseme detection.
33
34## Next upgrades
35
36- Add phoneme/viseme inference instead of pure amplitude
37- Add microphone input mode
38- Add drag-and-drop and avatar presets
39- Add eye blink / idle animation