nandi.uk / typeface
This repository has no description
0
Fork 0

Configure Feed

Select the types of activity you want to include in your feed.

typeface / README.md
1.0 kB

VRM Lip Sync MVP#

Minimal TypeScript web app that loads a VRM avatar and drives its mouth from uploaded audio using the Web Audio API.

The project now includes a default sample avatar at public/default-avatar.vrm, so you can run it immediately without finding your own model first.

It also includes a Gemini Live text-to-speech loop: enter a prompt, send it to Gemini, receive native audio, and feed that audio back into the avatar player for lip sync.

Run#

  1. Install dependencies: bun install
  2. Start dev server: bun run dev
  3. Open the local Vite URL.

Gemini setup#

Set your Gemini API key before starting Vite:

export VITE_GEMINI_API_KEY=your_key_here
bun run dev

Usage#

  1. Load a .vrm file.
  2. Load an audio file.
  3. Press Play.

The avatar uses amplitude-based mouth opening, not phoneme-level viseme detection.

Next upgrades#

  • Add phoneme/viseme inference instead of pure amplitude
  • Add microphone input mode
  • Add drag-and-drop and avatar presets
  • Add eye blink / idle animation