nandi.uk / typeface
This repository has no description
0
Fork 0

Configure Feed

Select the types of activity you want to include in your feed.

TypeScript 88.9%
CSS 9.7%
HTML 1.3%
2 1 0

Clone this repository

https://git.vm.fail/nandi.uk/typeface https://git.vm.fail/did:plc:mh52gbmx2dt2ljqnwo5phgtx
ssh://git@knot1.tangled.sh:2222/nandi.uk/typeface ssh://git@knot1.tangled.sh:2222/did:plc:mh52gbmx2dt2ljqnwo5phgtx

For self-hosted knots, clone URLs may differ based on your setup.


Download tar.gz Download .zip
README.md

VRM Lip Sync MVP#

Minimal TypeScript web app that loads a VRM avatar and drives its mouth from uploaded audio using the Web Audio API.

The project now includes a default sample avatar at public/default-avatar.vrm, so you can run it immediately without finding your own model first.

It also includes a Gemini Live text-to-speech loop: enter a prompt, send it to Gemini, receive native audio, and feed that audio back into the avatar player for lip sync.

Run#

  1. Install dependencies: bun install
  2. Start dev server: bun run dev
  3. Open the local Vite URL.

Gemini setup#

Set your Gemini API key before starting Vite:

export VITE_GEMINI_API_KEY=your_key_here
bun run dev

Usage#

  1. Load a .vrm file.
  2. Load an audio file.
  3. Press Play.

The avatar uses amplitude-based mouth opening, not phoneme-level viseme detection.

Next upgrades#

  • Add phoneme/viseme inference instead of pure amplitude
  • Add microphone input mode
  • Add drag-and-drop and avatar presets
  • Add eye blink / idle animation