nandi.uk
This repository has no description
1# Headless VTuber architecture notes
2
3## Recommended first build
4
5Use a split architecture:
6
71. Chat/runtime process on the headless machine
82. Avatar renderer as either:
9 - a headless browser scene with sprite layers and viseme/expression animation, or
10 - a Linux-native VTuber renderer if one exposes remote control hooks
113. ffmpeg composition and push to stream.place via WHIP or RTMP
12
13## Why not start with a full Live2D rig?
14
15Live2D-on-Linux headless is possible but likely slows iteration. First prove:
16
17- incoming chat
18- generated reply
19- synthesized audio
20- deterministic motion/expression events
21- continuous video stream
22
23Then swap the renderer.
24
25## Recommended prototype phases
26
27### Phase 1
28
29- simulate stream.place chat locally
30- generate reply text
31- synthesize or stub audio
32- emit avatar actions like `talk:start`, `emotion:happy`, `head:nod`
33- render a simple animated scene to a test stream or file
34
35### Phase 2
36
37- connect real stream.place chat feed
38- post replies back to chat
39- map reply prosody/emotion to avatar events
40
41### Phase 3
42
43- replace simple avatar scene with a real VTuber rig
44- add interrupt handling, queueing, moderation, and pacing
45
46## Candidate render paths
47
48### Path A: Browser/canvas renderer
49
50Pros:
51
52- easiest to automate from CLI
53- easy websocket control
54- easy capture with Chromium or ffmpeg from a virtual display
55- good enough for a first expressive avatar
56
57Cons:
58
59- not a full Live2D rig
60
61### Path B: OpenVT or custom Godot scene
62
63Pros:
64
65- closer to real VTuber motion
66- Linux-native
67
68Cons:
69
70- more integration work
71- remote-control API may need custom glue
72
73### Path C: OBS scene graph on another machine
74
75Pros:
76
77- simplest if final stream production happens elsewhere
78
79Cons:
80
81- not fully headless/self-contained