solpbc.org / solstone
personal memory agent
0
Fork 0

Configure Feed

Select the types of activity you want to include in your feed.

feat: place Parakeet STT on CPU when GPU cannot co-locate

On Linux hosts where the bundled brain is resident on a single discrete GPU that cannot also hold a worst-case ordinary-segment Parakeet run, transcribe.parakeet-cpp.device: auto now resolves to CPU at parakeet-server launch.

That decision uses measured floor-tier brain residency 4541 MiB + Parakeet worst case 5022 MiB + 1024 MiB margin = 10587 MiB. As a result, 6/8/10 GiB cards place STT on CPU, while 11/12 GiB and up keep it on the GPU.

Every other state fails toward today's behavior: unknown VRAM, probe failure, unmeasured capable-tier residency, 2+ discrete GPUs, integrated or unified memory, or an inactive bundled-brain lane keep auto resolving exactly as before.

The placement is announced through the supervisor reason line, GPU-check detail in sol check and the local provider fit report, and journal/health/parakeet-cpp.placement so transcribed events and transcript headers report the real device placement, cpu/gpu, rather than literal auto. The checks remain ok because this is a hardware-fit fact, not a failure.

No config keys were added; transcribe.parakeet-cpp.device still validates as auto | cpu only.

author
Jer Miller date (Jul 12, 2026, 2:27 PM -0600) commit 3c2d42e7 parent c389c1b9
+1077 -25
+1
AGENTS.md
··· 200 200 | Schedules (`config/schedules.json`) | `solstone/think/schedule_config.py` | 201 201 | Push devices (`config/push_devices.json`) | `solstone/think/push/devices.py` | 202 202 | Local inference operational telemetry (`health/local-inference/YYYYMMDD.jsonl`) | `solstone/think/providers/local_admission.py` | 203 + | Parakeet server placement record (`health/parakeet-cpp.placement`) | `solstone/think/providers/parakeet_server.py` | 203 204 | Hosted backup binding (`backup/hosted/binding.json`) | `solstone/think/backup/hosted.py` | 204 205 | Convey config (`config/convey.json`) | `solstone/convey/config.py` + `solstone/think/facets.py` | 205 206 | Chat config (`config/chat.json`) | `solstone/apps/chat/config.py` |
+2 -1
solstone/observe/transcribe/_parakeet_cpp.py
··· 219 219 """Return parakeet.cpp model metadata for transcript JSONL headers.""" 220 220 _require_linux() 221 221 device = _validate_config(config) 222 + placement = parakeet_server.read_parakeet_placement() 222 223 return { 223 224 "model": parakeet_readiness.PARAKEET_CPP_MODEL_FILENAME, 224 - "device": device, 225 + "device": placement or device, 225 226 "compute_type": _COMPUTE_TYPE, 226 227 "per_word_confidence": True, 227 228 }
+4 -3
solstone/observe/transcribe/failure-and-telemetry.md
··· 101 101 | `reason` | machine reason (table above) | deferred, failed | 102 102 | `error` | exception **type name** — never the message (see below) | failed | 103 103 | `backend` | STT backend name (`parakeet-cpp`, `gemini`, …) | whenever resolved | 104 - | `device` | resolved device (`auto` / `cpu`) | whenever known (see below) | 104 + | `device` | resolved placement (`cpu` / `gpu`) when a placement record exists; configured device otherwise | whenever known (see below) | 105 105 | `model` | model filename | success, and failures after the backend reported it | 106 106 | `audio_seconds` | original decoded length, 1 dp | whenever decoded | 107 107 | `reduced_seconds` | length after silence-trimming, 1 dp | when reduction ran | ··· 171 171 - **`model` on deferred events.** `get_model_info()` is cheap for the parakeet-cpp and 172 172 cloud backends, but on Apple Silicon it shells out to the CoreML helper (`--version`, 173 173 10 s timeout). Rather than hoist a subprocess probe onto a path whose whole point is 174 - *not* to do expensive work, deferred events omit `model`. `device` is still reported 175 - when the config names one. 174 + *not* to do expensive work, deferred events omit `model`. `device` reports the 175 + supervisor placement for parakeet-cpp when that record exists, and otherwise falls 176 + back to the configured value when the config names one. 176 177 177 178 ## Rollback 178 179
+1
solstone/think/backup/engine.py
··· 59 59 ".tmp*", 60 60 "supervisor.ready", 61 61 "supervisor.start_time", 62 + "parakeet-cpp.placement", 62 63 "scheduler.json", 63 64 "talents.json", 64 65 "agents.json",
+34 -3
solstone/think/check.py
··· 178 178 def _linux_gpu_check(probe: object) -> FitCheck: 179 179 try: 180 180 from solstone.think.providers import local_cuda, local_vulkan, memory 181 + from solstone.think.providers.parakeet_placement import cpu_placement_suffix 181 182 182 - if not bool(getattr(probe, "detected")): 183 + try: 183 184 devices = local_vulkan.detect_gpus() 184 185 probe_ok = local_vulkan.gpu_probe_ok() 185 - selected = local_vulkan.select_device(devices) if probe_ok else None 186 + except Exception: 187 + if not bool(getattr(probe, "detected")): 188 + raise 189 + # Fail toward current behavior: Vulkan failure on NVIDIA yields no placement line. 190 + devices = [] 191 + probe_ok = False 192 + selected = local_vulkan.select_device(devices) if probe_ok else None 193 + if not bool(getattr(probe, "detected")): 186 194 inaccessible = ( 187 195 not probe_ok or selected is None 188 196 ) and _render_nodes_present_but_inaccessible() ··· 223 231 return FitCheck( 224 232 "gpu", 225 233 "ok", 226 - f"Vulkan GPU {selected.name} with {memory.gb_label(vram_bytes)} GB", 234 + ( 235 + f"Vulkan GPU {selected.name} with {memory.gb_label(vram_bytes)} GB" 236 + + cpu_placement_suffix( 237 + devices=devices, 238 + selected=selected, 239 + local_vulkan=local_vulkan, 240 + unified_memory=False, 241 + # sol check runs before install and cannot rely on journal 242 + # config; use the bundled-brain default for this advisory. 243 + brain_lane_active=True, 244 + ) 245 + ), 227 246 required_bytes=GPU_MIN_BYTES, 228 247 available_bytes=vram_bytes, 229 248 ) ··· 257 276 detail = f"NVIDIA GPU with {memory.gb_label(gpu_bytes)} GB" 258 277 if getattr(probe, "memory_source") == local_cuda.MEMORY_SOURCE_SYSTEM_AVAILABLE: 259 278 detail = f"{detail} (unified memory)" 279 + detail += cpu_placement_suffix( 280 + devices=devices, 281 + selected=selected, 282 + local_vulkan=local_vulkan, 283 + unified_memory=( 284 + getattr(probe, "memory_source") 285 + == local_cuda.MEMORY_SOURCE_SYSTEM_AVAILABLE 286 + ), 287 + # sol check runs before install and cannot rely on journal config; use 288 + # the bundled-brain default for this advisory. 289 + brain_lane_active=True, 290 + ) 260 291 return FitCheck( 261 292 "gpu", 262 293 "ok",
+35 -3
solstone/think/providers/fit_report.py
··· 74 74 75 75 probe = None 76 76 choice = None 77 + devices: list[Any] = [] 78 + brain_lane_active = True 77 79 if sys.platform.startswith("linux"): 78 80 probe = local_cuda.probe_nvidia_gpu() 79 81 choice = local_cuda.select_local_backend( ··· 86 88 if choice.backend == "cuda" 87 89 else "llama-server tarball" 88 90 ) 91 + devices = local_vulkan.detect_gpus() 92 + try: 93 + from solstone.think.models import is_local_provider_needed 94 + from solstone.think.providers.local_endpoint import resolve_local_endpoint 95 + 96 + brain_lane_active = ( 97 + is_local_provider_needed() and resolve_local_endpoint().is_bundled 98 + ) 99 + except Exception: 100 + brain_lane_active = True 89 101 else: 90 102 unknown_server = "llama-server tarball" 91 103 ··· 103 115 104 116 if sys.platform.startswith("linux") and probe is not None and choice is not None: 105 117 checks.append( 106 - _local_gpu_check(probe, choice, local_vulkan.detect_gpus(), local_vulkan) 118 + _local_gpu_check( 119 + probe, 120 + choice, 121 + devices, 122 + local_vulkan, 123 + brain_lane_active=brain_lane_active, 124 + ) 107 125 ) 108 126 109 127 return FitReport(artifact="local provider artifacts", checks=tuple(checks)) ··· 388 406 choice: Any, 389 407 devices: list[Any], 390 408 local_vulkan: Any, 409 + *, 410 + brain_lane_active: bool, 391 411 ) -> FitCheck: 392 412 from solstone.think.providers import local_cuda 413 + from solstone.think.providers.parakeet_placement import cpu_placement_suffix 393 414 394 415 backend = getattr(choice, "backend") 395 416 reason = getattr(choice, "reason") ··· 401 422 ) 402 423 403 424 memory_source = getattr(probe, "memory_source") 425 + selected = local_vulkan.select_device(devices) 426 + placement_suffix = cpu_placement_suffix( 427 + devices=devices, 428 + selected=selected, 429 + local_vulkan=local_vulkan, 430 + unified_memory=memory_source == local_cuda.MEMORY_SOURCE_SYSTEM_AVAILABLE, 431 + brain_lane_active=brain_lane_active, 432 + ) 404 433 if memory_source == local_cuda.MEMORY_SOURCE_UNAVAILABLE: 405 434 return FitCheck( 406 435 "gpu", ··· 412 441 detail = f"CUDA backend selected: {reason}" 413 442 if memory_source == local_cuda.MEMORY_SOURCE_SYSTEM_AVAILABLE: 414 443 detail = f"{detail}; GPU tiering memory uses system MemAvailable" 444 + detail = f"{detail}{placement_suffix}" 415 445 return FitCheck("gpu", "ok", detail) 416 446 417 447 probe_ok = local_vulkan.gpu_probe_ok() ··· 421 451 "unknown", 422 452 f"Vulkan GPU probe did not complete; resolved backend is {backend}: {reason}", 423 453 ) 424 - selected = local_vulkan.select_device(devices) 425 454 if selected is None: 426 455 return FitCheck( 427 456 "gpu", ··· 431 460 return FitCheck( 432 461 "gpu", 433 462 "ok", 434 - f"Vulkan GPU selected: {selected.name}; resolved backend is {backend}: {reason}", 463 + ( 464 + f"Vulkan GPU selected: {selected.name}; resolved backend is {backend}: " 465 + f"{reason}{placement_suffix}" 466 + ), 435 467 ) 436 468 437 469
+10 -1
solstone/think/providers/local_server.py
··· 38 38 context_tokens: int 39 39 parallel_slots: int 40 40 prompt_cache_mib: int 41 + resident_mib: int | None 41 42 42 43 43 44 # Tunable estimates — keep all tier values in these two instances; do not 44 45 # scatter literals elsewhere. The threshold is the only other tunable. 45 46 _CAPABLE_TIER_MIN_VRAM_MIB = 16000 47 + # ``resident_mib`` is measured floor-tier brain residency under production 48 + # launch args. ``None`` on capable is load-bearing: unmeasured residency means 49 + # the co-location placement predicate can never fire at >=16 GiB. 46 50 _CAPABLE_TIER = ServerTier( 47 - name="capable", context_tokens=32768, parallel_slots=2, prompt_cache_mib=2048 51 + name="capable", 52 + context_tokens=32768, 53 + parallel_slots=2, 54 + prompt_cache_mib=2048, 55 + resident_mib=None, 48 56 ) 49 57 _FLOOR_TIER = ServerTier( 50 58 name="floor", 51 59 context_tokens=LOCAL_MIN_CONTEXT_TOKENS, 52 60 parallel_slots=1, 53 61 prompt_cache_mib=0, 62 + resident_mib=4541, 54 63 ) 55 64 56 65 # COPY REVIEW: placeholder owner-facing copy; founder-gated before ship.
+199
solstone/think/providers/parakeet_placement.py
··· 1 + # SPDX-License-Identifier: AGPL-3.0-only 2 + # Copyright (c) 2026 sol pbc 3 + 4 + """Pure placement decision for supervised parakeet.cpp co-location.""" 5 + 6 + from __future__ import annotations 7 + 8 + from collections.abc import Sequence 9 + from dataclasses import dataclass 10 + from typing import Any 11 + 12 + from solstone.think.providers.local_server import select_server_tier 13 + 14 + # Measured worst-case ordinary segment residency: 300 s is the observer-contract 15 + # cap. The 1024 MiB margin covers display framebuffer, compositor allocations, 16 + # driver overhead, and allocator fragmentation on the same monitor-driving GPU. 17 + # It intentionally uses 1024, not 512, to put 10 GiB cards (10240 MiB) on the 18 + # CPU side of the 10587 MiB threshold: without margin, only 677 MiB would remain 19 + # after the two measured residents, which is not reliable display-attached 20 + # operating slack. 21 + PARAKEET_WORST_CASE_MIB = 5022 22 + CO_FIT_MARGIN_MIB = 1024 23 + 24 + CPU_PLACEMENT_COPY = ( 25 + "sol thinks on your GPU; transcription runs on your CPU on this machine" 26 + ) 27 + _DISCRETE_CLASSIFICATION = "discrete" 28 + 29 + 30 + @dataclass(frozen=True) 31 + class ParakeetPlacementDecision: 32 + force_cpu: bool 33 + reason_code: str 34 + tier_name: str | None 35 + tier_resident_mib: int | None 36 + parakeet_worst_case_mib: int 37 + margin_mib: int 38 + required_mib: int | None 39 + vram_mib: int | None 40 + 41 + 42 + def _decision( 43 + *, 44 + force_cpu: bool, 45 + reason_code: str, 46 + tier_name: str | None, 47 + tier_resident_mib: int | None, 48 + required_mib: int | None, 49 + vram_mib: int | None, 50 + ) -> ParakeetPlacementDecision: 51 + return ParakeetPlacementDecision( 52 + force_cpu=force_cpu, 53 + reason_code=reason_code, 54 + tier_name=tier_name, 55 + tier_resident_mib=tier_resident_mib, 56 + parakeet_worst_case_mib=PARAKEET_WORST_CASE_MIB, 57 + margin_mib=CO_FIT_MARGIN_MIB, 58 + required_mib=required_mib, 59 + vram_mib=vram_mib, 60 + ) 61 + 62 + 63 + def is_discrete(device: Any, local_vulkan: Any) -> bool: 64 + """Return whether a pre-enumerated Vulkan device is classified discrete.""" 65 + return local_vulkan.classify(device) == _DISCRETE_CLASSIFICATION 66 + 67 + 68 + def discrete_hardware_gpu_count( 69 + devices: Sequence[Any], 70 + local_vulkan: Any, 71 + ) -> int: 72 + """Count hardware GPUs classified discrete from an existing Vulkan enumeration.""" 73 + return sum( 74 + 1 75 + for device in devices 76 + if local_vulkan.is_hardware_device(device) and is_discrete(device, local_vulkan) 77 + ) 78 + 79 + 80 + def cpu_placement_suffix( 81 + *, 82 + devices: Sequence[Any], 83 + selected: Any | None, 84 + local_vulkan: Any, 85 + unified_memory: bool, 86 + brain_lane_active: bool, 87 + ) -> str: 88 + """Return the advisory suffix when auto-placement resolves STT to CPU.""" 89 + if selected is None: 90 + return "" 91 + decision = decide_parakeet_auto_placement( 92 + vram_mib=getattr(selected, "vram_mib", None), 93 + selected_device_is_discrete=is_discrete(selected, local_vulkan), 94 + discrete_hardware_gpu_count=discrete_hardware_gpu_count(devices, local_vulkan), 95 + unified_memory=unified_memory, 96 + brain_lane_active=brain_lane_active, 97 + ) 98 + return f"; {CPU_PLACEMENT_COPY}" if decision.force_cpu else "" 99 + 100 + 101 + def decide_parakeet_auto_placement( 102 + vram_mib: int | None, 103 + selected_device_is_discrete: bool, 104 + discrete_hardware_gpu_count: int, 105 + unified_memory: bool, 106 + brain_lane_active: bool, 107 + ) -> ParakeetPlacementDecision: 108 + """Return whether parakeet.cpp auto-placement must use CPU. 109 + 110 + This is intentionally pure: callers provide probe/config facts, and this 111 + function performs only tier selection and arithmetic. 112 + """ 113 + if not brain_lane_active: 114 + return _decision( 115 + force_cpu=False, 116 + reason_code="brain_lane_inactive", 117 + tier_name=None, 118 + tier_resident_mib=None, 119 + required_mib=None, 120 + vram_mib=vram_mib, 121 + ) 122 + if not selected_device_is_discrete: 123 + return _decision( 124 + force_cpu=False, 125 + reason_code="selected_device_not_discrete", 126 + tier_name=None, 127 + tier_resident_mib=None, 128 + required_mib=None, 129 + vram_mib=vram_mib, 130 + ) 131 + if discrete_hardware_gpu_count != 1: 132 + return _decision( 133 + force_cpu=False, 134 + reason_code="discrete_gpu_count_not_one", 135 + tier_name=None, 136 + tier_resident_mib=None, 137 + required_mib=None, 138 + vram_mib=vram_mib, 139 + ) 140 + if unified_memory: 141 + return _decision( 142 + force_cpu=False, 143 + reason_code="unified_memory", 144 + tier_name=None, 145 + tier_resident_mib=None, 146 + required_mib=None, 147 + vram_mib=vram_mib, 148 + ) 149 + if vram_mib is None: 150 + return _decision( 151 + force_cpu=False, 152 + reason_code="vram_unknown", 153 + tier_name=None, 154 + tier_resident_mib=None, 155 + required_mib=None, 156 + vram_mib=None, 157 + ) 158 + 159 + tier = select_server_tier(vram_mib) 160 + if tier.resident_mib is None: 161 + return _decision( 162 + force_cpu=False, 163 + reason_code="tier_residency_unmeasured", 164 + tier_name=tier.name, 165 + tier_resident_mib=None, 166 + required_mib=None, 167 + vram_mib=vram_mib, 168 + ) 169 + 170 + required_mib = tier.resident_mib + PARAKEET_WORST_CASE_MIB + CO_FIT_MARGIN_MIB 171 + if vram_mib < required_mib: 172 + return _decision( 173 + force_cpu=True, 174 + reason_code="co_location_requires_cpu", 175 + tier_name=tier.name, 176 + tier_resident_mib=tier.resident_mib, 177 + required_mib=required_mib, 178 + vram_mib=vram_mib, 179 + ) 180 + return _decision( 181 + force_cpu=False, 182 + reason_code="co_location_fits_gpu", 183 + tier_name=tier.name, 184 + tier_resident_mib=tier.resident_mib, 185 + required_mib=required_mib, 186 + vram_mib=vram_mib, 187 + ) 188 + 189 + 190 + __all__ = [ 191 + "CO_FIT_MARGIN_MIB", 192 + "CPU_PLACEMENT_COPY", 193 + "PARAKEET_WORST_CASE_MIB", 194 + "ParakeetPlacementDecision", 195 + "cpu_placement_suffix", 196 + "decide_parakeet_auto_placement", 197 + "discrete_hardware_gpu_count", 198 + "is_discrete", 199 + ]
+34 -1
solstone/think/providers/parakeet_server.py
··· 6 6 from __future__ import annotations 7 7 8 8 from dataclasses import dataclass 9 + from pathlib import Path 9 10 10 11 from solstone.think import parakeet_readiness 11 12 from solstone.think.providers.parakeet_install import ParakeetProviderError 12 - from solstone.think.utils import read_service_port 13 + from solstone.think.utils import get_journal, read_service_port 13 14 14 15 STATE_READY = "ready" 15 16 STATE_FAILED = "failed" 16 17 17 18 _HOST = "127.0.0.1" 18 19 _SERVICE_NAME = "parakeet-cpp" 20 + _PLACEMENT_FILE = "parakeet-cpp.placement" 21 + _VALID_PLACEMENTS = {"cpu", "gpu"} 19 22 20 23 21 24 class ParakeetServerNotReady(ParakeetProviderError): ··· 43 46 return f"http://{_HOST}:{port}" 44 47 45 48 49 + def _placement_path() -> Path: 50 + return Path(get_journal()) / "health" / _PLACEMENT_FILE 51 + 52 + 53 + def write_parakeet_placement(device: str) -> None: 54 + """Persist the resolved parakeet.cpp serving placement for telemetry.""" 55 + if device not in _VALID_PLACEMENTS: 56 + raise ValueError(f"invalid parakeet placement: {device!r}") 57 + path = _placement_path() 58 + path.parent.mkdir(parents=True, exist_ok=True) 59 + path.write_text(device) 60 + 61 + 62 + def read_parakeet_placement() -> str | None: 63 + """Read the resolved parakeet.cpp serving placement, if valid.""" 64 + try: 65 + device = _placement_path().read_text().strip() 66 + except FileNotFoundError: 67 + return None 68 + return device if device in _VALID_PLACEMENTS else None 69 + 70 + 71 + def clear_parakeet_placement() -> None: 72 + """Remove any stale parakeet.cpp serving placement record.""" 73 + _placement_path().unlink(missing_ok=True) 74 + 75 + 46 76 def _probe_health(port: int, timeout_s: float = 1.0) -> tuple[str, str | None]: 47 77 import httpx 48 78 ··· 88 118 "STATE_READY", 89 119 "ParakeetServerInfo", 90 120 "ParakeetServerNotReady", 121 + "clear_parakeet_placement", 91 122 "connect", 92 123 "probe_state", 124 + "read_parakeet_placement", 125 + "write_parakeet_placement", 93 126 ]
+51 -1
solstone/think/supervisor.py
··· 64 64 evaluate_drain_gate, 65 65 load_processing_settings, 66 66 ) 67 + from solstone.think.providers import parakeet_server 67 68 from solstone.think.providers.memory import read_available_bytes 68 69 from solstone.think.providers.mlx_server import MLX_SERVER_PROCESS_NAME 69 70 from solstone.think.readiness import START_TIME_TOLERANCE_S, clear_ready, signal_ready ··· 2420 2421 2421 2422 def start_parakeet_server() -> RunnerManagedProcess | None: 2422 2423 """Launch the supervisor-owned parakeet-server when STT opts into it.""" 2424 + parakeet_server.clear_parakeet_placement() 2423 2425 if not linux_stt_uses_parakeet_cpp(): 2424 2426 return None 2425 2427 2426 2428 from solstone.think.providers import local_vulkan, parakeet_install 2429 + from solstone.think.providers.parakeet_placement import ( 2430 + decide_parakeet_auto_placement, 2431 + discrete_hardware_gpu_count, 2432 + is_discrete, 2433 + ) 2427 2434 2428 2435 config_device = _configured_parakeet_device() 2436 + effective_device = config_device 2429 2437 selected = None 2430 2438 if config_device == "auto": 2431 2439 devices = local_vulkan.detect_gpus() ··· 2440 2448 else "none" 2441 2449 ), 2442 2450 ) 2451 + selected_is_discrete = selected is not None and is_discrete( 2452 + selected, local_vulkan 2453 + ) 2454 + if selected is not None and selected_is_discrete: 2455 + from solstone.think.providers import local_cuda 2456 + from solstone.think.providers.local_endpoint import resolve_local_endpoint 2443 2457 2444 - plan = resolve_parakeet_server_launch_plan(config_device, selected) 2458 + discrete_count = discrete_hardware_gpu_count(devices, local_vulkan) 2459 + probe = local_cuda.probe_nvidia_gpu() 2460 + decision = decide_parakeet_auto_placement( 2461 + vram_mib=selected.vram_mib, 2462 + selected_device_is_discrete=selected_is_discrete, 2463 + discrete_hardware_gpu_count=discrete_count, 2464 + unified_memory=( 2465 + probe.memory_source == local_cuda.MEMORY_SOURCE_SYSTEM_AVAILABLE 2466 + ), 2467 + brain_lane_active=( 2468 + is_local_provider_needed() and resolve_local_endpoint().is_bundled 2469 + ), 2470 + ) 2471 + if decision.force_cpu: 2472 + logging.info( 2473 + "parakeet-server auto placement resolved to CPU: " 2474 + "tier=%s tier_resident_mib=%s " 2475 + "parakeet_worst_case_mib=%d margin_mib=%d " 2476 + "required_mib=%s gpu_vram_mib=%s placement=cpu", 2477 + decision.tier_name, 2478 + decision.tier_resident_mib, 2479 + decision.parakeet_worst_case_mib, 2480 + decision.margin_mib, 2481 + decision.required_mib, 2482 + decision.vram_mib, 2483 + ) 2484 + effective_device = "cpu" 2485 + 2486 + plan = resolve_parakeet_server_launch_plan(effective_device, selected) 2445 2487 try: 2446 2488 binary_path, gguf_path = parakeet_install.ensure_artifacts_installed( 2447 2489 plan.binary_backend ··· 2465 2507 env, 2466 2508 ) 2467 2509 if status == "ready": 2510 + parakeet_server.write_parakeet_placement( 2511 + "gpu" if plan.binary_backend == "vulkan" else "cpu" 2512 + ) 2468 2513 return managed 2469 2514 2470 2515 if plan.binary_backend == "vulkan" and status in {"crashed", "timeout"}: ··· 2499 2544 "continuing startup", 2500 2545 PARAKEET_SERVER_READY_TIMEOUT_S, 2501 2546 ) 2547 + parakeet_server.write_parakeet_placement("cpu") 2502 2548 return cpu_managed 2503 2549 2504 2550 if plan.binary_backend == "cpu": ··· 2514 2560 "continuing startup", 2515 2561 PARAKEET_SERVER_READY_TIMEOUT_S, 2516 2562 ) 2563 + parakeet_server.write_parakeet_placement("cpu") 2517 2564 return managed 2518 2565 2566 + parakeet_server.write_parakeet_placement( 2567 + "gpu" if plan.binary_backend == "vulkan" else "cpu" 2568 + ) 2519 2569 return managed 2520 2570 2521 2571
+2
tests/test_backup_engine.py
··· 245 245 "--exclude", 246 246 "supervisor.start_time", 247 247 "--exclude", 248 + "parakeet-cpp.placement", 249 + "--exclude", 248 250 "scheduler.json", 249 251 "--exclude", 250 252 "talents.json",
+100 -1
tests/test_check.py
··· 12 12 from solstone.think.providers import local_cuda, local_vulkan, memory 13 13 14 14 GB = 1024**3 15 + PLACEMENT_LINE = ( 16 + "sol thinks on your GPU; transcription runs on your CPU on this machine" 17 + ) 18 + 19 + 20 + @pytest.fixture(autouse=True) 21 + def _reset_vulkan_detect_cache(): 22 + local_vulkan.reset_detect_cache() 23 + yield 24 + local_vulkan.reset_detect_cache() 15 25 16 26 17 27 def _patch_platform( ··· 50 60 _patch_platform(monkeypatch) 51 61 _patch_memory(monkeypatch) 52 62 _patch_disk(monkeypatch) 63 + monkeypatch.setattr( 64 + local_vulkan, "detect_gpus", lambda: [_vulkan_device(vram_mib=24576)] 65 + ) 66 + monkeypatch.setattr(local_vulkan, "gpu_probe_ok", lambda: True) 53 67 54 68 55 69 def _nvidia_probe( ··· 90 104 *, 91 105 index: int = 0, 92 106 name: str = "Vulkan GPU", 107 + device_type: int = local_vulkan.VK_TYPE_DISCRETE, 93 108 vram_mib: int = 8192, 94 109 ) -> local_vulkan.VulkanDevice: 95 110 return local_vulkan.VulkanDevice( 96 111 index=index, 97 112 name=name, 98 - device_type=local_vulkan.VK_TYPE_DISCRETE, 113 + device_type=device_type, 99 114 vram_mib=vram_mib, 100 115 ) 101 116 ··· 168 183 output = capsys.readouterr().out 169 184 assert "solstone-journal" in output 170 185 assert "solstone-journal-cuda" not in output 186 + 187 + 188 + def test_linux_nvidia_small_single_discrete_mentions_cpu_transcription( 189 + monkeypatch: pytest.MonkeyPatch, 190 + ) -> None: 191 + _patch_linux_ok(monkeypatch) 192 + monkeypatch.setattr( 193 + local_cuda, 194 + "probe_nvidia_gpu", 195 + lambda: _nvidia_probe(vram_mib=6144), 196 + ) 197 + monkeypatch.setattr( 198 + local_vulkan, "detect_gpus", lambda: [_vulkan_device(vram_mib=6144)] 199 + ) 200 + 201 + result = check.build_check_report() 202 + 203 + gpu = _checks(result)["gpu"] 204 + assert gpu.severity == "ok" 205 + assert gpu.detail == f"NVIDIA GPU with 6 GB; {PLACEMENT_LINE}" 206 + 207 + 208 + @pytest.mark.parametrize( 209 + ("probe", "devices", "probe_ok"), 210 + [ 211 + (_nvidia_probe(vram_mib=12288), [_vulkan_device(vram_mib=12288)], True), 212 + ( 213 + _nvidia_probe(vram_mib=6144), 214 + [ 215 + _vulkan_device(index=0, vram_mib=6144), 216 + _vulkan_device(index=1, vram_mib=6144), 217 + ], 218 + True, 219 + ), 220 + ( 221 + _nvidia_probe( 222 + vram_mib=6144, 223 + memory_source=local_cuda.MEMORY_SOURCE_SYSTEM_AVAILABLE, 224 + ), 225 + [_vulkan_device(vram_mib=6144)], 226 + True, 227 + ), 228 + (_nvidia_probe(vram_mib=16384), [_vulkan_device(vram_mib=16384)], True), 229 + (_nvidia_probe(vram_mib=6144), [], False), 230 + ], 231 + ) 232 + def test_linux_nvidia_cpu_transcription_line_absent_outside_predicate( 233 + monkeypatch: pytest.MonkeyPatch, 234 + probe: local_cuda.NvidiaProbe, 235 + devices: list[local_vulkan.VulkanDevice], 236 + probe_ok: bool, 237 + ) -> None: 238 + _patch_linux_ok(monkeypatch) 239 + monkeypatch.setattr(local_cuda, "probe_nvidia_gpu", lambda: probe) 240 + monkeypatch.setattr(local_vulkan, "detect_gpus", lambda: devices) 241 + monkeypatch.setattr(local_vulkan, "gpu_probe_ok", lambda: probe_ok) 242 + 243 + result = check.build_check_report() 244 + 245 + gpu = _checks(result)["gpu"] 246 + assert gpu.severity == "ok" 247 + assert PLACEMENT_LINE not in gpu.detail 248 + 249 + 250 + def test_linux_nvidia_vulkan_detection_exception_keeps_current_detail( 251 + monkeypatch: pytest.MonkeyPatch, 252 + ) -> None: 253 + _patch_linux_ok(monkeypatch) 254 + monkeypatch.setattr( 255 + local_cuda, 256 + "probe_nvidia_gpu", 257 + lambda: _nvidia_probe(vram_mib=6144), 258 + ) 259 + monkeypatch.setattr( 260 + local_vulkan, 261 + "detect_gpus", 262 + lambda: (_ for _ in ()).throw(RuntimeError("vulkan failed")), 263 + ) 264 + 265 + result = check.build_check_report() 266 + 267 + gpu = _checks(result)["gpu"] 268 + assert gpu.severity == "ok" 269 + assert gpu.detail == "NVIDIA GPU with 6 GB" 171 270 172 271 173 272 def test_linux_vulkan_ok(
+98 -1
tests/test_fit_report.py
··· 7 7 8 8 import pytest 9 9 10 - from solstone.think.providers import fit_report, local_install 10 + from solstone.think.providers import fit_report, local_cuda, local_install, local_vulkan 11 11 from solstone.think.providers.local import LocalProviderError 12 12 from solstone.think.providers.memory import MemoryVerdict 13 + 14 + PLACEMENT_LINE = ( 15 + "sol thinks on your GPU; transcription runs on your CPU on this machine" 16 + ) 17 + 18 + 19 + @pytest.fixture(autouse=True) 20 + def _reset_vulkan_detect_cache(): 21 + local_vulkan.reset_detect_cache() 22 + yield 23 + local_vulkan.reset_detect_cache() 24 + 25 + 26 + def _nvidia_probe( 27 + *, 28 + vram_mib: int, 29 + memory_source: str = local_cuda.MEMORY_SOURCE_NVIDIA_VRAM, 30 + ) -> local_cuda.NvidiaProbe: 31 + return local_cuda.NvidiaProbe( 32 + index=0, 33 + compute_cap="sm_89", 34 + driver_cuda_version=13, 35 + vram_mib=vram_mib, 36 + tiering_memory_mib=vram_mib, 37 + memory_source=memory_source, 38 + detected=True, 39 + ) 40 + 41 + 42 + def _vulkan_device( 43 + *, 44 + index: int = 0, 45 + vram_mib: int, 46 + ) -> local_vulkan.VulkanDevice: 47 + return local_vulkan.VulkanDevice( 48 + index=index, 49 + name=f"Test GPU {index}", 50 + device_type=local_vulkan.VK_TYPE_DISCRETE, 51 + vram_mib=vram_mib, 52 + ) 53 + 54 + 55 + def _choice(backend: str = "cuda") -> local_cuda.BackendChoice: 56 + return local_cuda.BackendChoice(backend=backend, reason="test choice") 13 57 14 58 15 59 def test_overall_collapses_unknown_to_warning() -> None: ··· 35 79 ) 36 80 37 81 assert report.overall == "blocked" 82 + 83 + 84 + def test_local_gpu_check_mentions_cpu_transcription_on_small_bundled_brain() -> None: 85 + check = fit_report._local_gpu_check( 86 + _nvidia_probe(vram_mib=6144), 87 + _choice(), 88 + [_vulkan_device(vram_mib=6144)], 89 + local_vulkan, 90 + brain_lane_active=True, 91 + ) 92 + 93 + assert check.severity == "ok" 94 + assert check.detail == f"CUDA backend selected: test choice; {PLACEMENT_LINE}" 95 + 96 + 97 + @pytest.mark.parametrize( 98 + ("probe", "devices", "brain_lane_active"), 99 + [ 100 + (_nvidia_probe(vram_mib=6144), [_vulkan_device(vram_mib=6144)], False), 101 + ( 102 + _nvidia_probe(vram_mib=6144), 103 + [ 104 + _vulkan_device(index=0, vram_mib=6144), 105 + _vulkan_device(index=1, vram_mib=6144), 106 + ], 107 + True, 108 + ), 109 + ( 110 + _nvidia_probe( 111 + vram_mib=6144, 112 + memory_source=local_cuda.MEMORY_SOURCE_SYSTEM_AVAILABLE, 113 + ), 114 + [_vulkan_device(vram_mib=6144)], 115 + True, 116 + ), 117 + (_nvidia_probe(vram_mib=16384), [_vulkan_device(vram_mib=16384)], True), 118 + ], 119 + ) 120 + def test_local_gpu_check_omits_cpu_transcription_line_outside_predicate( 121 + probe: local_cuda.NvidiaProbe, 122 + devices: list[local_vulkan.VulkanDevice], 123 + brain_lane_active: bool, 124 + ) -> None: 125 + check = fit_report._local_gpu_check( 126 + probe, 127 + _choice(), 128 + devices, 129 + local_vulkan, 130 + brain_lane_active=brain_lane_active, 131 + ) 132 + 133 + assert check.severity == "ok" 134 + assert PLACEMENT_LINE not in check.detail 38 135 39 136 40 137 def test_disk_unknown_size_warns_when_known_size_fits(
+6
tests/test_local.py
··· 2116 2116 context_tokens=16384, 2117 2117 parallel_slots=1, 2118 2118 prompt_cache_mib=0, 2119 + resident_mib=4541, 2119 2120 ), 2120 2121 ), 2121 2122 ( ··· 2125 2126 context_tokens=16384, 2126 2127 parallel_slots=1, 2127 2128 prompt_cache_mib=0, 2129 + resident_mib=4541, 2128 2130 ), 2129 2131 ), 2130 2132 ( ··· 2134 2136 context_tokens=32768, 2135 2137 parallel_slots=2, 2136 2138 prompt_cache_mib=2048, 2139 + resident_mib=None, 2137 2140 ), 2138 2141 ), 2139 2142 ( ··· 2143 2146 context_tokens=32768, 2144 2147 parallel_slots=2, 2145 2148 prompt_cache_mib=2048, 2149 + resident_mib=None, 2146 2150 ), 2147 2151 ), 2148 2152 ] ··· 2152 2156 assert tier == expected 2153 2157 assert tier.context_tokens >= 16384 2154 2158 assert tier.context_tokens > 0 2159 + assert local_server._FLOOR_TIER.resident_mib == 4541 2160 + assert local_server._CAPABLE_TIER.resident_mib is None 2155 2161 2156 2162 2157 2163 @pytest.mark.parametrize(
+169
tests/test_parakeet_placement.py
··· 1 + # SPDX-License-Identifier: AGPL-3.0-only 2 + # Copyright (c) 2026 sol pbc 3 + 4 + from __future__ import annotations 5 + 6 + import pytest 7 + 8 + from solstone.think.providers import local_vulkan 9 + from solstone.think.providers.parakeet_placement import ( 10 + CO_FIT_MARGIN_MIB, 11 + PARAKEET_WORST_CASE_MIB, 12 + cpu_placement_suffix, 13 + decide_parakeet_auto_placement, 14 + discrete_hardware_gpu_count, 15 + is_discrete, 16 + ) 17 + 18 + PLACEMENT_LINE = ( 19 + "sol thinks on your GPU; transcription runs on your CPU on this machine" 20 + ) 21 + 22 + 23 + def _device( 24 + *, 25 + index: int = 0, 26 + name: str = "Test GPU", 27 + device_type: int = local_vulkan.VK_TYPE_DISCRETE, 28 + vram_mib: int = 6144, 29 + ) -> local_vulkan.VulkanDevice: 30 + return local_vulkan.VulkanDevice( 31 + index=index, 32 + name=name, 33 + device_type=device_type, 34 + vram_mib=vram_mib, 35 + ) 36 + 37 + 38 + def _decision( 39 + vram_mib: int | None, 40 + *, 41 + selected_device_is_discrete: bool = True, 42 + discrete_hardware_gpu_count: int = 1, 43 + unified_memory: bool = False, 44 + brain_lane_active: bool = True, 45 + ): 46 + return decide_parakeet_auto_placement( 47 + vram_mib=vram_mib, 48 + selected_device_is_discrete=selected_device_is_discrete, 49 + discrete_hardware_gpu_count=discrete_hardware_gpu_count, 50 + unified_memory=unified_memory, 51 + brain_lane_active=brain_lane_active, 52 + ) 53 + 54 + 55 + def test_floor_tier_small_cards_force_cpu() -> None: 56 + decision = _decision(6144) 57 + 58 + assert decision.force_cpu is True 59 + assert decision.reason_code == "co_location_requires_cpu" 60 + assert decision.tier_name == "floor" 61 + assert decision.tier_resident_mib == 4541 62 + assert decision.parakeet_worst_case_mib == PARAKEET_WORST_CASE_MIB 63 + assert decision.margin_mib == CO_FIT_MARGIN_MIB 64 + assert decision.required_mib == 10587 65 + assert decision.vram_mib == 6144 66 + 67 + 68 + @pytest.mark.parametrize("vram_mib", [10240, 11264, 12288]) 69 + def test_margin_places_real_floor_tier_cards(vram_mib: int) -> None: 70 + decision = _decision(vram_mib) 71 + 72 + assert decision.force_cpu is (vram_mib == 10240) 73 + 74 + 75 + def test_capable_tier_unmeasured_residency_keeps_gpu() -> None: 76 + decision = _decision(16000) 77 + 78 + assert decision.force_cpu is False 79 + assert decision.reason_code == "tier_residency_unmeasured" 80 + assert decision.tier_name == "capable" 81 + assert decision.tier_resident_mib is None 82 + assert decision.required_mib is None 83 + 84 + 85 + @pytest.mark.parametrize( 86 + ("kwargs", "reason_code"), 87 + [ 88 + ({"vram_mib": None}, "vram_unknown"), 89 + ({"vram_mib": 6144, "brain_lane_active": False}, "brain_lane_inactive"), 90 + ( 91 + {"vram_mib": 6144, "discrete_hardware_gpu_count": 2}, 92 + "discrete_gpu_count_not_one", 93 + ), 94 + ( 95 + {"vram_mib": 6144, "selected_device_is_discrete": False}, 96 + "selected_device_not_discrete", 97 + ), 98 + ({"vram_mib": 6144, "unified_memory": True}, "unified_memory"), 99 + ], 100 + ) 101 + def test_non_matching_inputs_keep_today(kwargs: dict, reason_code: str) -> None: 102 + decision = _decision(**kwargs) 103 + 104 + assert decision.force_cpu is False 105 + assert decision.reason_code == reason_code 106 + 107 + 108 + def test_is_discrete_centralizes_vulkan_classification() -> None: 109 + assert is_discrete(_device(), local_vulkan) is True 110 + assert ( 111 + is_discrete( 112 + _device(device_type=local_vulkan.VK_TYPE_INTEGRATED), 113 + local_vulkan, 114 + ) 115 + is False 116 + ) 117 + 118 + 119 + def test_discrete_hardware_gpu_count_ignores_integrated_and_software() -> None: 120 + devices = [ 121 + _device(index=0), 122 + _device(index=1, device_type=local_vulkan.VK_TYPE_INTEGRATED), 123 + _device(index=2, name="llvmpipe", device_type=local_vulkan.VK_TYPE_CPU), 124 + ] 125 + 126 + assert discrete_hardware_gpu_count(devices, local_vulkan) == 1 127 + 128 + 129 + def test_cpu_placement_suffix_owns_joiner_and_copy() -> None: 130 + selected = _device(vram_mib=6144) 131 + 132 + assert ( 133 + cpu_placement_suffix( 134 + devices=[selected], 135 + selected=selected, 136 + local_vulkan=local_vulkan, 137 + unified_memory=False, 138 + brain_lane_active=True, 139 + ) 140 + == f"; {PLACEMENT_LINE}" 141 + ) 142 + 143 + 144 + @pytest.mark.parametrize( 145 + ("selected", "unified_memory", "brain_lane_active"), 146 + [ 147 + (None, False, True), 148 + (_device(vram_mib=6144), True, True), 149 + (_device(vram_mib=6144), False, False), 150 + (_device(vram_mib=12288), False, True), 151 + ], 152 + ) 153 + def test_cpu_placement_suffix_absent_outside_predicate( 154 + selected: local_vulkan.VulkanDevice | None, 155 + unified_memory: bool, 156 + brain_lane_active: bool, 157 + ) -> None: 158 + devices = [selected] if selected is not None else [] 159 + 160 + assert ( 161 + cpu_placement_suffix( 162 + devices=devices, 163 + selected=selected, 164 + local_vulkan=local_vulkan, 165 + unified_memory=unified_memory, 166 + brain_lane_active=brain_lane_active, 167 + ) 168 + == "" 169 + )
+31
tests/test_parakeet_server.py
··· 5 5 6 6 import sys 7 7 import types 8 + from pathlib import Path 8 9 9 10 import pytest 10 11 ··· 22 23 fake_httpx = types.SimpleNamespace(get=get) 23 24 monkeypatch.setitem(sys.modules, "httpx", fake_httpx) 24 25 return fake_httpx 26 + 27 + 28 + def test_placement_record_round_trips_and_clears( 29 + monkeypatch: pytest.MonkeyPatch, 30 + tmp_path: Path, 31 + ) -> None: 32 + monkeypatch.setenv("SOLSTONE_JOURNAL", str(tmp_path / "journal")) 33 + 34 + assert parakeet_server.read_parakeet_placement() is None 35 + parakeet_server.clear_parakeet_placement() 36 + parakeet_server.write_parakeet_placement("gpu") 37 + assert parakeet_server.read_parakeet_placement() == "gpu" 38 + parakeet_server.write_parakeet_placement("cpu") 39 + assert parakeet_server.read_parakeet_placement() == "cpu" 40 + parakeet_server.clear_parakeet_placement() 41 + assert parakeet_server.read_parakeet_placement() is None 42 + 43 + 44 + def test_placement_record_validation( 45 + monkeypatch: pytest.MonkeyPatch, 46 + tmp_path: Path, 47 + ) -> None: 48 + monkeypatch.setenv("SOLSTONE_JOURNAL", str(tmp_path / "journal")) 49 + with pytest.raises(ValueError, match="invalid parakeet placement"): 50 + parakeet_server.write_parakeet_placement("vulkan") 51 + 52 + path = tmp_path / "journal" / "health" / "parakeet-cpp.placement" 53 + path.parent.mkdir(parents=True) 54 + path.write_text("vulkan") 55 + assert parakeet_server.read_parakeet_placement() is None 25 56 26 57 27 58 def test_no_port_probe_failed_and_connect_not_ready(monkeypatch: pytest.MonkeyPatch):
+10 -2
tests/test_supervisor.py
··· 3363 3363 from solstone.think.providers import local_server 3364 3364 3365 3365 floor = local_server.ServerTier( 3366 - name="floor", context_tokens=16384, parallel_slots=1, prompt_cache_mib=0 3366 + name="floor", 3367 + context_tokens=16384, 3368 + parallel_slots=1, 3369 + prompt_cache_mib=0, 3370 + resident_mib=4541, 3367 3371 ) 3368 3372 capable = local_server.ServerTier( 3369 - name="capable", context_tokens=32768, parallel_slots=2, prompt_cache_mib=2048 3373 + name="capable", 3374 + context_tokens=32768, 3375 + parallel_slots=2, 3376 + prompt_cache_mib=2048, 3377 + resident_mib=None, 3370 3378 ) 3371 3379 3372 3380 with caplog.at_level(logging.INFO):
+213 -4
tests/test_supervisor_parakeet.py
··· 3 3 4 4 from __future__ import annotations 5 5 6 + import logging 6 7 from pathlib import Path 7 8 from types import SimpleNamespace 8 9 9 10 import pytest 10 11 11 12 from solstone.think import supervisor 12 - from solstone.think.providers import local_vulkan, parakeet_install, parakeet_server 13 + from solstone.think.providers import ( 14 + local_cuda, 15 + local_vulkan, 16 + parakeet_install, 17 + parakeet_server, 18 + ) 19 + 20 + 21 + @pytest.fixture(autouse=True) 22 + def _reset_vulkan_detect_cache(): 23 + local_vulkan.reset_detect_cache() 24 + yield 25 + local_vulkan.reset_detect_cache() 13 26 14 27 15 28 class _FakeProcess: ··· 32 45 self.cleanup_called = True 33 46 34 47 48 + def _nvidia_probe( 49 + *, 50 + vram_mib: int, 51 + memory_source: str = local_cuda.MEMORY_SOURCE_NVIDIA_VRAM, 52 + ) -> local_cuda.NvidiaProbe: 53 + return local_cuda.NvidiaProbe( 54 + index=0, 55 + compute_cap="sm_75", 56 + driver_cuda_version=13, 57 + vram_mib=vram_mib, 58 + tiering_memory_mib=vram_mib, 59 + memory_source=memory_source, 60 + detected=True, 61 + ) 62 + 63 + 64 + def _patch_ready_parakeet_launch( 65 + monkeypatch, 66 + launches: list[dict[str, object]], 67 + *, 68 + poll_sequence: list[tuple[int | None, int | None]] | None = None, 69 + ) -> list[tuple[str, int]]: 70 + def fake_ensure(backend: str): 71 + return Path(f"/tmp/{backend}/parakeet-server"), Path("/tmp/model.gguf") 72 + 73 + monkeypatch.setattr(parakeet_install, "ensure_artifacts_installed", fake_ensure) 74 + monkeypatch.setattr(supervisor, "find_available_port", lambda: 45123) 75 + ports: list[tuple[str, int]] = [] 76 + monkeypatch.setattr( 77 + supervisor, 78 + "write_service_port", 79 + lambda service, port: ports.append((service, port)), 80 + ) 81 + monkeypatch.setattr(supervisor, "parakeet_physical_thread_count", lambda: 6) 82 + monkeypatch.setattr( 83 + supervisor, "_parakeet_runtime_library_dirs", lambda: [Path("/parakeet/lib")] 84 + ) 85 + monkeypatch.setattr( 86 + parakeet_server, "probe_state", lambda: (parakeet_server.STATE_READY, None) 87 + ) 88 + 89 + sequence = poll_sequence or [(None, None)] 90 + 91 + def fake_launch_process( 92 + name, cmd, *, restart=False, shutdown_timeout=15, ref=None, env=None 93 + ): 94 + index = min(len(launches), len(sequence) - 1) 95 + poll_value, returncode = sequence[index] 96 + managed = _FakeManaged(poll_value, returncode) 97 + launches.append( 98 + { 99 + "name": name, 100 + "cmd": cmd, 101 + "restart": restart, 102 + "env": env, 103 + "managed": managed, 104 + } 105 + ) 106 + return managed 107 + 108 + monkeypatch.setattr(supervisor, "_launch_process", fake_launch_process) 109 + return ports 110 + 111 + 35 112 def test_parakeet_server_is_sweepable_orphan_name() -> None: 36 113 assert ( 37 114 supervisor.PARAKEET_SERVER_PROCESS_NAME in supervisor._LOCAL_SERVER_PROCTITLES ··· 141 218 142 219 def test_start_parakeet_server_vulkan_crash_falls_back_to_cpu( 143 220 monkeypatch, 221 + tmp_path, 144 222 ) -> None: 223 + monkeypatch.setenv("SOLSTONE_JOURNAL", str(tmp_path / "journal")) 145 224 monkeypatch.delenv("GGML_VK_VISIBLE_DEVICES", raising=False) 146 225 monkeypatch.delenv("LD_LIBRARY_PATH", raising=False) 147 226 monkeypatch.setattr(supervisor.sys, "platform", "linux") 148 227 monkeypatch.setattr(supervisor, "linux_stt_uses_parakeet_cpp", lambda: True) 149 228 monkeypatch.setattr(supervisor, "_configured_parakeet_device", lambda: "auto") 229 + monkeypatch.setattr(supervisor, "is_local_provider_needed", lambda: True) 230 + monkeypatch.setattr( 231 + "solstone.think.providers.local_endpoint.resolve_local_endpoint", 232 + lambda: SimpleNamespace(is_bundled=True), 233 + ) 150 234 gpu = local_vulkan.VulkanDevice( 151 235 2, 152 236 "NVIDIA Test GPU", 153 237 local_vulkan.VK_TYPE_DISCRETE, 154 - 8192, 238 + 12288, 155 239 ) 156 240 monkeypatch.setattr(local_vulkan, "detect_gpus", lambda: [gpu]) 157 241 monkeypatch.setattr(local_vulkan, "select_device", lambda devices: devices[0]) 158 242 monkeypatch.setattr(local_vulkan, "classify", lambda _device: "discrete") 243 + monkeypatch.setattr( 244 + local_cuda, "probe_nvidia_gpu", lambda: _nvidia_probe(vram_mib=12288) 245 + ) 159 246 160 247 def fake_ensure(backend: str): 161 248 return Path(f"/tmp/{backend}/parakeet-server"), Path("/tmp/model.gguf") ··· 218 305 assert launches[0]["managed"].cleanup_called is True 219 306 assert terminated[0][0] is launches[0]["managed"] 220 307 assert ports == [("parakeet-cpp", 45123)] 308 + assert parakeet_server.read_parakeet_placement() == "cpu" 309 + 310 + 311 + def test_start_parakeet_server_forces_cpu_on_small_single_discrete_bundled_brain( 312 + monkeypatch, 313 + tmp_path, 314 + caplog, 315 + ) -> None: 316 + monkeypatch.setenv("SOLSTONE_JOURNAL", str(tmp_path / "journal")) 317 + monkeypatch.delenv("GGML_VK_VISIBLE_DEVICES", raising=False) 318 + monkeypatch.setattr(supervisor.sys, "platform", "linux") 319 + monkeypatch.setattr(supervisor, "linux_stt_uses_parakeet_cpp", lambda: True) 320 + monkeypatch.setattr(supervisor, "_configured_parakeet_device", lambda: "auto") 321 + monkeypatch.setattr(supervisor, "is_local_provider_needed", lambda: True) 322 + monkeypatch.setattr( 323 + "solstone.think.providers.local_endpoint.resolve_local_endpoint", 324 + lambda: SimpleNamespace(is_bundled=True), 325 + ) 326 + gpu = local_vulkan.VulkanDevice( 327 + 2, 328 + "NVIDIA Test GPU", 329 + local_vulkan.VK_TYPE_DISCRETE, 330 + 6144, 331 + ) 332 + monkeypatch.setattr(local_vulkan, "detect_gpus", lambda: [gpu]) 333 + monkeypatch.setattr(local_vulkan, "select_device", lambda devices: devices[0]) 334 + monkeypatch.setattr(local_vulkan, "classify", lambda _device: "discrete") 335 + monkeypatch.setattr( 336 + local_cuda, "probe_nvidia_gpu", lambda: _nvidia_probe(vram_mib=6144) 337 + ) 338 + launches: list[dict[str, object]] = [] 339 + ports = _patch_ready_parakeet_launch(monkeypatch, launches) 340 + 341 + caplog.set_level(logging.INFO) 342 + result = supervisor.start_parakeet_server() 343 + 344 + assert result is launches[0]["managed"] 345 + assert len(launches) == 1 346 + assert launches[0]["cmd"][0] == "/tmp/cpu/parakeet-server" 347 + assert "GGML_VK_VISIBLE_DEVICES" not in launches[0]["env"] 348 + assert ports == [("parakeet-cpp", 45123)] 349 + assert parakeet_server.read_parakeet_placement() == "cpu" 350 + assert ( 351 + "parakeet-server auto placement resolved to CPU: tier=floor " 352 + "tier_resident_mib=4541 parakeet_worst_case_mib=5022 margin_mib=1024 " 353 + "required_mib=10587 gpu_vram_mib=6144 placement=cpu" 354 + ) in caplog.text 355 + 356 + 357 + def test_start_parakeet_server_brain_lane_inactive_keeps_auto_vulkan( 358 + monkeypatch, 359 + tmp_path, 360 + ) -> None: 361 + monkeypatch.setenv("SOLSTONE_JOURNAL", str(tmp_path / "journal")) 362 + monkeypatch.setattr(supervisor.sys, "platform", "linux") 363 + monkeypatch.setattr(supervisor, "linux_stt_uses_parakeet_cpp", lambda: True) 364 + monkeypatch.setattr(supervisor, "_configured_parakeet_device", lambda: "auto") 365 + monkeypatch.setattr(supervisor, "is_local_provider_needed", lambda: False) 366 + gpu = local_vulkan.VulkanDevice( 367 + 2, 368 + "NVIDIA Test GPU", 369 + local_vulkan.VK_TYPE_DISCRETE, 370 + 6144, 371 + ) 372 + monkeypatch.setattr(local_vulkan, "detect_gpus", lambda: [gpu]) 373 + monkeypatch.setattr(local_vulkan, "select_device", lambda devices: devices[0]) 374 + monkeypatch.setattr(local_vulkan, "classify", lambda _device: "discrete") 375 + monkeypatch.setattr( 376 + local_cuda, "probe_nvidia_gpu", lambda: _nvidia_probe(vram_mib=6144) 377 + ) 378 + launches: list[dict[str, object]] = [] 379 + _patch_ready_parakeet_launch(monkeypatch, launches) 380 + 381 + result = supervisor.start_parakeet_server() 382 + 383 + assert result is launches[0]["managed"] 384 + assert len(launches) == 1 385 + assert launches[0]["cmd"][0] == "/tmp/vulkan/parakeet-server" 386 + assert launches[0]["env"]["GGML_VK_VISIBLE_DEVICES"] == "2" 387 + assert parakeet_server.read_parakeet_placement() == "gpu" 388 + 389 + 390 + def test_start_parakeet_server_explicit_cpu_skips_auto_placement( 391 + monkeypatch, 392 + tmp_path, 393 + ) -> None: 394 + monkeypatch.setenv("SOLSTONE_JOURNAL", str(tmp_path / "journal")) 395 + monkeypatch.setattr(supervisor.sys, "platform", "linux") 396 + monkeypatch.setattr(supervisor, "linux_stt_uses_parakeet_cpp", lambda: True) 397 + monkeypatch.setattr(supervisor, "_configured_parakeet_device", lambda: "cpu") 398 + monkeypatch.setattr( 399 + local_vulkan, 400 + "detect_gpus", 401 + lambda: pytest.fail("explicit CPU should not enumerate Vulkan devices"), 402 + ) 403 + monkeypatch.setattr( 404 + local_cuda, 405 + "probe_nvidia_gpu", 406 + lambda: pytest.fail("explicit CPU should not probe NVIDIA"), 407 + ) 408 + launches: list[dict[str, object]] = [] 409 + _patch_ready_parakeet_launch(monkeypatch, launches) 410 + 411 + result = supervisor.start_parakeet_server() 412 + 413 + assert result is launches[0]["managed"] 414 + assert launches[0]["cmd"][0] == "/tmp/cpu/parakeet-server" 415 + assert parakeet_server.read_parakeet_placement() == "cpu" 221 416 222 417 223 418 @pytest.mark.parametrize( ··· 287 482 assert supervisor.linux_stt_uses_parakeet_cpp() is expected 288 483 289 484 290 - def test_start_parakeet_server_early_returns_for_non_linux(monkeypatch) -> None: 485 + def test_start_parakeet_server_early_returns_for_non_linux( 486 + monkeypatch, tmp_path 487 + ) -> None: 488 + monkeypatch.setenv("SOLSTONE_JOURNAL", str(tmp_path / "journal")) 489 + parakeet_server.write_parakeet_placement("gpu") 291 490 monkeypatch.setattr(supervisor.sys, "platform", "darwin") 292 491 monkeypatch.setattr(supervisor.platform, "machine", lambda: "arm64") 293 492 monkeypatch.setattr( ··· 297 496 ) 298 497 299 498 assert supervisor.start_parakeet_server() is None 499 + assert parakeet_server.read_parakeet_placement() is None 300 500 301 501 302 - def test_start_parakeet_server_early_returns_for_other_backend(monkeypatch) -> None: 502 + def test_start_parakeet_server_early_returns_for_other_backend( 503 + monkeypatch, tmp_path 504 + ) -> None: 505 + monkeypatch.setenv("SOLSTONE_JOURNAL", str(tmp_path / "journal")) 506 + parakeet_server.write_parakeet_placement("gpu") 303 507 monkeypatch.setattr(supervisor.sys, "platform", "linux") 304 508 monkeypatch.setattr(supervisor.platform, "machine", lambda: "x86_64") 305 509 monkeypatch.setattr( ··· 309 513 ) 310 514 311 515 assert supervisor.start_parakeet_server() is None 516 + assert parakeet_server.read_parakeet_placement() is None 312 517 313 518 314 519 def test_start_parakeet_server_starts_background_install_when_missing( 315 520 monkeypatch, 521 + tmp_path, 316 522 ) -> None: 523 + monkeypatch.setenv("SOLSTONE_JOURNAL", str(tmp_path / "journal")) 524 + parakeet_server.write_parakeet_placement("gpu") 317 525 monkeypatch.setattr(supervisor, "linux_stt_uses_parakeet_cpp", lambda: True) 318 526 monkeypatch.setattr(supervisor, "_configured_parakeet_device", lambda: "cpu") 319 527 started: list[str] = [] ··· 353 561 354 562 assert supervisor.start_parakeet_server() is None 355 563 assert started == ["parakeet-cpp-provider-bootstrap"] 564 + assert parakeet_server.read_parakeet_placement() is None 356 565 357 566 358 567 def test_parakeet_bootstrap_worker_requests_start_after_install(monkeypatch) -> None:
+43
tests/test_transcribe_parakeet_cpp.py
··· 329 329 "compute_type": "q8_0", 330 330 "per_word_confidence": True, 331 331 } 332 + 333 + 334 + @pytest.mark.parametrize("placement", ["gpu", "cpu"]) 335 + def test_get_model_info_reports_supervisor_placement_record( 336 + monkeypatch: pytest.MonkeyPatch, 337 + tmp_path, 338 + placement: str, 339 + ) -> None: 340 + monkeypatch.setattr(parakeet_cpp.sys, "platform", "linux") 341 + monkeypatch.setenv("SOLSTONE_JOURNAL", str(tmp_path / "journal")) 342 + parakeet_cpp.parakeet_server.write_parakeet_placement(placement) 343 + 344 + info = parakeet_cpp.get_model_info({"device": "auto"}) 345 + 346 + assert info["device"] == placement 347 + assert info["device"] != "auto" 348 + 349 + 350 + def test_get_model_info_uses_configured_device_without_placement_record( 351 + monkeypatch: pytest.MonkeyPatch, 352 + tmp_path, 353 + ) -> None: 354 + monkeypatch.setattr(parakeet_cpp.sys, "platform", "linux") 355 + monkeypatch.setenv("SOLSTONE_JOURNAL", str(tmp_path / "journal")) 356 + 357 + info = parakeet_cpp.get_model_info({"device": "auto"}) 358 + 359 + assert info["device"] == "auto" 360 + 361 + 362 + def test_get_model_info_ignores_invalid_placement_record( 363 + monkeypatch: pytest.MonkeyPatch, 364 + tmp_path, 365 + ) -> None: 366 + monkeypatch.setattr(parakeet_cpp.sys, "platform", "linux") 367 + monkeypatch.setenv("SOLSTONE_JOURNAL", str(tmp_path / "journal")) 368 + placement_path = tmp_path / "journal" / "health" / "parakeet-cpp.placement" 369 + placement_path.parent.mkdir(parents=True) 370 + placement_path.write_text("vulkan") 371 + 372 + info = parakeet_cpp.get_model_info({"device": "cpu"}) 373 + 374 + assert info["device"] == "cpu"
+34 -4
tests/test_transcribe_telemetry.py
··· 51 51 backend_module = MagicMock() 52 52 backend_module.get_model_info.return_value = { 53 53 "model": "parakeet-v3-q8_0.gguf", 54 - "device": "auto", 54 + "device": "gpu", 55 55 "compute_type": "q8_0", 56 56 } 57 57 return backend_module 58 58 59 59 60 - def _run_success(raw_path: Path, audio_buffer, vad_result) -> dict: 60 + def _run_success( 61 + raw_path: Path, 62 + audio_buffer, 63 + vad_result, 64 + backend_module: MagicMock | None = None, 65 + ) -> dict: 61 66 """Run a successful process_audio and return the emitted event kwargs.""" 62 67 from solstone.observe.transcribe.main import process_audio 63 68 ··· 79 84 ), 80 85 patch( 81 86 "solstone.observe.transcribe.main.get_backend", 82 - return_value=_backend_module(), 87 + return_value=backend_module or _backend_module(), 83 88 ), 84 89 patch("solstone.observe.transcribe.main._embed_statements", return_value=None), 85 90 patch( ··· 110 115 assert all(isinstance(v, int) and v >= 0 for v in timings.values()) 111 116 112 117 assert kwargs["backend"] == "parakeet-cpp" 113 - assert kwargs["device"] == "auto" 118 + assert kwargs["device"] == "gpu" 114 119 assert kwargs["model"] == "parakeet-v3-q8_0.gguf" 120 + header = json.loads(raw_path.with_suffix(".jsonl").read_text().splitlines()[0]) 121 + assert header["device"] == "gpu" 115 122 assert kwargs["audio_seconds"] == 10.0 116 123 assert isinstance(kwargs["peak_rss_mib"], int) 117 124 assert kwargs["peak_rss_mib"] > 0 ··· 129 136 # And none of the enrichment-derived content fields ride along either. 130 137 for banned in ("text", "words", "statements", "topics", "setting", "emotions"): 131 138 assert banned not in kwargs 139 + 140 + 141 + @pytest.mark.parametrize("placement", ["gpu", "cpu"]) 142 + def test_success_event_and_header_use_parakeet_placement_record( 143 + monkeypatch: pytest.MonkeyPatch, 144 + raw_path: Path, 145 + audio_buffer: np.ndarray, 146 + vad_result: VadResult, 147 + placement: str, 148 + ) -> None: 149 + from solstone.observe.transcribe import _parakeet_cpp as parakeet_cpp 150 + 151 + monkeypatch.setattr(parakeet_cpp.sys, "platform", "linux") 152 + monkeypatch.setenv("SOLSTONE_JOURNAL", str(raw_path.parents[4])) 153 + parakeet_cpp.parakeet_server.write_parakeet_placement(placement) 154 + backend_module = MagicMock() 155 + backend_module.get_model_info.side_effect = parakeet_cpp.get_model_info 156 + 157 + kwargs = _run_success(raw_path, audio_buffer, vad_result, backend_module) 158 + 159 + assert kwargs["device"] == placement 160 + header = json.loads(raw_path.with_suffix(".jsonl").read_text().splitlines()[0]) 161 + assert header["device"] == placement 132 162 133 163 134 164 def test_failed_event_is_content_free_even_when_the_exception_message_is_not(