liveConfig gated only on S3_ENDPOINT, so a CI environment that defines
S3_ENDPOINT for other purposes un-skipped the live tests and they died
with "static credentials are empty". Require endpoint AND access key
AND secret before running; anything less skips as intended.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Before, the first error anywhere in the upload loop (e.g. R2's InvalidPart
rejection at the first 10-minute rollover) killed the loop for the rest of
the stream: one error line, then segments backed up silently and nothing
more was recorded or logged. Production showed exactly this signature —
one "started S3 multipart upload …-1.m4s" per stream, one InvalidPart ~10
minutes later, then silence.

Now the loop recovers: the broken object is aborted (so the backend
doesn't hold its parts) and abandoned with a loud log including the key
and how many bytes are lost, and the next segment starts a fresh object.
One bad object costs a gap in the recording instead of the whole stream.
Abandoned objects never get RecordComplete, so finalize's
completed_at-only query skips them naturally. Only the final
completing flush still surfaces through Close — there are no more
segments coming to recover with. A failure with no object in flight
(CreateMultipartUpload itself) logs the dropped segment explicitly.

The recording gate also gets audibility, after today's debugging session:

- "live recording disabled for streamer" is promoted Debug -> Info; it
fires once per stream session and is the answer to "why does this
streamer have no recording?"
- S3-not-configured gets a Debug line (runtime-flippable; Info would be
noise in the no-S3 dev default)
- a failed initial livestream-URI resolve now logs a Warn instead of
being swallowed, and "S3 upload enabled" carries the repoDID

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

R2 rejects CompleteMultipartUpload with "InvalidPart: All non-trailing
parts must have the same length" unless every part except the last has
the same size — a rule AWS/minio/rustfs don't enforce, which is why
live-to-rec worked locally against rustfs but failed against R2.

Two paths violated it:

- S3Uploader flushed whatever segments had accumulated past the 5MB
minimum, so part sizes tracked segment sizes. It now slices its buffer
at exact liveUploadPartSize (5MB) boundaries, holding the remainder
for the next flush; only an object's final part is short.

- ConcatWithHeader emitted one UploadPartCopy part per source object
(plus a ~5MB header part), so part sizes tracked object sizes. It now
slices the destination byte stream into uniform concatPartSize (16MB)
windows: a window that falls entirely inside one source object is a
server-side UploadPartCopy, and only windows containing the header or
an object boundary are downloaded and re-uploaded (~one window per
source object). Windows upload concurrently (copyConcurrency), like
multipartCopy already did.

A side effect of the windowing is that sub-5MB middle objects no longer
need a full-rewrite fallback — they just ride inside a mixed window —
so ErrConcatPartTooSmall and the fallback branch in assembleContentBlob
are gone. (MultipartWriter and multipartCopy already produced uniform
parts, which is why VOD uploads were unaffected.)

The fake S3 in concat_test now enforces R2's uniform-length rule next
to AWS's 5MB floor, and pkg/s3 gains env-gated live tests
(TestLiveS3Uploader / TestLiveConcatWithHeader) that run the real
uploader and concat against any S3-compatible endpoint:

set -a; . ~/r2.env; go test -count=1 -run TestLive -v ./pkg/s3

Both pass against R2 and rustfs, with sizes chosen to exercise real
copy windows and unequal part sizes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

ingest sparse video deadlock

The four MistServer-capture ingest tests read tmp-debug/nyc-*.mkv and
skipped everywhere but the machine that held the capture. The cuts are
now uploaded to the fixture bucket, so the tests run anywhere (CI
included) via remote.RemoteFixture, like the transcode-wedge fixtures:

- nyc-head / nyc-head-nojson (10.5MB each): M_JSON metadata-track canary
- nyc-tail135 (20MB): fast wedge check across the keyframe-only
transition
- nyc-full (124MB): the whole 164s capture, with and without transcode

Fixture downloads are content-addressed and cached, so a warm runner
pays nothing; a cold CI pod downloads ~165MB once per run.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

RunMKVIngestWorker uses ctx cancellation as its "flush the final GoP"
signal after EOS. But muxl's event parser treats a cancelled ctx as
"abandon the stream mid-read": it returns early, nobody reads or closes
the wasm's stdout pipe, and the wasm deadlocks against its next write —
the sign/unwrap call never returns, done never closes, and the worker
hangs forever at `cancel(); <-done`.

While ingest was clock-paced (appsink sync=true) every GoP had drained
long before EOS+cancel arrived, so the race never lost. Once ingest ran
at full speed, EOS+cancel landed while GoPs were still in flight, and
audioCompletionTarget then STARTED a muxl unwrap with an already-dead
ctx — a guaranteed deadlock. That's what wedged GitLab CI's
TestWorkerServesFramesOverSocket into the go test 30m timeout (GitHub's
slower runners kept winning the race).

Fix, in two spots:
- muxlSignSegmentElem: the signer + event drain run on
context.WithoutCancel. Cancel still triggers the flush — it closes the
input pipe, the signer sees EOF, signs the final GoP, and exits
cleanly — it just no longer aborts the parser mid-stream.
- workerSegmentSink: audioCompletionTarget's unwrap likewise runs
WithoutCancel (same reasoning as the transcoder it feeds).

Fallout, both good:
- The old race was silently DROPPING the final GoP even when it didn't
deadlock: the socket e2e test now serves 2 segments (was 1), the
135s-tail sample 31 (was 29).
- The "known issue" from the previous commit — the full 164s production
capture with node transcode keys progressively slowing until the
watchdog fired, then hanging in the post-cancel drain — was this same
bug. It now transcodes the whole capture in ~13s, so
TestMKVIngestMistFullSample's SP_SLOW_TESTS skip-gate is removed.

Upstream follow-up (streamplace/muxl): runWith should close its stdout
pipe reader when parseEvents returns early, so a cancelled ctx can't
permanently wedge a wasm instance; parseEvents' ctx.Done select can also
randomly drop tail events. vod/livehls call sites remain exposed to that
until it's fixed there.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Eli/idle finalize guard

Audit follow-up to the muxl-appsink sync fix: swept every appsink in the
tree for clock sync.

- thumbnailFromMP4 (thumbnail.go): the flat-MP4 thumbnail pipeline was
the only consumer-path appsink still defaulting to sync=true. In
practice videorate re-stamps the selected frame to PTS 0 so it usually
rendered immediately, but there's no reason to leave it on the clock —
and its twin Thumbnail() already sets sync=false.
- SelfTest (gstreamer.go): untimestamped buffers so sync never actually
waited; set sync=false for consistency so the tree greps clean.
- sp whip (cmd/whip.go): the ONE appsink pair where sync=true is
load-bearing — the client plays a file over WHIP as a live stream and
the appsink clock wait is the only realtime pacing (pion WriteSample
pushes immediately). Made sync=true explicit with a comment so a
future sweep doesn't "fix" it.

Already sync=false (no change): packetize, media_data_parser,
segment_conv, transcode, vod_pipeline, segmenter (via sink-added),
Thumbnail, muxl_segment.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

A production stream kept wedging in the MUXL-ize step: video flowed in on
the MKV endpoint but segments stopped coming out mid-stream, with no EOS,
until the ingest watchdog killed the worker.

Root cause: when a push falls behind, MistServer drops delta frames and the
video track degrades to keyframe-only (~1s between video frames) while audio
continues normally. The fMP4 muxer feeding the signer is a gst aggregator —
it consumes nothing until every sink pad has data — so during a video gap
the audio branch must buffer the whole gap while matroskademux walks the
byte stream to the next video frame. buildMKVIngestPipeline used bare
`queue` elements, whose default max-size-time is 1s: a >=1s video gap fills
the audio queue, blocks the demux, starves the muxer's video pad, and
deadlocks the graph permanently (confirmed with native thread stacks:
fmp4mux:src in gst_aggregator_wait_and_check, queue1:src blocked in
gst_aggregator_pad_chain_internal, demux parked on the full queue).

Fix: use the shared Queue2Big preset (no time/buffer cap, generous byte
cap) on both ingest branches — the same fix the transcode pipeline already
carries for its double-GoP variant of this deadlock. Also set sync=false on
the muxl appsink (every other appsink in the tree sets it); the default
sync=true was clock-pacing the entire ingest graph at realtime for no
benefit.

With the fix, the full 164s production capture (which goes keyframe-only at
~140s) segments to completion: 171/171 GoPs in ~4s, versus 145 then a
watchdog kill before.

Fallout handled: the 4-audio sample-stream.mkv "wedge fixture" stopped
wedging — its wedge was this same queue-cap deadlock all along — so the
watchdog/ban containment tests now synthesize an audio-only MKV instead
(the muxer's video pad never sees data or EOS: a true wedge no queue sizing
can fix). New TestMKVIngestSparseVideoNoWedge synthesizes the sparse-video
shape in-test (no binary fixture) and fails on the unfixed pipeline.

Known follow-up (pre-existing, out of scope here): with node transcode keys
a long full-speed feed progressively slows the transcode completion path
until the watchdog fires, and the worker then hangs in the post-cancel
drain (transcoder.Feed blocks on its jobs channel under a
context.WithoutCancel ctx, so RunMKVIngestWorker never returns). See the
skip note on TestMKVIngestMistFullSample.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

The idle-timeout finalizer (processFinalizeLivestreamTask) could set endedAt
on the record an actively-streaming user was publishing under, taking the
stream pre-live underneath a still-flowing ingest. Root cause: lastSeenAt
only advances via the per-segment heartbeat (StreamSession.doUpdateLivestream),
which is coupled to segment arrival and can lag behind actual ingestion. A
~60s ingest gap froze lastSeenAt; the 300s idle timer then fired while
segments were still flowing, and endedAt was written onto the live record.

Guard the finalizer: when lastSeenAt is stale but the record is still the
streamer's latest livestream (no newer record supersedes it), reschedule the
finalize for one more idle window instead of ending — giving a lagging
heartbeat a chance to catch up. If the stream is truly abandoned, the
heartbeat stays frozen and the next pass ends it. Superseded records (a newer
one exists) are still ended, since they're no longer live.

Also fix the related orphaned-finalize-task bug: startLivestream now ends any
prior un-ended livestream for the repo before creating the new one, so the
prior record's already-scheduled idle-timeout task hits its rec.EndedAt != nil
early-skip instead of later writing a stale endedAt onto a replaced record.

Verified against the 2026-07-06 production incident: segment manifests showed
the stream went pre-live exactly when the idle timer fired on the live record
(lastSeenAt frozen at the pre-gap heartbeat, endedAt = lastSeenAt at finalize).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Add "podcasting" to the activityLabel knownValues in
place.stream.defs, alongside just_chatting and the other known
activity labels. Regenerated docs via `make lexicons`; Go lexgen
needs no change (knownValues stay a plain string), while the JS
lexicon types now include "podcasting" in the label union. Also
adds the display entry to the hand-maintained ACTIVITY_LABELS
list in metadata-constants.ts.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

disk cleanup

The mistserver image build piped `curl … | tar xzv` straight from
r.mistserver.org, so any transient hiccup (it has been 502ing) either fed a
truncated stream to tar or failed outright with no retry — flaking CI's
"mistserver image" job repeatedly. Download to a file with retries
(`-f --retry 12 --retry-all-errors --retry-delay 10 --retry-max-time 240`)
and only extract once the transfer completes. Applied to both the ghcr and
compose Dockerfiles.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

CI caught a panic ("assignment to entry in nil map") in
TestFeedStreamTranscoderRebuildsOnNewSession: the transcoder feeds
distributeSegment from a background goroutine, and several MediaManager
constructors (isolated ingest/whip/rtmp workers, test helpers) build the struct
directly without seeding modBuffers, which MakeMediaManager does. Reads/deletes
on a nil map are safe but the assignment in feedModerationBuffer was not.

Lazily initialize the map under the existing lock so no constructor can trip
it, and add a regression test that feeds a MediaManager with a nil modBuffers.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Segments were archived to disk (segments/<did>/.../<ts>.m4s) purely so
moderation/report clips (makeClip -> ClipUser) and the now-removed legacy HLS
endpoints could read them back; live HLS is in-memory and VOD is S3/blob-store,
so nothing else needs them. On-disk archival was the thing filling the drive.

Moderation clipping can't come off S3: the live-rec uploader buffers >=5MB
parts / 10-min cutovers (the recent ~60s isn't flushed) and is gated to opt-in
recording users, while moderation must clip any user's last 60s on demand. So
instead keep a short in-memory ring of each user's recent canonical segments
(modClipWindow = 2m, matching the old on-disk moderationRetention) and clip
from that.

- pkg/media/moderation_buffer.go: per-user time+count-bounded segment ring,
fed for every validated segment; MediaManager.ClipUser reads from it.
- distributeSegment: feed the buffer instead of writing SegmentFileCreate.
The Segment DB row is still written (dedup, /segment metadata, playlists);
delete_after still governs row cleanup and reaps any legacy on-disk .m4s.
- Move both ClipUser callers (moderation report, /clip endpoint) onto
mm.ClipUser; delete the disk-reading clip_user.go free function.

The last-60s clip window is well within the buffer; a request for a longer
span yields only what's still retained (previously bounded by the 1h disk
retention anyway).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

The ffconcat -> latest.mp4 -> segment/:file trio served .m4s straight off
disk; live playback is all MUXL HLS now (in-memory window, pkg/livehls +
place_stream_playback_getlive). Nothing references these routes. Drops one of
the two remaining readers of on-disk segment archival. The MistServer
stream.mkv HEAD and the /segment/:id JSON metadata endpoint are unaffected.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>