Previously all DAG_STATUS events were skipped during historical replay
on WS connect, so reconnecting clients (React strict mode remount,
page navigation) lost current DAG state. Now sends only the most
recent DAG_STATUS event on connect.
The processing page only read dagStatus from the transcript room WS,
which loses events during page navigation and React strict mode
double-mounting (WS torn down and reconnected, historical replay
skips DAG_STATUS). Now also consumes useDagStatusMap() from
UserEventsProvider (user room), which uses a singleton WS that
survives remounts.
Priority: transcript room WS > user room WS > REST API.
Exercises the full broadcast → pub/sub → WebSocket delivery chain
that DEBUG.md identified as potentially broken. Covers send_json
direct delivery, broadcast_event() end-to-end, and event ordering.
Also patches broadcast.py's get_ws_manager (missing from conftest).
The UserEventsProvider only invalidated the list query on status changes.
The detail page's useTranscriptGet was never refreshed, so it never
redirected to /processing on reprocess.
Without this, the page never redirects to /processing when a reprocess
changes status from ended->processing, because the redirect logic only
watches the REST query data, not the WebSocket status state.
Remove 6 tests that covered malformed data shapes (data=None,
data=string, data=list, non-dict event elements) that cannot
occur since our own code always writes well-formed dicts via
model_dump(mode="json").
Add 4 new tests to test_dag_progress.py:
- test_production_dag_shape: Real 15-task pipeline topology with mixed
statuses, verifying all tasks present, topological order invariant,
and correct parent relationships (e.g. finalize has 4 parents)
- test_topological_sort_invariant_complex_dag: 7-node DAG with wide
branching/merging to stress-test that all parents precede children
- test_logging_throttled_by_interval: Mocks time.monotonic to verify
ctx.log() is throttled by interval while broadcasts are not
- test_uses_broadcast_event_not_append_event_and_broadcast: Verifies
progress uses transient broadcast_event, not persisted append variant
- Add TRANSCRIPT_DAG_STATUS handler to UserEventsProvider with
DagStatusContext and useDagStatusMap hook for live DAG task updates
- Clean up dagStatusMap entries when TRANSCRIPT_STATUS transitions
away from "processing"
- Create DagProgressDots component rendering color-coded dots per
DAG task (green=completed, blue pulsing=running, hollow=queued,
red=failed, gray=cancelled) with humanized tooltip names
- Wire dagStatusMap through browse page -> TranscriptCards ->
TranscriptStatusIcon, falling back to REST dag_status field
Add DAG_STATUS and DAG_TASK_PROGRESS event handlers to useWebSockets
hook with exported DagTask/DagTaskStatus types. Create DagProgressTable
component with status icons, live elapsed timers, progress bars, and
expandable error rows. Wire into processing page with REST fallback.
- Add broadcast_dag_status() to dag_progress.py: fetches Hatchet run
details, transforms to DagStatusData, and broadcasts DAG_STATUS event
via WebSocket. Fire-and-forget with exception swallowing.
- Modify with_error_handling decorator to call broadcast_dag_status on
both task success and failure.
- Add DAG_STATUS to USER_ROOM_EVENTS (broadcast.py) and reconnect
filter (transcripts_websocket.py) to avoid replaying stale DAG state.
- Add initial DAG broadcast at workflow dispatch (transcript_process.py).
- Extend make_audio_progress_logger with optional transcript_id param
for transient DAG_TASK_PROGRESS events during mixdown.
- All deferred imports for fork-safety, all broadcasts fire-and-forget.
Foundation for DAG progress reporting to frontend. Ported topo sort
and task extraction from render_hatchet_run.py (Zulip worktree) to
produce structured Pydantic models instead of markdown.
* feat: set Daily as default video platform
Daily.co has been battle-tested and is ready to be the default.
Whereby remains available for rooms that explicitly set it.
* feat: enforce Hatchet for all multitrack processing
Remove use_celery option from rooms - multitrack (Daily) recordings
now always use Hatchet workflows. Celery remains for single-track
(Whereby) file processing only.
- Remove use_celery column from room table
- Simplify dispatch logic to always use Hatchet for multitracks
- Update tests to mock Hatchet instead of Celery
* fix: update whereby test to patch Hatchet instead of removed Celery import
---------
Co-authored-by: Igor Loskutov <igor.loskutoff@gmail.com>
* fix websocket tests
* fix: restore timeout and fix celery test infrastructure
- Re-add timeout=1.0 to ws_manager pubsub loop (prevents CPU spin?)
- Use Redis for Celery tests (memory:// broker doesn't support chords)
- Add timeout param to in-memory subscriber mock
- Remove duplicate celery_includes fixture from rtc_ws tests
* fix: remove redundant inline imports in test files
* fix: update gitleaks ignore for moved s3_key line
---------
Co-authored-by: Igor Loskutov <igor.loskutoff@gmail.com>
Set duration early in get_participants from Daily API (seconds -> ms),
ensuring post_zulip has the value before mixdown_tracks completes.
Removes redundant duration update from mixdown_tracks.
Co-authored-by: Igor Loskutov <igor.loskutoff@gmail.com>
Topic detection was timing out on longer transcripts when LLM
responses are slow. This affects detect_chunk_topic and other
LLM-calling tasks that use TIMEOUT_MEDIUM.
Co-authored-by: Igor Loskutov <igor.loskutoff@gmail.com>
* set hatchet as default for multitracks
* fix: pipeline routing tests for hatchet-default branch
- Create room with use_celery=True to force Celery backend in tests
- Link transcript to room to enable multitrack pipeline routing
- Fixes test failures caused by missing HATCHET_CLIENT_TOKEN in test env
* Update server/reflector/services/transcript_process.py
Co-authored-by: pr-agent-monadical[bot] <198624643+pr-agent-monadical[bot]@users.noreply.github.com>
---------
Co-authored-by: Igor Loskutov <igor.loskutoff@gmail.com>
Co-authored-by: pr-agent-monadical[bot] <198624643+pr-agent-monadical[bot]@users.noreply.github.com>
