Compare commits

..

21 Commits

Author SHA1 Message Date
Igor Loskutov
1bf73c8199 sync with parent 2025-10-21 11:59:26 -04:00
d82abf65ba Emit multriack pipeline events 2025-10-21 16:31:31 +02:00
Igor Loskutov
7d239fe380 dailico track merge vibe 2025-10-21 10:30:19 -04:00
acb6e90f28 Generate waveforms for the mixed audio 2025-10-21 13:33:31 +02:00
Igor Loskutov
f844b9fc1f Merge branch 'igor/dailico-2' of github-monadical:Monadical-SAS/reflector into igor/dailico-2 2025-10-17 10:00:40 -04:00
96f05020cc Align tracks of a multitrack recording 2025-10-17 15:27:27 +02:00
fc79ff3114 Use explicit track keys for processing 2025-10-17 14:42:07 +02:00
Igor Loskutov
3641e2e599 apply platform from envs in priority: non-dry 2025-10-16 15:08:19 -04:00
c23518d2e3 Trigger multitrack processing for daily recordings 2025-10-16 20:05:26 +02:00
23edffe2a2 Mixdown with pyav filter graph 2025-10-16 17:14:55 +02:00
e59770ecc9 Mixdown audio tracks 2025-10-16 17:14:55 +02:00
6301f2afa6 Add multitrack pipeline 2025-10-16 17:14:55 +02:00
9ac7f0e8e2 chore(main): release 0.14.0 (#670) 2025-10-16 17:14:55 +02:00
Igor Loskutov
0a84a9351a stub processor (vibe) self-review 2025-10-10 20:41:08 -04:00
Igor Loskutov
ca22084845 stub processor (vibe) self-review 2025-10-10 18:45:19 -04:00
Igor Loskutov
f945f84be9 stub processor (vibe) 2025-10-10 18:05:31 -04:00
Igor Loskutov
4c523c8eec dont show recording ui on call 2025-10-10 12:45:10 -04:00
Igor Loskutov
0fcf8b6875 doc update (vibe) 2025-10-10 10:57:35 -04:00
Igor Loskutov
446cb748ae vibe dailyco 2025-10-09 17:04:16 -04:00
Igor Loskutov
3e1339a8ea vibe dailyco 2025-10-09 15:52:23 -04:00
Igor Loskutov
807819bb2f llm instructions 2025-10-08 13:06:04 -04:00
112 changed files with 7603 additions and 5403 deletions

View File

@@ -1,4 +1,4 @@
name: Build container/push to container registry name: Deploy to Amazon ECS
on: [workflow_dispatch] on: [workflow_dispatch]

View File

@@ -1,40 +1,5 @@
# Changelog # Changelog
## [0.18.0](https://github.com/Monadical-SAS/reflector/compare/v0.17.0...v0.18.0) (2025-11-14)
### Features
* daily QOL: participants dictionary ([#721](https://github.com/Monadical-SAS/reflector/issues/721)) ([b20cad7](https://github.com/Monadical-SAS/reflector/commit/b20cad76e69fb6a76405af299a005f1ddcf60eae))
### Bug Fixes
* add proccessing page to file upload and reprocessing ([#650](https://github.com/Monadical-SAS/reflector/issues/650)) ([28a7258](https://github.com/Monadical-SAS/reflector/commit/28a7258e45317b78e60e6397be2bc503647eaace))
* copy transcript ([#674](https://github.com/Monadical-SAS/reflector/issues/674)) ([a9a4f32](https://github.com/Monadical-SAS/reflector/commit/a9a4f32324f66c838e081eee42bb9502f38c1db1))
## [0.17.0](https://github.com/Monadical-SAS/reflector/compare/v0.16.0...v0.17.0) (2025-11-13)
### Features
* add API key management UI ([#716](https://github.com/Monadical-SAS/reflector/issues/716)) ([372202b](https://github.com/Monadical-SAS/reflector/commit/372202b0e1a86823900b0aa77be1bfbc2893d8a1))
* daily.co support as alternative to whereby ([#691](https://github.com/Monadical-SAS/reflector/issues/691)) ([1473fd8](https://github.com/Monadical-SAS/reflector/commit/1473fd82dc472c394cbaa2987212ad662a74bcac))
## [0.16.0](https://github.com/Monadical-SAS/reflector/compare/v0.15.0...v0.16.0) (2025-10-24)
### Features
* search date filter ([#710](https://github.com/Monadical-SAS/reflector/issues/710)) ([962c40e](https://github.com/Monadical-SAS/reflector/commit/962c40e2b6428ac42fd10aea926782d7a6f3f902))
## [0.15.0](https://github.com/Monadical-SAS/reflector/compare/v0.14.0...v0.15.0) (2025-10-20)
### Features
* api tokens ([#705](https://github.com/Monadical-SAS/reflector/issues/705)) ([9a258ab](https://github.com/Monadical-SAS/reflector/commit/9a258abc0209b0ac3799532a507ea6a9125d703a))
## [0.14.0](https://github.com/Monadical-SAS/reflector/compare/v0.13.1...v0.14.0) (2025-10-08) ## [0.14.0](https://github.com/Monadical-SAS/reflector/compare/v0.13.1...v0.14.0) (2025-10-08)

345
CODER_BRIEFING.md Normal file
View File

@@ -0,0 +1,345 @@
# Multi-Provider Video Platform Implementation - Coder Briefing
## Your Mission
Implement multi-provider video platform support in Reflector, allowing the system to work with both Whereby and Daily.co video conferencing providers. The goal is to abstract the current Whereby-only implementation and add Daily.co as a second provider, with the ability to switch between them via environment variables.
**Branch:** `igor/dailico-2` (you're already on it)
**Estimated Time:** 12-16 hours (senior engineer)
**Complexity:** Medium-High (requires careful integration with existing codebase)
---
## What You Have
### 1. **PLAN.md** - Your Technical Specification (2,452 lines)
- Complete step-by-step implementation guide
- All code examples you need
- Architecture diagrams and design rationale
- Testing strategy and success metrics
- **Read this first** to understand the overall approach
### 2. **IMPLEMENTATION_GUIDE.md** - Your Practical Guide
- What to copy vs. adapt vs. rewrite
- Common pitfalls and how to avoid them
- Verification checklists for each phase
- Decision trees for implementation choices
- **Use this as your day-to-day reference**
### 3. **Reference Implementation** - `./reflector-dailyco-reference/`
- Working implementation from 2.5 months ago
- Good architecture and patterns
- **BUT:** 91 commits behind current main, DO NOT merge directly
- Use for inspiration and code patterns only
---
## Critical Context: Why Not Just Merge?
The reference branch (`origin/igor/feat-dailyco`) was started on August 1, 2025 and is now severely diverged from main:
- **91 commits behind main**
- Main has 12x more changes (45,840 insertions vs 3,689)
- Main added: calendar integration, webhooks, full-text search, React Query migration, security fixes
- Reference removed: features that main still has and needs
**Merging would be a disaster.** We're implementing fresh on current main, using the reference for validated patterns.
---
## High-Level Approach
### Phase 1: Analysis (2 hours)
- Study current Whereby integration
- Define abstraction requirements
- Create standard data models
### Phase 2: Abstraction Layer (4-5 hours)
- Build platform abstraction (base class, registry, factory)
- Extract Whereby into the abstraction
- Update database schema (add `platform` field)
- Integrate into rooms.py **without breaking calendar/webhooks**
### Phase 3: Daily.co Implementation (4-5 hours)
- Implement Daily.co client
- Add webhook handler
- Create frontend components (rewrite API calls for React Query)
- Add recording processing
### Phase 4: Testing (2-3 hours)
- Unit tests for platform abstraction
- Integration tests for webhooks
- Manual testing with both providers
---
## Key Files You'll Touch
### Backend (New)
```
server/reflector/video_platforms/
├── __init__.py
├── base.py ← Abstract base class
├── models.py ← Platform, MeetingData, VideoPlatformConfig
├── registry.py ← Platform registration system
├── factory.py ← Client creation and config
├── whereby.py ← Whereby client wrapper
├── daily.py ← Daily.co client
└── mock.py ← Mock client for testing
server/reflector/views/daily.py ← Daily.co webhooks
server/tests/test_video_platforms.py ← Platform tests
server/tests/test_daily_webhook.py ← Webhook tests
```
### Backend (Modified - Careful!)
```
server/reflector/settings.py ← Add Daily.co settings
server/reflector/db/rooms.py ← Add platform field, PRESERVE calendar fields
server/reflector/db/meetings.py ← Add platform field
server/reflector/views/rooms.py ← Integrate abstraction, PRESERVE calendar/webhooks
server/reflector/worker/process.py ← Add process_recording_from_url task
server/reflector/app.py ← Register daily router
server/env.example ← Document new env vars
```
### Frontend (New)
```
www/app/[roomName]/components/
├── RoomContainer.tsx ← Platform router
├── DailyRoom.tsx ← Daily.co component (rewrite API calls!)
└── WherebyRoom.tsx ← Extract existing logic
```
### Frontend (Modified)
```
www/app/[roomName]/page.tsx ← Use RoomContainer
www/package.json ← Add @daily-co/daily-js
```
### Database
```
server/migrations/versions/XXXXXX_add_platform_support.py ← Generate fresh migration
```
---
## Critical Warnings ⚠️
### 1. **DO NOT Copy Database Migrations**
The reference migration has the wrong `down_revision` and is based on old schema.
```bash
# Instead:
cd server
uv run alembic revision -m "add_platform_support"
# Then edit the generated file
```
### 2. **DO NOT Remove Main's Features**
Main has calendar integration, webhooks, ICS sync that reference doesn't have.
When modifying `rooms.py`, only change meeting creation logic, preserve everything else.
### 3. **DO NOT Copy Frontend API Calls**
Reference uses old OpenAPI client. Main uses React Query.
Check how main currently makes API calls and replicate that pattern.
### 4. **DO NOT Copy package.json/migrations**
These files are severely outdated in reference.
### 5. **Preserve Type Safety**
Use `TYPE_CHECKING` imports to avoid circular dependencies:
```python
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from reflector.db.rooms import Room
```
---
## How to Start
### Day 1 Morning: Setup & Understanding (2-3 hours)
```bash
# 1. Verify you're on the right branch
git branch
# Should show: igor/dailico-2
# 2. Read the docs (in order)
# - PLAN.md (skim to understand scope, read Phase 1 carefully)
# - IMPLEMENTATION_GUIDE.md (read fully, bookmark it)
# 3. Study current Whereby integration
cat server/reflector/views/rooms.py | grep -A 20 "whereby"
cat www/app/[roomName]/page.tsx
# 4. Check reference implementation structure
ls -la reflector-dailyco-reference/server/reflector/video_platforms/
```
### Day 1 Afternoon: Phase 1 Execution (2-3 hours)
```bash
# 5. Copy video_platforms directory from reference
cp -r reflector-dailyco-reference/server/reflector/video_platforms/ \
server/reflector/
# 6. Review and fix imports
cd server
uv run ruff check reflector/video_platforms/
# 7. Add settings to settings.py (see PLAN.md Phase 2.7)
# 8. Test imports work
uv run python -c "from reflector.video_platforms import create_platform_client; print('OK')"
```
### Day 2: Phase 2 - Database & Integration (4-5 hours)
```bash
# 9. Generate migration
uv run alembic revision -m "add_platform_support"
# Edit the file following PLAN.md Phase 2.8
# 10. Update Room/Meeting models
# Add platform field, PRESERVE all existing fields
# 11. Integrate into rooms.py
# Carefully modify meeting creation, preserve calendar/webhooks
# 12. Add Daily.co webhook handler
cp reflector-dailyco-reference/server/reflector/views/daily.py \
server/reflector/views/
# Register in app.py
```
### Day 3: Phase 3 - Frontend & Testing (4-5 hours)
```bash
# 13. Create frontend components
mkdir -p www/app/[roomName]/components
# 14. Add Daily.co dependency
cd www
pnpm add @daily-co/daily-js@^0.81.0
# 15. Create RoomContainer, DailyRoom, WherebyRoom
# IMPORTANT: Rewrite API calls using React Query patterns
# 16. Regenerate types
pnpm openapi
# 17. Copy and adapt tests
cp reflector-dailyco-reference/server/tests/test_*.py server/tests/
# 18. Run tests
cd server
REDIS_HOST=localhost \
CELERY_BROKER_URL=redis://localhost:6379/1 \
uv run pytest tests/test_video_platforms.py -v
```
---
## Verification Checklist
After implementation, all of these must pass:
**Backend:**
- [ ] `cd server && uv run ruff check .` passes
- [ ] `uv run alembic upgrade head` works cleanly
- [ ] `uv run pytest tests/test_video_platforms.py` passes
- [ ] Can import: `from reflector.video_platforms import create_platform_client`
- [ ] Settings has all Daily.co variables
**Frontend:**
- [ ] `cd www && pnpm lint` passes
- [ ] No TypeScript errors
- [ ] `pnpm openapi` generates platform field
- [ ] No `@ts-ignore` for platform field
**Integration:**
- [ ] Whereby meetings still work (existing flow unchanged)
- [ ] Calendar/webhook features still work in rooms.py
- [ ] env.example documents all new variables
---
## When You're Stuck
### Check These Resources:
1. **PLAN.md** - Detailed code examples for your exact scenario
2. **IMPLEMENTATION_GUIDE.md** - Common pitfalls section
3. **Reference code** - See how it was solved before
4. **Git diff** - Compare reference to your implementation
### Compare Files:
```bash
# See what reference did
diff reflector-dailyco-reference/server/reflector/views/rooms.py \
server/reflector/views/rooms.py
# See what changed in main since reference branch
git log --oneline --since="2025-08-01" -- server/reflector/views/rooms.py
```
### Common Issues:
- **Circular imports:** Use `TYPE_CHECKING` pattern
- **Tests fail with postgres error:** Use `REDIS_HOST=localhost` env vars
- **Frontend API calls broken:** Check current React Query patterns in main
- **Migrations fail:** Ensure you generated fresh, not copied
---
## Success Looks Like
When you're done:
- ✅ All tests pass
- ✅ Linting passes
- ✅ Can create Whereby meetings (unchanged behavior)
- ✅ Can create Daily.co meetings (with env vars)
- ✅ Calendar/webhooks still work
- ✅ Frontend has no TypeScript errors
- ✅ Platform selection via environment variables works
---
## Communication
If you need clarification on requirements, have questions about architecture decisions, or find issues with the spec, document them clearly with:
- What you expected
- What you found
- Your proposed solution
The PLAN.md document is comprehensive but you may find edge cases. Use your engineering judgment and document decisions.
---
## Final Notes
**This is not a simple copy-paste job.** You're doing careful integration work where you need to:
- Understand the abstraction pattern (PLAN.md)
- Preserve all of main's features
- Adapt reference code to current patterns
- Think about edge cases and testing
Take your time with Phase 2 (rooms.py integration) - that's where most bugs will come from if you accidentally break calendar/webhook features.
**Good luck! You've got comprehensive specs, working reference code, and a clean starting point. You can do this.**
---
## Quick Reference
```bash
# Your workspace
├── PLAN.md ← Complete technical spec (read first)
├── IMPLEMENTATION_GUIDE.md ← Practical guide (bookmark this)
├── CODER_BRIEFING.md ← This file
└── reflector-dailyco-reference/ ← Reference implementation (inspiration only)
# Key commands
cd server && uv run ruff check . # Lint backend
cd www && pnpm lint # Lint frontend
cd server && uv run alembic revision -m "..." # Create migration
cd www && pnpm openapi # Regenerate types
cd server && uv run pytest -v # Run tests
```

489
IMPLEMENTATION_GUIDE.md Normal file
View File

@@ -0,0 +1,489 @@
# Daily.co Implementation Guide
## Overview
Implement multi-provider video platform support (Whereby + Daily.co) following PLAN.md.
## Reference Code Location
- **Reference branch:** `origin/igor/feat-dailyco` (on remote)
- **Worktree location:** `./reflector-dailyco-reference/`
- **Status:** Reference only - DO NOT merge or copy directly
## What Exists in Reference Branch (For Inspiration)
### ✅ Can Use As Reference (Well-Implemented)
```
server/reflector/video_platforms/
├── base.py ← Platform abstraction (good design, copy-safe)
├── models.py ← Data models (copy-safe)
├── registry.py ← Registry pattern (copy-safe)
├── factory.py ← Factory pattern (needs settings updates)
├── whereby.py ← Whereby client (needs adaptation)
├── daily.py ← Daily.co client (needs adaptation)
└── mock.py ← Mock client (copy-safe for tests)
server/reflector/views/daily.py ← Webhook handler (needs adaptation)
server/tests/test_video_platforms.py ← Tests (good reference)
server/tests/test_daily_webhook.py ← Tests (good reference)
www/app/[roomName]/components/
├── RoomContainer.tsx ← Platform router (needs React Query)
├── DailyRoom.tsx ← Daily component (needs React Query)
└── WherebyRoom.tsx ← Whereby extraction (needs React Query)
```
### ⚠️ Needs Significant Changes (Use Logic Only)
- `server/reflector/db/rooms.py` - Reference removed calendar/webhook fields that main has
- `server/reflector/db/meetings.py` - Same issue (missing user_id handling differences)
- `server/reflector/views/rooms.py` - Main has calendar integration, webhooks, ICS sync
- `server/reflector/worker/process.py` - Main has different recording flow
- Migration files - Must regenerate against current main schema
### ❌ Do NOT Use (Outdated/Incompatible)
- `package.json`/`pnpm-lock.yaml` - Main uses different dependency versions
- Frontend API client calls - Main uses React Query (reference uses old OpenAPI client)
- Database migrations - Must create new ones from scratch
- Any files that delete features present in main (search, calendar, webhooks)
## Key Differences: Reference vs Current Main
| Aspect | Reference Branch | Current Main | Action Required |
|--------|------------------|--------------|-----------------|
| **API client** | Old OpenAPI generated | React Query hooks | Rewrite all API calls |
| **Database schema** | Simplified (removed features) | Has calendar, webhooks, full-text search | Merge carefully, preserve main features |
| **Settings** | Aug 2025 structure | Current structure | Adapt carefully |
| **Migrations** | Branched from Aug 1 | Current main (91+ commits ahead) | Regenerate from scratch |
| **Frontend deps** | `@daily-co/daily-js@0.81.0` | Check current versions | Update to compatible versions |
| **Package manager** | yarn | pnpm (maybe both?) | Use what main uses |
## Branch Divergence Analysis
**The reference branch is 91 commits behind main and severely diverged:**
- Reference: 8 commits, 3,689 insertions, 425 deletions
- Main since divergence: 320 files changed, 45,840 insertions, 16,827 deletions
- **Main has 12x more changes**
**Major features in main that reference lacks:**
1. Calendar integration (ICS sync with rooms)
2. Self-hosted GPU API infrastructure
3. Frontend OpenAPI React Query migration
4. Full-text search (backend + frontend)
5. Webhook system for room events
6. Environment variable migration
7. Security fixes and auth improvements
8. Docker production frontend
9. Meeting user ID removal (schema change)
10. NextJS version upgrades
**High conflict risk files:**
- `server/reflector/views/rooms.py` - 12x more changes in main
- `server/reflector/db/rooms.py` - Main added 7+ fields
- `www/package.json` - NextJS major version bump
- Database migrations - 20+ new migrations in main
## Implementation Approach
### Phase 1: Copy Clean Abstractions (1-2 hours)
**Files to copy directly from reference:**
```bash
# Core abstraction (review but mostly safe to copy)
cp -r reflector-dailyco-reference/server/reflector/video_platforms/ \
server/reflector/
# BUT review each file for:
# - Import paths (make sure they match current main)
# - Settings references (adapt to current settings.py)
# - Type imports (ensure no circular dependencies)
```
**After copying, immediately:**
```bash
cd server
# Check for issues
uv run ruff check reflector/video_platforms/
# Fix any import errors or type issues
```
### Phase 2: Adapt to Current Main (2-3 hours)
**2.1 Settings Integration**
File: `server/reflector/settings.py`
Add at the appropriate location (near existing Whereby settings):
```python
# Daily.co API Integration (NEW)
DAILY_API_KEY: str | None = None
DAILY_WEBHOOK_SECRET: str | None = None
DAILY_SUBDOMAIN: str | None = None
AWS_DAILY_S3_BUCKET: str | None = None
AWS_DAILY_S3_REGION: str = "us-west-2"
AWS_DAILY_ROLE_ARN: str | None = None
# Platform Migration Feature Flags (NEW)
DAILY_MIGRATION_ENABLED: bool = False # Conservative default
DAILY_MIGRATION_ROOM_IDS: list[str] = []
DEFAULT_VIDEO_PLATFORM: Literal["whereby", "daily"] = "whereby"
```
**2.2 Database Migration**
⚠️ **CRITICAL: Do NOT copy migration from reference**
Generate new migration:
```bash
cd server
uv run alembic revision -m "add_platform_support"
```
Edit the generated migration file to add `platform` column:
```python
def upgrade():
with op.batch_alter_table("room", schema=None) as batch_op:
batch_op.add_column(
sa.Column("platform", sa.String(), nullable=False, server_default="whereby")
)
with op.batch_alter_table("meeting", schema=None) as batch_op:
batch_op.add_column(
sa.Column("platform", sa.String(), nullable=False, server_default="whereby")
)
```
**2.3 Update Database Models**
File: `server/reflector/db/rooms.py`
Add platform field (preserve all existing fields from main):
```python
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from reflector.video_platforms.models import Platform
class Room:
# ... ALL existing fields from main (calendar, webhooks, etc.) ...
# NEW: Platform field
platform: "Platform" = sqlalchemy.Column(
sqlalchemy.String,
nullable=False,
server_default="whereby",
)
```
File: `server/reflector/db/meetings.py`
Same approach - add platform field, preserve everything from main.
**2.4 Integrate Platform Abstraction into rooms.py**
⚠️ **This is the most delicate part - main has calendar/webhook features**
File: `server/reflector/views/rooms.py`
Strategy:
1. Add imports at top
2. Modify meeting creation logic only
3. Preserve all calendar/webhook/ICS logic from main
```python
# Add imports
from reflector.video_platforms import (
create_platform_client,
get_platform_for_room,
)
# In create_meeting endpoint:
# OLD: Direct Whereby API calls
# NEW: Platform abstraction
# Find the meeting creation section and replace:
platform = get_platform_for_room(room.id)
client = create_platform_client(platform)
meeting_data = await client.create_meeting(
room_name_prefix=room.name,
end_date=meeting_data.end_date,
room=room,
)
# Then create Meeting record with meeting_data.platform, meeting_data.meeting_id, etc.
```
**2.5 Add Daily.co Webhook Handler**
Copy from reference, minimal changes needed:
```bash
cp reflector-dailyco-reference/server/reflector/views/daily.py \
server/reflector/views/
```
Register in `server/reflector/app.py`:
```python
from reflector.views import daily
app.include_router(daily.router, prefix="/v1/daily", tags=["daily"])
```
**2.6 Add Recording Processing Task**
File: `server/reflector/worker/process.py`
Add the `process_recording_from_url` task from reference (copy the function).
### Phase 3: Frontend Adaptation (3-4 hours)
**3.1 Determine Current API Client Pattern**
First, check how main currently makes API calls:
```bash
cd www
grep -r "api\." app/ | head -20
# Look for patterns like: api.v1Something()
```
**3.2 Create Components**
Copy component structure from reference but **rewrite all API calls**:
```bash
mkdir -p www/app/[roomName]/components
```
Files to create:
- `RoomContainer.tsx` - Platform router (mostly copy-safe, just fix imports)
- `DailyRoom.tsx` - Needs React Query API calls
- `WherebyRoom.tsx` - Extract current room page logic
**Example React Query pattern** (adapt to your actual API):
```typescript
import { api } from '@/app/api/client'
// In DailyRoom.tsx
const handleConsent = async () => {
try {
await api.v1MeetingAudioConsent({
path: { meeting_id: meeting.id },
body: { consent: true },
})
// ...
} catch (error) {
// ...
}
}
```
**3.3 Add Daily.co Dependency**
Check current package manager:
```bash
cd www
ls package-lock.json yarn.lock pnpm-lock.yaml
```
Then install:
```bash
# If using pnpm
pnpm add @daily-co/daily-js@^0.81.0
# If using yarn
yarn add @daily-co/daily-js@^0.81.0
```
**3.4 Update TypeScript Types**
After backend changes, regenerate types:
```bash
cd www
pnpm openapi # or yarn openapi
```
This should pick up the new `platform` field on Meeting type.
### Phase 4: Testing (2-3 hours)
**4.1 Copy Test Structure**
```bash
cp reflector-dailyco-reference/server/tests/test_video_platforms.py \
server/tests/
cp reflector-dailyco-reference/server/tests/test_daily_webhook.py \
server/tests/
```
**4.2 Fix Test Imports and Fixtures**
Update imports to match current test infrastructure:
- Check `server/tests/conftest.py` for fixture patterns
- Update database access patterns if changed
- Fix any import errors
**4.3 Run Tests**
```bash
cd server
# Run with environment variables for Mac
REDIS_HOST=localhost \
CELERY_BROKER_URL=redis://localhost:6379/1 \
CELERY_RESULT_BACKEND=redis://localhost:6379/1 \
uv run pytest tests/test_video_platforms.py -v
```
### Phase 5: Environment Configuration
**Update `server/env.example`:**
Add at the end:
```bash
# Daily.co API Integration
DAILY_API_KEY=your-daily-api-key
DAILY_WEBHOOK_SECRET=your-daily-webhook-secret
DAILY_SUBDOMAIN=your-subdomain
AWS_DAILY_S3_BUCKET=your-daily-bucket
AWS_DAILY_S3_REGION=us-west-2
AWS_DAILY_ROLE_ARN=arn:aws:iam::ACCOUNT:role/DailyRecording
# Platform Selection
DAILY_MIGRATION_ENABLED=false # Master switch
DAILY_MIGRATION_ROOM_IDS=[] # Specific room IDs
DEFAULT_VIDEO_PLATFORM=whereby # Default platform
```
## Decision Tree: Copy vs Adapt vs Rewrite
```
┌─ Is it pure abstraction logic? (base.py, registry.py, models.py)
│ YES → Copy directly, review imports
│ NO → Continue ↓
├─ Does it touch database models?
│ YES → Adapt carefully, preserve main's fields
│ NO → Continue ↓
├─ Does it make API calls on frontend?
│ YES → Rewrite using React Query
│ NO → Continue ↓
├─ Is it a database migration?
│ YES → Generate fresh from current schema
│ NO → Continue ↓
└─ Does it touch rooms.py or core business logic?
YES → Merge carefully, preserve calendar/webhooks
NO → Safe to adapt from reference
```
## Verification Checklist
After each phase, verify:
**Phase 1 (Abstraction Layer):**
- [ ] `uv run ruff check server/reflector/video_platforms/` passes
- [ ] No circular import errors
- [ ] Can import `from reflector.video_platforms import create_platform_client`
**Phase 2 (Backend Integration):**
- [ ] `uv run ruff check server/` passes
- [ ] Migration file generated (not copied)
- [ ] Room and Meeting models have platform field
- [ ] rooms.py still has calendar/webhook features
**Phase 3 (Frontend):**
- [ ] `pnpm lint` passes
- [ ] No TypeScript errors
- [ ] No `@ts-ignore` for platform field
- [ ] API calls use React Query patterns
**Phase 4 (Testing):**
- [ ] Tests can be collected: `pytest tests/test_video_platforms.py --collect-only`
- [ ] Database fixtures work
- [ ] Mock platform works
**Phase 5 (Config):**
- [ ] env.example has Daily.co variables
- [ ] settings.py has all new variables
- [ ] No duplicate variable definitions
## Common Pitfalls
### 1. Database Schema Conflicts
**Problem:** Reference removed fields that main has (calendar, webhooks)
**Solution:** Always preserve main's fields, only add platform field
### 2. Migration Conflicts
**Problem:** Reference migration has wrong `down_revision`
**Solution:** Always generate fresh migration from current main
### 3. Frontend API Calls
**Problem:** Reference uses old API client patterns
**Solution:** Check current main's API usage, replicate that pattern
### 4. Import Errors
**Problem:** Circular imports with TYPE_CHECKING
**Solution:** Use `if TYPE_CHECKING:` for Room/Meeting imports in video_platforms
### 5. Test Database Issues
**Problem:** Tests fail with "could not translate host name 'postgres'"
**Solution:** Use environment variables: `REDIS_HOST=localhost DATABASE_URL=...`
### 6. Preserved Features Broken
**Problem:** Calendar/webhook features stop working
**Solution:** Carefully review rooms.py diff, only change meeting creation, not calendar logic
## File Modification Summary
**New files (can copy):**
- `server/reflector/video_platforms/*.py` (entire directory)
- `server/reflector/views/daily.py`
- `server/tests/test_video_platforms.py`
- `server/tests/test_daily_webhook.py`
- `www/app/[roomName]/components/RoomContainer.tsx`
- `www/app/[roomName]/components/DailyRoom.tsx`
- `www/app/[roomName]/components/WherebyRoom.tsx`
**Modified files (careful merging):**
- `server/reflector/settings.py` - Add Daily.co settings
- `server/reflector/db/rooms.py` - Add platform field
- `server/reflector/db/meetings.py` - Add platform field
- `server/reflector/views/rooms.py` - Integrate platform abstraction
- `server/reflector/worker/process.py` - Add process_recording_from_url
- `server/reflector/app.py` - Register daily router
- `server/env.example` - Add Daily.co variables
- `www/app/[roomName]/page.tsx` - Use RoomContainer
- `www/package.json` - Add @daily-co/daily-js
**Generated files (do not copy):**
- `server/migrations/versions/XXXXXX_add_platform_support.py` - Generate fresh
## Success Metrics
Implementation is complete when:
- [ ] All tests pass (including new platform tests)
- [ ] Linting passes (ruff, pnpm lint)
- [ ] Migration applies cleanly: `uv run alembic upgrade head`
- [ ] Can create Whereby meeting (existing flow unchanged)
- [ ] Can create Daily.co meeting (with env vars set)
- [ ] Frontend loads without TypeScript errors
- [ ] No features from main were accidentally removed
## Getting Help
**Reference documentation locations:**
- Implementation plan: `PLAN.md`
- Reference implementation: `./reflector-dailyco-reference/`
- Current main codebase: `./ ` (current directory)
**Compare implementations:**
```bash
# Compare specific files
diff reflector-dailyco-reference/server/reflector/video_platforms/base.py \
server/reflector/video_platforms/base.py
# See what changed in rooms.py between reference branch point and now
git log --oneline --since="2025-08-01" -- server/reflector/views/rooms.py
```
**Key insight:** The reference branch validates the approach and provides working code patterns, but you're implementing fresh against current main to avoid merge conflicts and preserve all new features.

2517
PLAN.md Normal file

File diff suppressed because it is too large Load Diff

613
server/DAILYCO_TEST.md Normal file
View File

@@ -0,0 +1,613 @@
# Daily.co Integration Test Plan
## ✅ IMPLEMENTATION STATUS: Real Transcription Active
**This test validates Daily.co multitrack recording integration with REAL transcription/diarization.**
The implementation includes complete audio processing pipeline:
- **Multitrack recordings** from Daily.co S3 (separate audio stream per participant)
- **PyAV-based audio mixdown** with PTS-based track alignment
- **Real transcription** via Modal GPU backend (Whisper)
- **Real diarization** via Modal GPU backend (speaker identification)
- **Per-track transcription** with timestamp synchronization
- **Complete database entities** (recording, transcript, topics, participants, words)
**Processing pipeline** (`PipelineMainMultitrack`):
1. Download all audio tracks from Daily.co S3
2. Align tracks by PTS (presentation timestamp) to handle late joiners
3. Mix tracks into single audio file for unified playback
4. Transcribe each track individually with proper offset handling
5. Perform diarization on mixed audio
6. Generate topics, summaries, and word-level timestamps
7. Convert audio to MP3 and generate waveform visualization
**Note:** A stub processor (`process_daily_recording`) exists for testing webhook flow without GPU costs, but the production code path uses `process_multitrack_recording` with full ML pipeline.
---
## Prerequisites
**1. Environment Variables** (check in `.env.development.local`):
```bash
# Daily.co API Configuration
DAILY_API_KEY=<key>
DAILY_SUBDOMAIN=monadical
DAILY_WEBHOOK_SECRET=<base64-encoded-secret>
AWS_DAILY_S3_BUCKET=reflector-dailyco-local
AWS_DAILY_S3_REGION=us-east-1
AWS_DAILY_ROLE_ARN=arn:aws:iam::950402358378:role/DailyCo
DAILY_MIGRATION_ENABLED=true
DAILY_MIGRATION_ROOM_IDS=["552640fd-16f2-4162-9526-8cf40cd2357e"]
# Transcription/Diarization Backend (Required for real processing)
DIARIZATION_BACKEND=modal
DIARIZATION_MODAL_API_KEY=<modal-api-key>
# TRANSCRIPTION_BACKEND is not explicitly set (uses default/modal)
```
**2. Services Running:**
```bash
docker compose ps # server, postgres, redis, worker, beat should be UP
```
**IMPORTANT:** Worker and beat services MUST be running for transcription processing:
```bash
docker compose up -d worker beat
```
**3. ngrok Tunnel for Webhooks:**
```bash
# Start ngrok (if not already running)
ngrok http 1250 --log=stdout > /tmp/ngrok.log 2>&1 &
# Get public URL
curl -s http://localhost:4040/api/tunnels | python3 -c "import sys, json; data=json.load(sys.stdin); print(data['tunnels'][0]['public_url'])"
```
**Current ngrok URL:** `https://0503947384a3.ngrok-free.app` (as of last registration)
**4. Webhook Created:**
```bash
cd server
uv run python scripts/recreate_daily_webhook.py https://0503947384a3.ngrok-free.app/v1/daily/webhook
# Verify: "Created webhook <uuid> (state: ACTIVE)"
```
**Current webhook status:** ✅ ACTIVE (webhook ID: dad5ad16-ceca-488e-8fc5-dae8650b51d0)
---
## Test 1: Database Configuration
**Check room platform:**
```bash
docker-compose exec -T postgres psql -U reflector -d reflector -c \
"SELECT id, name, platform, recording_type FROM room WHERE name = 'test2';"
```
**Expected:**
```
id: 552640fd-16f2-4162-9526-8cf40cd2357e
name: test2
platform: whereby # DB value (overridden by env var DAILY_MIGRATION_ROOM_IDS)
recording_type: cloud
```
**Clear old meetings:**
```bash
docker-compose exec -T postgres psql -U reflector -d reflector -c \
"UPDATE meeting SET is_active = false WHERE room_id = '552640fd-16f2-4162-9526-8cf40cd2357e';"
```
---
## Test 2: Meeting Creation with Auto-Recording
**Create meeting:**
```bash
curl -s -X POST http://localhost:1250/v1/rooms/test2/meeting \
-H "Content-Type: application/json" \
-d '{"allow_duplicated":false}' | python3 -m json.tool
```
**Expected Response:**
```json
{
"room_name": "test2-YYYYMMDDHHMMSS", // Includes "test2" prefix!
"room_url": "https://monadical.daily.co/test2-...?t=<JWT_TOKEN>", // Has token!
"platform": "daily",
"recording_type": "cloud" // DB value (Whereby-specific)
}
```
**Decode token to verify auto-recording:**
```bash
# Extract token from room_url, decode JWT payload
echo "<token>" | python3 -c "
import sys, json, base64
token = sys.stdin.read().strip()
payload = token.split('.')[1] + '=' * (4 - len(token.split('.')[1]) % 4)
print(json.dumps(json.loads(base64.b64decode(payload)), indent=2))
"
```
**Expected token payload:**
```json
{
"r": "test2-YYYYMMDDHHMMSS", // Room name
"sr": true, // start_recording: true ✅
"d": "...", // Domain ID
"iat": 1234567890
}
```
---
## Test 3: Daily.co API Verification
**Check room configuration:**
```bash
ROOM_NAME="<from previous step>"
curl -s -X GET "https://api.daily.co/v1/rooms/$ROOM_NAME" \
-H "Authorization: Bearer $DAILY_API_KEY" | python3 -m json.tool
```
**Expected config:**
```json
{
"config": {
"enable_recording": "raw-tracks", // ✅
"recordings_bucket": {
"bucket_name": "reflector-dailyco-local",
"bucket_region": "us-east-1",
"assume_role_arn": "arn:aws:iam::950402358378:role/DailyCo"
}
}
}
```
---
## Test 4: Browser UI Test (Playwright MCP)
**Using Claude Code MCP tools:**
**Load room:**
```
Use: mcp__playwright__browser_navigate
Input: {"url": "http://localhost:3000/test2"}
Then wait 12 seconds for iframe to load
```
**Verify Daily.co iframe loaded:**
```
Use: mcp__playwright__browser_snapshot
Expected in snapshot:
- iframe element with src containing "monadical.daily.co"
- Daily.co pre-call UI visible
```
**Take screenshot:**
```
Use: mcp__playwright__browser_take_screenshot
Input: {"filename": "test2-before-join.png"}
Expected: Daily.co pre-call UI with "Join" button visible
```
**Join meeting:**
```
Note: Daily.co iframe interaction requires clicking inside iframe.
Use: mcp__playwright__browser_click
Input: {"element": "Join button in Daily.co iframe", "ref": "<ref-from-snapshot>"}
Then wait 5 seconds for call to connect
```
**Verify in-call:**
```
Use: mcp__playwright__browser_take_screenshot
Input: {"filename": "test2-in-call.png"}
Expected: "Waiting for others to join" or participant video visible
```
**Leave meeting:**
```
Use: mcp__playwright__browser_click
Input: {"element": "Leave button in Daily.co iframe", "ref": "<ref-from-snapshot>"}
```
---
**Alternative: JavaScript snippets (for manual testing):**
```javascript
await page.goto('http://localhost:3000/test2');
await new Promise(f => setTimeout(f, 12000)); // Wait for load
// Verify iframe
const iframes = document.querySelectorAll('iframe');
// Expected: 1 iframe with src containing "monadical.daily.co"
// Screenshot
await page.screenshot({ path: 'test2-before-join.png' });
// Join
await page.locator('iframe').contentFrame().getByRole('button', { name: 'Join' }).click();
await new Promise(f => setTimeout(f, 5000));
// In-call screenshot
await page.screenshot({ path: 'test2-in-call.png' });
// Leave
await page.locator('iframe').contentFrame().getByRole('button', { name: 'Leave' }).click();
```
---
## Test 5: Webhook Verification
**Check server logs for webhooks:**
```bash
docker-compose logs --since 15m server 2>&1 | grep -i "participant joined\|recording started"
```
**Expected logs:**
```
[info] Participant joined | meeting_id=... | num_clients=1 | recording_type=cloud | recording_trigger=automatic-2nd-participant
[info] Recording started | meeting_id=... | recording_id=... | platform=daily
```
**Check Daily.co webhook delivery logs:**
```bash
curl -s -X GET "https://api.daily.co/v1/logs/webhooks?limit=20" \
-H "Authorization: Bearer $DAILY_API_KEY" | python3 -c "
import sys, json
logs = json.load(sys.stdin)
for log in logs[:10]:
req = json.loads(log['request'])
room = req.get('payload', {}).get('room') or req.get('payload', {}).get('room_name', 'N/A')
print(f\"{req['type']:30s} | room: {room:30s} | status: {log['status']}\")
"
```
**Expected output:**
```
participant.joined | room: test2-YYYYMMDDHHMMSS | status: 200
recording.started | room: test2-YYYYMMDDHHMMSS | status: 200
participant.left | room: test2-YYYYMMDDHHMMSS | status: 200
recording.ready-to-download | room: test2-YYYYMMDDHHMMSS | status: 200
```
**Check database updated:**
```bash
docker-compose exec -T postgres psql -U reflector -d reflector -c \
"SELECT room_name, num_clients FROM meeting WHERE room_name LIKE 'test2-%' ORDER BY end_date DESC LIMIT 1;"
```
**Expected:**
```
room_name: test2-YYYYMMDDHHMMSS
num_clients: 0 // After participant left
```
---
## Test 6: Recording in S3
**List recent recordings:**
```bash
curl -s -X GET "https://api.daily.co/v1/recordings" \
-H "Authorization: Bearer $DAILY_API_KEY" | python3 -c "
import sys, json
data = json.load(sys.stdin)
for rec in data.get('data', [])[:5]:
if 'test2-' in rec.get('room_name', ''):
print(f\"Room: {rec['room_name']}\")
print(f\"Status: {rec['status']}\")
print(f\"Duration: {rec.get('duration', 0)}s\")
print(f\"S3 key: {rec.get('s3key', 'N/A')}\")
print(f\"Tracks: {len(rec.get('tracks', []))} files\")
for track in rec.get('tracks', []):
print(f\" - {track['type']}: {track['s3Key'].split('/')[-1]} ({track['size']} bytes)\")
print()
"
```
**Expected output:**
```
Room: test2-20251009192341
Status: finished
Duration: ~30-120s
S3 key: monadical/test2-20251009192341/1760037914930
Tracks: 2 files
- audio: 1760037914930-<uuid>-cam-audio-1760037915265 (~400 KB)
- video: 1760037914930-<uuid>-cam-video-1760037915269 (~10-30 MB)
```
**Verify S3 path structure:**
- `monadical/` - Daily.co subdomain
- `test2-20251009192341/` - Reflector room name + timestamp
- `<timestamp>-<participant-uuid>-<media-type>-<track-start>.webm` - Individual track files
---
## Test 7: Database Check - Recording and Transcript
**Check recording created:**
```bash
docker-compose exec -T postgres psql -U reflector -d reflector -c \
"SELECT id, bucket_name, object_key, status, meeting_id, recorded_at
FROM recording
ORDER BY recorded_at DESC LIMIT 1;"
```
**Expected:**
```
id: <recording-id-from-webhook>
bucket_name: reflector-dailyco-local
object_key: monadical/test2-<timestamp>/<recording-timestamp>-<uuid>-cam-audio-<track-start>.webm
status: completed
meeting_id: <meeting-id>
recorded_at: <recent-timestamp>
```
**Check transcript created:**
```bash
docker compose exec -T postgres psql -U reflector -d reflector -c \
"SELECT id, title, status, duration, recording_id, meeting_id, room_id
FROM transcript
ORDER BY created_at DESC LIMIT 1;"
```
**Expected (REAL transcription):**
```
id: <transcript-id>
title: <AI-generated title based on actual conversation content>
status: uploaded (audio file processed and available)
duration: <actual meeting duration in seconds>
recording_id: <same-as-recording-id-above>
meeting_id: <meeting-id>
room_id: 552640fd-16f2-4162-9526-8cf40cd2357e
```
**Note:** Title and content will reflect the ACTUAL conversation, not mock data. Processing time depends on recording length and GPU backend availability (Modal).
**Verify audio file exists:**
```bash
ls -lh data/<transcript-id>/upload.webm
```
**Expected:**
```
-rw-r--r-- 1 user staff ~100-200K Oct 10 18:48 upload.webm
```
**Check transcript topics (REAL transcription):**
```bash
TRANSCRIPT_ID=$(docker compose exec -T postgres psql -U reflector -d reflector -t -c \
"SELECT id FROM transcript ORDER BY created_at DESC LIMIT 1;")
docker compose exec -T postgres psql -U reflector -d reflector -c \
"SELECT
jsonb_array_length(topics) as num_topics,
jsonb_array_length(participants) as num_participants,
short_summary,
title
FROM transcript
WHERE id = '$TRANSCRIPT_ID';"
```
**Expected (REAL data):**
```
num_topics: <varies based on conversation>
num_participants: <actual number of participants who spoke>
short_summary: <AI-generated summary of actual conversation>
title: <AI-generated title based on content>
```
**Check topics contain actual transcription:**
```bash
docker compose exec -T postgres psql -U reflector -d reflector -c \
"SELECT topics->0->'title', topics->0->'summary', topics->0->'transcript'
FROM transcript
ORDER BY created_at DESC LIMIT 1;" | head -20
```
**Expected output:** Will contain the ACTUAL transcribed conversation from the Daily.co meeting, not mock data.
**Check participants:**
```bash
docker compose exec -T postgres psql -U reflector -d reflector -c \
"SELECT participants FROM transcript ORDER BY created_at DESC LIMIT 1;" \
| python3 -c "import sys, json; data=json.loads(sys.stdin.read()); print(json.dumps(data, indent=2))"
```
**Expected (REAL diarization):**
```json
[
{
"id": "<uuid>",
"speaker": 0,
"name": "Speaker 1"
},
{
"id": "<uuid>",
"speaker": 1,
"name": "Speaker 2"
}
]
```
**Note:** Speaker names will be generic ("Speaker 1", "Speaker 2", etc.) as determined by the diarization backend. Number of participants depends on how many actually spoke during the meeting.
**Check word-level data:**
```bash
docker compose exec -T postgres psql -U reflector -d reflector -c \
"SELECT jsonb_array_length(topics->0->'words') as num_words_first_topic
FROM transcript
ORDER BY created_at DESC LIMIT 1;"
```
**Expected:**
```
num_words_first_topic: <varies based on actual conversation length and topic chunking>
```
**Verify speaker diarization in words:**
```bash
docker compose exec -T postgres psql -U reflector -d reflector -c \
"SELECT
topics->0->'words'->0->>'text' as first_word,
topics->0->'words'->0->>'speaker' as speaker,
topics->0->'words'->0->>'start' as start_time,
topics->0->'words'->0->>'end' as end_time
FROM transcript
ORDER BY created_at DESC LIMIT 1;"
```
**Expected (REAL transcription):**
```
first_word: <actual first word from transcription>
speaker: 0, 1, 2, ... (actual speaker ID from diarization)
start_time: <actual timestamp in seconds>
end_time: <actual end timestamp>
```
**Note:** All timestamps and speaker IDs are from real transcription/diarization, synchronized across tracks.
---
## Test 8: Recording Type Verification
**Check what Daily.co received:**
```bash
curl -s -X GET "https://api.daily.co/v1/rooms/test2-<timestamp>" \
-H "Authorization: Bearer $DAILY_API_KEY" | python3 -m json.tool | grep "enable_recording"
```
**Expected:**
```json
"enable_recording": "raw-tracks"
```
**NOT:** `"enable_recording": "cloud"` (that would be wrong - we want raw tracks)
---
## Troubleshooting
### Issue: No webhooks received
**Check webhook state:**
```bash
curl -s -X GET "https://api.daily.co/v1/webhooks" \
-H "Authorization: Bearer $DAILY_API_KEY" | python3 -m json.tool
```
**If state is FAILED:**
```bash
cd server
uv run python scripts/recreate_daily_webhook.py https://<ngrok-url>/v1/daily/webhook
```
### Issue: Webhooks return 422
**Check server logs:**
```bash
docker-compose logs --tail=50 server | grep "Failed to parse webhook event"
```
**Common cause:** Event structure mismatch. Daily.co events use:
```json
{
"version": "1.0.0",
"type": "participant.joined",
"payload": {...}, // NOT "data"
"event_ts": 123.456 // NOT "ts"
}
```
### Issue: Recording not starting
1. **Check token has `sr: true`:**
- Decode JWT token from room_url query param
- Should contain `"sr": true`
2. **Check Daily.co room config:**
- `enable_recording` must be set (not false)
- For raw-tracks: must be exactly `"raw-tracks"`
3. **Check participant actually joined:**
- Logs should show "Participant joined"
- Must click "Join" button, not just pre-call screen
### Issue: Recording in S3 but wrong format
**Daily.co recording types:**
- `"cloud"` → Single MP4 file (`download_link` in webhook)
- `"raw-tracks"` → Multiple WebM files (`tracks` array in webhook)
- `"raw-tracks-audio-only"` → Only audio WebM files
**Current implementation:** Always uses `"raw-tracks"` (better for transcription)
---
## Quick Validation Commands
**One-liner to verify everything:**
```bash
# 1. Check room exists
docker-compose exec -T postgres psql -U reflector -d reflector -c \
"SELECT name, platform FROM room WHERE name = 'test2';" && \
# 2. Create meeting
MEETING=$(curl -s -X POST http://localhost:1250/v1/rooms/test2/meeting \
-H "Content-Type: application/json" -d '{"allow_duplicated":false}') && \
echo "$MEETING" | python3 -c "import sys,json; m=json.load(sys.stdin); print(f'Room: {m[\"room_name\"]}\nURL: {m[\"room_url\"][:80]}...')" && \
# 3. Check Daily.co config
ROOM_NAME=$(echo "$MEETING" | python3 -c "import sys,json; print(json.load(sys.stdin)['room_name'])") && \
curl -s -X GET "https://api.daily.co/v1/rooms/$ROOM_NAME" \
-H "Authorization: Bearer $DAILY_API_KEY" | python3 -c "import sys,json; print(f'Recording: {json.load(sys.stdin)[\"config\"][\"enable_recording\"]}')"
```
**Expected output:**
```
name: test2, platform: whereby
Room: test2-20251009192341
URL: https://monadical.daily.co/test2-20251009192341?t=eyJhbGc...
Recording: raw-tracks
```
---
## Success Criteria Checklist
- [x] Room name includes Reflector room prefix (`test2-...`)
- [x] Meeting URL contains JWT token (`?t=...`)
- [x] Token has `sr: true` (auto-recording enabled)
- [x] Daily.co room config: `enable_recording: "raw-tracks"`
- [x] Browser loads Daily.co interface (not Whereby)
- [x] Recording auto-starts when participant joins
- [x] Webhooks received: participant.joined, recording.started, participant.left, recording.ready-to-download
- [x] Recording status: `finished`
- [x] S3 contains 2 files: audio (.webm) and video (.webm)
- [x] S3 path: `monadical/test2-{timestamp}/{recording-start-ts}-{participant-uuid}-cam-{audio|video}-{track-start-ts}`
- [x] Database `num_clients` increments/decrements correctly
- [x] **Database recording entry created** with correct S3 path and status `completed`
- [ ] **Database transcript entry created** with status `uploaded`
- [ ] **Audio file downloaded** to `data/{transcript_id}/upload.webm`
- [ ] **Transcript has REAL data**: AI-generated title based on conversation
- [ ] **Transcript has topics** generated from actual content
- [ ] **Transcript has participants** with proper speaker diarization
- [ ] **Topics contain word-level data** with accurate timestamps and speaker IDs
- [ ] **Total duration** matches actual meeting length
- [ ] **MP3 and waveform files generated** by file processing pipeline
- [ ] **Frontend transcript page loads** without "Failed to load audio" error
- [ ] **Audio player functional** with working playback and waveform visualization
- [ ] **Multitrack processing completed** without errors in worker logs
- [ ] **Modal GPU backends accessible** (transcription and diarization)

View File

@@ -6,7 +6,7 @@ ENV PYTHONUNBUFFERED=1 \
# builder install base dependencies # builder install base dependencies
WORKDIR /tmp WORKDIR /tmp
RUN apt-get update && apt-get install -y curl && apt-get clean RUN apt-get update && apt-get install -y curl ffmpeg && apt-get clean
ADD https://astral.sh/uv/install.sh /uv-installer.sh ADD https://astral.sh/uv/install.sh /uv-installer.sh
RUN sh /uv-installer.sh && rm /uv-installer.sh RUN sh /uv-installer.sh && rm /uv-installer.sh
ENV PATH="/root/.local/bin/:$PATH" ENV PATH="/root/.local/bin/:$PATH"

View File

@@ -1,29 +1,3 @@
## API Key Management
### Finding Your User ID
```bash
# Get your OAuth sub (user ID) - requires authentication
curl -H "Authorization: Bearer <your_jwt>" http://localhost:1250/v1/me
# Returns: {"sub": "your-oauth-sub-here", "email": "...", ...}
```
### Creating API Keys
```bash
curl -X POST http://localhost:1250/v1/user/api-keys \
-H "Authorization: Bearer <your_jwt>" \
-H "Content-Type: application/json" \
-d '{"name": "My API Key"}'
```
### Using API Keys
```bash
# Use X-API-Key header instead of Authorization
curl -H "X-API-Key: <your_api_key>" http://localhost:1250/v1/transcripts
```
## AWS S3/SQS usage clarification ## AWS S3/SQS usage clarification
Whereby.com uploads recordings directly to our S3 bucket when meetings end. Whereby.com uploads recordings directly to our S3 bucket when meetings end.

View File

@@ -1,234 +0,0 @@
# Reflector Architecture: Whereby + Daily.co Recording Storage
## System Overview
```mermaid
graph TB
subgraph "Actors"
APP[Our App<br/>Reflector]
WHEREBY[Whereby Service<br/>External]
DAILY[Daily.co Service<br/>External]
end
subgraph "AWS S3 Buckets"
TRANSCRIPT_BUCKET[Transcript Bucket<br/>reflector-transcripts<br/>Output: Processed MP3s]
WHEREBY_BUCKET[Whereby Bucket<br/>reflector-whereby-recordings<br/>Input: Raw MP4s]
DAILY_BUCKET[Daily.co Bucket<br/>reflector-dailyco-recordings<br/>Input: Raw WebM tracks]
end
subgraph "AWS Infrastructure"
SQS[SQS Queue<br/>Whereby notifications]
end
subgraph "Database"
DB[(PostgreSQL<br/>Recordings, Transcripts, Meetings)]
end
APP -->|Write processed| TRANSCRIPT_BUCKET
APP -->|Read/Delete| WHEREBY_BUCKET
APP -->|Read/Delete| DAILY_BUCKET
APP -->|Poll| SQS
APP -->|Store metadata| DB
WHEREBY -->|Write recordings| WHEREBY_BUCKET
WHEREBY_BUCKET -->|S3 Event| SQS
WHEREBY -->|Participant webhooks<br/>room.client.joined/left| APP
DAILY -->|Write recordings| DAILY_BUCKET
DAILY -->|Recording webhook<br/>recording.ready-to-download| APP
```
**Note on Webhook vs S3 Event for Recording Processing:**
- **Whereby**: Uses S3 Events → SQS for recording availability (S3 as source of truth, no race conditions)
- **Daily.co**: Uses webhooks for recording availability (more immediate, built-in reliability)
- **Both**: Use webhooks for participant tracking (real-time updates)
## Credentials & Permissions
```mermaid
graph LR
subgraph "Master Credentials"
MASTER[TRANSCRIPT_STORAGE_AWS_*<br/>Access Key ID + Secret]
end
subgraph "Whereby Upload Credentials"
WHEREBY_CREDS[AWS_WHEREBY_ACCESS_KEY_*<br/>Access Key ID + Secret]
end
subgraph "Daily.co Upload Role"
DAILY_ROLE[DAILY_STORAGE_AWS_ROLE_ARN<br/>IAM Role ARN]
end
subgraph "Our App Uses"
MASTER -->|Read/Write/Delete| TRANSCRIPT_BUCKET[Transcript Bucket]
MASTER -->|Read/Delete| WHEREBY_BUCKET[Whereby Bucket]
MASTER -->|Read/Delete| DAILY_BUCKET[Daily.co Bucket]
MASTER -->|Poll/Delete| SQS[SQS Queue]
end
subgraph "We Give To Services"
WHEREBY_CREDS -->|Passed in API call| WHEREBY_SERVICE[Whereby Service]
WHEREBY_SERVICE -->|Write Only| WHEREBY_BUCKET
DAILY_ROLE -->|Passed in API call| DAILY_SERVICE[Daily.co Service]
DAILY_SERVICE -->|Assume Role| DAILY_ROLE
DAILY_SERVICE -->|Write Only| DAILY_BUCKET
end
```
# Video Platform Recording Integration
This document explains how Reflector receives and identifies multitrack audio recordings from different video platforms.
## Platform Comparison
| Platform | Delivery Method | Track Identification |
|----------|----------------|---------------------|
| **Daily.co** | Webhook | Explicit track list in payload |
| **Whereby** | SQS (S3 notifications) | Single file per notification |
---
## Daily.co (Webhook-based)
Daily.co uses **webhooks** to notify Reflector when recordings are ready.
### How It Works
1. **Daily.co sends webhook** when recording is ready
- Event type: `recording.ready-to-download`
- Endpoint: `/v1/daily/webhook` (`reflector/views/daily.py:46-102`)
2. **Webhook payload explicitly includes track list**:
```json
{
"recording_id": "7443ee0a-dab1-40eb-b316-33d6c0d5ff88",
"room_name": "daily-20251020193458",
"tracks": [
{
"type": "audio",
"s3Key": "monadical/daily-20251020193458/1760988935484-52f7f48b-fbab-431f-9a50-87b9abfc8255-cam-audio-1760988935922",
"size": 831843
},
{
"type": "audio",
"s3Key": "monadical/daily-20251020193458/1760988935484-a37c35e3-6f8e-4274-a482-e9d0f102a732-cam-audio-1760988943823",
"size": 408438
},
{
"type": "video",
"s3Key": "monadical/daily-20251020193458/...-video.webm",
"size": 30000000
}
]
}
```
3. **System extracts audio tracks** (`daily.py:211`):
```python
track_keys = [t.s3Key for t in tracks if t.type == "audio"]
```
4. **Triggers multitrack processing** (`daily.py:213-218`):
```python
process_multitrack_recording.delay(
bucket_name=bucket_name, # reflector-dailyco-local
room_name=room_name, # daily-20251020193458
recording_id=recording_id, # 7443ee0a-dab1-40eb-b316-33d6c0d5ff88
track_keys=track_keys # Only audio s3Keys
)
```
### Key Advantage: No Ambiguity
Even though multiple meetings may share the same S3 bucket/folder (`monadical/`), **there's no ambiguity** because:
- Each webhook payload contains the exact `s3Key` list for that specific `recording_id`
- No need to scan folders or guess which files belong together
- Each track's s3Key includes the room timestamp subfolder (e.g., `daily-20251020193458/`)
The room name includes timestamp (`daily-20251020193458`) to keep recordings organized, but **the webhook's explicit track list is what prevents mixing files from different meetings**.
### Track Timeline Extraction
Daily.co provides timing information in two places:
**1. PyAV WebM Metadata (current approach)**:
```python
# Read from WebM container stream metadata
stream.start_time = 8.130s # Meeting-relative timing
```
**2. Filename Timestamps (alternative approach, commit 3bae9076)**:
```
Filename format: {recording_start_ts}-{uuid}-cam-audio-{track_start_ts}.webm
Example: 1760988935484-52f7f48b-fbab-431f-9a50-87b9abfc8255-cam-audio-1760988935922.webm
Parse timestamps:
- recording_start_ts: 1760988935484 (Unix ms)
- track_start_ts: 1760988935922 (Unix ms)
- offset: (1760988935922 - 1760988935484) / 1000 = 0.438s
```
**Time Difference (PyAV vs Filename)**:
```
Track 0:
Filename offset: 438ms
PyAV metadata: 229ms
Difference: 209ms
Track 1:
Filename offset: 8339ms
PyAV metadata: 8130ms
Difference: 209ms
```
**Consistent 209ms delta** suggests network/encoding delay between file upload initiation (filename) and actual audio stream start (metadata).
**Current implementation uses PyAV metadata** because:
- More accurate (represents when audio actually started)
- Padding BEFORE transcription produces correct Whisper timestamps automatically
- No manual offset adjustment needed during transcript merge
### Why Re-encoding During Padding
Padding coincidentally involves re-encoding, which is important for Daily.co + Whisper:
**Problem:** Daily.co skips frames in recordings when microphone is muted or paused
- WebM containers have gaps where audio frames should be
- Whisper doesn't understand these gaps and produces incorrect timestamps
- Example: 5s of audio with 2s muted → file has frames only for 3s, Whisper thinks duration is 3s
**Solution:** Re-encoding via PyAV filter graph (`adelay` + `aresample`)
- Restores missing frames as silence
- Produces continuous audio stream without gaps
- Whisper now sees correct duration and produces accurate timestamps
**Why combined with padding:**
- Already re-encoding for padding (adding initial silence)
- More performant to do both operations in single PyAV pipeline
- Padded values needed for mixdown anyway (creating final MP3)
Implementation: `main_multitrack_pipeline.py:_apply_audio_padding_streaming()`
---
## Whereby (SQS-based)
Whereby uses **AWS SQS** (via S3 notifications) to notify Reflector when files are uploaded.
### How It Works
1. **Whereby uploads recording** to S3
2. **S3 sends notification** to SQS queue (one notification per file)
3. **Reflector polls SQS queue** (`worker/process.py:process_messages()`)
4. **System processes single file** (`worker/process.py:process_recording()`)
### Key Difference from Daily.co
**Whereby (SQS):** System receives S3 notification "file X was created" - only knows about one file at a time, would need to scan folder to find related files
**Daily.co (Webhook):** Daily explicitly tells system which files belong together in the webhook payload
---

View File

@@ -79,22 +79,19 @@ DIARIZATION_URL=https://monadical-sas--reflector-diarizer-web.modal.run
## Whereby ## Whereby
#WHEREBY_API_KEY=your-whereby-api-key #WHEREBY_API_KEY=your-whereby-api-key
#WHEREBY_WEBHOOK_SECRET=your-whereby-webhook-secret #WHEREBY_WEBHOOK_SECRET=your-whereby-webhook-secret
#WHEREBY_STORAGE_AWS_ACCESS_KEY_ID=your-aws-key #AWS_WHEREBY_ACCESS_KEY_ID=your-aws-key
#WHEREBY_STORAGE_AWS_SECRET_ACCESS_KEY=your-aws-secret #AWS_WHEREBY_ACCESS_KEY_SECRET=your-aws-secret
#AWS_PROCESS_RECORDING_QUEUE_URL=https://sqs.us-west-2.amazonaws.com/... #AWS_PROCESS_RECORDING_QUEUE_URL=https://sqs.us-west-2.amazonaws.com/...
## Daily.co ## Daily.co
#DAILY_API_KEY=your-daily-api-key #DAILY_API_KEY=your-daily-api-key
#DAILY_WEBHOOK_SECRET=your-daily-webhook-secret #DAILY_WEBHOOK_SECRET=your-daily-webhook-secret
#DAILY_SUBDOMAIN=your-subdomain #DAILY_SUBDOMAIN=your-subdomain
#DAILY_WEBHOOK_UUID= # Auto-populated by recreate_daily_webhook.py script #AWS_DAILY_S3_BUCKET=your-daily-bucket
#DAILYCO_STORAGE_AWS_ROLE_ARN=... # IAM role ARN for Daily.co S3 access #AWS_DAILY_S3_REGION=us-west-2
#DAILYCO_STORAGE_AWS_BUCKET_NAME=reflector-dailyco #AWS_DAILY_ROLE_ARN=arn:aws:iam::ACCOUNT:role/DailyRecording
#DAILYCO_STORAGE_AWS_REGION=us-west-2
## Whereby (optional separate bucket) ## Platform Selection
#WHEREBY_STORAGE_AWS_BUCKET_NAME=reflector-whereby #DAILY_MIGRATION_ENABLED=false # Enable Daily.co support
#WHEREBY_STORAGE_AWS_REGION=us-east-1 #DAILY_MIGRATION_ROOM_IDS=[] # Specific rooms to use Daily
## Platform Configuration
#DEFAULT_VIDEO_PLATFORM=whereby # Default platform for new rooms #DEFAULT_VIDEO_PLATFORM=whereby # Default platform for new rooms

View File

@@ -1,7 +1,7 @@
"""add_platform_support """add_platform_support
Revision ID: 1e49625677e4 Revision ID: 1e49625677e4
Revises: 9e3f7b2a4c8e Revises: dc035ff72fd5
Create Date: 2025-10-08 13:17:29.943612 Create Date: 2025-10-08 13:17:29.943612
""" """
@@ -13,7 +13,7 @@ from alembic import op
# revision identifiers, used by Alembic. # revision identifiers, used by Alembic.
revision: str = "1e49625677e4" revision: str = "1e49625677e4"
down_revision: Union[str, None] = "9e3f7b2a4c8e" down_revision: Union[str, None] = "dc035ff72fd5"
branch_labels: Union[str, Sequence[str], None] = None branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None depends_on: Union[str, Sequence[str], None] = None
@@ -25,8 +25,8 @@ def upgrade() -> None:
sa.Column( sa.Column(
"platform", "platform",
sa.String(), sa.String(),
nullable=True, nullable=False,
server_default=None, server_default="whereby",
) )
) )

View File

@@ -1,79 +0,0 @@
"""add daily participant session table with immutable left_at
Revision ID: 2b92a1b03caa
Revises: f8294b31f022
Create Date: 2025-11-13 20:29:30.486577
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "2b92a1b03caa"
down_revision: Union[str, None] = "f8294b31f022"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# Create table
op.create_table(
"daily_participant_session",
sa.Column("id", sa.String(), nullable=False),
sa.Column("meeting_id", sa.String(), nullable=False),
sa.Column("room_id", sa.String(), nullable=False),
sa.Column("session_id", sa.String(), nullable=False),
sa.Column("user_id", sa.String(), nullable=True),
sa.Column("user_name", sa.String(), nullable=False),
sa.Column("joined_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("left_at", sa.DateTime(timezone=True), nullable=True),
sa.ForeignKeyConstraint(["meeting_id"], ["meeting.id"], ondelete="CASCADE"),
sa.ForeignKeyConstraint(["room_id"], ["room.id"], ondelete="CASCADE"),
sa.PrimaryKeyConstraint("id"),
)
with op.batch_alter_table("daily_participant_session", schema=None) as batch_op:
batch_op.create_index(
"idx_daily_session_meeting_left", ["meeting_id", "left_at"], unique=False
)
batch_op.create_index("idx_daily_session_room", ["room_id"], unique=False)
# Create trigger function to prevent left_at from being updated once set
op.execute("""
CREATE OR REPLACE FUNCTION prevent_left_at_update()
RETURNS TRIGGER AS $$
BEGIN
IF OLD.left_at IS NOT NULL THEN
RAISE EXCEPTION 'left_at is immutable once set';
END IF;
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
""")
# Create trigger
op.execute("""
CREATE TRIGGER prevent_left_at_update_trigger
BEFORE UPDATE ON daily_participant_session
FOR EACH ROW
EXECUTE FUNCTION prevent_left_at_update();
""")
def downgrade() -> None:
# Drop trigger
op.execute(
"DROP TRIGGER IF EXISTS prevent_left_at_update_trigger ON daily_participant_session;"
)
# Drop trigger function
op.execute("DROP FUNCTION IF EXISTS prevent_left_at_update();")
# Drop indexes and table
with op.batch_alter_table("daily_participant_session", schema=None) as batch_op:
batch_op.drop_index("idx_daily_session_room")
batch_op.drop_index("idx_daily_session_meeting_left")
op.drop_table("daily_participant_session")

View File

@@ -1,38 +0,0 @@
"""add user api keys
Revision ID: 9e3f7b2a4c8e
Revises: dc035ff72fd5
Create Date: 2025-10-17 00:00:00.000000
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "9e3f7b2a4c8e"
down_revision: Union[str, None] = "dc035ff72fd5"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.create_table(
"user_api_key",
sa.Column("id", sa.String(), nullable=False),
sa.Column("user_id", sa.String(), nullable=False),
sa.Column("key_hash", sa.String(), nullable=False),
sa.Column("name", sa.String(), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.PrimaryKeyConstraint("id"),
)
with op.batch_alter_table("user_api_key", schema=None) as batch_op:
batch_op.create_index("idx_user_api_key_hash", ["key_hash"], unique=True)
batch_op.create_index("idx_user_api_key_user_id", ["user_id"], unique=False)
def downgrade() -> None:
op.drop_table("user_api_key")

View File

@@ -1,28 +0,0 @@
"""add_track_keys
Revision ID: f8294b31f022
Revises: 1e49625677e4
Create Date: 2025-10-27 18:52:17.589167
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "f8294b31f022"
down_revision: Union[str, None] = "1e49625677e4"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
with op.batch_alter_table("recording", schema=None) as batch_op:
batch_op.add_column(sa.Column("track_keys", sa.JSON(), nullable=True))
def downgrade() -> None:
with op.batch_alter_table("recording", schema=None) as batch_op:
batch_op.drop_column("track_keys")

View File

@@ -27,7 +27,6 @@ from reflector.views.transcripts_upload import router as transcripts_upload_rout
from reflector.views.transcripts_webrtc import router as transcripts_webrtc_router from reflector.views.transcripts_webrtc import router as transcripts_webrtc_router
from reflector.views.transcripts_websocket import router as transcripts_websocket_router from reflector.views.transcripts_websocket import router as transcripts_websocket_router
from reflector.views.user import router as user_router from reflector.views.user import router as user_router
from reflector.views.user_api_keys import router as user_api_keys_router
from reflector.views.user_websocket import router as user_ws_router from reflector.views.user_websocket import router as user_ws_router
from reflector.views.whereby import router as whereby_router from reflector.views.whereby import router as whereby_router
from reflector.views.zulip import router as zulip_router from reflector.views.zulip import router as zulip_router
@@ -93,7 +92,6 @@ app.include_router(transcripts_websocket_router, prefix="/v1")
app.include_router(transcripts_webrtc_router, prefix="/v1") app.include_router(transcripts_webrtc_router, prefix="/v1")
app.include_router(transcripts_process_router, prefix="/v1") app.include_router(transcripts_process_router, prefix="/v1")
app.include_router(user_router, prefix="/v1") app.include_router(user_router, prefix="/v1")
app.include_router(user_api_keys_router, prefix="/v1")
app.include_router(user_ws_router, prefix="/v1") app.include_router(user_ws_router, prefix="/v1")
app.include_router(zulip_router, prefix="/v1") app.include_router(zulip_router, prefix="/v1")
app.include_router(whereby_router, prefix="/v1") app.include_router(whereby_router, prefix="/v1")

View File

@@ -1,16 +1,14 @@
from typing import Annotated, List, Optional from typing import Annotated, Optional
from fastapi import Depends, HTTPException from fastapi import Depends, HTTPException
from fastapi.security import APIKeyHeader, OAuth2PasswordBearer from fastapi.security import OAuth2PasswordBearer
from jose import JWTError, jwt from jose import JWTError, jwt
from pydantic import BaseModel from pydantic import BaseModel
from reflector.db.user_api_keys import user_api_keys_controller
from reflector.logger import logger from reflector.logger import logger
from reflector.settings import settings from reflector.settings import settings
oauth2_scheme = OAuth2PasswordBearer(tokenUrl="token", auto_error=False) oauth2_scheme = OAuth2PasswordBearer(tokenUrl="token", auto_error=False)
api_key_header = APIKeyHeader(name="X-API-Key", auto_error=False)
jwt_public_key = open(f"reflector/auth/jwt/keys/{settings.AUTH_JWT_PUBLIC_KEY}").read() jwt_public_key = open(f"reflector/auth/jwt/keys/{settings.AUTH_JWT_PUBLIC_KEY}").read()
jwt_algorithm = settings.AUTH_JWT_ALGORITHM jwt_algorithm = settings.AUTH_JWT_ALGORITHM
@@ -28,7 +26,7 @@ class JWTException(Exception):
class UserInfo(BaseModel): class UserInfo(BaseModel):
sub: str sub: str
email: Optional[str] = None email: str
def __getitem__(self, key): def __getitem__(self, key):
return getattr(self, key) return getattr(self, key)
@@ -60,53 +58,34 @@ def authenticated(token: Annotated[str, Depends(oauth2_scheme)]):
return None return None
async def _authenticate_user( def current_user(
jwt_token: Optional[str], token: Annotated[Optional[str], Depends(oauth2_scheme)],
api_key: Optional[str], jwtauth: JWTAuth = Depends(),
jwtauth: JWTAuth, ):
) -> UserInfo | None: if token is None:
user_infos: List[UserInfo] = [] raise HTTPException(status_code=401, detail="Not authenticated")
if api_key:
user_api_key = await user_api_keys_controller.verify_key(api_key)
if user_api_key:
user_infos.append(UserInfo(sub=user_api_key.user_id, email=None))
if jwt_token:
try: try:
payload = jwtauth.verify_token(jwt_token) payload = jwtauth.verify_token(token)
sub = payload["sub"] sub = payload["sub"]
email = payload["email"] email = payload["email"]
user_infos.append(UserInfo(sub=sub, email=email)) return UserInfo(sub=sub, email=email)
except JWTError as e: except JWTError as e:
logger.error(f"JWT error: {e}") logger.error(f"JWT error: {e}")
raise HTTPException(status_code=401, detail="Invalid authentication") raise HTTPException(status_code=401, detail="Invalid authentication")
if len(user_infos) == 0:
def current_user_optional(
token: Annotated[Optional[str], Depends(oauth2_scheme)],
jwtauth: JWTAuth = Depends(),
):
# we accept no token, but if one is provided, it must be a valid one.
if token is None:
return None return None
try:
if len(set([x.sub for x in user_infos])) > 1: payload = jwtauth.verify_token(token)
raise JWTException( sub = payload["sub"]
status_code=401, email = payload["email"]
detail="Invalid authentication: more than one user provided", return UserInfo(sub=sub, email=email)
) except JWTError as e:
logger.error(f"JWT error: {e}")
return user_infos[0] raise HTTPException(status_code=401, detail="Invalid authentication")
async def current_user(
jwt_token: Annotated[Optional[str], Depends(oauth2_scheme)],
api_key: Annotated[Optional[str], Depends(api_key_header)],
jwtauth: JWTAuth = Depends(),
):
user = await _authenticate_user(jwt_token, api_key, jwtauth)
if user is None:
raise HTTPException(status_code=401, detail="Not authenticated")
return user
async def current_user_optional(
jwt_token: Annotated[Optional[str], Depends(oauth2_scheme)],
api_key: Annotated[Optional[str], Depends(api_key_header)],
jwtauth: JWTAuth = Depends(),
):
return await _authenticate_user(jwt_token, api_key, jwtauth)

View File

@@ -25,12 +25,10 @@ def get_database() -> databases.Database:
# import models # import models
import reflector.db.calendar_events # noqa import reflector.db.calendar_events # noqa
import reflector.db.daily_participant_sessions # noqa
import reflector.db.meetings # noqa import reflector.db.meetings # noqa
import reflector.db.recordings # noqa import reflector.db.recordings # noqa
import reflector.db.rooms # noqa import reflector.db.rooms # noqa
import reflector.db.transcripts # noqa import reflector.db.transcripts # noqa
import reflector.db.user_api_keys # noqa
kwargs = {} kwargs = {}
if "postgres" not in settings.DATABASE_URL: if "postgres" not in settings.DATABASE_URL:

View File

@@ -1,169 +0,0 @@
"""Daily.co participant session tracking.
Stores webhook data for participant.joined and participant.left events to provide
historical session information (Daily.co API only returns current participants).
"""
from datetime import datetime
import sqlalchemy as sa
from pydantic import BaseModel
from sqlalchemy.dialects.postgresql import insert
from reflector.db import get_database, metadata
from reflector.utils.string import NonEmptyString
daily_participant_sessions = sa.Table(
"daily_participant_session",
metadata,
sa.Column("id", sa.String, primary_key=True),
sa.Column(
"meeting_id",
sa.String,
sa.ForeignKey("meeting.id", ondelete="CASCADE"),
nullable=False,
),
sa.Column(
"room_id",
sa.String,
sa.ForeignKey("room.id", ondelete="CASCADE"),
nullable=False,
),
sa.Column("session_id", sa.String, nullable=False),
sa.Column("user_id", sa.String, nullable=True),
sa.Column("user_name", sa.String, nullable=False),
sa.Column("joined_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("left_at", sa.DateTime(timezone=True), nullable=True),
sa.Index("idx_daily_session_meeting_left", "meeting_id", "left_at"),
sa.Index("idx_daily_session_room", "room_id"),
)
class DailyParticipantSession(BaseModel):
"""Daily.co participant session record.
Tracks when a participant joined and left a meeting. Populated from webhooks:
- participant.joined: Creates record with left_at=None
- participant.left: Updates record with left_at
ID format: {meeting_id}:{user_id}:{joined_at_ms}
- Ensures idempotency (duplicate webhooks don't create duplicates)
- Allows same user to rejoin (different joined_at = different session)
Duration is calculated as: left_at - joined_at (not stored)
"""
id: NonEmptyString
meeting_id: NonEmptyString
room_id: NonEmptyString
session_id: NonEmptyString # Daily.co's session_id (identifies room session)
user_id: NonEmptyString | None = None
user_name: str
joined_at: datetime
left_at: datetime | None = None
class DailyParticipantSessionController:
"""Controller for Daily.co participant session persistence."""
async def get_by_id(self, id: str) -> DailyParticipantSession | None:
"""Get a session by its ID."""
query = daily_participant_sessions.select().where(
daily_participant_sessions.c.id == id
)
result = await get_database().fetch_one(query)
return DailyParticipantSession(**result) if result else None
async def get_open_session(
self, meeting_id: NonEmptyString, session_id: NonEmptyString
) -> DailyParticipantSession | None:
"""Get the open (not left) session for a user in a meeting."""
query = daily_participant_sessions.select().where(
sa.and_(
daily_participant_sessions.c.meeting_id == meeting_id,
daily_participant_sessions.c.session_id == session_id,
daily_participant_sessions.c.left_at.is_(None),
)
)
results = await get_database().fetch_all(query)
if len(results) > 1:
raise ValueError(
f"Multiple open sessions for daily session {session_id} in meeting {meeting_id}: "
f"found {len(results)} sessions"
)
return DailyParticipantSession(**results[0]) if results else None
async def upsert_joined(self, session: DailyParticipantSession) -> None:
"""Insert or update when participant.joined webhook arrives.
Idempotent: Duplicate webhooks with same ID are safely ignored.
Out-of-order: If left webhook arrived first, preserves left_at.
"""
query = insert(daily_participant_sessions).values(**session.model_dump())
query = query.on_conflict_do_update(
index_elements=["id"],
set_={"user_name": session.user_name},
)
await get_database().execute(query)
async def upsert_left(self, session: DailyParticipantSession) -> None:
"""Update session when participant.left webhook arrives.
Finds the open session for this user in this meeting and updates left_at.
Works around Daily.co webhook timestamp inconsistency (joined_at differs by ~4ms between webhooks).
Handles three cases:
1. Normal flow: open session exists → updates left_at
2. Out-of-order: left arrives first → creates new record with left data
3. Duplicate: left arrives again → idempotent (DB trigger prevents left_at modification)
"""
if session.left_at is None:
raise ValueError("left_at is required for upsert_left")
if session.left_at <= session.joined_at:
raise ValueError(
f"left_at ({session.left_at}) must be after joined_at ({session.joined_at})"
)
# Find existing open session (works around timestamp mismatch in webhooks)
existing = await self.get_open_session(session.meeting_id, session.session_id)
if existing:
# Update existing open session
query = (
daily_participant_sessions.update()
.where(daily_participant_sessions.c.id == existing.id)
.values(left_at=session.left_at)
)
await get_database().execute(query)
else:
# Out-of-order or first webhook: insert new record
query = insert(daily_participant_sessions).values(**session.model_dump())
query = query.on_conflict_do_nothing(index_elements=["id"])
await get_database().execute(query)
async def get_by_meeting(self, meeting_id: str) -> list[DailyParticipantSession]:
"""Get all participant sessions for a meeting (active and ended)."""
query = daily_participant_sessions.select().where(
daily_participant_sessions.c.meeting_id == meeting_id
)
results = await get_database().fetch_all(query)
return [DailyParticipantSession(**result) for result in results]
async def get_active_by_meeting(
self, meeting_id: str
) -> list[DailyParticipantSession]:
"""Get only active (not left) participant sessions for a meeting."""
query = daily_participant_sessions.select().where(
sa.and_(
daily_participant_sessions.c.meeting_id == meeting_id,
daily_participant_sessions.c.left_at.is_(None),
)
)
results = await get_database().fetch_all(query)
return [DailyParticipantSession(**result) for result in results]
daily_participant_sessions_controller = DailyParticipantSessionController()

View File

@@ -7,10 +7,8 @@ from sqlalchemy.dialects.postgresql import JSONB
from reflector.db import get_database, metadata from reflector.db import get_database, metadata
from reflector.db.rooms import Room from reflector.db.rooms import Room
from reflector.schemas.platform import WHEREBY_PLATFORM, Platform from reflector.platform_types import Platform
from reflector.utils import generate_uuid4 from reflector.utils import generate_uuid4
from reflector.utils.string import assert_equal
from reflector.video_platforms.factory import get_platform
meetings = sa.Table( meetings = sa.Table(
"meeting", "meeting",
@@ -62,7 +60,7 @@ meetings = sa.Table(
"platform", "platform",
sa.String, sa.String,
nullable=False, nullable=False,
server_default=assert_equal(WHEREBY_PLATFORM, "whereby"), server_default="whereby",
), ),
sa.Index("idx_meeting_room_id", "room_id"), sa.Index("idx_meeting_room_id", "room_id"),
sa.Index("idx_meeting_calendar_event", "calendar_event_id"), sa.Index("idx_meeting_calendar_event", "calendar_event_id"),
@@ -110,7 +108,7 @@ class Meeting(BaseModel):
is_active: bool = True is_active: bool = True
calendar_event_id: str | None = None calendar_event_id: str | None = None
calendar_metadata: dict[str, Any] | None = None calendar_metadata: dict[str, Any] | None = None
platform: Platform = WHEREBY_PLATFORM platform: Platform = "whereby"
class MeetingController: class MeetingController:
@@ -125,6 +123,7 @@ class MeetingController:
room: Room, room: Room,
calendar_event_id: str | None = None, calendar_event_id: str | None = None,
calendar_metadata: dict[str, Any] | None = None, calendar_metadata: dict[str, Any] | None = None,
platform: Platform = "whereby",
): ):
meeting = Meeting( meeting = Meeting(
id=id, id=id,
@@ -140,7 +139,7 @@ class MeetingController:
recording_trigger=room.recording_trigger, recording_trigger=room.recording_trigger,
calendar_event_id=calendar_event_id, calendar_event_id=calendar_event_id,
calendar_metadata=calendar_metadata, calendar_metadata=calendar_metadata,
platform=get_platform(room.platform), platform=platform,
) )
query = meetings.insert().values(**meeting.model_dump()) query = meetings.insert().values(**meeting.model_dump())
await get_database().execute(query) await get_database().execute(query)
@@ -148,8 +147,7 @@ class MeetingController:
async def get_all_active(self) -> list[Meeting]: async def get_all_active(self) -> list[Meeting]:
query = meetings.select().where(meetings.c.is_active) query = meetings.select().where(meetings.c.is_active)
results = await get_database().fetch_all(query) return await get_database().fetch_all(query)
return [Meeting(**result) for result in results]
async def get_by_room_name( async def get_by_room_name(
self, self,
@@ -159,14 +157,16 @@ class MeetingController:
Get a meeting by room name. Get a meeting by room name.
For backward compatibility, returns the most recent meeting. For backward compatibility, returns the most recent meeting.
""" """
end_date = getattr(meetings.c, "end_date")
query = ( query = (
meetings.select() meetings.select()
.where(meetings.c.room_name == room_name) .where(meetings.c.room_name == room_name)
.order_by(meetings.c.end_date.desc()) .order_by(end_date.desc())
) )
result = await get_database().fetch_one(query) result = await get_database().fetch_one(query)
if not result: if not result:
return None return None
return Meeting(**result) return Meeting(**result)
async def get_active(self, room: Room, current_time: datetime) -> Meeting | None: async def get_active(self, room: Room, current_time: datetime) -> Meeting | None:
@@ -189,6 +189,7 @@ class MeetingController:
result = await get_database().fetch_one(query) result = await get_database().fetch_one(query)
if not result: if not result:
return None return None
return Meeting(**result) return Meeting(**result)
async def get_all_active_for_room( async def get_all_active_for_room(
@@ -228,27 +229,17 @@ class MeetingController:
return None return None
return Meeting(**result) return Meeting(**result)
async def get_by_id( async def get_by_id(self, meeting_id: str, **kwargs) -> Meeting | None:
self, meeting_id: str, room: Room | None = None
) -> Meeting | None:
query = meetings.select().where(meetings.c.id == meeting_id) query = meetings.select().where(meetings.c.id == meeting_id)
if room:
query = query.where(meetings.c.room_id == room.id)
result = await get_database().fetch_one(query) result = await get_database().fetch_one(query)
if not result: if not result:
return None return None
return Meeting(**result) return Meeting(**result)
async def get_by_calendar_event( async def get_by_calendar_event(self, calendar_event_id: str) -> Meeting | None:
self, calendar_event_id: str, room: Room
) -> Meeting | None:
query = meetings.select().where( query = meetings.select().where(
meetings.c.calendar_event_id == calendar_event_id meetings.c.calendar_event_id == calendar_event_id
) )
if room:
query = query.where(meetings.c.room_id == room.id)
result = await get_database().fetch_one(query) result = await get_database().fetch_one(query)
if not result: if not result:
return None return None
@@ -258,7 +249,7 @@ class MeetingController:
query = meetings.update().where(meetings.c.id == meeting_id).values(**kwargs) query = meetings.update().where(meetings.c.id == meeting_id).values(**kwargs)
await get_database().execute(query) await get_database().execute(query)
async def increment_num_clients(self, meeting_id: str) -> None: async def increment_num_clients(self, meeting_id: str):
"""Atomically increment participant count.""" """Atomically increment participant count."""
query = ( query = (
meetings.update() meetings.update()
@@ -267,7 +258,7 @@ class MeetingController:
) )
await get_database().execute(query) await get_database().execute(query)
async def decrement_num_clients(self, meeting_id: str) -> None: async def decrement_num_clients(self, meeting_id: str):
"""Atomically decrement participant count (min 0).""" """Atomically decrement participant count (min 0)."""
query = ( query = (
meetings.update() meetings.update()

View File

@@ -21,7 +21,6 @@ recordings = sa.Table(
server_default="pending", server_default="pending",
), ),
sa.Column("meeting_id", sa.String), sa.Column("meeting_id", sa.String),
sa.Column("track_keys", sa.JSON, nullable=True),
sa.Index("idx_recording_meeting_id", "meeting_id"), sa.Index("idx_recording_meeting_id", "meeting_id"),
) )
@@ -29,13 +28,10 @@ recordings = sa.Table(
class Recording(BaseModel): class Recording(BaseModel):
id: str = Field(default_factory=generate_uuid4) id: str = Field(default_factory=generate_uuid4)
bucket_name: str bucket_name: str
# for single-track
object_key: str object_key: str
recorded_at: datetime recorded_at: datetime
status: Literal["pending", "processing", "completed", "failed"] = "pending" status: Literal["pending", "processing", "completed", "failed"] = "pending"
meeting_id: str | None = None meeting_id: str | None = None
# for multitrack reprocessing
track_keys: list[str] | None = None
class RecordingController: class RecordingController:

View File

@@ -1,7 +1,7 @@
import secrets import secrets
from datetime import datetime, timezone from datetime import datetime, timezone
from sqlite3 import IntegrityError from sqlite3 import IntegrityError
from typing import Literal from typing import Literal, Optional
import sqlalchemy import sqlalchemy
from fastapi import HTTPException from fastapi import HTTPException
@@ -9,7 +9,7 @@ from pydantic import BaseModel, Field
from sqlalchemy.sql import false, or_ from sqlalchemy.sql import false, or_
from reflector.db import get_database, metadata from reflector.db import get_database, metadata
from reflector.schemas.platform import Platform from reflector.platform_types import Platform
from reflector.utils import generate_uuid4 from reflector.utils import generate_uuid4
rooms = sqlalchemy.Table( rooms = sqlalchemy.Table(
@@ -54,8 +54,8 @@ rooms = sqlalchemy.Table(
sqlalchemy.Column( sqlalchemy.Column(
"platform", "platform",
sqlalchemy.String, sqlalchemy.String,
nullable=True, nullable=False,
server_default=None, server_default="whereby",
), ),
sqlalchemy.Index("idx_room_is_shared", "is_shared"), sqlalchemy.Index("idx_room_is_shared", "is_shared"),
sqlalchemy.Index("idx_room_ics_enabled", "ics_enabled"), sqlalchemy.Index("idx_room_ics_enabled", "ics_enabled"),
@@ -84,7 +84,7 @@ class Room(BaseModel):
ics_enabled: bool = False ics_enabled: bool = False
ics_last_sync: datetime | None = None ics_last_sync: datetime | None = None
ics_last_etag: str | None = None ics_last_etag: str | None = None
platform: Platform | None = None platform: Platform = "whereby"
class RoomController: class RoomController:
@@ -138,7 +138,7 @@ class RoomController:
ics_url: str | None = None, ics_url: str | None = None,
ics_fetch_interval: int = 300, ics_fetch_interval: int = 300,
ics_enabled: bool = False, ics_enabled: bool = False,
platform: Platform | None = None, platform: Optional[Platform] = None,
): ):
""" """
Add a new room Add a new room
@@ -162,7 +162,7 @@ class RoomController:
ics_url=ics_url, ics_url=ics_url,
ics_fetch_interval=ics_fetch_interval, ics_fetch_interval=ics_fetch_interval,
ics_enabled=ics_enabled, ics_enabled=ics_enabled,
platform=platform, platform=platform or "whereby",
) )
query = rooms.insert().values(**room.model_dump()) query = rooms.insert().values(**room.model_dump())
try: try:

View File

@@ -135,8 +135,6 @@ class SearchParameters(BaseModel):
user_id: str | None = None user_id: str | None = None
room_id: str | None = None room_id: str | None = None
source_kind: SourceKind | None = None source_kind: SourceKind | None = None
from_datetime: datetime | None = None
to_datetime: datetime | None = None
class SearchResultDB(BaseModel): class SearchResultDB(BaseModel):
@@ -404,14 +402,6 @@ class SearchController:
base_query = base_query.where( base_query = base_query.where(
transcripts.c.source_kind == params.source_kind transcripts.c.source_kind == params.source_kind
) )
if params.from_datetime:
base_query = base_query.where(
transcripts.c.created_at >= params.from_datetime
)
if params.to_datetime:
base_query = base_query.where(
transcripts.c.created_at <= params.to_datetime
)
if params.query_text is not None: if params.query_text is not None:
order_by = sqlalchemy.desc(sqlalchemy.text("rank")) order_by = sqlalchemy.desc(sqlalchemy.text("rank"))

View File

@@ -21,7 +21,7 @@ from reflector.db.utils import is_postgresql
from reflector.logger import logger from reflector.logger import logger
from reflector.processors.types import Word as ProcessorWord from reflector.processors.types import Word as ProcessorWord
from reflector.settings import settings from reflector.settings import settings
from reflector.storage import get_transcripts_storage from reflector.storage import get_recordings_storage, get_transcripts_storage
from reflector.utils import generate_uuid4 from reflector.utils import generate_uuid4
from reflector.utils.webvtt import topics_to_webvtt from reflector.utils.webvtt import topics_to_webvtt
@@ -186,7 +186,6 @@ class TranscriptParticipant(BaseModel):
id: str = Field(default_factory=generate_uuid4) id: str = Field(default_factory=generate_uuid4)
speaker: int | None speaker: int | None
name: str name: str
user_id: str | None = None
class Transcript(BaseModel): class Transcript(BaseModel):
@@ -624,9 +623,7 @@ class TranscriptController:
) )
if recording: if recording:
try: try:
await get_transcripts_storage().delete_file( await get_recordings_storage().delete_file(recording.object_key)
recording.object_key, bucket=recording.bucket_name
)
except Exception as e: except Exception as e:
logger.warning( logger.warning(
"Failed to delete recording object from S3", "Failed to delete recording object from S3",
@@ -728,13 +725,11 @@ class TranscriptController:
""" """
Download audio from storage Download audio from storage
""" """
storage = get_transcripts_storage() transcript.audio_mp3_filename.write_bytes(
try: await get_transcripts_storage().get_file(
with open(transcript.audio_mp3_filename, "wb") as f: transcript.storage_audio_path,
await storage.stream_to_fileobj(transcript.storage_audio_path, f) )
except Exception: )
transcript.audio_mp3_filename.unlink(missing_ok=True)
raise
async def upsert_participant( async def upsert_participant(
self, self,

View File

@@ -1,91 +0,0 @@
import hmac
import secrets
from datetime import datetime, timezone
from hashlib import sha256
import sqlalchemy
from pydantic import BaseModel, Field
from reflector.db import get_database, metadata
from reflector.settings import settings
from reflector.utils import generate_uuid4
from reflector.utils.string import NonEmptyString
user_api_keys = sqlalchemy.Table(
"user_api_key",
metadata,
sqlalchemy.Column("id", sqlalchemy.String, primary_key=True),
sqlalchemy.Column("user_id", sqlalchemy.String, nullable=False),
sqlalchemy.Column("key_hash", sqlalchemy.String, nullable=False),
sqlalchemy.Column("name", sqlalchemy.String, nullable=True),
sqlalchemy.Column("created_at", sqlalchemy.DateTime(timezone=True), nullable=False),
sqlalchemy.Index("idx_user_api_key_hash", "key_hash", unique=True),
sqlalchemy.Index("idx_user_api_key_user_id", "user_id"),
)
class UserApiKey(BaseModel):
id: NonEmptyString = Field(default_factory=generate_uuid4)
user_id: NonEmptyString
key_hash: NonEmptyString
name: NonEmptyString | None = None
created_at: datetime = Field(default_factory=lambda: datetime.now(timezone.utc))
class UserApiKeyController:
@staticmethod
def generate_key() -> NonEmptyString:
return secrets.token_urlsafe(48)
@staticmethod
def hash_key(key: NonEmptyString) -> str:
return hmac.new(
settings.SECRET_KEY.encode(), key.encode(), digestmod=sha256
).hexdigest()
@classmethod
async def create_key(
cls,
user_id: NonEmptyString,
name: NonEmptyString | None = None,
) -> tuple[UserApiKey, NonEmptyString]:
plaintext = cls.generate_key()
api_key = UserApiKey(
user_id=user_id,
key_hash=cls.hash_key(plaintext),
name=name,
)
query = user_api_keys.insert().values(**api_key.model_dump())
await get_database().execute(query)
return api_key, plaintext
@classmethod
async def verify_key(cls, plaintext_key: NonEmptyString) -> UserApiKey | None:
key_hash = cls.hash_key(plaintext_key)
query = user_api_keys.select().where(
user_api_keys.c.key_hash == key_hash,
)
result = await get_database().fetch_one(query)
return UserApiKey(**result) if result else None
@staticmethod
async def list_by_user_id(user_id: NonEmptyString) -> list[UserApiKey]:
query = (
user_api_keys.select()
.where(user_api_keys.c.user_id == user_id)
.order_by(user_api_keys.c.created_at.desc())
)
results = await get_database().fetch_all(query)
return [UserApiKey(**r) for r in results]
@staticmethod
async def delete_key(key_id: NonEmptyString, user_id: NonEmptyString) -> bool:
query = user_api_keys.delete().where(
(user_api_keys.c.id == key_id) & (user_api_keys.c.user_id == user_id)
)
result = await get_database().execute(query)
# asyncpg returns None for DELETE, consider it success if no exception
return result is None or result > 0
user_api_keys_controller = UserApiKeyController()

View File

@@ -0,0 +1,84 @@
# Multitrack Pipeline Fix Summary
## Problem
Whisper timestamps were incorrect because it ignores leading silence in audio files. Daily.co tracks can have arbitrary amounts of silence before speech starts.
## Solution
**Pad tracks BEFORE transcription using stream metadata `start_time`**
This makes Whisper timestamps automatically correct relative to recording start.
## Key Changes in `main_multitrack_pipeline_fixed.py`
### 1. Added `pad_track_for_transcription()` method (lines 55-172)
```python
async def pad_track_for_transcription(
self,
track_data: bytes,
track_idx: int,
storage,
) -> tuple[bytes, str]:
```
- Extracts stream metadata `start_time` using PyAV
- Creates PyAV filter graph with `adelay` filter to add padding
- Stores padded track to S3 and returns URL
- Uses same audio processing library (PyAV) already in the pipeline
### 2. Modified `process()` method
#### REMOVED (lines 255-302):
- Entire filename parsing for offsets - NOT NEEDED ANYMORE
- The complex regex parsing of Daily.co filenames
- Offset adjustment after transcription
#### ADDED (lines 371-382):
- Padding step BEFORE transcription:
```python
# PAD TRACKS BEFORE TRANSCRIPTION - THIS IS THE KEY FIX!
padded_track_urls: list[str] = []
for idx, data in enumerate(track_datas):
if not data:
padded_track_urls.append("")
continue
_, padded_url = await self.pad_track_for_transcription(
data, idx, storage
)
padded_track_urls.append(padded_url)
```
#### MODIFIED (lines 385-435):
- Transcribe PADDED tracks instead of raw tracks
- Removed all timestamp offset adjustment code
- Just set speaker ID - timestamps already correct!
```python
# NO OFFSET ADJUSTMENT NEEDED!
# Timestamps are already correct because we transcribed padded tracks
# Just set speaker ID
for w in t.words:
w.speaker = idx
```
## Why This Works
1. **Stream metadata is authoritative**: Daily.co sets `start_time` in the WebM container
2. **PyAV respects metadata**: `audio_stream.start_time * audio_stream.time_base` gives seconds
3. **Padding before transcription**: Whisper sees continuous audio from time 0
4. **Automatic alignment**: Word at 51s in padded track = 51s in recording
## Testing
Process the test recording (daily-20251020193458) and verify:
- Participant 0 words appear at ~2s
- Participant 1 words appear at ~51s
- No word interleaving
- Correct chronological order
## Files
- **Original**: `main_multitrack_pipeline.py`
- **Fixed**: `main_multitrack_pipeline_fixed.py`
- **Test data**: `/Users/firfi/work/clients/monadical/reflector/1760988935484-*.webm`

View File

@@ -1 +0,0 @@
"""Pipeline modules for audio processing."""

View File

@@ -23,18 +23,23 @@ from reflector.db.transcripts import (
transcripts_controller, transcripts_controller,
) )
from reflector.logger import logger from reflector.logger import logger
from reflector.pipelines import topic_processing
from reflector.pipelines.main_live_pipeline import ( from reflector.pipelines.main_live_pipeline import (
PipelineMainBase, PipelineMainBase,
broadcast_to_sockets, broadcast_to_sockets,
task_cleanup_consent, task_cleanup_consent,
task_pipeline_post_to_zulip, task_pipeline_post_to_zulip,
) )
from reflector.pipelines.transcription_helpers import transcribe_file_with_processor from reflector.processors import (
from reflector.processors import AudioFileWriterProcessor AudioFileWriterProcessor,
TranscriptFinalSummaryProcessor,
TranscriptFinalTitleProcessor,
TranscriptTopicDetectorProcessor,
)
from reflector.processors.audio_waveform_processor import AudioWaveformProcessor from reflector.processors.audio_waveform_processor import AudioWaveformProcessor
from reflector.processors.file_diarization import FileDiarizationInput from reflector.processors.file_diarization import FileDiarizationInput
from reflector.processors.file_diarization_auto import FileDiarizationAutoProcessor from reflector.processors.file_diarization_auto import FileDiarizationAutoProcessor
from reflector.processors.file_transcript import FileTranscriptInput
from reflector.processors.file_transcript_auto import FileTranscriptAutoProcessor
from reflector.processors.transcript_diarization_assembler import ( from reflector.processors.transcript_diarization_assembler import (
TranscriptDiarizationAssemblerInput, TranscriptDiarizationAssemblerInput,
TranscriptDiarizationAssemblerProcessor, TranscriptDiarizationAssemblerProcessor,
@@ -51,6 +56,19 @@ from reflector.storage import get_transcripts_storage
from reflector.worker.webhook import send_transcript_webhook from reflector.worker.webhook import send_transcript_webhook
class EmptyPipeline:
"""Empty pipeline for processors that need a pipeline reference"""
def __init__(self, logger: structlog.BoundLogger):
self.logger = logger
def get_pref(self, k, d=None):
return d
async def emit(self, event):
pass
class PipelineMainFile(PipelineMainBase): class PipelineMainFile(PipelineMainBase):
""" """
Optimized file processing pipeline. Optimized file processing pipeline.
@@ -63,7 +81,7 @@ class PipelineMainFile(PipelineMainBase):
def __init__(self, transcript_id: str): def __init__(self, transcript_id: str):
super().__init__(transcript_id=transcript_id) super().__init__(transcript_id=transcript_id)
self.logger = logger.bind(transcript_id=self.transcript_id) self.logger = logger.bind(transcript_id=self.transcript_id)
self.empty_pipeline = topic_processing.EmptyPipeline(logger=self.logger) self.empty_pipeline = EmptyPipeline(logger=self.logger)
def _handle_gather_exceptions(self, results: list, operation: str) -> None: def _handle_gather_exceptions(self, results: list, operation: str) -> None:
"""Handle exceptions from asyncio.gather with return_exceptions=True""" """Handle exceptions from asyncio.gather with return_exceptions=True"""
@@ -244,7 +262,24 @@ class PipelineMainFile(PipelineMainBase):
async def transcribe_file(self, audio_url: str, language: str) -> TranscriptType: async def transcribe_file(self, audio_url: str, language: str) -> TranscriptType:
"""Transcribe complete file""" """Transcribe complete file"""
return await transcribe_file_with_processor(audio_url, language) processor = FileTranscriptAutoProcessor()
input_data = FileTranscriptInput(audio_url=audio_url, language=language)
# Store result for retrieval
result: TranscriptType | None = None
async def capture_result(transcript):
nonlocal result
result = transcript
processor.on(capture_result)
await processor.push(input_data)
await processor.flush()
if not result:
raise ValueError("No transcript captured")
return result
async def diarize_file(self, audio_url: str) -> list[DiarizationSegment] | None: async def diarize_file(self, audio_url: str) -> list[DiarizationSegment] | None:
"""Get diarization for file""" """Get diarization for file"""
@@ -287,31 +322,63 @@ class PipelineMainFile(PipelineMainBase):
async def detect_topics( async def detect_topics(
self, transcript: TranscriptType, target_language: str self, transcript: TranscriptType, target_language: str
) -> list[TitleSummary]: ) -> list[TitleSummary]:
return await topic_processing.detect_topics( """Detect topics from complete transcript"""
transcript, chunk_size = 300
target_language, topics: list[TitleSummary] = []
on_topic_callback=self.on_topic,
empty_pipeline=self.empty_pipeline, async def on_topic(topic: TitleSummary):
topics.append(topic)
return await self.on_topic(topic)
topic_detector = TranscriptTopicDetectorProcessor(callback=on_topic)
topic_detector.set_pipeline(self.empty_pipeline)
for i in range(0, len(transcript.words), chunk_size):
chunk_words = transcript.words[i : i + chunk_size]
if not chunk_words:
continue
chunk_transcript = TranscriptType(
words=chunk_words, translation=transcript.translation
) )
await topic_detector.push(chunk_transcript)
await topic_detector.flush()
return topics
async def generate_title(self, topics: list[TitleSummary]): async def generate_title(self, topics: list[TitleSummary]):
return await topic_processing.generate_title( """Generate title from topics"""
topics, if not topics:
on_title_callback=self.on_title, self.logger.warning("No topics for title generation")
empty_pipeline=self.empty_pipeline, return
logger=self.logger,
) processor = TranscriptFinalTitleProcessor(callback=self.on_title)
processor.set_pipeline(self.empty_pipeline)
for topic in topics:
await processor.push(topic)
await processor.flush()
async def generate_summaries(self, topics: list[TitleSummary]): async def generate_summaries(self, topics: list[TitleSummary]):
"""Generate long and short summaries from topics"""
if not topics:
self.logger.warning("No topics for summary generation")
return
transcript = await self.get_transcript() transcript = await self.get_transcript()
return await topic_processing.generate_summaries( processor = TranscriptFinalSummaryProcessor(
topics, transcript=transcript,
transcript, callback=self.on_long_summary,
on_long_summary_callback=self.on_long_summary, on_short_summary=self.on_short_summary,
on_short_summary_callback=self.on_short_summary,
empty_pipeline=self.empty_pipeline,
logger=self.logger,
) )
processor.set_pipeline(self.empty_pipeline)
for topic in topics:
await processor.push(topic)
await processor.flush()
@shared_task @shared_task
@@ -359,12 +426,7 @@ async def task_pipeline_file_process(*, transcript_id: str):
await pipeline.process(audio_file) await pipeline.process(audio_file)
except Exception as e: except Exception:
logger.error(
f"File pipeline failed for transcript {transcript_id}: {type(e).__name__}: {str(e)}",
exc_info=True,
transcript_id=transcript_id,
)
await pipeline.set_status(transcript_id, "error") await pipeline.set_status(transcript_id, "error")
raise raise

View File

@@ -17,6 +17,7 @@ from contextlib import asynccontextmanager
from typing import Generic from typing import Generic
import av import av
import boto3
from celery import chord, current_task, group, shared_task from celery import chord, current_task, group, shared_task
from pydantic import BaseModel from pydantic import BaseModel
from structlog import BoundLogger as Logger from structlog import BoundLogger as Logger
@@ -583,7 +584,6 @@ async def cleanup_consent(transcript: Transcript, logger: Logger):
consent_denied = False consent_denied = False
recording = None recording = None
meeting = None
try: try:
if transcript.recording_id: if transcript.recording_id:
recording = await recordings_controller.get_by_id(transcript.recording_id) recording = await recordings_controller.get_by_id(transcript.recording_id)
@@ -594,8 +594,8 @@ async def cleanup_consent(transcript: Transcript, logger: Logger):
meeting.id meeting.id
) )
except Exception as e: except Exception as e:
logger.error(f"Failed to fetch consent: {e}", exc_info=e) logger.error(f"Failed to get fetch consent: {e}", exc_info=e)
raise consent_denied = True
if not consent_denied: if not consent_denied:
logger.info("Consent approved, keeping all files") logger.info("Consent approved, keeping all files")
@@ -603,24 +603,25 @@ async def cleanup_consent(transcript: Transcript, logger: Logger):
logger.info("Consent denied, cleaning up all related audio files") logger.info("Consent denied, cleaning up all related audio files")
deletion_errors = [] if recording and recording.bucket_name and recording.object_key:
if recording and recording.bucket_name: s3_whereby = boto3.client(
keys_to_delete = [] "s3",
if recording.track_keys: aws_access_key_id=settings.AWS_WHEREBY_ACCESS_KEY_ID,
keys_to_delete = recording.track_keys aws_secret_access_key=settings.AWS_WHEREBY_ACCESS_KEY_SECRET,
elif recording.object_key: )
keys_to_delete = [recording.object_key]
master_storage = get_transcripts_storage()
for key in keys_to_delete:
try: try:
await master_storage.delete_file(key, bucket=recording.bucket_name) s3_whereby.delete_object(
logger.info(f"Deleted recording file: {recording.bucket_name}/{key}") Bucket=recording.bucket_name, Key=recording.object_key
)
logger.info(
f"Deleted original Whereby recording: {recording.bucket_name}/{recording.object_key}"
)
except Exception as e: except Exception as e:
error_msg = f"Failed to delete {key}: {e}" logger.error(f"Failed to delete Whereby recording: {e}", exc_info=e)
logger.error(error_msg, exc_info=e)
deletion_errors.append(error_msg)
# non-transactional, files marked for deletion not actually deleted is possible
await transcripts_controller.update(transcript, {"audio_deleted": True})
# 2. Delete processed audio from transcript storage S3 bucket
if transcript.audio_location == "storage": if transcript.audio_location == "storage":
storage = get_transcripts_storage() storage = get_transcripts_storage()
try: try:
@@ -629,28 +630,18 @@ async def cleanup_consent(transcript: Transcript, logger: Logger):
f"Deleted processed audio from storage: {transcript.storage_audio_path}" f"Deleted processed audio from storage: {transcript.storage_audio_path}"
) )
except Exception as e: except Exception as e:
error_msg = f"Failed to delete processed audio: {e}" logger.error(f"Failed to delete processed audio: {e}", exc_info=e)
logger.error(error_msg, exc_info=e)
deletion_errors.append(error_msg)
# 3. Delete local audio files
try: try:
if hasattr(transcript, "audio_mp3_filename") and transcript.audio_mp3_filename: if hasattr(transcript, "audio_mp3_filename") and transcript.audio_mp3_filename:
transcript.audio_mp3_filename.unlink(missing_ok=True) transcript.audio_mp3_filename.unlink(missing_ok=True)
if hasattr(transcript, "audio_wav_filename") and transcript.audio_wav_filename: if hasattr(transcript, "audio_wav_filename") and transcript.audio_wav_filename:
transcript.audio_wav_filename.unlink(missing_ok=True) transcript.audio_wav_filename.unlink(missing_ok=True)
except Exception as e: except Exception as e:
error_msg = f"Failed to delete local audio files: {e}" logger.error(f"Failed to delete local audio files: {e}", exc_info=e)
logger.error(error_msg, exc_info=e)
deletion_errors.append(error_msg)
if deletion_errors: logger.info("Consent cleanup done")
logger.warning(
f"Consent cleanup completed with {len(deletion_errors)} errors",
errors=deletion_errors,
)
else:
await transcripts_controller.update(transcript, {"audio_deleted": True})
logger.info("Consent cleanup done - all audio deleted")
@get_transcript @get_transcript

View File

@@ -0,0 +1,510 @@
import asyncio
import io
from fractions import Fraction
import av
import boto3
import structlog
from av.audio.resampler import AudioResampler
from celery import chain, shared_task
from reflector.asynctask import asynctask
from reflector.db.transcripts import (
TranscriptStatus,
TranscriptText,
transcripts_controller,
)
from reflector.logger import logger
from reflector.pipelines.main_file_pipeline import task_send_webhook_if_needed
from reflector.pipelines.main_live_pipeline import (
PipelineMainBase,
task_cleanup_consent,
task_pipeline_post_to_zulip,
)
from reflector.processors import (
AudioFileWriterProcessor,
TranscriptFinalSummaryProcessor,
TranscriptFinalTitleProcessor,
TranscriptTopicDetectorProcessor,
)
from reflector.processors.file_transcript import FileTranscriptInput
from reflector.processors.file_transcript_auto import FileTranscriptAutoProcessor
from reflector.processors.types import TitleSummary
from reflector.processors.types import (
Transcript as TranscriptType,
)
from reflector.settings import settings
from reflector.storage import get_transcripts_storage
class EmptyPipeline:
def __init__(self, logger: structlog.BoundLogger):
self.logger = logger
def get_pref(self, k, d=None):
return d
async def emit(self, event):
pass
class PipelineMainMultitrack(PipelineMainBase):
"""Process multiple participant tracks for a transcript without mixing audio."""
def __init__(self, transcript_id: str):
super().__init__(transcript_id=transcript_id)
self.logger = logger.bind(transcript_id=self.transcript_id)
self.empty_pipeline = EmptyPipeline(logger=self.logger)
async def mixdown_tracks(
self,
track_datas: list[bytes],
writer: AudioFileWriterProcessor,
offsets_seconds: list[float] | None = None,
) -> None:
"""
Minimal multi-track mixdown using a PyAV filter graph (amix), no resampling.
"""
# Discover target sample rate from first decodable frame
target_sample_rate: int | None = None
for data in track_datas:
if not data:
continue
try:
container = av.open(io.BytesIO(data))
try:
for frame in container.decode(audio=0):
target_sample_rate = frame.sample_rate
break
finally:
container.close()
except Exception:
continue
if target_sample_rate:
break
if not target_sample_rate:
self.logger.warning("Mixdown skipped - no decodable audio frames found")
return
# Build PyAV filter graph:
# N abuffer (s32/stereo)
# -> optional adelay per input (for alignment)
# -> amix (s32)
# -> aformat(s16)
# -> sink
graph = av.filter.Graph()
inputs = []
valid_track_datas = [d for d in track_datas if d]
# Align offsets list with the filtered inputs (skip empties)
input_offsets_seconds = None
if offsets_seconds is not None:
input_offsets_seconds = [
offsets_seconds[i] for i, d in enumerate(track_datas) if d
]
for idx, data in enumerate(valid_track_datas):
args = (
f"time_base=1/{target_sample_rate}:"
f"sample_rate={target_sample_rate}:"
f"sample_fmt=s32:"
f"channel_layout=stereo"
)
in_ctx = graph.add("abuffer", args=args, name=f"in{idx}")
inputs.append(in_ctx)
if not inputs:
self.logger.warning("Mixdown skipped - no valid inputs for graph")
return
mixer = graph.add("amix", args=f"inputs={len(inputs)}:normalize=0", name="mix")
fmt = graph.add(
"aformat",
args=(
f"sample_fmts=s32:channel_layouts=stereo:sample_rates={target_sample_rate}"
),
name="fmt",
)
sink = graph.add("abuffersink", name="out")
# Optional per-input delay before mixing
delays_ms: list[int] = []
if input_offsets_seconds is not None:
base = min(input_offsets_seconds) if input_offsets_seconds else 0.0
delays_ms = [
max(0, int(round((o - base) * 1000))) for o in input_offsets_seconds
]
else:
delays_ms = [0 for _ in inputs]
for idx, in_ctx in enumerate(inputs):
delay_ms = delays_ms[idx] if idx < len(delays_ms) else 0
if delay_ms > 0:
# adelay requires one value per channel; use same for stereo
adelay = graph.add(
"adelay",
args=f"delays={delay_ms}|{delay_ms}:all=1",
name=f"delay{idx}",
)
in_ctx.link_to(adelay)
adelay.link_to(mixer, 0, idx)
else:
in_ctx.link_to(mixer, 0, idx)
mixer.link_to(fmt)
fmt.link_to(sink)
graph.configure()
# Open containers for decoding
containers = []
for i, d in enumerate(valid_track_datas):
try:
c = av.open(io.BytesIO(d))
containers.append(c)
except Exception as e:
self.logger.warning(
"Mixdown: failed to open container", input=i, error=str(e)
)
containers.append(None)
# Filter out Nones for decoders
containers = [c for c in containers if c is not None]
decoders = [c.decode(audio=0) for c in containers]
active = [True] * len(decoders)
# Per-input resamplers to enforce s32/stereo at the same rate (no resample of rate)
resamplers = [
AudioResampler(format="s32", layout="stereo", rate=target_sample_rate)
for _ in decoders
]
try:
# Round-robin feed frames into graph, pull mixed frames as they become available
while any(active):
for i, (dec, is_active) in enumerate(zip(decoders, active)):
if not is_active:
continue
try:
frame = next(dec)
except StopIteration:
active[i] = False
continue
# Enforce same sample rate; convert format/layout to s16/stereo (no resample)
if frame.sample_rate != target_sample_rate:
# Skip frames with differing rate
continue
out_frames = resamplers[i].resample(frame) or []
for rf in out_frames:
rf.sample_rate = target_sample_rate
rf.time_base = Fraction(1, target_sample_rate)
inputs[i].push(rf)
# Drain available mixed frames
while True:
try:
mixed = sink.pull()
except Exception:
break
mixed.sample_rate = target_sample_rate
mixed.time_base = Fraction(1, target_sample_rate)
await writer.push(mixed)
# Signal EOF to inputs and drain remaining
for in_ctx in inputs:
in_ctx.push(None)
while True:
try:
mixed = sink.pull()
except Exception:
break
mixed.sample_rate = target_sample_rate
mixed.time_base = Fraction(1, target_sample_rate)
await writer.push(mixed)
finally:
for c in containers:
c.close()
async def set_status(self, transcript_id: str, status: TranscriptStatus):
async with self.lock_transaction():
return await transcripts_controller.set_status(transcript_id, status)
async def process(self, bucket_name: str, track_keys: list[str]):
transcript = await self.get_transcript()
s3 = boto3.client(
"s3",
region_name=settings.RECORDING_STORAGE_AWS_REGION,
aws_access_key_id=settings.RECORDING_STORAGE_AWS_ACCESS_KEY_ID,
aws_secret_access_key=settings.RECORDING_STORAGE_AWS_SECRET_ACCESS_KEY,
)
storage = get_transcripts_storage()
# Pre-download bytes for all tracks for mixing and transcription
track_datas: list[bytes] = []
for key in track_keys:
try:
obj = s3.get_object(Bucket=bucket_name, Key=key)
track_datas.append(obj["Body"].read())
except Exception as e:
self.logger.warning(
"Skipping track - cannot read S3 object", key=key, error=str(e)
)
track_datas.append(b"")
# Extract offsets from Daily.co filename timestamps
# Format: {rec_start_ts}-{uuid}-{media_type}-{track_start_ts}.{ext}
# Example: 1760988935484-uuid-cam-audio-1760988935922
import re
offsets_seconds: list[float] = []
recording_start_ts: int | None = None
for key in track_keys:
# Parse Daily.co raw-tracks filename pattern
match = re.search(r"(\d+)-([0-9a-f-]{36})-(cam-audio)-(\d+)", key)
if not match:
self.logger.warning(
"Track key doesn't match Daily.co pattern, using 0.0 offset",
key=key,
)
offsets_seconds.append(0.0)
continue
rec_start_ts = int(match.group(1))
track_start_ts = int(match.group(4))
# Validate all tracks belong to same recording
if recording_start_ts is None:
recording_start_ts = rec_start_ts
elif rec_start_ts != recording_start_ts:
self.logger.error(
"Track belongs to different recording",
key=key,
expected_start=recording_start_ts,
got_start=rec_start_ts,
)
offsets_seconds.append(0.0)
continue
# Calculate offset in seconds
offset_ms = track_start_ts - rec_start_ts
offset_s = offset_ms / 1000.0
self.logger.info(
"Parsed track offset from filename",
key=key,
recording_start=rec_start_ts,
track_start=track_start_ts,
offset_seconds=offset_s,
)
offsets_seconds.append(max(0.0, offset_s))
# Mixdown all available tracks into transcript.audio_mp3_filename, preserving sample rate
try:
mp3_writer = AudioFileWriterProcessor(
path=str(transcript.audio_mp3_filename)
)
await self.mixdown_tracks(track_datas, mp3_writer, offsets_seconds)
await mp3_writer.flush()
except Exception as e:
self.logger.error("Mixdown failed", error=str(e))
speaker_transcripts: list[TranscriptType] = []
for idx, key in enumerate(track_keys):
ext = ".mp4"
try:
obj = s3.get_object(Bucket=bucket_name, Key=key)
data = obj["Body"].read()
except Exception as e:
self.logger.error(
"Skipping track - cannot read S3 object", key=key, error=str(e)
)
continue
storage_path = f"file_pipeline/{transcript.id}/tracks/track_{idx}{ext}"
try:
await storage.put_file(storage_path, data)
audio_url = await storage.get_file_url(storage_path)
except Exception as e:
self.logger.error(
"Skipping track - cannot upload to storage", key=key, error=str(e)
)
continue
try:
t = await self.transcribe_file(audio_url, transcript.source_language)
except Exception as e:
self.logger.error(
"Transcription via default backend failed, trying local whisper",
key=key,
url=audio_url,
error=str(e),
)
try:
fallback = FileTranscriptAutoProcessor(name="whisper")
result = None
async def capture_result(r):
nonlocal result
result = r
fallback.on(capture_result)
await fallback.push(
FileTranscriptInput(
audio_url=audio_url, language=transcript.source_language
)
)
await fallback.flush()
if not result:
raise Exception("No transcript captured in fallback")
t = result
except Exception as e2:
self.logger.error(
"Skipping track - transcription failed after fallback",
key=key,
url=audio_url,
error=str(e2),
)
continue
if not t.words:
continue
# Shift word timestamps by the track's offset so all are relative to 00:00
track_offset = offsets_seconds[idx] if idx < len(offsets_seconds) else 0.0
for w in t.words:
try:
if hasattr(w, "start") and w.start is not None:
w.start = float(w.start) + track_offset
if hasattr(w, "end") and w.end is not None:
w.end = float(w.end) + track_offset
except Exception:
pass
w.speaker = idx
speaker_transcripts.append(t)
if not speaker_transcripts:
raise Exception("No valid track transcriptions")
merged_words = []
for t in speaker_transcripts:
merged_words.extend(t.words)
merged_words.sort(key=lambda w: w.start)
merged_transcript = TranscriptType(words=merged_words, translation=None)
await transcripts_controller.append_event(
transcript,
event="TRANSCRIPT",
data=TranscriptText(
text=merged_transcript.text, translation=merged_transcript.translation
),
)
topics = await self.detect_topics(merged_transcript, transcript.target_language)
await asyncio.gather(
self.generate_title(topics),
self.generate_summaries(topics),
return_exceptions=False,
)
await self.set_status(transcript.id, "ended")
async def transcribe_file(self, audio_url: str, language: str) -> TranscriptType:
processor = FileTranscriptAutoProcessor()
input_data = FileTranscriptInput(audio_url=audio_url, language=language)
result: TranscriptType | None = None
async def capture_result(transcript):
nonlocal result
result = transcript
processor.on(capture_result)
await processor.push(input_data)
await processor.flush()
if not result:
raise ValueError("No transcript captured")
return result
async def detect_topics(
self, transcript: TranscriptType, target_language: str
) -> list[TitleSummary]:
chunk_size = 300
topics: list[TitleSummary] = []
async def on_topic(topic: TitleSummary):
topics.append(topic)
return await self.on_topic(topic)
topic_detector = TranscriptTopicDetectorProcessor(callback=on_topic)
topic_detector.set_pipeline(self.empty_pipeline)
for i in range(0, len(transcript.words), chunk_size):
chunk_words = transcript.words[i : i + chunk_size]
if not chunk_words:
continue
chunk_transcript = TranscriptType(
words=chunk_words, translation=transcript.translation
)
await topic_detector.push(chunk_transcript)
await topic_detector.flush()
return topics
async def generate_title(self, topics: list[TitleSummary]):
if not topics:
self.logger.warning("No topics for title generation")
return
processor = TranscriptFinalTitleProcessor(callback=self.on_title)
processor.set_pipeline(self.empty_pipeline)
for topic in topics:
await processor.push(topic)
await processor.flush()
async def generate_summaries(self, topics: list[TitleSummary]):
if not topics:
self.logger.warning("No topics for summary generation")
return
transcript = await self.get_transcript()
processor = TranscriptFinalSummaryProcessor(
transcript=transcript,
callback=self.on_long_summary,
on_short_summary=self.on_short_summary,
)
processor.set_pipeline(self.empty_pipeline)
for topic in topics:
await processor.push(topic)
await processor.flush()
@shared_task
@asynctask
async def task_pipeline_multitrack_process(
*, transcript_id: str, bucket_name: str, track_keys: list[str]
):
pipeline = PipelineMainMultitrack(transcript_id=transcript_id)
try:
await pipeline.set_status(transcript_id, "processing")
await pipeline.process(bucket_name, track_keys)
except Exception:
await pipeline.set_status(transcript_id, "error")
raise
post_chain = chain(
task_cleanup_consent.si(transcript_id=transcript_id),
task_pipeline_post_to_zulip.si(transcript_id=transcript_id),
task_send_webhook_if_needed.si(transcript_id=transcript_id),
)
post_chain.delay()

View File

@@ -1,10 +1,10 @@
import asyncio import asyncio
import math import io
import tempfile
from fractions import Fraction from fractions import Fraction
from pathlib import Path
import av import av
import boto3
import structlog
from av.audio.resampler import AudioResampler from av.audio.resampler import AudioResampler
from celery import chain, shared_task from celery import chain, shared_task
@@ -15,7 +15,6 @@ from reflector.db.transcripts import (
transcripts_controller, transcripts_controller,
) )
from reflector.logger import logger from reflector.logger import logger
from reflector.pipelines import topic_processing
from reflector.pipelines.main_file_pipeline import task_send_webhook_if_needed from reflector.pipelines.main_file_pipeline import task_send_webhook_if_needed
from reflector.pipelines.main_live_pipeline import ( from reflector.pipelines.main_live_pipeline import (
PipelineMainBase, PipelineMainBase,
@@ -23,325 +22,213 @@ from reflector.pipelines.main_live_pipeline import (
task_cleanup_consent, task_cleanup_consent,
task_pipeline_post_to_zulip, task_pipeline_post_to_zulip,
) )
from reflector.pipelines.transcription_helpers import transcribe_file_with_processor from reflector.processors import (
from reflector.processors import AudioFileWriterProcessor AudioFileWriterProcessor,
TranscriptFinalSummaryProcessor,
TranscriptFinalTitleProcessor,
TranscriptTopicDetectorProcessor,
)
from reflector.processors.audio_waveform_processor import AudioWaveformProcessor from reflector.processors.audio_waveform_processor import AudioWaveformProcessor
from reflector.processors.file_transcript import FileTranscriptInput
from reflector.processors.file_transcript_auto import FileTranscriptAutoProcessor
from reflector.processors.types import TitleSummary from reflector.processors.types import TitleSummary
from reflector.processors.types import Transcript as TranscriptType from reflector.processors.types import (
from reflector.storage import Storage, get_transcripts_storage Transcript as TranscriptType,
from reflector.utils.string import NonEmptyString )
from reflector.settings import settings
from reflector.storage import get_transcripts_storage
# Audio encoding constants
OPUS_STANDARD_SAMPLE_RATE = 48000
OPUS_DEFAULT_BIT_RATE = 128000
# Storage operation constants class EmptyPipeline:
PRESIGNED_URL_EXPIRATION_SECONDS = 7200 # 2 hours def __init__(self, logger: structlog.BoundLogger):
self.logger = logger
def get_pref(self, k, d=None):
return d
async def emit(self, event):
pass
class PipelineMainMultitrack(PipelineMainBase): class PipelineMainMultitrack(PipelineMainBase):
"""Process multiple participant tracks for a transcript without mixing audio."""
def __init__(self, transcript_id: str): def __init__(self, transcript_id: str):
super().__init__(transcript_id=transcript_id) super().__init__(transcript_id=transcript_id)
self.logger = logger.bind(transcript_id=self.transcript_id) self.logger = logger.bind(transcript_id=self.transcript_id)
self.empty_pipeline = topic_processing.EmptyPipeline(logger=self.logger) self.empty_pipeline = EmptyPipeline(logger=self.logger)
async def pad_track_for_transcription( async def pad_track_for_transcription(
self, self,
track_url: NonEmptyString, track_data: bytes,
track_idx: int, track_idx: int,
storage: Storage, storage,
) -> NonEmptyString: ) -> tuple[bytes, str]:
""" """
Pad a single track with silence based on stream metadata start_time. Pad a single track with silence based on stream metadata start_time.
Downloads from S3 presigned URL, processes via PyAV using tempfile, uploads to S3. This ensures Whisper timestamps will be relative to recording start.
Returns presigned URL of padded track (or original URL if no padding needed). Uses ffmpeg subprocess approach proven to work with python-raw-tracks-align.
Memory usage: Returns: (padded_data, storage_url)
- Pattern: fixed_overhead(2-5MB) for PyAV codec/filters
- PyAV streams input efficiently (no full download, verified)
- Output written to tempfile (disk-based, not memory)
- Upload streams from file handle (boto3 chunks, typically 5-10MB)
Daily.co raw-tracks timing - Two approaches:
CURRENT APPROACH (PyAV metadata):
The WebM stream.start_time field encodes MEETING-RELATIVE timing:
- t=0: When Daily.co recording started (first participant joined)
- start_time=8.13s: This participant's track began 8.13s after recording started
- Purpose: Enables track alignment without external manifest files
This is NOT:
- Stream-internal offset (first packet timestamp relative to stream start)
- Absolute/wall-clock time
- Recording duration
ALTERNATIVE APPROACH (filename parsing):
Daily.co filenames contain Unix timestamps (milliseconds):
Format: {recording_start_ts}-{participant_id}-cam-audio-{track_start_ts}.webm
Example: 1760988935484-52f7f48b-fbab-431f-9a50-87b9abfc8255-cam-audio-1760988935922.webm
Can calculate offset: (track_start_ts - recording_start_ts) / 1000
- Track 0: (1760988935922 - 1760988935484) / 1000 = 0.438s
- Track 1: (1760988943823 - 1760988935484) / 1000 = 8.339s
TIME DIFFERENCE: PyAV metadata vs filename timestamps differ by ~209ms:
- Track 0: filename=438ms, metadata=229ms (diff: 209ms)
- Track 1: filename=8339ms, metadata=8130ms (diff: 209ms)
Consistent delta suggests network/encoding delay. PyAV metadata is ground truth
(represents when audio stream actually started vs when file upload initiated).
Example with 2 participants:
Track A: start_time=0.2s → Joined 200ms after recording began
Track B: start_time=8.1s → Joined 8.1 seconds later
After padding:
Track A: [0.2s silence] + [speech...]
Track B: [8.1s silence] + [speech...]
Whisper transcription timestamps are now synchronized:
Track A word at 5.0s → happened at meeting t=5.0s
Track B word at 10.0s → happened at meeting t=10.0s
Merging just sorts by timestamp - no offset calculation needed.
Padding coincidentally involves re-encoding. It's important when we work with Daily.co + Whisper.
This is because Daily.co returns recordings with skipped frames e.g. when microphone muted.
Daily.co doesn't understand those frames and ignores them, causing timestamp issues in transcription.
Re-encoding restores those frames. We do padding and re-encoding together just because it's convenient and more performant:
we need padded values for mix mp3 anyways
""" """
import json
import math
import subprocess
import tempfile
if not track_data:
return b"", ""
transcript = await self.get_transcript() transcript = await self.get_transcript()
try: # Create temp files for ffmpeg processing
# PyAV streams input from S3 URL efficiently (2-5MB fixed overhead for codec/filters) with tempfile.NamedTemporaryFile(suffix=".webm", delete=False) as input_file:
with av.open(track_url) as in_container: input_file.write(track_data)
start_time_seconds = self._extract_stream_start_time_from_container( input_file_path = input_file.name
in_container, track_idx
)
if start_time_seconds <= 0: output_file_path = input_file_path.replace(".webm", "_padded.webm")
self.logger.info(
f"Track {track_idx} requires no padding (start_time={start_time_seconds}s)",
track_idx=track_idx,
)
return track_url
# Use tempfile instead of BytesIO for better memory efficiency
# Reduces peak memory usage during encoding/upload
with tempfile.NamedTemporaryFile(
suffix=".webm", delete=False
) as temp_file:
temp_path = temp_file.name
try: try:
self._apply_audio_padding_to_file( # Get stream metadata using ffprobe
in_container, temp_path, start_time_seconds, track_idx ffprobe_cmd = [
"ffprobe",
"-v",
"error",
"-show_entries",
"stream=start_time",
"-of",
"json",
input_file_path,
]
result = subprocess.run(
ffprobe_cmd, capture_output=True, text=True, check=True
) )
metadata = json.loads(result.stdout)
storage_path = ( # Extract start_time from stream metadata
f"file_pipeline/{transcript.id}/tracks/padded_{track_idx}.webm"
)
# Upload using file handle for streaming
with open(temp_path, "rb") as padded_file:
await storage.put_file(storage_path, padded_file)
finally:
# Clean up temp file
Path(temp_path).unlink(missing_ok=True)
padded_url = await storage.get_file_url(
storage_path,
operation="get_object",
expires_in=PRESIGNED_URL_EXPIRATION_SECONDS,
)
self.logger.info(
f"Successfully padded track {track_idx}",
track_idx=track_idx,
start_time_seconds=start_time_seconds,
padded_url=padded_url,
)
return padded_url
except Exception as e:
self.logger.error(
f"Failed to process track {track_idx}",
track_idx=track_idx,
url=track_url,
error=str(e),
exc_info=True,
)
raise Exception(
f"Track {track_idx} padding failed - transcript would have incorrect timestamps"
) from e
def _extract_stream_start_time_from_container(
self, container, track_idx: int
) -> float:
"""
Extract meeting-relative start time from WebM stream metadata.
Uses PyAV to read stream.start_time from WebM container.
More accurate than filename timestamps by ~209ms due to network/encoding delays.
"""
start_time_seconds = 0.0
try:
audio_streams = [s for s in container.streams if s.type == "audio"]
stream = audio_streams[0] if audio_streams else container.streams[0]
# 1) Try stream-level start_time (most reliable for Daily.co tracks)
if stream.start_time is not None and stream.time_base is not None:
start_time_seconds = float(stream.start_time * stream.time_base)
# 2) Fallback to container-level start_time (in av.time_base units)
if (start_time_seconds <= 0) and (container.start_time is not None):
start_time_seconds = float(container.start_time * av.time_base)
# 3) Fallback to first packet DTS in stream.time_base
if start_time_seconds <= 0:
for packet in container.demux(stream):
if packet.dts is not None:
start_time_seconds = float(packet.dts * stream.time_base)
break
except Exception as e:
self.logger.warning(
"PyAV metadata read failed; assuming 0 start_time",
track_idx=track_idx,
error=str(e),
)
start_time_seconds = 0.0 start_time_seconds = 0.0
if metadata.get("streams") and len(metadata["streams"]) > 0:
start_time_str = metadata["streams"][0].get("start_time", "0")
start_time_seconds = float(start_time_str)
self.logger.info( self.logger.info(
f"Track {track_idx} stream metadata: start_time={start_time_seconds:.3f}s", f"Track {track_idx} stream metadata: start_time={start_time_seconds:.3f}s",
track_idx=track_idx, track_idx=track_idx,
) )
return start_time_seconds
def _apply_audio_padding_to_file( # If no padding needed, use original
self, if start_time_seconds <= 0:
in_container, storage_path = f"file_pipeline/{transcript.id}/tracks/original_track_{track_idx}.webm"
output_path: str, await storage.put_file(storage_path, track_data)
start_time_seconds: float, url = await storage.get_file_url(storage_path)
track_idx: int, return track_data, url
) -> None:
"""Apply silence padding to audio track using PyAV filter graph, writing to file""" # Calculate delay in milliseconds
delay_ms = math.floor(start_time_seconds * 1000) delay_ms = math.floor(start_time_seconds * 1000)
# Run ffmpeg to pad the audio while maintaining WebM/Opus format for Modal compatibility
# ffmpeg quirk: aresample needs to come before adelay in the filter chain
ffmpeg_cmd = [
"ffmpeg",
"-hide_banner",
"-loglevel",
"error",
"-y", # overwrite output
"-i",
input_file_path,
"-af",
f"aresample=async=1,adelay={delay_ms}:all=true",
"-c:a",
"libopus", # Keep Opus codec for Modal compatibility
"-b:a",
"128k", # Standard bitrate for Opus
output_file_path,
]
self.logger.info( self.logger.info(
f"Padding track {track_idx} with {delay_ms}ms delay using PyAV", f"Padding track {track_idx} with {delay_ms}ms delay using ffmpeg",
track_idx=track_idx, track_idx=track_idx,
delay_ms=delay_ms, delay_ms=delay_ms,
command=" ".join(ffmpeg_cmd),
) )
try: result = subprocess.run(ffmpeg_cmd, capture_output=True, text=True)
with av.open(output_path, "w", format="webm") as out_container: if result.returncode != 0:
in_stream = next(
(s for s in in_container.streams if s.type == "audio"), None
)
if in_stream is None:
raise Exception("No audio stream in input")
out_stream = out_container.add_stream(
"libopus", rate=OPUS_STANDARD_SAMPLE_RATE
)
out_stream.bit_rate = OPUS_DEFAULT_BIT_RATE
graph = av.filter.Graph()
abuf_args = (
f"time_base=1/{OPUS_STANDARD_SAMPLE_RATE}:"
f"sample_rate={OPUS_STANDARD_SAMPLE_RATE}:"
f"sample_fmt=s16:"
f"channel_layout=stereo"
)
src = graph.add("abuffer", args=abuf_args, name="src")
aresample_f = graph.add("aresample", args="async=1", name="ares")
# adelay requires one delay value per channel separated by '|'
delays_arg = f"{delay_ms}|{delay_ms}"
adelay_f = graph.add(
"adelay", args=f"delays={delays_arg}:all=1", name="delay"
)
sink = graph.add("abuffersink", name="sink")
src.link_to(aresample_f)
aresample_f.link_to(adelay_f)
adelay_f.link_to(sink)
graph.configure()
resampler = AudioResampler(
format="s16", layout="stereo", rate=OPUS_STANDARD_SAMPLE_RATE
)
# Decode -> resample -> push through graph -> encode Opus
for frame in in_container.decode(in_stream):
out_frames = resampler.resample(frame) or []
for rframe in out_frames:
rframe.sample_rate = OPUS_STANDARD_SAMPLE_RATE
rframe.time_base = Fraction(1, OPUS_STANDARD_SAMPLE_RATE)
src.push(rframe)
while True:
try:
f_out = sink.pull()
except Exception:
break
f_out.sample_rate = OPUS_STANDARD_SAMPLE_RATE
f_out.time_base = Fraction(1, OPUS_STANDARD_SAMPLE_RATE)
for packet in out_stream.encode(f_out):
out_container.mux(packet)
src.push(None)
while True:
try:
f_out = sink.pull()
except Exception:
break
f_out.sample_rate = OPUS_STANDARD_SAMPLE_RATE
f_out.time_base = Fraction(1, OPUS_STANDARD_SAMPLE_RATE)
for packet in out_stream.encode(f_out):
out_container.mux(packet)
for packet in out_stream.encode(None):
out_container.mux(packet)
except Exception as e:
self.logger.error( self.logger.error(
"PyAV padding failed for track", f"ffmpeg padding failed for track {track_idx}",
track_idx=track_idx,
stderr=result.stderr,
returncode=result.returncode,
)
raise Exception(f"ffmpeg padding failed: {result.stderr}")
# Read the padded output
with open(output_file_path, "rb") as f:
padded_data = f.read()
# Store padded track
storage_path = (
f"file_pipeline/{transcript.id}/tracks/padded_track_{track_idx}.webm"
)
await storage.put_file(storage_path, padded_data)
padded_url = await storage.get_file_url(storage_path)
self.logger.info(
f"Successfully padded track {track_idx} with {start_time_seconds:.3f}s offset, stored at {storage_path}",
track_idx=track_idx, track_idx=track_idx,
delay_ms=delay_ms, delay_ms=delay_ms,
error=str(e), padded_url=padded_url,
exc_info=True, padded_size=len(padded_data),
) )
raise
return padded_data, padded_url
finally:
# Clean up temp files
import os
try:
os.unlink(input_file_path)
except:
pass
try:
os.unlink(output_file_path)
except:
pass
async def mixdown_tracks( async def mixdown_tracks(
self, self,
track_urls: list[str], track_datas: list[bytes],
writer: AudioFileWriterProcessor, writer: AudioFileWriterProcessor,
offsets_seconds: list[float] | None = None, offsets_seconds: list[float] | None = None,
) -> None: ) -> None:
"""Multi-track mixdown using PyAV filter graph (amix), reading from S3 presigned URLs""" """
Minimal multi-track mixdown using a PyAV filter graph (amix), no resampling.
"""
# Discover target sample rate from first decodable frame
target_sample_rate: int | None = None target_sample_rate: int | None = None
for url in track_urls: for data in track_datas:
if not url: if not data:
continue continue
container = None
try: try:
container = av.open(url) container = av.open(io.BytesIO(data))
try:
for frame in container.decode(audio=0): for frame in container.decode(audio=0):
target_sample_rate = frame.sample_rate target_sample_rate = frame.sample_rate
break break
finally:
container.close()
except Exception: except Exception:
continue continue
finally:
if container is not None:
container.close()
if target_sample_rate: if target_sample_rate:
break break
if not target_sample_rate: if not target_sample_rate:
self.logger.error("Mixdown failed - no decodable audio frames found") self.logger.warning("Mixdown skipped - no decodable audio frames found")
raise Exception("Mixdown failed: No decodable audio frames in any track") return
# Build PyAV filter graph: # Build PyAV filter graph:
# N abuffer (s32/stereo) # N abuffer (s32/stereo)
# -> optional adelay per input (for alignment) # -> optional adelay per input (for alignment)
@@ -350,13 +237,14 @@ class PipelineMainMultitrack(PipelineMainBase):
# -> sink # -> sink
graph = av.filter.Graph() graph = av.filter.Graph()
inputs = [] inputs = []
valid_track_urls = [url for url in track_urls if url] valid_track_datas = [d for d in track_datas if d]
# Align offsets list with the filtered inputs (skip empties)
input_offsets_seconds = None input_offsets_seconds = None
if offsets_seconds is not None: if offsets_seconds is not None:
input_offsets_seconds = [ input_offsets_seconds = [
offsets_seconds[i] for i, url in enumerate(track_urls) if url offsets_seconds[i] for i, d in enumerate(track_datas) if d
] ]
for idx, url in enumerate(valid_track_urls): for idx, data in enumerate(valid_track_datas):
args = ( args = (
f"time_base=1/{target_sample_rate}:" f"time_base=1/{target_sample_rate}:"
f"sample_rate={target_sample_rate}:" f"sample_rate={target_sample_rate}:"
@@ -367,8 +255,8 @@ class PipelineMainMultitrack(PipelineMainBase):
inputs.append(in_ctx) inputs.append(in_ctx)
if not inputs: if not inputs:
self.logger.error("Mixdown failed - no valid inputs for graph") self.logger.warning("Mixdown skipped - no valid inputs for graph")
raise Exception("Mixdown failed: No valid inputs for filter graph") return
mixer = graph.add("amix", args=f"inputs={len(inputs)}:normalize=0", name="mix") mixer = graph.add("amix", args=f"inputs={len(inputs)}:normalize=0", name="mix")
@@ -409,32 +297,29 @@ class PipelineMainMultitrack(PipelineMainBase):
fmt.link_to(sink) fmt.link_to(sink)
graph.configure() graph.configure()
# Open containers for decoding
containers = [] containers = []
for i, d in enumerate(valid_track_datas):
try: try:
# Open all containers with cleanup guaranteed c = av.open(io.BytesIO(d))
for i, url in enumerate(valid_track_urls):
try:
c = av.open(url)
containers.append(c) containers.append(c)
except Exception as e: except Exception as e:
self.logger.warning( self.logger.warning(
"Mixdown: failed to open container from URL", "Mixdown: failed to open container", input=i, error=str(e)
input=i,
url=url,
error=str(e),
) )
containers.append(None)
if not containers: # Filter out Nones for decoders
self.logger.error("Mixdown failed - no valid containers opened") containers = [c for c in containers if c is not None]
raise Exception("Mixdown failed: Could not open any track containers")
decoders = [c.decode(audio=0) for c in containers] decoders = [c.decode(audio=0) for c in containers]
active = [True] * len(decoders) active = [True] * len(decoders)
# Per-input resamplers to enforce s32/stereo at the same rate (no resample of rate)
resamplers = [ resamplers = [
AudioResampler(format="s32", layout="stereo", rate=target_sample_rate) AudioResampler(format="s32", layout="stereo", rate=target_sample_rate)
for _ in decoders for _ in decoders
] ]
try:
# Round-robin feed frames into graph, pull mixed frames as they become available
while any(active): while any(active):
for i, (dec, is_active) in enumerate(zip(decoders, active)): for i, (dec, is_active) in enumerate(zip(decoders, active)):
if not is_active: if not is_active:
@@ -445,7 +330,9 @@ class PipelineMainMultitrack(PipelineMainBase):
active[i] = False active[i] = False
continue continue
# Enforce same sample rate; convert format/layout to s16/stereo (no resample)
if frame.sample_rate != target_sample_rate: if frame.sample_rate != target_sample_rate:
# Skip frames with differing rate
continue continue
out_frames = resamplers[i].resample(frame) or [] out_frames = resamplers[i].resample(frame) or []
for rf in out_frames: for rf in out_frames:
@@ -453,6 +340,7 @@ class PipelineMainMultitrack(PipelineMainBase):
rf.time_base = Fraction(1, target_sample_rate) rf.time_base = Fraction(1, target_sample_rate)
inputs[i].push(rf) inputs[i].push(rf)
# Drain available mixed frames
while True: while True:
try: try:
mixed = sink.pull() mixed = sink.pull()
@@ -462,6 +350,7 @@ class PipelineMainMultitrack(PipelineMainBase):
mixed.time_base = Fraction(1, target_sample_rate) mixed.time_base = Fraction(1, target_sample_rate)
await writer.push(mixed) await writer.push(mixed)
# Signal EOF to inputs and drain remaining
for in_ctx in inputs: for in_ctx in inputs:
in_ctx.push(None) in_ctx.push(None)
while True: while True:
@@ -473,13 +362,8 @@ class PipelineMainMultitrack(PipelineMainBase):
mixed.time_base = Fraction(1, target_sample_rate) mixed.time_base = Fraction(1, target_sample_rate)
await writer.push(mixed) await writer.push(mixed)
finally: finally:
# Cleanup all containers, even if processing failed
for c in containers: for c in containers:
if c is not None:
try:
c.close() c.close()
except Exception:
pass # Best effort cleanup
@broadcast_to_sockets @broadcast_to_sockets
async def set_status(self, transcript_id: str, status: TranscriptStatus): async def set_status(self, transcript_id: str, status: TranscriptStatus):
@@ -496,74 +380,85 @@ class PipelineMainMultitrack(PipelineMainBase):
async def process(self, bucket_name: str, track_keys: list[str]): async def process(self, bucket_name: str, track_keys: list[str]):
transcript = await self.get_transcript() transcript = await self.get_transcript()
async with self.transaction():
await transcripts_controller.update( s3 = boto3.client(
transcript, "s3",
{ region_name=settings.RECORDING_STORAGE_AWS_REGION,
"events": [], aws_access_key_id=settings.RECORDING_STORAGE_AWS_ACCESS_KEY_ID,
"topics": [], aws_secret_access_key=settings.RECORDING_STORAGE_AWS_SECRET_ACCESS_KEY,
},
) )
source_storage = get_transcripts_storage() storage = get_transcripts_storage()
transcript_storage = source_storage
track_urls: list[str] = [] # Pre-download bytes for all tracks for mixing and transcription
track_datas: list[bytes] = []
for key in track_keys: for key in track_keys:
url = await source_storage.get_file_url( try:
key, obj = s3.get_object(Bucket=bucket_name, Key=key)
operation="get_object", track_datas.append(obj["Body"].read())
expires_in=PRESIGNED_URL_EXPIRATION_SECONDS, except Exception as e:
bucket=bucket_name, self.logger.warning(
) "Skipping track - cannot read S3 object", key=key, error=str(e)
track_urls.append(url)
self.logger.info(
f"Generated presigned URL for track from {bucket_name}",
key=key,
) )
track_datas.append(b"")
created_padded_files = set() # PAD TRACKS FIRST - this creates full-length tracks with correct timeline
padded_track_datas: list[bytes] = []
padded_track_urls: list[str] = [] padded_track_urls: list[str] = []
for idx, url in enumerate(track_urls): for idx, data in enumerate(track_datas):
padded_url = await self.pad_track_for_transcription( if not data:
url, idx, transcript_storage padded_track_datas.append(b"")
) padded_track_urls.append("")
padded_track_urls.append(padded_url) continue
if padded_url != url:
storage_path = f"file_pipeline/{transcript.id}/tracks/padded_{idx}.webm"
created_padded_files.add(storage_path)
self.logger.info(f"Track {idx} processed, padded URL: {padded_url}")
padded_data, padded_url = await self.pad_track_for_transcription(
data, idx, storage
)
padded_track_datas.append(padded_data)
padded_track_urls.append(padded_url)
self.logger.info(f"Padded track {idx} for transcription: {padded_url}")
# Mixdown PADDED tracks (already aligned with timeline) into transcript.audio_mp3_filename
try:
# Ensure data directory exists
transcript.data_path.mkdir(parents=True, exist_ok=True) transcript.data_path.mkdir(parents=True, exist_ok=True)
mp3_writer = AudioFileWriterProcessor( mp3_writer = AudioFileWriterProcessor(
path=str(transcript.audio_mp3_filename), path=str(transcript.audio_mp3_filename),
on_duration=self.on_duration, on_duration=self.on_duration,
) )
await self.mixdown_tracks(padded_track_urls, mp3_writer, offsets_seconds=None) # Use PADDED tracks with NO additional offsets (already aligned by padding)
await self.mixdown_tracks(
padded_track_datas, mp3_writer, offsets_seconds=None
)
await mp3_writer.flush() await mp3_writer.flush()
if not transcript.audio_mp3_filename.exists(): # Upload the mixed audio to S3 for web playback
raise Exception( if transcript.audio_mp3_filename.exists():
"Mixdown failed - no MP3 file generated. Cannot proceed without playable audio." mp3_data = transcript.audio_mp3_filename.read_bytes()
)
storage_path = f"{transcript.id}/audio.mp3" storage_path = f"{transcript.id}/audio.mp3"
# Use file handle streaming to avoid loading entire MP3 into memory await storage.put_file(storage_path, mp3_data)
mp3_size = transcript.audio_mp3_filename.stat().st_size mp3_url = await storage.get_file_url(storage_path)
with open(transcript.audio_mp3_filename, "rb") as mp3_file:
await transcript_storage.put_file(storage_path, mp3_file)
mp3_url = await transcript_storage.get_file_url(storage_path)
await transcripts_controller.update(transcript, {"audio_location": "storage"}) # Update transcript to indicate audio is in storage
await transcripts_controller.update(
transcript, {"audio_location": "storage"}
)
self.logger.info( self.logger.info(
f"Uploaded mixed audio to storage", f"Uploaded mixed audio to storage",
storage_path=storage_path, storage_path=storage_path,
size=mp3_size, size=len(mp3_data),
url=mp3_url, url=mp3_url,
) )
else:
self.logger.warning("Mixdown file does not exist after processing")
except Exception as e:
self.logger.error("Mixdown failed", error=str(e), exc_info=True)
# Generate waveform from the mixed audio file
if transcript.audio_mp3_filename.exists():
try:
self.logger.info("Generating waveform from mixed audio") self.logger.info("Generating waveform from mixed audio")
waveform_processor = AudioWaveformProcessor( waveform_processor = AudioWaveformProcessor(
audio_path=transcript.audio_mp3_filename, audio_path=transcript.audio_mp3_filename,
@@ -573,17 +468,60 @@ class PipelineMainMultitrack(PipelineMainBase):
waveform_processor.set_pipeline(self.empty_pipeline) waveform_processor.set_pipeline(self.empty_pipeline)
await waveform_processor.flush() await waveform_processor.flush()
self.logger.info("Waveform generated successfully") self.logger.info("Waveform generated successfully")
except Exception as e:
self.logger.error(
"Waveform generation failed", error=str(e), exc_info=True
)
# Transcribe PADDED tracks - timestamps will be automatically correct!
speaker_transcripts: list[TranscriptType] = [] speaker_transcripts: list[TranscriptType] = []
for idx, padded_url in enumerate(padded_track_urls): for idx, padded_url in enumerate(padded_track_urls):
if not padded_url: if not padded_url:
continue continue
try:
# Transcribe the PADDED track
t = await self.transcribe_file(padded_url, transcript.source_language) t = await self.transcribe_file(padded_url, transcript.source_language)
except Exception as e:
self.logger.error(
"Transcription via default backend failed, trying local whisper",
track_idx=idx,
url=padded_url,
error=str(e),
)
try:
fallback = FileTranscriptAutoProcessor(name="whisper")
result = None
async def capture_result(r):
nonlocal result
result = r
fallback.on(capture_result)
await fallback.push(
FileTranscriptInput(
audio_url=padded_url, language=transcript.source_language
)
)
await fallback.flush()
if not result:
raise Exception("No transcript captured in fallback")
t = result
except Exception as e2:
self.logger.error(
"Skipping track - transcription failed after fallback",
track_idx=idx,
url=padded_url,
error=str(e2),
)
continue
if not t.words: if not t.words:
continue continue
# NO OFFSET ADJUSTMENT NEEDED!
# Timestamps are already correct because we transcribed padded tracks
# Just set speaker ID
for w in t.words: for w in t.words:
w.speaker = idx w.speaker = idx
@@ -593,33 +531,10 @@ class PipelineMainMultitrack(PipelineMainBase):
track_idx=idx, track_idx=idx,
) )
valid_track_count = len([url for url in padded_track_urls if url])
if valid_track_count > 0 and len(speaker_transcripts) != valid_track_count:
raise Exception(
f"Only {len(speaker_transcripts)}/{valid_track_count} tracks transcribed successfully. "
f"All tracks must succeed to avoid incomplete transcripts."
)
if not speaker_transcripts: if not speaker_transcripts:
raise Exception("No valid track transcriptions") raise Exception("No valid track transcriptions")
self.logger.info(f"Cleaning up {len(created_padded_files)} temporary S3 files") # Merge all words and sort by timestamp
cleanup_tasks = []
for storage_path in created_padded_files:
cleanup_tasks.append(transcript_storage.delete_file(storage_path))
if cleanup_tasks:
cleanup_results = await asyncio.gather(
*cleanup_tasks, return_exceptions=True
)
for storage_path, result in zip(created_padded_files, cleanup_results):
if isinstance(result, Exception):
self.logger.warning(
"Failed to cleanup temporary padded track",
storage_path=storage_path,
error=str(result),
)
merged_words = [] merged_words = []
for t in speaker_transcripts: for t in speaker_transcripts:
merged_words.extend(t.words) merged_words.extend(t.words)
@@ -629,6 +544,7 @@ class PipelineMainMultitrack(PipelineMainBase):
merged_transcript = TranscriptType(words=merged_words, translation=None) merged_transcript = TranscriptType(words=merged_words, translation=None)
# Emit TRANSCRIPT event through the shared handler (persists and broadcasts)
await self.on_transcript(merged_transcript) await self.on_transcript(merged_transcript)
topics = await self.detect_topics(merged_transcript, transcript.target_language) topics = await self.detect_topics(merged_transcript, transcript.target_language)
@@ -641,36 +557,80 @@ class PipelineMainMultitrack(PipelineMainBase):
await self.set_status(transcript.id, "ended") await self.set_status(transcript.id, "ended")
async def transcribe_file(self, audio_url: str, language: str) -> TranscriptType: async def transcribe_file(self, audio_url: str, language: str) -> TranscriptType:
return await transcribe_file_with_processor(audio_url, language) processor = FileTranscriptAutoProcessor()
input_data = FileTranscriptInput(audio_url=audio_url, language=language)
result: TranscriptType | None = None
async def capture_result(transcript):
nonlocal result
result = transcript
processor.on(capture_result)
await processor.push(input_data)
await processor.flush()
if not result:
raise ValueError("No transcript captured")
return result
async def detect_topics( async def detect_topics(
self, transcript: TranscriptType, target_language: str self, transcript: TranscriptType, target_language: str
) -> list[TitleSummary]: ) -> list[TitleSummary]:
return await topic_processing.detect_topics( chunk_size = 300
transcript, topics: list[TitleSummary] = []
target_language,
on_topic_callback=self.on_topic, async def on_topic(topic: TitleSummary):
empty_pipeline=self.empty_pipeline, topics.append(topic)
return await self.on_topic(topic)
topic_detector = TranscriptTopicDetectorProcessor(callback=on_topic)
topic_detector.set_pipeline(self.empty_pipeline)
for i in range(0, len(transcript.words), chunk_size):
chunk_words = transcript.words[i : i + chunk_size]
if not chunk_words:
continue
chunk_transcript = TranscriptType(
words=chunk_words, translation=transcript.translation
) )
await topic_detector.push(chunk_transcript)
await topic_detector.flush()
return topics
async def generate_title(self, topics: list[TitleSummary]): async def generate_title(self, topics: list[TitleSummary]):
return await topic_processing.generate_title( if not topics:
topics, self.logger.warning("No topics for title generation")
on_title_callback=self.on_title, return
empty_pipeline=self.empty_pipeline,
logger=self.logger, processor = TranscriptFinalTitleProcessor(callback=self.on_title)
) processor.set_pipeline(self.empty_pipeline)
for topic in topics:
await processor.push(topic)
await processor.flush()
async def generate_summaries(self, topics: list[TitleSummary]): async def generate_summaries(self, topics: list[TitleSummary]):
if not topics:
self.logger.warning("No topics for summary generation")
return
transcript = await self.get_transcript() transcript = await self.get_transcript()
return await topic_processing.generate_summaries( processor = TranscriptFinalSummaryProcessor(
topics, transcript=transcript,
transcript, callback=self.on_long_summary,
on_long_summary_callback=self.on_long_summary, on_short_summary=self.on_short_summary,
on_short_summary_callback=self.on_short_summary,
empty_pipeline=self.empty_pipeline,
logger=self.logger,
) )
processor.set_pipeline(self.empty_pipeline)
for topic in topics:
await processor.push(topic)
await processor.flush()
@shared_task @shared_task

View File

@@ -0,0 +1,629 @@
import asyncio
import io
from fractions import Fraction
import av
import boto3
import structlog
from av.audio.resampler import AudioResampler
from celery import chain, shared_task
from reflector.asynctask import asynctask
from reflector.db.transcripts import (
TranscriptStatus,
TranscriptText,
transcripts_controller,
)
from reflector.logger import logger
from reflector.pipelines.main_file_pipeline import task_send_webhook_if_needed
from reflector.pipelines.main_live_pipeline import (
PipelineMainBase,
task_cleanup_consent,
task_pipeline_post_to_zulip,
)
from reflector.processors import (
AudioFileWriterProcessor,
TranscriptFinalSummaryProcessor,
TranscriptFinalTitleProcessor,
TranscriptTopicDetectorProcessor,
)
from reflector.processors.file_transcript import FileTranscriptInput
from reflector.processors.file_transcript_auto import FileTranscriptAutoProcessor
from reflector.processors.types import TitleSummary
from reflector.processors.types import (
Transcript as TranscriptType,
)
from reflector.settings import settings
from reflector.storage import get_transcripts_storage
class EmptyPipeline:
def __init__(self, logger: structlog.BoundLogger):
self.logger = logger
def get_pref(self, k, d=None):
return d
async def emit(self, event):
pass
class PipelineMainMultitrack(PipelineMainBase):
"""Process multiple participant tracks for a transcript without mixing audio."""
def __init__(self, transcript_id: str):
super().__init__(transcript_id=transcript_id)
self.logger = logger.bind(transcript_id=self.transcript_id)
self.empty_pipeline = EmptyPipeline(logger=self.logger)
async def pad_track_for_transcription(
self,
track_data: bytes,
track_idx: int,
storage,
) -> tuple[bytes, str]:
"""
Pad a single track with silence based on stream metadata start_time.
This ensures Whisper timestamps will be relative to recording start.
Returns: (padded_data, storage_url)
"""
if not track_data:
return b"", ""
transcript = await self.get_transcript()
# Get stream metadata start_time using PyAV
container = av.open(io.BytesIO(track_data))
try:
audio_stream = container.streams.audio[0]
# Extract start_time from stream metadata
if (
audio_stream.start_time is not None
and audio_stream.time_base is not None
):
start_time_seconds = float(
audio_stream.start_time * audio_stream.time_base
)
else:
start_time_seconds = 0.0
sample_rate = audio_stream.sample_rate
codec_name = audio_stream.codec.name
finally:
container.close()
self.logger.info(
f"Track {track_idx} stream metadata: start_time={start_time_seconds:.3f}s, sample_rate={sample_rate}",
track_idx=track_idx,
)
# If no padding needed, use original
if start_time_seconds <= 0:
storage_path = (
f"file_pipeline/{transcript.id}/tracks/original_track_{track_idx}.webm"
)
await storage.put_file(storage_path, track_data)
url = await storage.get_file_url(storage_path)
return track_data, url
# Create PyAV filter graph for padding
graph = av.filter.Graph()
# Input buffer
in_args = (
f"time_base=1/{sample_rate}:"
f"sample_rate={sample_rate}:"
f"sample_fmt=s16:"
f"channel_layout=stereo"
)
input_buffer = graph.add("abuffer", args=in_args, name="in")
# Add delay filter for padding
delay_ms = int(start_time_seconds * 1000)
delay_filter = graph.add(
"adelay", args=f"delays={delay_ms}|{delay_ms}:all=1", name="delay"
)
# Output sink
sink = graph.add("abuffersink", name="out")
# Link filters
input_buffer.link_to(delay_filter)
delay_filter.link_to(sink)
graph.configure()
# Process audio through filter
output_bytes = io.BytesIO()
output_container = av.open(output_bytes, "w", format="webm")
output_stream = output_container.add_stream("libopus", rate=sample_rate)
output_stream.channels = 2
# Reopen input for processing
input_container = av.open(io.BytesIO(track_data))
resampler = AudioResampler(format="s16", layout="stereo", rate=sample_rate)
try:
# Process frames
for frame in input_container.decode(audio=0):
# Resample to match filter requirements
resampled_frames = resampler.resample(frame)
for resampled_frame in resampled_frames:
resampled_frame.pts = frame.pts
resampled_frame.time_base = Fraction(1, sample_rate)
input_buffer.push(resampled_frame)
# Pull from filter and encode
while True:
try:
out_frame = sink.pull()
out_frame.pts = out_frame.pts if out_frame.pts else 0
out_frame.time_base = Fraction(1, sample_rate)
for packet in output_stream.encode(out_frame):
output_container.mux(packet)
except av.BlockingIOError:
break
# Flush
input_buffer.push(None)
while True:
try:
out_frame = sink.pull()
for packet in output_stream.encode(out_frame):
output_container.mux(packet)
except (av.BlockingIOError, av.EOFError):
break
# Flush encoder
for packet in output_stream.encode(None):
output_container.mux(packet)
finally:
input_container.close()
output_container.close()
padded_data = output_bytes.getvalue()
# Store padded track
storage_path = (
f"file_pipeline/{transcript.id}/tracks/padded_track_{track_idx}.webm"
)
await storage.put_file(storage_path, padded_data)
padded_url = await storage.get_file_url(storage_path)
self.logger.info(
f"Padded track {track_idx} with {start_time_seconds:.3f}s offset, stored at {storage_path}",
track_idx=track_idx,
delay_ms=delay_ms,
padded_url=padded_url,
)
return padded_data, padded_url
async def mixdown_tracks(
self,
track_datas: list[bytes],
writer: AudioFileWriterProcessor,
offsets_seconds: list[float] | None = None,
) -> None:
"""
Minimal multi-track mixdown using a PyAV filter graph (amix), no resampling.
"""
# Discover target sample rate from first decodable frame
target_sample_rate: int | None = None
for data in track_datas:
if not data:
continue
try:
container = av.open(io.BytesIO(data))
try:
for frame in container.decode(audio=0):
target_sample_rate = frame.sample_rate
break
finally:
container.close()
except Exception:
continue
if target_sample_rate:
break
if not target_sample_rate:
self.logger.warning("Mixdown skipped - no decodable audio frames found")
return
# Build PyAV filter graph:
# N abuffer (s32/stereo)
# -> optional adelay per input (for alignment)
# -> amix (s32)
# -> aformat(s16)
# -> sink
graph = av.filter.Graph()
inputs = []
valid_track_datas = [d for d in track_datas if d]
# Align offsets list with the filtered inputs (skip empties)
input_offsets_seconds = None
if offsets_seconds is not None:
input_offsets_seconds = [
offsets_seconds[i] for i, d in enumerate(track_datas) if d
]
for idx, data in enumerate(valid_track_datas):
args = (
f"time_base=1/{target_sample_rate}:"
f"sample_rate={target_sample_rate}:"
f"sample_fmt=s32:"
f"channel_layout=stereo"
)
in_ctx = graph.add("abuffer", args=args, name=f"in{idx}")
inputs.append(in_ctx)
if not inputs:
self.logger.warning("Mixdown skipped - no valid inputs for graph")
return
mixer = graph.add("amix", args=f"inputs={len(inputs)}:normalize=0", name="mix")
fmt = graph.add(
"aformat",
args=(
f"sample_fmts=s32:channel_layouts=stereo:sample_rates={target_sample_rate}"
),
name="fmt",
)
sink = graph.add("abuffersink", name="out")
# Optional per-input delay before mixing
delays_ms: list[int] = []
if input_offsets_seconds is not None:
base = min(input_offsets_seconds) if input_offsets_seconds else 0.0
delays_ms = [
max(0, int(round((o - base) * 1000))) for o in input_offsets_seconds
]
else:
delays_ms = [0 for _ in inputs]
for idx, in_ctx in enumerate(inputs):
delay_ms = delays_ms[idx] if idx < len(delays_ms) else 0
if delay_ms > 0:
# adelay requires one value per channel; use same for stereo
adelay = graph.add(
"adelay",
args=f"delays={delay_ms}|{delay_ms}:all=1",
name=f"delay{idx}",
)
in_ctx.link_to(adelay)
adelay.link_to(mixer, 0, idx)
else:
in_ctx.link_to(mixer, 0, idx)
mixer.link_to(fmt)
fmt.link_to(sink)
graph.configure()
# Open containers for decoding
containers = []
for i, d in enumerate(valid_track_datas):
try:
c = av.open(io.BytesIO(d))
containers.append(c)
except Exception as e:
self.logger.warning(
"Mixdown: failed to open container", input=i, error=str(e)
)
containers.append(None)
# Filter out Nones for decoders
containers = [c for c in containers if c is not None]
decoders = [c.decode(audio=0) for c in containers]
active = [True] * len(decoders)
# Per-input resamplers to enforce s32/stereo at the same rate (no resample of rate)
resamplers = [
AudioResampler(format="s32", layout="stereo", rate=target_sample_rate)
for _ in decoders
]
try:
# Round-robin feed frames into graph, pull mixed frames as they become available
while any(active):
for i, (dec, is_active) in enumerate(zip(decoders, active)):
if not is_active:
continue
try:
frame = next(dec)
except StopIteration:
active[i] = False
continue
# Enforce same sample rate; convert format/layout to s16/stereo (no resample)
if frame.sample_rate != target_sample_rate:
# Skip frames with differing rate
continue
out_frames = resamplers[i].resample(frame) or []
for rf in out_frames:
rf.sample_rate = target_sample_rate
rf.time_base = Fraction(1, target_sample_rate)
inputs[i].push(rf)
# Drain available mixed frames
while True:
try:
mixed = sink.pull()
except Exception:
break
mixed.sample_rate = target_sample_rate
mixed.time_base = Fraction(1, target_sample_rate)
await writer.push(mixed)
# Signal EOF to inputs and drain remaining
for in_ctx in inputs:
in_ctx.push(None)
while True:
try:
mixed = sink.pull()
except Exception:
break
mixed.sample_rate = target_sample_rate
mixed.time_base = Fraction(1, target_sample_rate)
await writer.push(mixed)
finally:
for c in containers:
c.close()
async def set_status(self, transcript_id: str, status: TranscriptStatus):
async with self.lock_transaction():
return await transcripts_controller.set_status(transcript_id, status)
async def process(self, bucket_name: str, track_keys: list[str]):
transcript = await self.get_transcript()
s3 = boto3.client(
"s3",
region_name=settings.RECORDING_STORAGE_AWS_REGION,
aws_access_key_id=settings.RECORDING_STORAGE_AWS_ACCESS_KEY_ID,
aws_secret_access_key=settings.RECORDING_STORAGE_AWS_SECRET_ACCESS_KEY,
)
storage = get_transcripts_storage()
# Pre-download bytes for all tracks for mixing and transcription
track_datas: list[bytes] = []
for key in track_keys:
try:
obj = s3.get_object(Bucket=bucket_name, Key=key)
track_datas.append(obj["Body"].read())
except Exception as e:
self.logger.warning(
"Skipping track - cannot read S3 object", key=key, error=str(e)
)
track_datas.append(b"")
# REMOVED: Filename offset extraction - not needed anymore!
# We use stream metadata start_time for padding instead
# Get stream metadata start_times for mixing (still useful for mixdown)
stream_start_times: list[float] = []
for data in track_datas:
if not data:
stream_start_times.append(0.0)
continue
container = av.open(io.BytesIO(data))
try:
audio_stream = container.streams.audio[0]
if (
audio_stream.start_time is not None
and audio_stream.time_base is not None
):
start_time = float(audio_stream.start_time * audio_stream.time_base)
else:
start_time = 0.0
stream_start_times.append(start_time)
finally:
container.close()
# Mixdown all available tracks into transcript.audio_mp3_filename, using stream metadata offsets
try:
mp3_writer = AudioFileWriterProcessor(
path=str(transcript.audio_mp3_filename)
)
await self.mixdown_tracks(track_datas, mp3_writer, stream_start_times)
await mp3_writer.flush()
except Exception as e:
self.logger.error("Mixdown failed", error=str(e))
# PAD TRACKS BEFORE TRANSCRIPTION - THIS IS THE KEY FIX!
padded_track_urls: list[str] = []
for idx, data in enumerate(track_datas):
if not data:
padded_track_urls.append("")
continue
_, padded_url = await self.pad_track_for_transcription(data, idx, storage)
padded_track_urls.append(padded_url)
self.logger.info(f"Padded track {idx} for transcription: {padded_url}")
# Transcribe PADDED tracks - timestamps will be automatically correct!
speaker_transcripts: list[TranscriptType] = []
for idx, padded_url in enumerate(padded_track_urls):
if not padded_url:
continue
try:
# Transcribe the PADDED track
t = await self.transcribe_file(padded_url, transcript.source_language)
except Exception as e:
self.logger.error(
"Transcription via default backend failed, trying local whisper",
track_idx=idx,
url=padded_url,
error=str(e),
)
try:
fallback = FileTranscriptAutoProcessor(name="whisper")
result = None
async def capture_result(r):
nonlocal result
result = r
fallback.on(capture_result)
await fallback.push(
FileTranscriptInput(
audio_url=padded_url, language=transcript.source_language
)
)
await fallback.flush()
if not result:
raise Exception("No transcript captured in fallback")
t = result
except Exception as e2:
self.logger.error(
"Skipping track - transcription failed after fallback",
track_idx=idx,
url=padded_url,
error=str(e2),
)
continue
if not t.words:
continue
# NO OFFSET ADJUSTMENT NEEDED!
# Timestamps are already correct because we transcribed padded tracks
# Just set speaker ID
for w in t.words:
w.speaker = idx
speaker_transcripts.append(t)
self.logger.info(
f"Track {idx} transcribed successfully with {len(t.words)} words",
track_idx=idx,
)
if not speaker_transcripts:
raise Exception("No valid track transcriptions")
# Merge all words and sort by timestamp
merged_words = []
for t in speaker_transcripts:
merged_words.extend(t.words)
merged_words.sort(
key=lambda w: w.start if hasattr(w, "start") and w.start is not None else 0
)
merged_transcript = TranscriptType(words=merged_words, translation=None)
await transcripts_controller.append_event(
transcript,
event="TRANSCRIPT",
data=TranscriptText(
text=merged_transcript.text, translation=merged_transcript.translation
),
)
topics = await self.detect_topics(merged_transcript, transcript.target_language)
await asyncio.gather(
self.generate_title(topics),
self.generate_summaries(topics),
return_exceptions=False,
)
await self.set_status(transcript.id, "ended")
async def transcribe_file(self, audio_url: str, language: str) -> TranscriptType:
processor = FileTranscriptAutoProcessor()
input_data = FileTranscriptInput(audio_url=audio_url, language=language)
result: TranscriptType | None = None
async def capture_result(transcript):
nonlocal result
result = transcript
processor.on(capture_result)
await processor.push(input_data)
await processor.flush()
if not result:
raise ValueError("No transcript captured")
return result
async def detect_topics(
self, transcript: TranscriptType, target_language: str
) -> list[TitleSummary]:
chunk_size = 300
topics: list[TitleSummary] = []
async def on_topic(topic: TitleSummary):
topics.append(topic)
return await self.on_topic(topic)
topic_detector = TranscriptTopicDetectorProcessor(callback=on_topic)
topic_detector.set_pipeline(self.empty_pipeline)
for i in range(0, len(transcript.words), chunk_size):
chunk_words = transcript.words[i : i + chunk_size]
if not chunk_words:
continue
chunk_transcript = TranscriptType(
words=chunk_words, translation=transcript.translation
)
await topic_detector.push(chunk_transcript)
await topic_detector.flush()
return topics
async def generate_title(self, topics: list[TitleSummary]):
if not topics:
self.logger.warning("No topics for title generation")
return
processor = TranscriptFinalTitleProcessor(callback=self.on_title)
processor.set_pipeline(self.empty_pipeline)
for topic in topics:
await processor.push(topic)
await processor.flush()
async def generate_summaries(self, topics: list[TitleSummary]):
if not topics:
self.logger.warning("No topics for summary generation")
return
transcript = await self.get_transcript()
processor = TranscriptFinalSummaryProcessor(
transcript=transcript,
callback=self.on_long_summary,
on_short_summary=self.on_short_summary,
)
processor.set_pipeline(self.empty_pipeline)
for topic in topics:
await processor.push(topic)
await processor.flush()
@shared_task
@asynctask
async def task_pipeline_multitrack_process(
*, transcript_id: str, bucket_name: str, track_keys: list[str]
):
pipeline = PipelineMainMultitrack(transcript_id=transcript_id)
try:
await pipeline.set_status(transcript_id, "processing")
await pipeline.process(bucket_name, track_keys)
except Exception:
await pipeline.set_status(transcript_id, "error")
raise
post_chain = chain(
task_cleanup_consent.si(transcript_id=transcript_id),
task_pipeline_post_to_zulip.si(transcript_id=transcript_id),
task_send_webhook_if_needed.si(transcript_id=transcript_id),
)
post_chain.delay()

View File

@@ -1,109 +0,0 @@
"""
Topic processing utilities
==========================
Shared topic detection, title generation, and summarization logic
used across file and multitrack pipelines.
"""
from typing import Callable
import structlog
from reflector.db.transcripts import Transcript
from reflector.processors import (
TranscriptFinalSummaryProcessor,
TranscriptFinalTitleProcessor,
TranscriptTopicDetectorProcessor,
)
from reflector.processors.types import TitleSummary
from reflector.processors.types import Transcript as TranscriptType
class EmptyPipeline:
def __init__(self, logger: structlog.BoundLogger):
self.logger = logger
def get_pref(self, k, d=None):
return d
async def emit(self, event):
pass
async def detect_topics(
transcript: TranscriptType,
target_language: str,
*,
on_topic_callback: Callable,
empty_pipeline: EmptyPipeline,
) -> list[TitleSummary]:
chunk_size = 300
topics: list[TitleSummary] = []
async def on_topic(topic: TitleSummary):
topics.append(topic)
return await on_topic_callback(topic)
topic_detector = TranscriptTopicDetectorProcessor(callback=on_topic)
topic_detector.set_pipeline(empty_pipeline)
for i in range(0, len(transcript.words), chunk_size):
chunk_words = transcript.words[i : i + chunk_size]
if not chunk_words:
continue
chunk_transcript = TranscriptType(
words=chunk_words, translation=transcript.translation
)
await topic_detector.push(chunk_transcript)
await topic_detector.flush()
return topics
async def generate_title(
topics: list[TitleSummary],
*,
on_title_callback: Callable,
empty_pipeline: EmptyPipeline,
logger: structlog.BoundLogger,
):
if not topics:
logger.warning("No topics for title generation")
return
processor = TranscriptFinalTitleProcessor(callback=on_title_callback)
processor.set_pipeline(empty_pipeline)
for topic in topics:
await processor.push(topic)
await processor.flush()
async def generate_summaries(
topics: list[TitleSummary],
transcript: Transcript,
*,
on_long_summary_callback: Callable,
on_short_summary_callback: Callable,
empty_pipeline: EmptyPipeline,
logger: structlog.BoundLogger,
):
if not topics:
logger.warning("No topics for summary generation")
return
processor = TranscriptFinalSummaryProcessor(
transcript=transcript,
callback=on_long_summary_callback,
on_short_summary=on_short_summary_callback,
)
processor.set_pipeline(empty_pipeline)
for topic in topics:
await processor.push(topic)
await processor.flush()

View File

@@ -1,34 +0,0 @@
from reflector.processors.file_transcript import FileTranscriptInput
from reflector.processors.file_transcript_auto import FileTranscriptAutoProcessor
from reflector.processors.types import Transcript as TranscriptType
async def transcribe_file_with_processor(
audio_url: str,
language: str,
processor_name: str | None = None,
) -> TranscriptType:
processor = (
FileTranscriptAutoProcessor(name=processor_name)
if processor_name
else FileTranscriptAutoProcessor()
)
input_data = FileTranscriptInput(audio_url=audio_url, language=language)
result: TranscriptType | None = None
async def capture_result(transcript):
nonlocal result
result = transcript
processor.on(capture_result)
await processor.push(input_data)
await processor.flush()
if not result:
processor_label = processor_name or "default"
raise ValueError(
f"No transcript captured from {processor_label} processor for audio: {audio_url}"
)
return result

View File

@@ -0,0 +1,9 @@
"""Platform type definitions.
This module exists solely to define the Platform literal type without any imports,
preventing circular import issues when used across the codebase.
"""
from typing import Literal
Platform = Literal["whereby", "daily"]

View File

@@ -56,16 +56,6 @@ class FileTranscriptModalProcessor(FileTranscriptProcessor):
}, },
follow_redirects=True, follow_redirects=True,
) )
if response.status_code != 200:
error_body = response.text
self.logger.error(
"Modal API error",
audio_url=data.audio_url,
status_code=response.status_code,
error_body=error_body,
)
response.raise_for_status() response.raise_for_status()
result = response.json() result = response.json()

View File

@@ -165,7 +165,6 @@ class SummaryBuilder:
self.llm: LLM = llm self.llm: LLM = llm
self.model_name: str = llm.model_name self.model_name: str = llm.model_name
self.logger = logger or structlog.get_logger() self.logger = logger or structlog.get_logger()
self.participant_instructions: str | None = None
if filename: if filename:
self.read_transcript_from_file(filename) self.read_transcript_from_file(filename)
@@ -192,61 +191,14 @@ class SummaryBuilder:
self, prompt: str, output_cls: Type[T], tone_name: str | None = None self, prompt: str, output_cls: Type[T], tone_name: str | None = None
) -> T: ) -> T:
"""Generic function to get structured output from LLM for non-function-calling models.""" """Generic function to get structured output from LLM for non-function-calling models."""
# Add participant instructions to the prompt if available
enhanced_prompt = self._enhance_prompt_with_participants(prompt)
return await self.llm.get_structured_response( return await self.llm.get_structured_response(
enhanced_prompt, [self.transcript], output_cls, tone_name=tone_name prompt, [self.transcript], output_cls, tone_name=tone_name
) )
async def _get_response(
self, prompt: str, texts: list[str], tone_name: str | None = None
) -> str:
"""Get text response with automatic participant instructions injection."""
enhanced_prompt = self._enhance_prompt_with_participants(prompt)
return await self.llm.get_response(enhanced_prompt, texts, tone_name=tone_name)
def _enhance_prompt_with_participants(self, prompt: str) -> str:
"""Add participant instructions to any prompt if participants are known."""
if self.participant_instructions:
self.logger.debug("Adding participant instructions to prompt")
return f"{prompt}\n\n{self.participant_instructions}"
return prompt
# ---------------------------------------------------------------------------- # ----------------------------------------------------------------------------
# Participants # Participants
# ---------------------------------------------------------------------------- # ----------------------------------------------------------------------------
def set_known_participants(self, participants: list[str]) -> None:
"""
Set known participants directly without LLM identification.
This is used when participants are already identified and stored.
They are appended at the end of the transcript, providing more context for the assistant.
"""
if not participants:
self.logger.warning("No participants provided")
return
self.logger.info(
"Using known participants",
participants=participants,
)
participants_md = self.format_list_md(participants)
self.transcript += f"\n\n# Participants\n\n{participants_md}"
# Set instructions that will be automatically added to all prompts
participants_list = ", ".join(participants)
self.participant_instructions = dedent(
f"""
# IMPORTANT: Participant Names
The following participants are identified in this conversation: {participants_list}
You MUST use these specific participant names when referring to people in your response.
Do NOT use generic terms like "a participant", "someone", "attendee", "Speaker 1", "Speaker 2", etc.
Always refer to people by their actual names (e.g., "John suggested..." not "A participant suggested...").
"""
).strip()
async def identify_participants(self) -> None: async def identify_participants(self) -> None:
""" """
From a transcript, try to identify the participants using TreeSummarize with structured output. From a transcript, try to identify the participants using TreeSummarize with structured output.
@@ -280,19 +232,6 @@ class SummaryBuilder:
if unique_participants: if unique_participants:
participants_md = self.format_list_md(unique_participants) participants_md = self.format_list_md(unique_participants)
self.transcript += f"\n\n# Participants\n\n{participants_md}" self.transcript += f"\n\n# Participants\n\n{participants_md}"
# Set instructions that will be automatically added to all prompts
participants_list = ", ".join(unique_participants)
self.participant_instructions = dedent(
f"""
# IMPORTANT: Participant Names
The following participants are identified in this conversation: {participants_list}
You MUST use these specific participant names when referring to people in your response.
Do NOT use generic terms like "a participant", "someone", "attendee", "Speaker 1", "Speaker 2", etc.
Always refer to people by their actual names (e.g., "John suggested..." not "A participant suggested...").
"""
).strip()
else: else:
self.logger.warning("No participants identified in the transcript") self.logger.warning("No participants identified in the transcript")
@@ -379,13 +318,13 @@ class SummaryBuilder:
for subject in self.subjects: for subject in self.subjects:
detailed_prompt = DETAILED_SUBJECT_PROMPT_TEMPLATE.format(subject=subject) detailed_prompt = DETAILED_SUBJECT_PROMPT_TEMPLATE.format(subject=subject)
detailed_response = await self._get_response( detailed_response = await self.llm.get_response(
detailed_prompt, [self.transcript], tone_name="Topic assistant" detailed_prompt, [self.transcript], tone_name="Topic assistant"
) )
paragraph_prompt = PARAGRAPH_SUMMARY_PROMPT paragraph_prompt = PARAGRAPH_SUMMARY_PROMPT
paragraph_response = await self._get_response( paragraph_response = await self.llm.get_response(
paragraph_prompt, [str(detailed_response)], tone_name="Topic summarizer" paragraph_prompt, [str(detailed_response)], tone_name="Topic summarizer"
) )
@@ -406,7 +345,7 @@ class SummaryBuilder:
recap_prompt = RECAP_PROMPT recap_prompt = RECAP_PROMPT
recap_response = await self._get_response( recap_response = await self.llm.get_response(
recap_prompt, [summaries_text], tone_name="Recap summarizer" recap_prompt, [summaries_text], tone_name="Recap summarizer"
) )

View File

@@ -26,25 +26,7 @@ class TranscriptFinalSummaryProcessor(Processor):
async def get_summary_builder(self, text) -> SummaryBuilder: async def get_summary_builder(self, text) -> SummaryBuilder:
builder = SummaryBuilder(self.llm, logger=self.logger) builder = SummaryBuilder(self.llm, logger=self.logger)
builder.set_transcript(text) builder.set_transcript(text)
# Use known participants if available, otherwise identify them
if self.transcript and self.transcript.participants:
# Extract participant names from the stored participants
participant_names = [p.name for p in self.transcript.participants if p.name]
if participant_names:
self.logger.info(
f"Using {len(participant_names)} known participants from transcript"
)
builder.set_known_participants(participant_names)
else:
self.logger.info(
"Participants field exists but is empty, identifying participants"
)
await builder.identify_participants() await builder.identify_participants()
else:
self.logger.info("No participants stored, identifying participants")
await builder.identify_participants()
await builder.generate_summary() await builder.generate_summary()
return builder return builder
@@ -67,30 +49,18 @@ class TranscriptFinalSummaryProcessor(Processor):
speakermap = {} speakermap = {}
if self.transcript: if self.transcript:
speakermap = { speakermap = {
p.speaker: p.name participant["speaker"]: participant["name"]
for p in (self.transcript.participants or []) for participant in self.transcript.participants
if p.speaker is not None and p.name
} }
self.logger.info(
f"Built speaker map with {len(speakermap)} participants",
speakermap=speakermap,
)
# build the transcript as a single string # build the transcript as a single string
# Replace speaker IDs with actual participant names if available # XXX: unsure if the participants name as replaced directly in speaker ?
text_transcript = [] text_transcript = []
unique_speakers = set()
for topic in self.chunks: for topic in self.chunks:
for segment in topic.transcript.as_segments(): for segment in topic.transcript.as_segments():
name = speakermap.get(segment.speaker, f"Speaker {segment.speaker}") name = speakermap.get(segment.speaker, f"Speaker {segment.speaker}")
unique_speakers.add((segment.speaker, name))
text_transcript.append(f"{name}: {segment.text}") text_transcript.append(f"{name}: {segment.text}")
self.logger.info(
f"Built transcript with {len(unique_speakers)} unique speakers",
speakers=list(unique_speakers),
)
text_transcript = "\n".join(text_transcript) text_transcript = "\n".join(text_transcript)
last_chunk = self.chunks[-1] last_chunk = self.chunks[-1]

View File

@@ -1,6 +1,6 @@
from textwrap import dedent from textwrap import dedent
from pydantic import AliasChoices, BaseModel, Field from pydantic import BaseModel, ConfigDict, Field
from reflector.llm import LLM from reflector.llm import LLM
from reflector.processors.base import Processor from reflector.processors.base import Processor
@@ -34,13 +34,13 @@ TOPIC_PROMPT = dedent(
class TopicResponse(BaseModel): class TopicResponse(BaseModel):
"""Structured response for topic detection""" """Structured response for topic detection"""
model_config = ConfigDict(populate_by_name=True)
title: str = Field( title: str = Field(
description="A descriptive title for the topic being discussed", description="A descriptive title for the topic being discussed", alias="Title"
validation_alias=AliasChoices("title", "Title"),
) )
summary: str = Field( summary: str = Field(
description="A concise 1-2 sentence summary of the discussion", description="A concise 1-2 sentence summary of the discussion", alias="Summary"
validation_alias=AliasChoices("summary", "Summary"),
) )

View File

@@ -1,5 +0,0 @@
from typing import Literal
Platform = Literal["whereby", "daily"]
WHEREBY_PLATFORM: Platform = "whereby"
DAILY_PLATFORM: Platform = "daily"

View File

@@ -1,7 +1,7 @@
from pydantic.types import PositiveInt from pydantic.types import PositiveInt
from pydantic_settings import BaseSettings, SettingsConfigDict from pydantic_settings import BaseSettings, SettingsConfigDict
from reflector.schemas.platform import WHEREBY_PLATFORM, Platform from reflector.platform_types import Platform
from reflector.utils.string import NonEmptyString from reflector.utils.string import NonEmptyString
@@ -48,17 +48,14 @@ class Settings(BaseSettings):
TRANSCRIPT_STORAGE_AWS_ACCESS_KEY_ID: str | None = None TRANSCRIPT_STORAGE_AWS_ACCESS_KEY_ID: str | None = None
TRANSCRIPT_STORAGE_AWS_SECRET_ACCESS_KEY: str | None = None TRANSCRIPT_STORAGE_AWS_SECRET_ACCESS_KEY: str | None = None
# Platform-specific recording storage (follows {PREFIX}_STORAGE_AWS_{CREDENTIAL} pattern) # Recording storage
# Whereby storage configuration RECORDING_STORAGE_BACKEND: str | None = None
WHEREBY_STORAGE_AWS_BUCKET_NAME: str | None = None
WHEREBY_STORAGE_AWS_REGION: str | None = None
WHEREBY_STORAGE_AWS_ACCESS_KEY_ID: str | None = None
WHEREBY_STORAGE_AWS_SECRET_ACCESS_KEY: str | None = None
# Daily.co storage configuration # Recording storage configuration for AWS
DAILYCO_STORAGE_AWS_BUCKET_NAME: str | None = None RECORDING_STORAGE_AWS_BUCKET_NAME: str = "recording-bucket"
DAILYCO_STORAGE_AWS_REGION: str | None = None RECORDING_STORAGE_AWS_REGION: str = "us-east-1"
DAILYCO_STORAGE_AWS_ROLE_ARN: str | None = None RECORDING_STORAGE_AWS_ACCESS_KEY_ID: str | None = None
RECORDING_STORAGE_AWS_SECRET_ACCESS_KEY: str | None = None
# Translate into the target language # Translate into the target language
TRANSLATION_BACKEND: str = "passthrough" TRANSLATION_BACKEND: str = "passthrough"
@@ -128,6 +125,8 @@ class Settings(BaseSettings):
WHEREBY_API_URL: str = "https://api.whereby.dev/v1" WHEREBY_API_URL: str = "https://api.whereby.dev/v1"
WHEREBY_API_KEY: NonEmptyString | None = None WHEREBY_API_KEY: NonEmptyString | None = None
WHEREBY_WEBHOOK_SECRET: str | None = None WHEREBY_WEBHOOK_SECRET: str | None = None
AWS_WHEREBY_ACCESS_KEY_ID: str | None = None
AWS_WHEREBY_ACCESS_KEY_SECRET: str | None = None
AWS_PROCESS_RECORDING_QUEUE_URL: str | None = None AWS_PROCESS_RECORDING_QUEUE_URL: str | None = None
SQS_POLLING_TIMEOUT_SECONDS: int = 60 SQS_POLLING_TIMEOUT_SECONDS: int = 60
@@ -135,12 +134,14 @@ class Settings(BaseSettings):
DAILY_API_KEY: str | None = None DAILY_API_KEY: str | None = None
DAILY_WEBHOOK_SECRET: str | None = None DAILY_WEBHOOK_SECRET: str | None = None
DAILY_SUBDOMAIN: str | None = None DAILY_SUBDOMAIN: str | None = None
DAILY_WEBHOOK_UUID: str | None = ( AWS_DAILY_S3_BUCKET: str | None = None
None # Webhook UUID for this environment. Not used by production code AWS_DAILY_S3_REGION: str = "us-west-2"
) AWS_DAILY_ROLE_ARN: str | None = None
# Platform Configuration # Platform Migration Feature Flags
DEFAULT_VIDEO_PLATFORM: Platform = WHEREBY_PLATFORM DAILY_MIGRATION_ENABLED: bool = False
DAILY_MIGRATION_ROOM_IDS: list[str] = []
DEFAULT_VIDEO_PLATFORM: Platform = "whereby"
# Zulip integration # Zulip integration
ZULIP_REALM: str | None = None ZULIP_REALM: str | None = None

View File

@@ -3,13 +3,6 @@ from reflector.settings import settings
def get_transcripts_storage() -> Storage: def get_transcripts_storage() -> Storage:
"""
Get storage for processed transcript files (master credentials).
Also use this for ALL our file operations with bucket override:
master = get_transcripts_storage()
master.delete_file(key, bucket=recording.bucket_name)
"""
assert settings.TRANSCRIPT_STORAGE_BACKEND assert settings.TRANSCRIPT_STORAGE_BACKEND
return Storage.get_instance( return Storage.get_instance(
name=settings.TRANSCRIPT_STORAGE_BACKEND, name=settings.TRANSCRIPT_STORAGE_BACKEND,
@@ -17,53 +10,8 @@ def get_transcripts_storage() -> Storage:
) )
def get_whereby_storage() -> Storage: def get_recordings_storage() -> Storage:
"""
Get storage config for Whereby (for passing to Whereby API).
Usage:
whereby_storage = get_whereby_storage()
key_id, secret = whereby_storage.key_credentials
whereby_api.create_meeting(
bucket=whereby_storage.bucket_name,
access_key_id=key_id,
secret=secret,
)
Do NOT use for our file operations - use get_transcripts_storage() instead.
"""
if not settings.WHEREBY_STORAGE_AWS_BUCKET_NAME:
raise ValueError(
"WHEREBY_STORAGE_AWS_BUCKET_NAME required for Whereby with AWS storage"
)
return Storage.get_instance( return Storage.get_instance(
name="aws", name=settings.RECORDING_STORAGE_BACKEND,
settings_prefix="WHEREBY_STORAGE_", settings_prefix="RECORDING_STORAGE_",
)
def get_dailyco_storage() -> Storage:
"""
Get storage config for Daily.co (for passing to Daily API).
Usage:
daily_storage = get_dailyco_storage()
daily_api.create_meeting(
bucket=daily_storage.bucket_name,
region=daily_storage.region,
role_arn=daily_storage.role_credential,
)
Do NOT use for our file operations - use get_transcripts_storage() instead.
"""
# Fail fast if platform-specific config missing
if not settings.DAILYCO_STORAGE_AWS_BUCKET_NAME:
raise ValueError(
"DAILYCO_STORAGE_AWS_BUCKET_NAME required for Daily.co with AWS storage"
)
return Storage.get_instance(
name="aws",
settings_prefix="DAILYCO_STORAGE_",
) )

View File

@@ -1,23 +1,10 @@
import importlib import importlib
from typing import BinaryIO, Union
from pydantic import BaseModel from pydantic import BaseModel
from reflector.settings import settings from reflector.settings import settings
class StorageError(Exception):
"""Base exception for storage operations."""
pass
class StoragePermissionError(StorageError):
"""Exception raised when storage operation fails due to permission issues."""
pass
class FileResult(BaseModel): class FileResult(BaseModel):
filename: str filename: str
url: str url: str
@@ -49,113 +36,26 @@ class Storage:
return cls._registry[name](**config) return cls._registry[name](**config)
# Credential properties for API passthrough async def put_file(self, filename: str, data: bytes) -> FileResult:
@property return await self._put_file(filename, data)
def bucket_name(self) -> str:
"""Default bucket name for this storage instance.""" async def _put_file(self, filename: str, data: bytes) -> FileResult:
raise NotImplementedError raise NotImplementedError
@property async def delete_file(self, filename: str):
def region(self) -> str: return await self._delete_file(filename)
"""AWS region for this storage instance."""
async def _delete_file(self, filename: str):
raise NotImplementedError raise NotImplementedError
@property async def get_file_url(self, filename: str) -> str:
def access_key_id(self) -> str | None: return await self._get_file_url(filename)
"""AWS access key ID (None for role-based auth). Prefer key_credentials property."""
return None
@property async def _get_file_url(self, filename: str) -> str:
def secret_access_key(self) -> str | None:
"""AWS secret access key (None for role-based auth). Prefer key_credentials property."""
return None
@property
def role_arn(self) -> str | None:
"""AWS IAM role ARN for role-based auth (None for key-based auth). Prefer role_credential property."""
return None
@property
def key_credentials(self) -> tuple[str, str]:
"""
Get (access_key_id, secret_access_key) for key-based auth.
Raises ValueError if storage uses IAM role instead.
"""
raise NotImplementedError raise NotImplementedError
@property async def get_file(self, filename: str):
def role_credential(self) -> str: return await self._get_file(filename)
"""
Get IAM role ARN for role-based auth. async def _get_file(self, filename: str):
Raises ValueError if storage uses access keys instead.
"""
raise NotImplementedError
async def put_file(
self, filename: str, data: Union[bytes, BinaryIO], *, bucket: str | None = None
) -> FileResult:
"""Upload data. bucket: override instance default if provided."""
return await self._put_file(filename, data, bucket=bucket)
async def _put_file(
self, filename: str, data: Union[bytes, BinaryIO], *, bucket: str | None = None
) -> FileResult:
raise NotImplementedError
async def delete_file(self, filename: str, *, bucket: str | None = None):
"""Delete file. bucket: override instance default if provided."""
return await self._delete_file(filename, bucket=bucket)
async def _delete_file(self, filename: str, *, bucket: str | None = None):
raise NotImplementedError
async def get_file_url(
self,
filename: str,
operation: str = "get_object",
expires_in: int = 3600,
*,
bucket: str | None = None,
) -> str:
"""Generate presigned URL. bucket: override instance default if provided."""
return await self._get_file_url(filename, operation, expires_in, bucket=bucket)
async def _get_file_url(
self,
filename: str,
operation: str = "get_object",
expires_in: int = 3600,
*,
bucket: str | None = None,
) -> str:
raise NotImplementedError
async def get_file(self, filename: str, *, bucket: str | None = None):
"""Download file. bucket: override instance default if provided."""
return await self._get_file(filename, bucket=bucket)
async def _get_file(self, filename: str, *, bucket: str | None = None):
raise NotImplementedError
async def list_objects(
self, prefix: str = "", *, bucket: str | None = None
) -> list[str]:
"""List object keys. bucket: override instance default if provided."""
return await self._list_objects(prefix, bucket=bucket)
async def _list_objects(
self, prefix: str = "", *, bucket: str | None = None
) -> list[str]:
raise NotImplementedError
async def stream_to_fileobj(
self, filename: str, fileobj: BinaryIO, *, bucket: str | None = None
):
"""Stream file directly to file object without loading into memory.
bucket: override instance default if provided."""
return await self._stream_to_fileobj(filename, fileobj, bucket=bucket)
async def _stream_to_fileobj(
self, filename: str, fileobj: BinaryIO, *, bucket: str | None = None
):
raise NotImplementedError raise NotImplementedError

View File

@@ -1,236 +1,79 @@
from functools import wraps
from typing import BinaryIO, Union
import aioboto3 import aioboto3
from botocore.config import Config
from botocore.exceptions import ClientError
from reflector.logger import logger from reflector.logger import logger
from reflector.storage.base import FileResult, Storage, StoragePermissionError from reflector.storage.base import FileResult, Storage
def handle_s3_client_errors(operation_name: str):
"""Decorator to handle S3 ClientError with bucket-aware messaging.
Args:
operation_name: Human-readable operation name for error messages (e.g., "upload", "delete")
"""
def decorator(func):
@wraps(func)
async def wrapper(self, *args, **kwargs):
bucket = kwargs.get("bucket")
try:
return await func(self, *args, **kwargs)
except ClientError as e:
error_code = e.response.get("Error", {}).get("Code")
if error_code in ("AccessDenied", "NoSuchBucket"):
actual_bucket = bucket or self._bucket_name
bucket_context = (
f"overridden bucket '{actual_bucket}'"
if bucket
else f"default bucket '{actual_bucket}'"
)
raise StoragePermissionError(
f"S3 {operation_name} failed for {bucket_context}: {error_code}. "
f"Check TRANSCRIPT_STORAGE_AWS_* credentials have permission."
) from e
raise
return wrapper
return decorator
class AwsStorage(Storage): class AwsStorage(Storage):
"""AWS S3 storage with bucket override for multi-platform recording architecture.
Master credentials access all buckets via optional bucket parameter in operations."""
def __init__( def __init__(
self, self,
aws_access_key_id: str,
aws_secret_access_key: str,
aws_bucket_name: str, aws_bucket_name: str,
aws_region: str, aws_region: str,
aws_access_key_id: str | None = None,
aws_secret_access_key: str | None = None,
aws_role_arn: str | None = None,
): ):
if not aws_access_key_id:
raise ValueError("Storage `aws_storage` require `aws_access_key_id`")
if not aws_secret_access_key:
raise ValueError("Storage `aws_storage` require `aws_secret_access_key`")
if not aws_bucket_name: if not aws_bucket_name:
raise ValueError("Storage `aws_storage` require `aws_bucket_name`") raise ValueError("Storage `aws_storage` require `aws_bucket_name`")
if not aws_region: if not aws_region:
raise ValueError("Storage `aws_storage` require `aws_region`") raise ValueError("Storage `aws_storage` require `aws_region`")
if not aws_access_key_id and not aws_role_arn:
raise ValueError(
"Storage `aws_storage` require either `aws_access_key_id` or `aws_role_arn`"
)
if aws_role_arn and (aws_access_key_id or aws_secret_access_key):
raise ValueError(
"Storage `aws_storage` cannot use both `aws_role_arn` and access keys"
)
super().__init__() super().__init__()
self._bucket_name = aws_bucket_name self.aws_bucket_name = aws_bucket_name
self._region = aws_region
self._access_key_id = aws_access_key_id
self._secret_access_key = aws_secret_access_key
self._role_arn = aws_role_arn
self.aws_folder = "" self.aws_folder = ""
if "/" in aws_bucket_name: if "/" in aws_bucket_name:
self._bucket_name, self.aws_folder = aws_bucket_name.split("/", 1) self.aws_bucket_name, self.aws_folder = aws_bucket_name.split("/", 1)
self.boto_config = Config(retries={"max_attempts": 3, "mode": "adaptive"})
self.session = aioboto3.Session( self.session = aioboto3.Session(
aws_access_key_id=aws_access_key_id, aws_access_key_id=aws_access_key_id,
aws_secret_access_key=aws_secret_access_key, aws_secret_access_key=aws_secret_access_key,
region_name=aws_region, region_name=aws_region,
) )
self.base_url = f"https://{self._bucket_name}.s3.amazonaws.com/" self.base_url = f"https://{aws_bucket_name}.s3.amazonaws.com/"
# Implement credential properties async def _put_file(self, filename: str, data: bytes) -> FileResult:
@property bucket = self.aws_bucket_name
def bucket_name(self) -> str: folder = self.aws_folder
return self._bucket_name logger.info(f"Uploading {filename} to S3 {bucket}/{folder}")
s3filename = f"{folder}/{filename}" if folder else filename
@property async with self.session.client("s3") as client:
def region(self) -> str: await client.put_object(
return self._region Bucket=bucket,
Key=s3filename,
@property Body=data,
def access_key_id(self) -> str | None:
return self._access_key_id
@property
def secret_access_key(self) -> str | None:
return self._secret_access_key
@property
def role_arn(self) -> str | None:
return self._role_arn
@property
def key_credentials(self) -> tuple[str, str]:
"""Get (access_key_id, secret_access_key) for key-based auth."""
if self._role_arn:
raise ValueError(
"Storage uses IAM role authentication. "
"Use role_credential property instead of key_credentials."
) )
if not self._access_key_id or not self._secret_access_key:
raise ValueError("Storage access key credentials not configured")
return (self._access_key_id, self._secret_access_key)
@property async def _get_file_url(self, filename: str) -> FileResult:
def role_credential(self) -> str: bucket = self.aws_bucket_name
"""Get IAM role ARN for role-based auth."""
if self._access_key_id or self._secret_access_key:
raise ValueError(
"Storage uses access key authentication. "
"Use key_credentials property instead of role_credential."
)
if not self._role_arn:
raise ValueError("Storage IAM role ARN not configured")
return self._role_arn
@handle_s3_client_errors("upload")
async def _put_file(
self, filename: str, data: Union[bytes, BinaryIO], *, bucket: str | None = None
) -> FileResult:
actual_bucket = bucket or self._bucket_name
folder = self.aws_folder folder = self.aws_folder
s3filename = f"{folder}/{filename}" if folder else filename s3filename = f"{folder}/{filename}" if folder else filename
logger.info(f"Uploading {filename} to S3 {actual_bucket}/{folder}") async with self.session.client("s3") as client:
async with self.session.client("s3", config=self.boto_config) as client:
if isinstance(data, bytes):
await client.put_object(Bucket=actual_bucket, Key=s3filename, Body=data)
else:
# boto3 reads file-like object in chunks
# avoids creating extra memory copy vs bytes.getvalue() approach
await client.upload_fileobj(data, Bucket=actual_bucket, Key=s3filename)
url = await self._get_file_url(filename, bucket=bucket)
return FileResult(filename=filename, url=url)
@handle_s3_client_errors("presign")
async def _get_file_url(
self,
filename: str,
operation: str = "get_object",
expires_in: int = 3600,
*,
bucket: str | None = None,
) -> str:
actual_bucket = bucket or self._bucket_name
folder = self.aws_folder
s3filename = f"{folder}/{filename}" if folder else filename
async with self.session.client("s3", config=self.boto_config) as client:
presigned_url = await client.generate_presigned_url( presigned_url = await client.generate_presigned_url(
operation, "get_object",
Params={"Bucket": actual_bucket, "Key": s3filename}, Params={"Bucket": bucket, "Key": s3filename},
ExpiresIn=expires_in, ExpiresIn=3600,
) )
return presigned_url return presigned_url
@handle_s3_client_errors("delete") async def _delete_file(self, filename: str):
async def _delete_file(self, filename: str, *, bucket: str | None = None): bucket = self.aws_bucket_name
actual_bucket = bucket or self._bucket_name
folder = self.aws_folder folder = self.aws_folder
logger.info(f"Deleting {filename} from S3 {actual_bucket}/{folder}") logger.info(f"Deleting {filename} from S3 {bucket}/{folder}")
s3filename = f"{folder}/{filename}" if folder else filename s3filename = f"{folder}/{filename}" if folder else filename
async with self.session.client("s3", config=self.boto_config) as client: async with self.session.client("s3") as client:
await client.delete_object(Bucket=actual_bucket, Key=s3filename) await client.delete_object(Bucket=bucket, Key=s3filename)
@handle_s3_client_errors("download") async def _get_file(self, filename: str):
async def _get_file(self, filename: str, *, bucket: str | None = None): bucket = self.aws_bucket_name
actual_bucket = bucket or self._bucket_name
folder = self.aws_folder folder = self.aws_folder
logger.info(f"Downloading {filename} from S3 {actual_bucket}/{folder}") logger.info(f"Downloading {filename} from S3 {bucket}/{folder}")
s3filename = f"{folder}/{filename}" if folder else filename s3filename = f"{folder}/{filename}" if folder else filename
async with self.session.client("s3", config=self.boto_config) as client: async with self.session.client("s3") as client:
response = await client.get_object(Bucket=actual_bucket, Key=s3filename) response = await client.get_object(Bucket=bucket, Key=s3filename)
return await response["Body"].read() return await response["Body"].read()
@handle_s3_client_errors("list_objects")
async def _list_objects(
self, prefix: str = "", *, bucket: str | None = None
) -> list[str]:
actual_bucket = bucket or self._bucket_name
folder = self.aws_folder
# Combine folder and prefix
s3prefix = f"{folder}/{prefix}" if folder else prefix
logger.info(f"Listing objects from S3 {actual_bucket} with prefix '{s3prefix}'")
keys = []
async with self.session.client("s3", config=self.boto_config) as client:
paginator = client.get_paginator("list_objects_v2")
async for page in paginator.paginate(Bucket=actual_bucket, Prefix=s3prefix):
if "Contents" in page:
for obj in page["Contents"]:
# Strip folder prefix from keys if present
key = obj["Key"]
if folder:
if key.startswith(f"{folder}/"):
key = key[len(folder) + 1 :]
elif key == folder:
# Skip folder marker itself
continue
keys.append(key)
return keys
@handle_s3_client_errors("stream")
async def _stream_to_fileobj(
self, filename: str, fileobj: BinaryIO, *, bucket: str | None = None
):
"""Stream file from S3 directly to file object without loading into memory."""
actual_bucket = bucket or self._bucket_name
folder = self.aws_folder
logger.info(f"Streaming {filename} from S3 {actual_bucket}/{folder}")
s3filename = f"{folder}/{filename}" if folder else filename
async with self.session.client("s3", config=self.boto_config) as client:
await client.download_fileobj(
Bucket=actual_bucket, Key=s3filename, Fileobj=fileobj
)
Storage.register("aws", AwsStorage) Storage.register("aws", AwsStorage)

View File

@@ -1,26 +0,0 @@
from reflector.utils.string import NonEmptyString
DailyRoomName = str
def extract_base_room_name(daily_room_name: DailyRoomName) -> NonEmptyString:
"""
Extract base room name from Daily.co timestamped room name.
Daily.co creates rooms with timestamp suffix: {base_name}-YYYYMMDDHHMMSS
This function removes the timestamp to get the original room name.
Examples:
"daily-20251020193458""daily"
"daily-2-20251020193458""daily-2"
"my-room-name-20251020193458""my-room-name"
Args:
daily_room_name: Full Daily.co room name with optional timestamp
Returns:
Base room name without timestamp suffix
"""
base_name = daily_room_name.rsplit("-", 1)[0]
assert base_name, f"Extracted base name is empty from: {daily_room_name}"
return base_name

View File

@@ -1,9 +0,0 @@
from datetime import datetime, timezone
def parse_datetime_with_timezone(iso_string: str) -> datetime:
"""Parse ISO datetime string and ensure timezone awareness (defaults to UTC if naive)."""
dt = datetime.fromisoformat(iso_string)
if dt.tzinfo is None:
dt = dt.replace(tzinfo=timezone.utc)
return dt

View File

@@ -1,4 +1,4 @@
from typing import Annotated, TypeVar from typing import Annotated
from pydantic import Field, TypeAdapter, constr from pydantic import Field, TypeAdapter, constr
@@ -21,12 +21,3 @@ def try_parse_non_empty_string(s: str) -> NonEmptyString | None:
if not s: if not s:
return None return None
return parse_non_empty_string(s) return parse_non_empty_string(s)
T = TypeVar("T", bound=str)
def assert_equal[T](s1: T, s2: T) -> T:
if s1 != s2:
raise ValueError(f"assert_equal: {s1} != {s2}")
return s1

View File

@@ -1,37 +0,0 @@
"""URL manipulation utilities."""
from urllib.parse import parse_qs, urlencode, urlparse, urlunparse
def add_query_param(url: str, key: str, value: str) -> str:
"""
Add or update a query parameter in a URL.
Properly handles URLs with or without existing query parameters,
preserving fragments and encoding special characters.
Args:
url: The URL to modify
key: The query parameter name
value: The query parameter value
Returns:
The URL with the query parameter added or updated
Examples:
>>> add_query_param("https://example.com/room", "t", "token123")
'https://example.com/room?t=token123'
>>> add_query_param("https://example.com/room?existing=param", "t", "token123")
'https://example.com/room?existing=param&t=token123'
"""
parsed = urlparse(url)
query_params = parse_qs(parsed.query, keep_blank_values=True)
query_params[key] = [value]
new_query = urlencode(query_params, doseq=True)
new_parsed = parsed._replace(query=new_query)
return urlunparse(new_parsed)

View File

@@ -1,3 +1,10 @@
# Video Platform Abstraction Layer
"""
This module provides an abstraction layer for different video conferencing platforms.
It allows seamless switching between providers (Whereby, Daily.co, etc.) without
changing the core application logic.
"""
from .base import VideoPlatformClient from .base import VideoPlatformClient
from .models import MeetingData, VideoPlatformConfig from .models import MeetingData, VideoPlatformConfig
from .registry import get_platform_client, register_platform from .registry import get_platform_client, register_platform

View File

@@ -2,18 +2,17 @@ from abc import ABC, abstractmethod
from datetime import datetime from datetime import datetime
from typing import TYPE_CHECKING, Any, Dict, Optional from typing import TYPE_CHECKING, Any, Dict, Optional
from ..schemas.platform import Platform from reflector.platform_types import Platform
from ..utils.string import NonEmptyString
from .models import MeetingData, SessionData, VideoPlatformConfig from .models import MeetingData, VideoPlatformConfig
if TYPE_CHECKING: if TYPE_CHECKING:
from reflector.db.rooms import Room from reflector.db.rooms import Room
# separator doesn't guarantee there's no more "ROOM_PREFIX_SEPARATOR" strings in room name
ROOM_PREFIX_SEPARATOR = "-"
class VideoPlatformClient(ABC): class VideoPlatformClient(ABC):
"""Abstract base class for video platform integrations."""
PLATFORM_NAME: Platform PLATFORM_NAME: Platform
def __init__(self, config: VideoPlatformConfig): def __init__(self, config: VideoPlatformConfig):
@@ -21,30 +20,36 @@ class VideoPlatformClient(ABC):
@abstractmethod @abstractmethod
async def create_meeting( async def create_meeting(
self, room_name_prefix: NonEmptyString, end_date: datetime, room: "Room" self, room_name_prefix: str, end_date: datetime, room: "Room"
) -> MeetingData: ) -> MeetingData:
"""Create a new meeting room."""
pass pass
@abstractmethod @abstractmethod
async def get_room_sessions(self, room_name: str) -> list[SessionData]: async def get_room_sessions(self, room_name: str) -> Dict[str, Any]:
"""Get session history for a room.""" """Get session information for a room."""
pass pass
@abstractmethod @abstractmethod
async def delete_room(self, room_name: str) -> bool: async def delete_room(self, room_name: str) -> bool:
"""Delete a room. Returns True if successful."""
pass pass
@abstractmethod @abstractmethod
async def upload_logo(self, room_name: str, logo_path: str) -> bool: async def upload_logo(self, room_name: str, logo_path: str) -> bool:
"""Upload a logo to the room. Returns True if successful."""
pass pass
@abstractmethod @abstractmethod
def verify_webhook_signature( def verify_webhook_signature(
self, body: bytes, signature: str, timestamp: Optional[str] = None self, body: bytes, signature: str, timestamp: Optional[str] = None
) -> bool: ) -> bool:
"""Verify webhook signature for security."""
pass pass
def format_recording_config(self, room: "Room") -> Dict[str, Any]: def format_recording_config(self, room: "Room") -> Dict[str, Any]:
"""Format recording configuration for the platform.
Can be overridden by specific implementations."""
if room.recording_type == "cloud" and self.config.s3_bucket: if room.recording_type == "cloud" and self.config.s3_bucket:
return { return {
"type": room.recording_type, "type": room.recording_type,

View File

@@ -1,4 +1,3 @@
import base64
import hmac import hmac
from datetime import datetime from datetime import datetime
from hashlib import sha256 from hashlib import sha256
@@ -7,18 +6,11 @@ from typing import Any, Dict, Optional
import httpx import httpx
from reflector.db.daily_participant_sessions import (
daily_participant_sessions_controller,
)
from reflector.db.rooms import Room from reflector.db.rooms import Room
from reflector.logger import logger from reflector.platform_types import Platform
from reflector.storage import get_dailyco_storage
from ..schemas.platform import Platform from .base import VideoPlatformClient
from ..utils.daily import DailyRoomName from .models import MeetingData, RecordingType, VideoPlatformConfig
from ..utils.string import NonEmptyString
from .base import ROOM_PREFIX_SEPARATOR, VideoPlatformClient
from .models import MeetingData, RecordingType, SessionData, VideoPlatformConfig
class DailyClient(VideoPlatformClient): class DailyClient(VideoPlatformClient):
@@ -37,17 +29,14 @@ class DailyClient(VideoPlatformClient):
} }
async def create_meeting( async def create_meeting(
self, room_name_prefix: NonEmptyString, end_date: datetime, room: Room self, room_name_prefix: str, end_date: datetime, room: Room
) -> MeetingData: ) -> MeetingData:
""" """Create a Daily.co room."""
Daily.co rooms vs meetings:
- We create a NEW Daily.co room for each Reflector meeting
- Daily.co meeting/session starts automatically when first participant joins
- Room auto-deletes after exp time
- Meeting.room_name stores the timestamped Daily.co room name
"""
timestamp = datetime.now().strftime(self.TIMESTAMP_FORMAT) timestamp = datetime.now().strftime(self.TIMESTAMP_FORMAT)
room_name = f"{room_name_prefix}{ROOM_PREFIX_SEPARATOR}{timestamp}" if room_name_prefix:
room_name = f"{room_name_prefix}-{timestamp}"
else:
room_name = f"room-{timestamp}"
data = { data = {
"name": room_name, "name": room_name,
@@ -64,16 +53,18 @@ class DailyClient(VideoPlatformClient):
}, },
} }
# Only configure recordings_bucket if recording is enabled # Configure S3 bucket for recordings
if room.recording_type != self.RECORDING_NONE: # NOTE: Not checking room.recording_type - figure out later if conditional needed
daily_storage = get_dailyco_storage() assert self.config.s3_bucket, "S3 bucket must be configured"
assert daily_storage.bucket_name, "S3 bucket must be configured"
data["properties"]["recordings_bucket"] = { data["properties"]["recordings_bucket"] = {
"bucket_name": daily_storage.bucket_name, "bucket_name": self.config.s3_bucket,
"bucket_region": daily_storage.region, "bucket_region": self.config.s3_region,
"assume_role_arn": daily_storage.role_credential, "assume_role_arn": self.config.aws_role_arn,
"allow_api_access": True, "allow_api_access": True,
} }
from reflector.logger import logger
async with httpx.AsyncClient() as client: async with httpx.AsyncClient() as client:
response = await client.post( response = await client.post(
f"{self.BASE_URL}/rooms", f"{self.BASE_URL}/rooms",
@@ -91,6 +82,7 @@ class DailyClient(VideoPlatformClient):
response.raise_for_status() response.raise_for_status()
result = response.json() result = response.json()
# Format response to match our standard
room_url = result["url"] room_url = result["url"]
return MeetingData( return MeetingData(
@@ -102,49 +94,19 @@ class DailyClient(VideoPlatformClient):
extra_data=result, extra_data=result,
) )
async def get_room_sessions(self, room_name: str) -> list[SessionData]: async def get_room_sessions(self, room_name: str) -> Dict[str, Any]:
"""Get room session history from database (webhook-stored sessions). """Get Daily.co room information."""
async with httpx.AsyncClient() as client:
Daily.co doesn't provide historical session API, so we query our database response = await client.get(
where participant.joined/left webhooks are stored. f"{self.BASE_URL}/rooms/{room_name}",
""" headers=self.headers,
from reflector.db.meetings import meetings_controller timeout=self.TIMEOUT,
meeting = await meetings_controller.get_by_room_name(room_name)
if not meeting:
return []
sessions = await daily_participant_sessions_controller.get_by_meeting(
meeting.id
) )
response.raise_for_status()
return [ return response.json()
SessionData(
session_id=s.id,
started_at=s.joined_at,
ended_at=s.left_at,
)
for s in sessions
]
async def get_room_presence(self, room_name: str) -> Dict[str, Any]: async def get_room_presence(self, room_name: str) -> Dict[str, Any]:
"""Get room presence/session data for a Daily.co room. """Get real-time participant data - Daily.co specific feature."""
Example response:
{
"total_count": 1,
"data": [
{
"room": "w2pp2cf4kltgFACPKXmX",
"id": "d61cd7b2-a273-42b4-89bd-be763fd562c1",
"userId": "pbZ+ismP7dk=",
"userName": "Moishe",
"joinTime": "2023-01-01T20:53:19.000Z",
"duration": 2312
}
]
}
"""
async with httpx.AsyncClient() as client: async with httpx.AsyncClient() as client:
response = await client.get( response = await client.get(
f"{self.BASE_URL}/rooms/{room_name}/presence", f"{self.BASE_URL}/rooms/{room_name}/presence",
@@ -154,58 +116,19 @@ class DailyClient(VideoPlatformClient):
response.raise_for_status() response.raise_for_status()
return response.json() return response.json()
async def get_meeting_participants(self, meeting_id: str) -> Dict[str, Any]:
"""Get participant data for a specific Daily.co meeting.
Example response:
{
"data": [
{
"user_id": "4q47OTmqa/w=",
"participant_id": "d61cd7b2-a273-42b4-89bd-be763fd562c1",
"user_name": "Lindsey",
"join_time": 1672786813,
"duration": 150
},
{
"user_id": "pbZ+ismP7dk=",
"participant_id": "b3d56359-14d7-46af-ac8b-18f8c991f5f6",
"user_name": "Moishe",
"join_time": 1672786797,
"duration": 165
}
]
}
"""
async with httpx.AsyncClient() as client:
response = await client.get(
f"{self.BASE_URL}/meetings/{meeting_id}/participants",
headers=self.headers,
timeout=self.TIMEOUT,
)
response.raise_for_status()
return response.json()
async def get_recording(self, recording_id: str) -> Dict[str, Any]:
async with httpx.AsyncClient() as client:
response = await client.get(
f"{self.BASE_URL}/recordings/{recording_id}",
headers=self.headers,
timeout=self.TIMEOUT,
)
response.raise_for_status()
return response.json()
async def delete_room(self, room_name: str) -> bool: async def delete_room(self, room_name: str) -> bool:
"""Delete a Daily.co room."""
async with httpx.AsyncClient() as client: async with httpx.AsyncClient() as client:
response = await client.delete( response = await client.delete(
f"{self.BASE_URL}/rooms/{room_name}", f"{self.BASE_URL}/rooms/{room_name}",
headers=self.headers, headers=self.headers,
timeout=self.TIMEOUT, timeout=self.TIMEOUT,
) )
# Daily.co returns 200 for success, 404 if room doesn't exist
return response.status_code in (HTTPStatus.OK, HTTPStatus.NOT_FOUND) return response.status_code in (HTTPStatus.OK, HTTPStatus.NOT_FOUND)
async def upload_logo(self, room_name: str, logo_path: str) -> bool: async def upload_logo(self, room_name: str, logo_path: str) -> bool:
"""Daily.co doesn't support custom logos per room - this is a no-op."""
return True return True
def verify_webhook_signature( def verify_webhook_signature(
@@ -223,6 +146,8 @@ class DailyClient(VideoPlatformClient):
return False return False
try: try:
import base64
secret_bytes = base64.b64decode(self.config.webhook_secret) secret_bytes = base64.b64decode(self.config.webhook_secret)
signed_content = timestamp.encode() + b"." + body signed_content = timestamp.encode() + b"." + body
@@ -231,25 +156,17 @@ class DailyClient(VideoPlatformClient):
expected_b64 = base64.b64encode(expected).decode() expected_b64 = base64.b64encode(expected).decode()
return hmac.compare_digest(expected_b64, signature) return hmac.compare_digest(expected_b64, signature)
except Exception as e: except Exception:
logger.error("Daily.co webhook signature verification failed", exc_info=e)
return False return False
async def create_meeting_token( async def create_meeting_token(self, room_name: str, enable_recording: bool) -> str:
self, """Create meeting token for auto-recording."""
room_name: DailyRoomName,
enable_recording: bool,
user_id: Optional[str] = None,
) -> str:
data = {"properties": {"room_name": room_name}} data = {"properties": {"room_name": room_name}}
if enable_recording: if enable_recording:
data["properties"]["start_cloud_recording"] = True data["properties"]["start_cloud_recording"] = True
data["properties"]["enable_recording_ui"] = False data["properties"]["enable_recording_ui"] = False
if user_id:
data["properties"]["user_id"] = user_id
async with httpx.AsyncClient() as client: async with httpx.AsyncClient() as client:
response = await client.post( response = await client.post(
f"{self.BASE_URL}/meeting-tokens", f"{self.BASE_URL}/meeting-tokens",

View File

@@ -1,30 +1,29 @@
"""Factory for creating video platform clients based on configuration."""
from typing import Optional from typing import Optional
from reflector.settings import settings from reflector.settings import settings
from reflector.storage import get_dailyco_storage, get_whereby_storage
from ..schemas.platform import WHEREBY_PLATFORM, Platform from .base import Platform, VideoPlatformClient, VideoPlatformConfig
from .base import VideoPlatformClient, VideoPlatformConfig
from .registry import get_platform_client from .registry import get_platform_client
def get_platform_config(platform: Platform) -> VideoPlatformConfig: def get_platform_config(platform: Platform) -> VideoPlatformConfig:
if platform == WHEREBY_PLATFORM: """Get configuration for a specific platform."""
if platform == "whereby":
if not settings.WHEREBY_API_KEY: if not settings.WHEREBY_API_KEY:
raise ValueError( raise ValueError(
"WHEREBY_API_KEY is required when platform='whereby'. " "WHEREBY_API_KEY is required when platform='whereby'. "
"Set WHEREBY_API_KEY environment variable." "Set WHEREBY_API_KEY environment variable."
) )
whereby_storage = get_whereby_storage()
key_id, secret = whereby_storage.key_credentials
return VideoPlatformConfig( return VideoPlatformConfig(
api_key=settings.WHEREBY_API_KEY, api_key=settings.WHEREBY_API_KEY,
webhook_secret=settings.WHEREBY_WEBHOOK_SECRET or "", webhook_secret=settings.WHEREBY_WEBHOOK_SECRET or "",
api_url=settings.WHEREBY_API_URL, api_url=settings.WHEREBY_API_URL,
s3_bucket=whereby_storage.bucket_name, s3_bucket=settings.RECORDING_STORAGE_AWS_BUCKET_NAME,
s3_region=whereby_storage.region, s3_region=settings.RECORDING_STORAGE_AWS_REGION,
aws_access_key_id=key_id, aws_access_key_id=settings.AWS_WHEREBY_ACCESS_KEY_ID,
aws_access_key_secret=secret, aws_access_key_secret=settings.AWS_WHEREBY_ACCESS_KEY_SECRET,
) )
elif platform == "daily": elif platform == "daily":
if not settings.DAILY_API_KEY: if not settings.DAILY_API_KEY:
@@ -37,26 +36,45 @@ def get_platform_config(platform: Platform) -> VideoPlatformConfig:
"DAILY_SUBDOMAIN is required when platform='daily'. " "DAILY_SUBDOMAIN is required when platform='daily'. "
"Set DAILY_SUBDOMAIN environment variable." "Set DAILY_SUBDOMAIN environment variable."
) )
daily_storage = get_dailyco_storage()
return VideoPlatformConfig( return VideoPlatformConfig(
api_key=settings.DAILY_API_KEY, api_key=settings.DAILY_API_KEY,
webhook_secret=settings.DAILY_WEBHOOK_SECRET or "", webhook_secret=settings.DAILY_WEBHOOK_SECRET or "",
subdomain=settings.DAILY_SUBDOMAIN, subdomain=settings.DAILY_SUBDOMAIN,
s3_bucket=daily_storage.bucket_name, s3_bucket=settings.AWS_DAILY_S3_BUCKET,
s3_region=daily_storage.region, s3_region=settings.AWS_DAILY_S3_REGION,
aws_role_arn=daily_storage.role_credential, aws_role_arn=settings.AWS_DAILY_ROLE_ARN,
) )
else: else:
raise ValueError(f"Unknown platform: {platform}") raise ValueError(f"Unknown platform: {platform}")
def create_platform_client(platform: Platform) -> VideoPlatformClient: def create_platform_client(platform: Platform) -> VideoPlatformClient:
"""Create a video platform client instance."""
config = get_platform_config(platform) config = get_platform_config(platform)
return get_platform_client(platform, config) return get_platform_client(platform, config)
def get_platform(room_platform: Optional[Platform] = None) -> Platform: def get_platform_for_room(
room_id: Optional[str] = None, room_platform: Optional[Platform] = None
) -> Platform:
"""Determine which platform to use for a room.
Priority order (highest to lowest):
1. DAILY_MIGRATION_ROOM_IDS - env var override for testing/migration
2. room_platform - database persisted platform choice
3. DEFAULT_VIDEO_PLATFORM - env var fallback
"""
# If Daily migration is disabled, always use Whereby
if not settings.DAILY_MIGRATION_ENABLED:
return "whereby"
# Highest priority: If room is in migration list, use Daily (env var override)
if room_id and room_id in settings.DAILY_MIGRATION_ROOM_IDS:
return "daily"
# Second priority: Use room's persisted platform from database
if room_platform: if room_platform:
return room_platform return room_platform
# Fallback: Use default platform from env var
return settings.DEFAULT_VIDEO_PLATFORM return settings.DEFAULT_VIDEO_PLATFORM

View File

@@ -1,44 +1,31 @@
from datetime import datetime """Video platform data models.
Standard data models used across all video platform implementations.
"""
from typing import Any, Dict, Literal, Optional from typing import Any, Dict, Literal, Optional
from pydantic import BaseModel, Field from pydantic import BaseModel, Field
from reflector.schemas.platform import WHEREBY_PLATFORM, Platform from reflector.platform_types import Platform
from reflector.utils.string import NonEmptyString
RecordingType = Literal["none", "local", "cloud"] RecordingType = Literal["none", "local", "cloud"]
class SessionData(BaseModel):
"""Platform-agnostic session data.
Represents a participant session in a meeting room, regardless of platform.
Used to determine if a meeting is still active or has ended.
"""
session_id: NonEmptyString = Field(description="Unique session identifier")
started_at: datetime = Field(description="When session started (UTC)")
ended_at: datetime | None = Field(
description="When session ended (UTC), None if still active"
)
class MeetingData(BaseModel): class MeetingData(BaseModel):
"""Standardized meeting data returned by all providers."""
platform: Platform platform: Platform
meeting_id: NonEmptyString = Field( meeting_id: str = Field(description="Platform-specific meeting identifier")
description="Platform-specific meeting identifier" room_url: str = Field(description="URL for participants to join")
) host_room_url: str = Field(description="URL for hosts (may be same as room_url)")
room_url: NonEmptyString = Field(description="URL for participants to join") room_name: str = Field(description="Human-readable room name")
host_room_url: NonEmptyString = Field(
description="URL for hosts (may be same as room_url)"
)
room_name: NonEmptyString = Field(description="Human-readable room name")
extra_data: Dict[str, Any] = Field(default_factory=dict) extra_data: Dict[str, Any] = Field(default_factory=dict)
class Config: class Config:
json_schema_extra = { json_schema_extra = {
"example": { "example": {
"platform": WHEREBY_PLATFORM, "platform": "whereby",
"meeting_id": "12345678", "meeting_id": "12345678",
"room_url": "https://subdomain.whereby.com/room-20251008120000", "room_url": "https://subdomain.whereby.com/room-20251008120000",
"host_room_url": "https://subdomain.whereby.com/room-20251008120000?roomKey=abc123", "host_room_url": "https://subdomain.whereby.com/room-20251008120000?roomKey=abc123",
@@ -48,6 +35,8 @@ class MeetingData(BaseModel):
class VideoPlatformConfig(BaseModel): class VideoPlatformConfig(BaseModel):
"""Platform-agnostic configuration model."""
api_key: str api_key: str
webhook_secret: str webhook_secret: str
api_url: Optional[str] = None api_url: Optional[str] = None

View File

@@ -1,18 +1,20 @@
from typing import Dict, Type from typing import Dict, Type
from ..schemas.platform import DAILY_PLATFORM, WHEREBY_PLATFORM, Platform from .base import Platform, VideoPlatformClient, VideoPlatformConfig
from .base import VideoPlatformClient, VideoPlatformConfig
# Registry of available video platforms
_PLATFORMS: Dict[Platform, Type[VideoPlatformClient]] = {} _PLATFORMS: Dict[Platform, Type[VideoPlatformClient]] = {}
def register_platform(name: Platform, client_class: Type[VideoPlatformClient]): def register_platform(name: Platform, client_class: Type[VideoPlatformClient]):
"""Register a video platform implementation."""
_PLATFORMS[name] = client_class _PLATFORMS[name] = client_class
def get_platform_client( def get_platform_client(
platform: Platform, config: VideoPlatformConfig platform: Platform, config: VideoPlatformConfig
) -> VideoPlatformClient: ) -> VideoPlatformClient:
"""Get a video platform client instance."""
if platform not in _PLATFORMS: if platform not in _PLATFORMS:
raise ValueError(f"Unknown video platform: {platform}") raise ValueError(f"Unknown video platform: {platform}")
@@ -21,15 +23,17 @@ def get_platform_client(
def get_available_platforms() -> list[Platform]: def get_available_platforms() -> list[Platform]:
"""Get list of available platform names."""
return list(_PLATFORMS.keys()) return list(_PLATFORMS.keys())
# Auto-register built-in platforms
def _register_builtin_platforms(): def _register_builtin_platforms():
from .daily import DailyClient # noqa: PLC0415 from .daily import DailyClient # noqa: PLC0415
from .whereby import WherebyClient # noqa: PLC0415 from .whereby import WherebyClient # noqa: PLC0415
register_platform(WHEREBY_PLATFORM, WherebyClient) register_platform("whereby", WherebyClient)
register_platform(DAILY_PLATFORM, DailyClient) register_platform("daily", DailyClient)
_register_builtin_platforms() _register_builtin_platforms()

View File

@@ -4,22 +4,19 @@ import re
import time import time
from datetime import datetime from datetime import datetime
from hashlib import sha256 from hashlib import sha256
from typing import Optional from typing import Any, Dict, Optional
import httpx import httpx
from reflector.db.rooms import Room from reflector.db.rooms import Room
from reflector.storage import get_whereby_storage
from ..schemas.platform import WHEREBY_PLATFORM, Platform from .base import MeetingData, Platform, VideoPlatformClient, VideoPlatformConfig
from ..utils.string import NonEmptyString
from .base import VideoPlatformClient
from .models import MeetingData, SessionData, VideoPlatformConfig
from .whereby_utils import whereby_room_name_prefix
class WherebyClient(VideoPlatformClient): class WherebyClient(VideoPlatformClient):
PLATFORM_NAME: Platform = WHEREBY_PLATFORM """Whereby video platform implementation."""
PLATFORM_NAME: Platform = "whereby"
TIMEOUT = 10 # seconds TIMEOUT = 10 # seconds
MAX_ELAPSED_TIME = 60 * 1000 # 1 minute in milliseconds MAX_ELAPSED_TIME = 60 * 1000 # 1 minute in milliseconds
@@ -31,28 +28,27 @@ class WherebyClient(VideoPlatformClient):
} }
async def create_meeting( async def create_meeting(
self, room_name_prefix: NonEmptyString, end_date: datetime, room: Room self, room_name_prefix: str, end_date: datetime, room: Room
) -> MeetingData: ) -> MeetingData:
"""Create a Whereby meeting."""
data = { data = {
"isLocked": room.is_locked, "isLocked": room.is_locked,
"roomNamePrefix": whereby_room_name_prefix(room_name_prefix), "roomNamePrefix": room_name_prefix,
"roomNamePattern": "uuid", "roomNamePattern": "uuid",
"roomMode": room.room_mode, "roomMode": room.room_mode,
"endDate": end_date.isoformat(), "endDate": end_date.isoformat(),
"fields": ["hostRoomUrl"], "fields": ["hostRoomUrl"],
} }
# Add recording configuration if cloud recording is enabled
if room.recording_type == "cloud": if room.recording_type == "cloud":
# Get storage config for passing credentials to Whereby API
whereby_storage = get_whereby_storage()
key_id, secret = whereby_storage.key_credentials
data["recording"] = { data["recording"] = {
"type": room.recording_type, "type": room.recording_type,
"destination": { "destination": {
"provider": "s3", "provider": "s3",
"bucket": whereby_storage.bucket_name, "bucket": self.config.s3_bucket,
"accessKeyId": key_id, "accessKeyId": self.config.aws_access_key_id,
"accessKeySecret": secret, "accessKeySecret": self.config.aws_access_key_secret,
"fileFormat": "mp4", "fileFormat": "mp4",
}, },
"startTrigger": room.recording_trigger, "startTrigger": room.recording_trigger,
@@ -77,55 +73,23 @@ class WherebyClient(VideoPlatformClient):
extra_data=result, extra_data=result,
) )
async def get_room_sessions(self, room_name: str) -> list[SessionData]: async def get_room_sessions(self, room_name: str) -> Dict[str, Any]:
"""Get room session history from Whereby API. """Get Whereby room session information."""
Whereby API returns: [{"sessionId": "...", "startedAt": "...", "endedAt": "..." | null}, ...]
"""
async with httpx.AsyncClient() as client: async with httpx.AsyncClient() as client:
"""
{
"cursor": "text",
"results": [
{
"roomSessionId": "e2f29530-46ec-4cee-8b27-e565cb5bb2e9",
"roomName": "/room-prefix-793e9ec1-c686-423d-9043-9b7a10c553fd",
"startedAt": "2025-01-01T00:00:00.000Z",
"endedAt": "2025-01-01T01:00:00.000Z",
"totalParticipantMinutes": 124,
"totalRecorderMinutes": 120,
"totalStreamerMinutes": 120,
"totalUniqueParticipants": 4,
"totalUniqueRecorders": 3,
"totalUniqueStreamers": 2
}
]
}"""
response = await client.get( response = await client.get(
f"{self.config.api_url}/insights/room-sessions?roomName={room_name}", f"{self.config.api_url}/insights/room-sessions?roomName={room_name}",
headers=self.headers, headers=self.headers,
timeout=self.TIMEOUT, timeout=self.TIMEOUT,
) )
response.raise_for_status() response.raise_for_status()
results = response.json().get("results", []) return response.json()
return [
SessionData(
session_id=s["roomSessionId"],
started_at=datetime.fromisoformat(
s["startedAt"].replace("Z", "+00:00")
),
ended_at=datetime.fromisoformat(s["endedAt"].replace("Z", "+00:00"))
if s.get("endedAt")
else None,
)
for s in results
]
async def delete_room(self, room_name: str) -> bool: async def delete_room(self, room_name: str) -> bool:
"""Whereby doesn't support room deletion - meetings expire automatically."""
return True return True
async def upload_logo(self, room_name: str, logo_path: str) -> bool: async def upload_logo(self, room_name: str, logo_path: str) -> bool:
"""Upload logo to Whereby room."""
async with httpx.AsyncClient() as client: async with httpx.AsyncClient() as client:
with open(logo_path, "rb") as f: with open(logo_path, "rb") as f:
response = await client.put( response = await client.put(
@@ -142,6 +106,7 @@ class WherebyClient(VideoPlatformClient):
def verify_webhook_signature( def verify_webhook_signature(
self, body: bytes, signature: str, timestamp: Optional[str] = None self, body: bytes, signature: str, timestamp: Optional[str] = None
) -> bool: ) -> bool:
"""Verify Whereby webhook signature."""
if not signature: if not signature:
return False return False
@@ -151,11 +116,13 @@ class WherebyClient(VideoPlatformClient):
ts, sig = matches.groups() ts, sig = matches.groups()
# Check timestamp to prevent replay attacks
current_time = int(time.time() * 1000) current_time = int(time.time() * 1000)
diff_time = current_time - int(ts) * 1000 diff_time = current_time - int(ts) * 1000
if diff_time >= self.MAX_ELAPSED_TIME: if diff_time >= self.MAX_ELAPSED_TIME:
return False return False
# Verify signature
body_dict = json.loads(body) body_dict = json.loads(body)
signed_payload = f"{ts}.{json.dumps(body_dict, separators=(',', ':'))}" signed_payload = f"{ts}.{json.dumps(body_dict, separators=(',', ':'))}"
hmac_obj = hmac.new( hmac_obj = hmac.new(

View File

@@ -1,38 +0,0 @@
import re
from datetime import datetime
from reflector.utils.datetime import parse_datetime_with_timezone
from reflector.utils.string import NonEmptyString, parse_non_empty_string
from reflector.video_platforms.base import ROOM_PREFIX_SEPARATOR
def parse_whereby_recording_filename(
object_key: NonEmptyString,
) -> (NonEmptyString, datetime):
filename = parse_non_empty_string(object_key.rsplit(".", 1)[0])
timestamp_pattern = r"(\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}Z)"
match = re.search(timestamp_pattern, filename)
if not match:
raise ValueError(f"No ISO timestamp found in filename: {object_key}")
timestamp_str = match.group(1)
timestamp_start = match.start(1)
room_name_part = filename[:timestamp_start]
if room_name_part.endswith(ROOM_PREFIX_SEPARATOR):
room_name_part = room_name_part[: -len(ROOM_PREFIX_SEPARATOR)]
else:
raise ValueError(
f"room name {room_name_part} doesnt have {ROOM_PREFIX_SEPARATOR} at the end of filename: {object_key}"
)
return parse_non_empty_string(room_name_part), parse_datetime_with_timezone(
timestamp_str
)
def whereby_room_name_prefix(room_name_prefix: NonEmptyString) -> NonEmptyString:
return room_name_prefix + ROOM_PREFIX_SEPARATOR
# room name comes with "/" from whereby api but lacks "/" e.g. in recording filenames
def room_name_to_whereby_api_room_name(room_name: NonEmptyString) -> NonEmptyString:
return f"/{room_name}"

View File

@@ -1,34 +1,31 @@
"""Daily.co webhook handler endpoint."""
import json import json
from datetime import datetime, timezone
from typing import Any, Dict, Literal from typing import Any, Dict, Literal
from fastapi import APIRouter, HTTPException, Request from fastapi import APIRouter, HTTPException, Request
from pydantic import BaseModel from pydantic import BaseModel
from reflector.db import get_database
from reflector.db.daily_participant_sessions import (
DailyParticipantSession,
daily_participant_sessions_controller,
)
from reflector.db.meetings import meetings_controller from reflector.db.meetings import meetings_controller
from reflector.logger import logger as _logger from reflector.logger import logger
from reflector.settings import settings from reflector.settings import settings
from reflector.utils.daily import DailyRoomName
from reflector.video_platforms.factory import create_platform_client from reflector.video_platforms.factory import create_platform_client
from reflector.worker.process import process_multitrack_recording from reflector.worker.process import process_multitrack_recording
router = APIRouter() router = APIRouter()
logger = _logger.bind(platform="daily")
class DailyTrack(BaseModel): class DailyTrack(BaseModel):
"""Daily.co recording track (audio or video file)."""
type: Literal["audio", "video"] type: Literal["audio", "video"]
s3Key: str s3Key: str
size: int size: int
class DailyWebhookEvent(BaseModel): class DailyWebhookEvent(BaseModel):
"""Daily webhook event structure."""
version: str version: str
type: str type: str
id: str id: str
@@ -36,7 +33,7 @@ class DailyWebhookEvent(BaseModel):
event_ts: float event_ts: float
def _extract_room_name(event: DailyWebhookEvent) -> DailyRoomName | None: def _extract_room_name(event: DailyWebhookEvent) -> str | None:
"""Extract room name from Daily event payload. """Extract room name from Daily event payload.
Daily.co API inconsistency: Daily.co API inconsistency:
@@ -50,24 +47,6 @@ def _extract_room_name(event: DailyWebhookEvent) -> DailyRoomName | None:
async def webhook(request: Request): async def webhook(request: Request):
"""Handle Daily webhook events. """Handle Daily webhook events.
Example webhook payload:
{
"version": "1.0.0",
"type": "recording.ready-to-download",
"id": "rec-rtd-c3df927c-f738-4471-a2b7-066fa7e95a6b-1692124192",
"payload": {
"recording_id": "08fa0b24-9220-44c5-846c-3f116cf8e738",
"room_name": "Xcm97xRZ08b2dePKb78g",
"start_ts": 1692124183,
"status": "finished",
"max_participants": 1,
"duration": 9,
"share_token": "ntDCL5k98Ulq", #gitleaks:allow
"s3_key": "api-test-1j8fizhzd30c/Xcm97xRZ08b2dePKb78g/1692124183028"
},
"event_ts": 1692124192
}
Daily.co circuit-breaker: After 3+ failed responses (4xx/5xx), webhook Daily.co circuit-breaker: After 3+ failed responses (4xx/5xx), webhook
state→FAILED, stops sending events. Reset: scripts/recreate_daily_webhook.py state→FAILED, stops sending events. Reset: scripts/recreate_daily_webhook.py
""" """
@@ -90,11 +69,13 @@ async def webhook(request: Request):
) )
raise HTTPException(status_code=401, detail="Invalid webhook signature") raise HTTPException(status_code=401, detail="Invalid webhook signature")
# Parse the JSON body
try: try:
body_json = json.loads(body) body_json = json.loads(body)
except json.JSONDecodeError: except json.JSONDecodeError:
raise HTTPException(status_code=422, detail="Invalid JSON") raise HTTPException(status_code=422, detail="Invalid JSON")
# Handle Daily's test event during webhook creation
if body_json.get("test") == "test": if body_json.get("test") == "test":
logger.info("Received Daily webhook test event") logger.info("Received Daily webhook test event")
return {"status": "ok"} return {"status": "ok"}
@@ -117,157 +98,44 @@ async def webhook(request: Request):
await _handle_recording_ready(event) await _handle_recording_ready(event)
elif event.type == "recording.error": elif event.type == "recording.error":
await _handle_recording_error(event) await _handle_recording_error(event)
else:
logger.warning(
"Unhandled Daily webhook event type",
event_type=event.type,
payload=event.payload,
)
return {"status": "ok"} return {"status": "ok"}
"""
{
"version": "1.0.0",
"type": "participant.joined",
"id": "ptcpt-join-6497c79b-f326-4942-aef8-c36a29140ad1-1708972279961",
"payload": {
"room": "test",
"user_id": "6497c79b-f326-4942-aef8-c36a29140ad1",
"user_name": "testuser",
"session_id": "0c0d2dda-f21d-4cf9-ab56-86bf3c407ffa",
"joined_at": 1708972279.96,
"will_eject_at": 1708972299.541,
"owner": false,
"permissions": {
"hasPresence": true,
"canSend": true,
"canReceive": { "base": true },
"canAdmin": false
}
},
"event_ts": 1708972279.961
}
"""
async def _handle_participant_joined(event: DailyWebhookEvent): async def _handle_participant_joined(event: DailyWebhookEvent):
daily_room_name = _extract_room_name(event) """Handle participant joined event."""
if not daily_room_name: room_name = _extract_room_name(event)
if not room_name:
logger.warning("participant.joined: no room in payload", payload=event.payload) logger.warning("participant.joined: no room in payload", payload=event.payload)
return return
meeting = await meetings_controller.get_by_room_name(daily_room_name) meeting = await meetings_controller.get_by_room_name(room_name)
if not meeting: if meeting:
logger.warning(
"participant.joined: meeting not found", room_name=daily_room_name
)
return
payload = event.payload
logger.warning({"payload": payload})
joined_at = datetime.fromtimestamp(payload["joined_at"], tz=timezone.utc)
session_id = f"{meeting.id}:{payload['session_id']}"
session = DailyParticipantSession(
id=session_id,
meeting_id=meeting.id,
room_id=meeting.room_id,
session_id=payload["session_id"],
user_id=payload.get("user_id", None),
user_name=payload["user_name"],
joined_at=joined_at,
left_at=None,
)
# num_clients serves as a projection/cache of active session count for Daily.co
# Both operations must succeed or fail together to maintain consistency
async with get_database().transaction():
await meetings_controller.increment_num_clients(meeting.id) await meetings_controller.increment_num_clients(meeting.id)
await daily_participant_sessions_controller.upsert_joined(session)
logger.info( logger.info(
"Participant joined", "Participant joined",
meeting_id=meeting.id, meeting_id=meeting.id,
room_name=daily_room_name, room_name=room_name,
user_id=payload.get("user_id", None), recording_type=meeting.recording_type,
user_name=payload.get("user_name"), recording_trigger=meeting.recording_trigger,
session_id=session_id,
) )
else:
logger.warning("participant.joined: meeting not found", room_name=room_name)
"""
{
"version": "1.0.0",
"type": "participant.left",
"id": "ptcpt-left-16168c97-f973-4eae-9642-020fe3fda5db-1708972302986",
"payload": {
"room": "test",
"user_id": "16168c97-f973-4eae-9642-020fe3fda5db",
"user_name": "bipol",
"session_id": "0c0d2dda-f21d-4cf9-ab56-86bf3c407ffa",
"joined_at": 1708972291.567,
"will_eject_at": null,
"owner": false,
"permissions": {
"hasPresence": true,
"canSend": true,
"canReceive": { "base": true },
"canAdmin": false
},
"duration": 11.419000148773193
},
"event_ts": 1708972302.986
}
"""
async def _handle_participant_left(event: DailyWebhookEvent): async def _handle_participant_left(event: DailyWebhookEvent):
"""Handle participant left event."""
room_name = _extract_room_name(event) room_name = _extract_room_name(event)
if not room_name: if not room_name:
logger.warning("participant.left: no room in payload", payload=event.payload)
return return
meeting = await meetings_controller.get_by_room_name(room_name) meeting = await meetings_controller.get_by_room_name(room_name)
if not meeting: if meeting:
logger.warning("participant.left: meeting not found", room_name=room_name)
return
payload = event.payload
joined_at = datetime.fromtimestamp(payload["joined_at"], tz=timezone.utc)
left_at = datetime.fromtimestamp(event.event_ts, tz=timezone.utc)
session_id = f"{meeting.id}:{payload['session_id']}"
session = DailyParticipantSession(
id=session_id,
meeting_id=meeting.id,
room_id=meeting.room_id,
session_id=payload["session_id"],
user_id=payload.get("user_id", None),
user_name=payload["user_name"],
joined_at=joined_at,
left_at=left_at,
)
# num_clients serves as a projection/cache of active session count for Daily.co
# Both operations must succeed or fail together to maintain consistency
async with get_database().transaction():
await meetings_controller.decrement_num_clients(meeting.id) await meetings_controller.decrement_num_clients(meeting.id)
await daily_participant_sessions_controller.upsert_left(session)
logger.info(
"Participant left",
meeting_id=meeting.id,
room_name=room_name,
user_id=payload.get("user_id", None),
duration=payload.get("duration"),
session_id=session_id,
)
async def _handle_recording_started(event: DailyWebhookEvent): async def _handle_recording_started(event: DailyWebhookEvent):
"""Handle recording started event."""
room_name = _extract_room_name(event) room_name = _extract_room_name(event)
if not room_name: if not room_name:
logger.warning( logger.warning(
@@ -314,6 +182,7 @@ async def _handle_recording_ready(event: DailyWebhookEvent):
) )
return return
# Validate tracks structure
try: try:
tracks = [DailyTrack(**t) for t in tracks_raw] tracks = [DailyTrack(**t) for t in tracks_raw]
except Exception as e: except Exception as e:
@@ -332,10 +201,10 @@ async def _handle_recording_ready(event: DailyWebhookEvent):
platform="daily", platform="daily",
) )
bucket_name = settings.DAILYCO_STORAGE_AWS_BUCKET_NAME bucket_name = settings.AWS_DAILY_S3_BUCKET
if not bucket_name: if not bucket_name:
logger.error( logger.error(
"DAILYCO_STORAGE_AWS_BUCKET_NAME not configured; cannot process Daily recording" "AWS_DAILY_S3_BUCKET not configured; cannot process Daily recording"
) )
return return
@@ -343,13 +212,14 @@ async def _handle_recording_ready(event: DailyWebhookEvent):
process_multitrack_recording.delay( process_multitrack_recording.delay(
bucket_name=bucket_name, bucket_name=bucket_name,
daily_room_name=room_name, room_name=room_name,
recording_id=recording_id, recording_id=recording_id,
track_keys=track_keys, track_keys=track_keys,
) )
async def _handle_recording_error(event: DailyWebhookEvent): async def _handle_recording_error(event: DailyWebhookEvent):
"""Handle recording error event."""
room_name = _extract_room_name(event) room_name = _extract_room_name(event)
error = event.payload.get("error", "Unknown error") error = event.payload.get("error", "Unknown error")

View File

@@ -15,13 +15,12 @@ from reflector.db.calendar_events import calendar_events_controller
from reflector.db.meetings import meetings_controller from reflector.db.meetings import meetings_controller
from reflector.db.rooms import rooms_controller from reflector.db.rooms import rooms_controller
from reflector.redis_cache import RedisAsyncLock from reflector.redis_cache import RedisAsyncLock
from reflector.schemas.platform import Platform
from reflector.services.ics_sync import ics_sync_service from reflector.services.ics_sync import ics_sync_service
from reflector.settings import settings from reflector.settings import settings
from reflector.utils.url import add_query_param from reflector.video_platforms.base import Platform
from reflector.video_platforms.factory import ( from reflector.video_platforms.factory import (
create_platform_client, create_platform_client,
get_platform, get_platform_for_room,
) )
from reflector.worker.webhook import test_webhook from reflector.worker.webhook import test_webhook
@@ -46,7 +45,7 @@ class Room(BaseModel):
ics_enabled: bool = False ics_enabled: bool = False
ics_last_sync: Optional[datetime] = None ics_last_sync: Optional[datetime] = None
ics_last_etag: Optional[str] = None ics_last_etag: Optional[str] = None
platform: Platform platform: Platform = "whereby"
class RoomDetails(Room): class RoomDetails(Room):
@@ -74,7 +73,7 @@ class Meeting(BaseModel):
is_active: bool = True is_active: bool = True
calendar_event_id: str | None = None calendar_event_id: str | None = None
calendar_metadata: dict[str, Any] | None = None calendar_metadata: dict[str, Any] | None = None
platform: Platform platform: Platform = "whereby"
class CreateRoom(BaseModel): class CreateRoom(BaseModel):
@@ -174,6 +173,14 @@ class CalendarEventResponse(BaseModel):
router = APIRouter() router = APIRouter()
def parse_datetime_with_timezone(iso_string: str) -> datetime:
"""Parse ISO datetime string and ensure timezone awareness (defaults to UTC if naive)."""
dt = datetime.fromisoformat(iso_string)
if dt.tzinfo is None:
dt = dt.replace(tzinfo=timezone.utc)
return dt
@router.get("/rooms", response_model=Page[RoomDetails]) @router.get("/rooms", response_model=Page[RoomDetails])
async def rooms_list( async def rooms_list(
user: Annotated[Optional[auth.UserInfo], Depends(auth.current_user_optional)], user: Annotated[Optional[auth.UserInfo], Depends(auth.current_user_optional)],
@@ -191,7 +198,7 @@ async def rooms_list(
) )
for room in paginated.items: for room in paginated.items:
room.platform = get_platform(room.platform) room.platform = get_platform_for_room(room.id, room.platform)
return paginated return paginated
@@ -207,7 +214,7 @@ async def rooms_get(
raise HTTPException(status_code=404, detail="Room not found") raise HTTPException(status_code=404, detail="Room not found")
if not room.is_shared and (user_id is None or room.user_id != user_id): if not room.is_shared and (user_id is None or room.user_id != user_id):
raise HTTPException(status_code=403, detail="Room access denied") raise HTTPException(status_code=403, detail="Room access denied")
room.platform = get_platform(room.platform) room.platform = get_platform_for_room(room.id, room.platform)
return room return room
@@ -229,7 +236,7 @@ async def rooms_get_by_name(
room_dict["webhook_url"] = None room_dict["webhook_url"] = None
room_dict["webhook_secret"] = None room_dict["webhook_secret"] = None
room_dict["platform"] = get_platform(room.platform) room_dict["platform"] = get_platform_for_room(room.id, room.platform)
return RoomDetails(**room_dict) return RoomDetails(**room_dict)
@@ -275,7 +282,7 @@ async def rooms_update(
raise HTTPException(status_code=403, detail="Not authorized") raise HTTPException(status_code=403, detail="Not authorized")
values = info.dict(exclude_unset=True) values = info.dict(exclude_unset=True)
await rooms_controller.update(room, values) await rooms_controller.update(room, values)
room.platform = get_platform(room.platform) room.platform = get_platform_for_room(room.id, room.platform)
return room return room
@@ -323,13 +330,16 @@ async def rooms_create_meeting(
if meeting is None: if meeting is None:
end_date = current_time + timedelta(hours=8) end_date = current_time + timedelta(hours=8)
platform = get_platform(room.platform) # Determine which platform to use
platform = get_platform_for_room(room.id, room.platform)
client = create_platform_client(platform) client = create_platform_client(platform)
# Create meeting via platform abstraction
meeting_data = await client.create_meeting( meeting_data = await client.create_meeting(
room.name, end_date=end_date, room=room room.name, end_date=end_date, room=room
) )
# Upload logo if supported by platform
await client.upload_logo(meeting_data.room_name, "./images/logo.png") await client.upload_logo(meeting_data.room_name, "./images/logo.png")
meeting = await meetings_controller.create( meeting = await meetings_controller.create(
@@ -340,6 +350,7 @@ async def rooms_create_meeting(
start_date=current_time, start_date=current_time,
end_date=end_date, end_date=end_date,
room=room, room=room,
platform=platform,
) )
except LockError: except LockError:
logger.warning("Failed to acquire lock for room %s within timeout", room_name) logger.warning("Failed to acquire lock for room %s within timeout", room_name)
@@ -347,17 +358,17 @@ async def rooms_create_meeting(
status_code=503, detail="Meeting creation in progress, please try again" status_code=503, detail="Meeting creation in progress, please try again"
) )
meeting.platform = get_platform_for_room(room.id, room.platform)
if meeting.platform == "daily" and room.recording_trigger != "none": if meeting.platform == "daily" and room.recording_trigger != "none":
client = create_platform_client(meeting.platform) client = create_platform_client(meeting.platform)
token = await client.create_meeting_token( token = await client.create_meeting_token(
meeting.room_name, meeting.room_name, enable_recording=True
enable_recording=True,
user_id=user_id,
) )
meeting = meeting.model_copy() meeting = meeting.model_copy()
meeting.room_url = add_query_param(meeting.room_url, "t", token) meeting.room_url += f"?t={token}"
if meeting.host_room_url: if meeting.host_room_url:
meeting.host_room_url = add_query_param(meeting.host_room_url, "t", token) meeting.host_room_url += f"?t={token}"
if user_id != room.user_id: if user_id != room.user_id:
meeting.host_room_url = "" meeting.host_room_url = ""
@@ -513,7 +524,7 @@ async def rooms_list_active_meetings(
room=room, current_time=current_time room=room, current_time=current_time
) )
effective_platform = get_platform(room.platform) effective_platform = get_platform_for_room(room.id, room.platform)
for meeting in meetings: for meeting in meetings:
meeting.platform = effective_platform meeting.platform = effective_platform
@@ -537,10 +548,17 @@ async def rooms_get_meeting(
if not room: if not room:
raise HTTPException(status_code=404, detail="Room not found") raise HTTPException(status_code=404, detail="Room not found")
meeting = await meetings_controller.get_by_id(meeting_id, room=room) meeting = await meetings_controller.get_by_id(meeting_id)
if not meeting: if not meeting:
raise HTTPException(status_code=404, detail="Meeting not found") raise HTTPException(status_code=404, detail="Meeting not found")
if meeting.room_id != room.id:
raise HTTPException(
status_code=403, detail="Meeting does not belong to this room"
)
meeting.platform = get_platform_for_room(room.id, room.platform)
if user_id != room.user_id and not room.is_shared: if user_id != room.user_id and not room.is_shared:
meeting.host_room_url = "" meeting.host_room_url = ""
@@ -559,11 +577,16 @@ async def rooms_join_meeting(
if not room: if not room:
raise HTTPException(status_code=404, detail="Room not found") raise HTTPException(status_code=404, detail="Room not found")
meeting = await meetings_controller.get_by_id(meeting_id, room=room) meeting = await meetings_controller.get_by_id(meeting_id)
if not meeting: if not meeting:
raise HTTPException(status_code=404, detail="Meeting not found") raise HTTPException(status_code=404, detail="Meeting not found")
if meeting.room_id != room.id:
raise HTTPException(
status_code=403, detail="Meeting does not belong to this room"
)
if not meeting.is_active: if not meeting.is_active:
raise HTTPException(status_code=400, detail="Meeting is not active") raise HTTPException(status_code=400, detail="Meeting is not active")
@@ -571,6 +594,8 @@ async def rooms_join_meeting(
if meeting.end_date <= current_time: if meeting.end_date <= current_time:
raise HTTPException(status_code=400, detail="Meeting has ended") raise HTTPException(status_code=400, detail="Meeting has ended")
meeting.platform = get_platform_for_room(room.id, room.platform)
if user_id != room.user_id: if user_id != room.user_id:
meeting.host_room_url = "" meeting.host_room_url = ""

View File

@@ -5,7 +5,7 @@ from fastapi import APIRouter, Depends, HTTPException, Query
from fastapi_pagination import Page from fastapi_pagination import Page
from fastapi_pagination.ext.databases import apaginate from fastapi_pagination.ext.databases import apaginate
from jose import jwt from jose import jwt
from pydantic import AwareDatetime, BaseModel, Field, constr, field_serializer from pydantic import BaseModel, Field, constr, field_serializer
import reflector.auth as auth import reflector.auth as auth
from reflector.db import get_database from reflector.db import get_database
@@ -133,21 +133,6 @@ SearchOffsetParam = Annotated[
SearchOffsetBase, Query(description="Number of results to skip") SearchOffsetBase, Query(description="Number of results to skip")
] ]
SearchFromDatetimeParam = Annotated[
AwareDatetime | None,
Query(
alias="from",
description="Filter transcripts created on or after this datetime (ISO 8601 with timezone)",
),
]
SearchToDatetimeParam = Annotated[
AwareDatetime | None,
Query(
alias="to",
description="Filter transcripts created on or before this datetime (ISO 8601 with timezone)",
),
]
class SearchResponse(BaseModel): class SearchResponse(BaseModel):
results: list[SearchResult] results: list[SearchResult]
@@ -189,23 +174,18 @@ async def transcripts_search(
offset: SearchOffsetParam = 0, offset: SearchOffsetParam = 0,
room_id: Optional[str] = None, room_id: Optional[str] = None,
source_kind: Optional[SourceKind] = None, source_kind: Optional[SourceKind] = None,
from_datetime: SearchFromDatetimeParam = None,
to_datetime: SearchToDatetimeParam = None,
user: Annotated[ user: Annotated[
Optional[auth.UserInfo], Depends(auth.current_user_optional) Optional[auth.UserInfo], Depends(auth.current_user_optional)
] = None, ] = None,
): ):
"""Full-text search across transcript titles and content.""" """
Full-text search across transcript titles and content.
"""
if not user and not settings.PUBLIC_MODE: if not user and not settings.PUBLIC_MODE:
raise HTTPException(status_code=401, detail="Not authenticated") raise HTTPException(status_code=401, detail="Not authenticated")
user_id = user["sub"] if user else None user_id = user["sub"] if user else None
if from_datetime and to_datetime and from_datetime > to_datetime:
raise HTTPException(
status_code=400, detail="'from' must be less than or equal to 'to'"
)
search_params = SearchParameters( search_params = SearchParameters(
query_text=parse_search_query_param(q), query_text=parse_search_query_param(q),
limit=limit, limit=limit,
@@ -213,8 +193,6 @@ async def transcripts_search(
user_id=user_id, user_id=user_id,
room_id=room_id, room_id=room_id,
source_kind=source_kind, source_kind=source_kind,
from_datetime=from_datetime,
to_datetime=to_datetime,
) )
results, total = await search_controller.search_transcripts(search_params) results, total = await search_controller.search_transcripts(search_params)

View File

@@ -5,12 +5,8 @@ from fastapi import APIRouter, Depends, HTTPException
from pydantic import BaseModel from pydantic import BaseModel
import reflector.auth as auth import reflector.auth as auth
from reflector.db.recordings import recordings_controller
from reflector.db.transcripts import transcripts_controller from reflector.db.transcripts import transcripts_controller
from reflector.pipelines.main_file_pipeline import task_pipeline_file_process from reflector.pipelines.main_file_pipeline import task_pipeline_file_process
from reflector.pipelines.main_multitrack_pipeline import (
task_pipeline_multitrack_process,
)
router = APIRouter() router = APIRouter()
@@ -37,43 +33,13 @@ async def transcript_process(
status_code=400, detail="Recording is not ready for processing" status_code=400, detail="Recording is not ready for processing"
) )
# avoid duplicate scheduling for either pipeline
if task_is_scheduled_or_active( if task_is_scheduled_or_active(
"reflector.pipelines.main_file_pipeline.task_pipeline_file_process", "reflector.pipelines.main_file_pipeline.task_pipeline_file_process",
transcript_id=transcript_id, transcript_id=transcript_id,
) or task_is_scheduled_or_active(
"reflector.pipelines.main_multitrack_pipeline.task_pipeline_multitrack_process",
transcript_id=transcript_id,
): ):
return ProcessStatus(status="already running") return ProcessStatus(status="already running")
# Determine processing mode strictly from DB to avoid S3 scans # schedule a background task process the file
bucket_name = None
track_keys: list[str] = []
if transcript.recording_id:
recording = await recordings_controller.get_by_id(transcript.recording_id)
if recording:
bucket_name = recording.bucket_name
track_keys = recording.track_keys
if track_keys is not None and len(track_keys) == 0:
raise HTTPException(
status_code=500,
detail="No track keys found, must be either > 0 or None",
)
if track_keys is not None and not bucket_name:
raise HTTPException(
status_code=500, detail="Bucket name must be specified"
)
if track_keys:
task_pipeline_multitrack_process.delay(
transcript_id=transcript_id,
bucket_name=bucket_name,
track_keys=track_keys,
)
else:
# Default single-file pipeline
task_pipeline_file_process.delay(transcript_id=transcript_id) task_pipeline_file_process.delay(transcript_id=transcript_id)
return ProcessStatus(status="ok") return ProcessStatus(status="ok")

View File

@@ -11,6 +11,7 @@ router = APIRouter()
class UserInfo(BaseModel): class UserInfo(BaseModel):
sub: str sub: str
email: Optional[str] email: Optional[str]
email_verified: Optional[bool]
@router.get("/me") @router.get("/me")

View File

@@ -1,62 +0,0 @@
from datetime import datetime
from typing import Annotated
import structlog
from fastapi import APIRouter, Depends, HTTPException
from pydantic import BaseModel
import reflector.auth as auth
from reflector.db.user_api_keys import user_api_keys_controller
from reflector.utils.string import NonEmptyString
router = APIRouter()
logger = structlog.get_logger(__name__)
class CreateApiKeyRequest(BaseModel):
name: NonEmptyString | None = None
class ApiKeyResponse(BaseModel):
id: NonEmptyString
user_id: NonEmptyString
name: NonEmptyString | None
created_at: datetime
class CreateApiKeyResponse(ApiKeyResponse):
key: NonEmptyString
@router.post("/user/api-keys", response_model=CreateApiKeyResponse)
async def create_api_key(
req: CreateApiKeyRequest,
user: Annotated[auth.UserInfo, Depends(auth.current_user)],
):
api_key_model, plaintext = await user_api_keys_controller.create_key(
user_id=user["sub"],
name=req.name,
)
return CreateApiKeyResponse(
**api_key_model.model_dump(),
key=plaintext,
)
@router.get("/user/api-keys", response_model=list[ApiKeyResponse])
async def list_api_keys(
user: Annotated[auth.UserInfo, Depends(auth.current_user)],
):
api_keys = await user_api_keys_controller.list_by_user_id(user["sub"])
return [ApiKeyResponse(**k.model_dump()) for k in api_keys]
@router.delete("/user/api-keys/{key_id}")
async def delete_api_key(
key_id: NonEmptyString,
user: Annotated[auth.UserInfo, Depends(auth.current_user)],
):
deleted = await user_api_keys_controller.delete_key(key_id, user["sub"])
if not deleted:
raise HTTPException(status_code=404)
return {"status": "ok"}

114
server/reflector/whereby.py Normal file
View File

@@ -0,0 +1,114 @@
import logging
from datetime import datetime
import httpx
from reflector.db.rooms import Room
from reflector.settings import settings
from reflector.utils.string import parse_non_empty_string
logger = logging.getLogger(__name__)
def _get_headers():
api_key = parse_non_empty_string(
settings.WHEREBY_API_KEY, "WHEREBY_API_KEY value is required."
)
return {
"Content-Type": "application/json; charset=utf-8",
"Authorization": f"Bearer {api_key}",
}
TIMEOUT = 10 # seconds
def _get_whereby_s3_auth():
errors = []
try:
bucket_name = parse_non_empty_string(
settings.RECORDING_STORAGE_AWS_BUCKET_NAME,
"RECORDING_STORAGE_AWS_BUCKET_NAME value is required.",
)
except Exception as e:
errors.append(e)
try:
key_id = parse_non_empty_string(
settings.AWS_WHEREBY_ACCESS_KEY_ID,
"AWS_WHEREBY_ACCESS_KEY_ID value is required.",
)
except Exception as e:
errors.append(e)
try:
key_secret = parse_non_empty_string(
settings.AWS_WHEREBY_ACCESS_KEY_SECRET,
"AWS_WHEREBY_ACCESS_KEY_SECRET value is required.",
)
except Exception as e:
errors.append(e)
if len(errors) > 0:
raise Exception(
f"Failed to get Whereby auth settings: {', '.join(str(e) for e in errors)}"
)
return bucket_name, key_id, key_secret
async def create_meeting(room_name_prefix: str, end_date: datetime, room: Room):
s3_bucket_name, s3_key_id, s3_key_secret = _get_whereby_s3_auth()
data = {
"isLocked": room.is_locked,
"roomNamePrefix": room_name_prefix,
"roomNamePattern": "uuid",
"roomMode": room.room_mode,
"endDate": end_date.isoformat(),
"recording": {
"type": room.recording_type,
"destination": {
"provider": "s3",
"bucket": s3_bucket_name,
"accessKeyId": s3_key_id,
"accessKeySecret": s3_key_secret,
"fileFormat": "mp4",
},
"startTrigger": room.recording_trigger,
},
"fields": ["hostRoomUrl"],
}
async with httpx.AsyncClient() as client:
response = await client.post(
f"{settings.WHEREBY_API_URL}/meetings",
headers=_get_headers(),
json=data,
timeout=TIMEOUT,
)
if response.status_code == 403:
logger.warning(
f"Failed to create meeting: access denied on Whereby: {response.text}"
)
response.raise_for_status()
return response.json()
async def get_room_sessions(room_name: str):
async with httpx.AsyncClient() as client:
response = await client.get(
f"{settings.WHEREBY_API_URL}/insights/room-sessions?roomName={room_name}",
headers=_get_headers(),
timeout=TIMEOUT,
)
response.raise_for_status()
return response.json()
async def upload_logo(room_name: str, logo_path: str):
async with httpx.AsyncClient() as client:
with open(logo_path, "rb") as f:
response = await client.put(
f"{settings.WHEREBY_API_URL}/rooms{room_name}/theme/logo",
headers={
"Authorization": f"Bearer {settings.WHEREBY_API_KEY}",
},
timeout=TIMEOUT,
files={"image": f},
)
response.raise_for_status()

View File

@@ -19,7 +19,7 @@ from reflector.db.meetings import meetings
from reflector.db.recordings import recordings from reflector.db.recordings import recordings
from reflector.db.transcripts import transcripts, transcripts_controller from reflector.db.transcripts import transcripts, transcripts_controller
from reflector.settings import settings from reflector.settings import settings
from reflector.storage import get_transcripts_storage from reflector.storage import get_recordings_storage
logger = structlog.get_logger(__name__) logger = structlog.get_logger(__name__)
@@ -53,8 +53,8 @@ async def delete_single_transcript(
) )
if recording: if recording:
try: try:
await get_transcripts_storage().delete_file( await get_recordings_storage().delete_file(
recording["object_key"], bucket=recording["bucket_name"] recording["object_key"]
) )
except Exception as storage_error: except Exception as storage_error:
logger.warning( logger.warning(

View File

@@ -0,0 +1,179 @@
"""Stub data for Daily.co testing - Fish conversation"""
import re
from typing import Any
from reflector.processors.types import Word
from reflector.utils import generate_uuid4
# Constants for stub data generation
MIN_WORD_DURATION = 0.3 # Base duration per word in seconds
WORD_LENGTH_MULTIPLIER = 0.05 # Additional duration per character
NUM_STUB_TOPICS = 3 # Number of topics to generate
# The fish argument text - 2 speakers arguing about eating fish
FISH_TEXT = """Fish for dinner are nothing wrong with you? There's nothing wrong with me. Wrong with you? Would you shut up? There's nothing wrong with me. I'm just trying to. There's nothing wrong with me. I'm trying to eat a fish. Wrong with you trying to eat a fish and it falls off the plate. Would you shut up? You're bothering me. More than a fish is bothering me. Would you shut up and leave me alone? What's your problem? I'm just trying to eat a fish is wrong with you. I'm only trying to eat a fish. Would you shut up? Wrong with you. There's nothing wrong with me. There's nothing wrong with me. Wrong with you. There's nothing wrong with me. Wrong with you. There's nothing wrong with me. Would you shut up and let me eat my fish? Wrong with you. Shut up! What is wrong with you? Would you just shut up? What's your problem? Would you shut up with you? What is wrong with you? Wrong with me? I'm just trying to get my attention. Did you shut up? You're bothering me. Would you shut up? You're beginning to bug me. What's your problem? Just trying to eat my fish. Stay on the plate. Would you shut up? Just trying to eat my fish.
I'm gonna hit you with my problem. You're worse than this fish. You're more of a problem than a fish. What's your problem? Would you shut up? Would you shut your mouth? I want to eat my fish. Shut up! I can't even think. What's your problem? Trying to eat my fish is wrong with you. I don't have a problem. What is wrong with you? I have a problem. What's your problem? I don't have a problem. Can't you hear me with you? Can't you hear me? I don't have a problem. I want to eat my fish. Your problem? Just want to eat. What is wrong with you? Shut up! What is wrong with you? You just shut up! What's your problem? What is wrong with you anyway? What is wrong with you? I won't stay on the plate. You shut up! What is wrong with you? Would you just shut up? Let me eat my fish. What's your problem? Shut up and leave me alone! I can't even think. Wrong with you. I don't have a problem. Problem? I don't have a problem. Wrong with you. I don't have a problem with you. That's your problem. Don't have a problem? I want to eat my fish.
What is wrong with you? What's your problem? Problem? I just want to eat my fish. Wrong with you. What's wrong with you? I don't have a problem. You shut up! What's wrong with you? Just shut up! What's wrong with you? Shut up! What is wrong with you? I'm trying to eat a fish. I'm trying to eat a fish and it falls off the plate. Would you shut up? What is wrong with you? Would you shut up? Is wrong with you? Would you just shut up? What is wrong with you? Would you just shut? Is wrong with you? What's your problem? You just shut. What is wrong with you? Trying to eat my fish. Would you be quiet? What's your problem? Would you just shut up? Eat my fish. I can't even eat it. Don't stay on the plate. What's your problem? Would you shut up? What is wrong with you? What is wrong with you? Would you just shut up? What's your problem? What is wrong with you? I'm gonna hit you with my fish if you don't shut up. What's your problem? Would you shut up? What's wrong with you? What is wrong? Shut up! What's your problem?"""
def parse_fish_text() -> list[Word]:
"""Parse fish text into words with timestamps and speakers.
Returns list of Word objects with text, start/end timestamps, and speaker ID.
Speaker assignment heuristic:
- Speaker 0 (eating fish): "fish", "eat", "trying", "problem", "I"
- Speaker 1 (annoying): "wrong with you", "shut up", "What's your problem"
"""
# Split into sentences (rough)
sentences = re.split(r"([.!?])", FISH_TEXT)
# Reconstruct sentences with punctuation
full_sentences = []
for i in range(0, len(sentences) - 1, 2):
if sentences[i].strip():
full_sentences.append(
sentences[i].strip()
+ (sentences[i + 1] if i + 1 < len(sentences) else "")
)
words: list[Word] = []
current_time = 0.0
for sentence in full_sentences:
if not sentence.strip():
continue
# TODO: Delete this heuristic-based speaker detection when real diarization is implemented.
# This overly complex pattern matching is only for stub test data.
# Real implementation should use actual speaker diarization from audio processing.
# Determine speaker based on content
sentence_lower = sentence.lower()
# Speaker 1 patterns (annoying person)
if any(
p in sentence_lower
for p in [
"wrong with you",
"shut up",
"what's your problem",
"what is wrong",
"would you shut",
"you shut",
]
):
speaker = 1
# Speaker 0 patterns (trying to eat)
elif any(
p in sentence_lower
for p in [
"i'm trying",
"i'm just",
"i want to eat",
"eat my fish",
"trying to eat",
"nothing wrong with me",
"i don't have a problem",
"just trying",
"leave me alone",
"can't even",
"i'm gonna hit",
]
):
speaker = 0
# Default: alternate or use context
else:
# For short phrases, guess based on keywords
if "fish" in sentence_lower and "eat" in sentence_lower:
speaker = 0
elif "problem" in sentence_lower and "your" not in sentence_lower:
speaker = 0
else:
speaker = 1
# Split sentence into words
sentence_words = sentence.split()
for word in sentence_words:
word_duration = MIN_WORD_DURATION + (len(word) * WORD_LENGTH_MULTIPLIER)
words.append(
Word(
text=word + " ", # Add space
start=current_time,
end=current_time + word_duration,
speaker=speaker,
)
)
current_time += word_duration
return words
def generate_fake_topics(words: list[Word]) -> list[dict[str, Any]]:
"""Generate fake topics from words.
Splits into equal topics based on word count.
Returns list of topic dicts for database storage.
"""
if not words:
return []
chunk_size = len(words) // NUM_STUB_TOPICS
topics: list[dict[str, Any]] = []
for i in range(NUM_STUB_TOPICS):
start_idx = i * chunk_size
end_idx = (i + 1) * chunk_size if i < NUM_STUB_TOPICS - 1 else len(words)
if start_idx >= len(words):
break
chunk_words = words[start_idx:end_idx]
topic = {
"id": generate_uuid4(),
"title": f"Fish Argument Part {i+1}",
"summary": f"Argument about eating fish continues (part {i+1})",
"timestamp": chunk_words[0].start,
"duration": chunk_words[-1].end - chunk_words[0].start,
"transcript": "".join(w.text for w in chunk_words),
"words": [w.model_dump() for w in chunk_words],
}
topics.append(topic)
return topics
def generate_fake_participants() -> list[dict[str, Any]]:
"""Generate fake participants for stub transcript."""
return [
{"id": generate_uuid4(), "speaker": 0, "name": "Fish Eater"},
{"id": generate_uuid4(), "speaker": 1, "name": "Annoying Person"},
]
def get_stub_transcript_data() -> dict[str, Any]:
"""Get complete stub transcript data for Daily.co testing.
Returns dict with topics, participants, title, summaries, duration.
All data is fake/predetermined for testing webhook flow without GPU processing.
"""
words = parse_fish_text()
topics = generate_fake_topics(words)
participants = generate_fake_participants()
return {
"topics": topics,
"participants": participants,
"title": "The Great Fish Eating Argument",
"short_summary": "Two people argue about eating fish",
"long_summary": "An extended argument between someone trying to eat fish and another person who won't stop asking what's wrong. The fish keeps falling off the plate.",
"duration": words[-1].end if words else 0.0,
}

View File

@@ -7,10 +7,10 @@ from celery.utils.log import get_task_logger
from reflector.asynctask import asynctask from reflector.asynctask import asynctask
from reflector.db.calendar_events import calendar_events_controller from reflector.db.calendar_events import calendar_events_controller
from reflector.db.meetings import meetings_controller from reflector.db.meetings import meetings_controller
from reflector.db.rooms import Room, rooms_controller from reflector.db.rooms import rooms_controller
from reflector.redis_cache import RedisAsyncLock from reflector.redis_cache import RedisAsyncLock
from reflector.services.ics_sync import SyncStatus, ics_sync_service from reflector.services.ics_sync import SyncStatus, ics_sync_service
from reflector.video_platforms.factory import create_platform_client, get_platform from reflector.video_platforms.factory import create_platform_client
logger = structlog.wrap_logger(get_task_logger(__name__)) logger = structlog.wrap_logger(get_task_logger(__name__))
@@ -86,17 +86,17 @@ def _should_sync(room) -> bool:
MEETING_DEFAULT_DURATION = timedelta(hours=1) MEETING_DEFAULT_DURATION = timedelta(hours=1)
async def create_upcoming_meetings_for_event(event, create_window, room: Room): async def create_upcoming_meetings_for_event(event, create_window, room_id, room):
if event.start_time <= create_window: if event.start_time <= create_window:
return return
existing_meeting = await meetings_controller.get_by_calendar_event(event.id, room) existing_meeting = await meetings_controller.get_by_calendar_event(event.id)
if existing_meeting: if existing_meeting:
return return
logger.info( logger.info(
"Pre-creating meeting for calendar event", "Pre-creating meeting for calendar event",
room_id=room.id, room_id=room_id,
event_id=event.id, event_id=event.id,
event_title=event.title, event_title=event.title,
) )
@@ -104,10 +104,12 @@ async def create_upcoming_meetings_for_event(event, create_window, room: Room):
try: try:
end_date = event.end_time or (event.start_time + MEETING_DEFAULT_DURATION) end_date = event.end_time or (event.start_time + MEETING_DEFAULT_DURATION)
client = create_platform_client(get_platform(room.platform)) # Use platform abstraction to create meeting
platform = room.platform
client = create_platform_client(platform)
meeting_data = await client.create_meeting( meeting_data = await client.create_meeting(
room.name, "",
end_date=end_date, end_date=end_date,
room=room, room=room,
) )
@@ -127,6 +129,7 @@ async def create_upcoming_meetings_for_event(event, create_window, room: Room):
"description": event.description, "description": event.description,
"attendees": event.attendees, "attendees": event.attendees,
}, },
platform=platform,
) )
logger.info( logger.info(
@@ -138,7 +141,7 @@ async def create_upcoming_meetings_for_event(event, create_window, room: Room):
except Exception as e: except Exception as e:
logger.error( logger.error(
"Failed to pre-create meeting", "Failed to pre-create meeting",
room_id=room.id, room_id=room_id,
event_id=event.id, event_id=event.id,
error=str(e), error=str(e),
) )
@@ -168,7 +171,9 @@ async def create_upcoming_meetings():
) )
for event in events: for event in events:
await create_upcoming_meetings_for_event(event, create_window, room) await create_upcoming_meetings_for_event(
event, create_window, room.id, room
)
logger.info("Completed pre-creation check for upcoming meetings") logger.info("Completed pre-creation check for upcoming meetings")
except Exception as e: except Exception as e:

View File

@@ -15,32 +15,28 @@ from redis.exceptions import LockError
from reflector.db.meetings import meetings_controller from reflector.db.meetings import meetings_controller
from reflector.db.recordings import Recording, recordings_controller from reflector.db.recordings import Recording, recordings_controller
from reflector.db.rooms import rooms_controller from reflector.db.rooms import rooms_controller
from reflector.db.transcripts import ( from reflector.db.transcripts import SourceKind, transcripts_controller
SourceKind,
TranscriptParticipant,
transcripts_controller,
)
from reflector.pipelines.main_file_pipeline import task_pipeline_file_process from reflector.pipelines.main_file_pipeline import task_pipeline_file_process
from reflector.pipelines.main_live_pipeline import asynctask from reflector.pipelines.main_live_pipeline import asynctask
from reflector.pipelines.main_multitrack_pipeline import ( from reflector.pipelines.main_multitrack_pipeline import (
task_pipeline_multitrack_process, task_pipeline_multitrack_process,
) )
from reflector.pipelines.topic_processing import EmptyPipeline
from reflector.processors import AudioFileWriterProcessor
from reflector.processors.audio_waveform_processor import AudioWaveformProcessor
from reflector.redis_cache import get_redis_client from reflector.redis_cache import get_redis_client
from reflector.settings import settings from reflector.settings import settings
from reflector.storage import get_transcripts_storage from reflector.whereby import get_room_sessions
from reflector.utils.daily import DailyRoomName, extract_base_room_name from reflector.worker.daily_stub_data import get_stub_transcript_data
from reflector.video_platforms.factory import create_platform_client
from reflector.video_platforms.whereby_utils import (
parse_whereby_recording_filename,
room_name_to_whereby_api_room_name,
)
logger = structlog.wrap_logger(get_task_logger(__name__)) logger = structlog.wrap_logger(get_task_logger(__name__))
def parse_datetime_with_timezone(iso_string: str) -> datetime:
"""Parse ISO datetime string and ensure timezone awareness (defaults to UTC if naive)."""
dt = datetime.fromisoformat(iso_string)
if dt.tzinfo is None:
dt = dt.replace(tzinfo=timezone.utc)
return dt
@shared_task @shared_task
def process_messages(): def process_messages():
queue_url = settings.AWS_PROCESS_RECORDING_QUEUE_URL queue_url = settings.AWS_PROCESS_RECORDING_QUEUE_URL
@@ -82,16 +78,14 @@ def process_messages():
logger.error("process_messages", error=str(e)) logger.error("process_messages", error=str(e))
# only whereby supported.
@shared_task @shared_task
@asynctask @asynctask
async def process_recording(bucket_name: str, object_key: str): async def process_recording(bucket_name: str, object_key: str):
logger.info("Processing recording: %s/%s", bucket_name, object_key) logger.info("Processing recording: %s/%s", bucket_name, object_key)
room_name_part, recorded_at = parse_whereby_recording_filename(object_key) # extract a guid and a datetime from the object key
room_name = f"/{object_key[:36]}"
# we store whereby api room names, NOT whereby room names recorded_at = parse_datetime_with_timezone(object_key[37:57])
room_name = room_name_to_whereby_api_room_name(room_name_part)
meeting = await meetings_controller.get_by_room_name(room_name) meeting = await meetings_controller.get_by_room_name(room_name)
room = await rooms_controller.get_by_id(meeting.room_id) room = await rooms_controller.get_by_id(meeting.room_id)
@@ -113,7 +107,6 @@ async def process_recording(bucket_name: str, object_key: str):
transcript, transcript,
{ {
"topics": [], "topics": [],
"participants": [],
}, },
) )
else: else:
@@ -133,15 +126,15 @@ async def process_recording(bucket_name: str, object_key: str):
upload_filename = transcript.data_path / f"upload{extension}" upload_filename = transcript.data_path / f"upload{extension}"
upload_filename.parent.mkdir(parents=True, exist_ok=True) upload_filename.parent.mkdir(parents=True, exist_ok=True)
storage = get_transcripts_storage() s3 = boto3.client(
"s3",
region_name=settings.TRANSCRIPT_STORAGE_AWS_REGION,
aws_access_key_id=settings.TRANSCRIPT_STORAGE_AWS_ACCESS_KEY_ID,
aws_secret_access_key=settings.TRANSCRIPT_STORAGE_AWS_SECRET_ACCESS_KEY,
)
try:
with open(upload_filename, "wb") as f: with open(upload_filename, "wb") as f:
await storage.stream_to_fileobj(object_key, f, bucket=bucket_name) s3.download_fileobj(bucket_name, object_key, f)
except Exception:
# Clean up partial file on stream failure
upload_filename.unlink(missing_ok=True)
raise
container = av.open(upload_filename.as_posix()) container = av.open(upload_filename.as_posix())
try: try:
@@ -162,14 +155,14 @@ async def process_recording(bucket_name: str, object_key: str):
@asynctask @asynctask
async def process_multitrack_recording( async def process_multitrack_recording(
bucket_name: str, bucket_name: str,
daily_room_name: DailyRoomName, room_name: str,
recording_id: str, recording_id: str,
track_keys: list[str], track_keys: list[str],
): ):
logger.info( logger.info(
"Processing multitrack recording", "Processing multitrack recording",
bucket=bucket_name, bucket=bucket_name,
room_name=daily_room_name, room_name=room_name,
recording_id=recording_id, recording_id=recording_id,
provided_keys=len(track_keys), provided_keys=len(track_keys),
) )
@@ -178,38 +171,33 @@ async def process_multitrack_recording(
logger.warning("No audio track keys provided") logger.warning("No audio track keys provided")
return return
tz = timezone.utc recorded_at = datetime.now(timezone.utc)
recorded_at = datetime.now(tz)
try: try:
if track_keys: if track_keys:
folder = os.path.basename(os.path.dirname(track_keys[0])) folder = os.path.basename(os.path.dirname(track_keys[0]))
ts_match = re.search(r"(\d{14})$", folder) ts_match = re.search(r"(\d{14})$", folder)
if ts_match: if ts_match:
ts = ts_match.group(1) ts = ts_match.group(1)
recorded_at = datetime.strptime(ts, "%Y%m%d%H%M%S").replace(tzinfo=tz) recorded_at = datetime.strptime(ts, "%Y%m%d%H%M%S").replace(
except Exception as e: tzinfo=timezone.utc
logger.warning(
f"Could not parse recorded_at from keys, using now() {recorded_at}",
e,
exc_info=True,
) )
except Exception:
logger.warning("Could not parse recorded_at from keys, using now()")
meeting = await meetings_controller.get_by_room_name(daily_room_name) room_name = room_name.split("-", 1)[0]
room = await rooms_controller.get_by_name(room_name)
room_name_base = extract_base_room_name(daily_room_name)
room = await rooms_controller.get_by_name(room_name_base)
if not room: if not room:
raise Exception(f"Room not found: {room_name_base}") raise Exception(f"Room not found: {room_name}")
if not meeting: meeting = await meetings_controller.create(
raise Exception(f"Meeting not found: {room_name_base}") id=recording_id,
room_name=room_name,
logger.info( room_url=room.name,
"Found existing Meeting for recording", host_room_url=room.name,
meeting_id=meeting.id, start_date=recorded_at,
room_name=daily_room_name, end_date=recorded_at,
recording_id=recording_id, room=room,
platform=room.platform,
) )
recording = await recordings_controller.get_by_id(recording_id) recording = await recordings_controller.get_by_id(recording_id)
@@ -222,12 +210,8 @@ async def process_multitrack_recording(
object_key=object_key_dir, object_key=object_key_dir,
recorded_at=recorded_at, recorded_at=recorded_at,
meeting_id=meeting.id, meeting_id=meeting.id,
track_keys=track_keys,
) )
) )
else:
# Recording already exists; assume metadata was set at creation time
pass
transcript = await transcripts_controller.get_by_recording_id(recording.id) transcript = await transcripts_controller.get_by_recording_id(recording.id)
if transcript: if transcript:
@@ -235,7 +219,6 @@ async def process_multitrack_recording(
transcript, transcript,
{ {
"topics": [], "topics": [],
"participants": [],
}, },
) )
else: else:
@@ -251,65 +234,6 @@ async def process_multitrack_recording(
room_id=room.id, room_id=room.id,
) )
try:
daily_client = create_platform_client("daily")
id_to_name = {}
id_to_user_id = {}
mtg_session_id = None
try:
rec_details = await daily_client.get_recording(recording_id)
mtg_session_id = rec_details.get("mtgSessionId")
except Exception as e:
logger.warning(
"Failed to fetch Daily recording details",
error=str(e),
recording_id=recording_id,
exc_info=True,
)
if mtg_session_id:
try:
payload = await daily_client.get_meeting_participants(mtg_session_id)
for p in payload.get("data", []):
pid = p.get("participant_id")
name = p.get("user_name")
user_id = p.get("user_id")
if pid and name:
id_to_name[pid] = name
if pid and user_id:
id_to_user_id[pid] = user_id
except Exception as e:
logger.warning(
"Failed to fetch Daily meeting participants",
error=str(e),
mtg_session_id=mtg_session_id,
exc_info=True,
)
else:
logger.warning(
"No mtgSessionId found for recording; participant names may be generic",
recording_id=recording_id,
)
for idx, key in enumerate(track_keys):
base = os.path.basename(key)
m = re.search(r"\d{13,}-([0-9a-fA-F-]{36})-cam-audio-", base)
participant_id = m.group(1) if m else None
default_name = f"Speaker {idx}"
name = id_to_name.get(participant_id, default_name)
user_id = id_to_user_id.get(participant_id)
participant = TranscriptParticipant(
id=participant_id, speaker=idx, name=name, user_id=user_id
)
await transcripts_controller.upsert_participant(transcript, participant)
except Exception as e:
logger.warning("Failed to map participant names", error=str(e), exc_info=True)
task_pipeline_multitrack_process.delay( task_pipeline_multitrack_process.delay(
transcript_id=transcript.id, transcript_id=transcript.id,
bucket_name=bucket_name, bucket_name=bucket_name,
@@ -335,15 +259,15 @@ async def process_meetings():
Uses distributed locking to prevent race conditions when multiple workers Uses distributed locking to prevent race conditions when multiple workers
process the same meeting simultaneously. process the same meeting simultaneously.
""" """
logger.info("Processing meetings")
meetings = await meetings_controller.get_all_active() meetings = await meetings_controller.get_all_active()
logger.info(f"Processing {len(meetings)} meetings")
current_time = datetime.now(timezone.utc) current_time = datetime.now(timezone.utc)
redis_client = get_redis_client() redis_client = get_redis_client()
processed_count = 0 processed_count = 0
skipped_count = 0 skipped_count = 0
for meeting in meetings: for meeting in meetings:
logger_ = logger.bind(meeting_id=meeting.id, room_name=meeting.room_name) logger_ = logger.bind(meeting_id=meeting.id, room_name=meeting.room_name)
logger_.info("Processing meeting")
lock_key = f"meeting_process_lock:{meeting.id}" lock_key = f"meeting_process_lock:{meeting.id}"
lock = redis_client.lock(lock_key, timeout=120) lock = redis_client.lock(lock_key, timeout=120)
@@ -359,23 +283,21 @@ async def process_meetings():
if end_date.tzinfo is None: if end_date.tzinfo is None:
end_date = end_date.replace(tzinfo=timezone.utc) end_date = end_date.replace(tzinfo=timezone.utc)
client = create_platform_client(meeting.platform) # This API call could be slow, extend lock if needed
room_sessions = await client.get_room_sessions(meeting.room_name) response = await get_room_sessions(meeting.room_name)
try: try:
# Extend lock after operation to ensure we still hold it # Extend lock after slow operation to ensure we still hold it
lock.extend(120, replace_ttl=True) lock.extend(120, replace_ttl=True)
except LockError: except LockError:
logger_.warning("Lost lock for meeting, skipping") logger_.warning("Lost lock for meeting, skipping")
continue continue
room_sessions = response.get("results", [])
has_active_sessions = room_sessions and any( has_active_sessions = room_sessions and any(
s.ended_at is None for s in room_sessions rs["endedAt"] is None for rs in room_sessions
) )
has_had_sessions = bool(room_sessions) has_had_sessions = bool(room_sessions)
logger_.info(
f"found {has_active_sessions} active sessions, had {has_had_sessions}"
)
if has_active_sessions: if has_active_sessions:
logger_.debug("Meeting still has active sessions, keep it") logger_.debug("Meeting still has active sessions, keep it")
@@ -404,7 +326,7 @@ async def process_meetings():
except LockError: except LockError:
pass # Lock already released or expired pass # Lock already released or expired
logger.debug( logger.info(
"Processed meetings finished", "Processed meetings finished",
processed_count=processed_count, processed_count=processed_count,
skipped_count=skipped_count, skipped_count=skipped_count,
@@ -422,6 +344,10 @@ async def convert_audio_and_waveform(transcript) -> None:
transcript_id=transcript.id, transcript_id=transcript.id,
) )
# Import processors we need
from reflector.processors import AudioFileWriterProcessor
from reflector.processors.audio_waveform_processor import AudioWaveformProcessor
upload_path = transcript.data_path / "upload.webm" upload_path = transcript.data_path / "upload.webm"
mp3_path = transcript.audio_mp3_filename mp3_path = transcript.audio_mp3_filename
@@ -440,11 +366,21 @@ async def convert_audio_and_waveform(transcript) -> None:
mp3_size=mp3_path.stat().st_size, mp3_size=mp3_path.stat().st_size,
) )
# Generate waveform
waveform_processor = AudioWaveformProcessor( waveform_processor = AudioWaveformProcessor(
audio_path=mp3_path, audio_path=mp3_path,
waveform_path=transcript.audio_waveform_filename, waveform_path=transcript.audio_waveform_filename,
) )
waveform_processor.set_pipeline(EmptyPipeline(logger))
# Create minimal pipeline object for processor (matching EmptyPipeline from main_file_pipeline.py)
class MinimalPipeline:
def __init__(self, logger_instance):
self.logger = logger_instance
def get_pref(self, k, d=None):
return d
waveform_processor.set_pipeline(MinimalPipeline(logger))
await waveform_processor.flush() await waveform_processor.flush()
logger.info( logger.info(
@@ -468,30 +404,198 @@ async def convert_audio_and_waveform(transcript) -> None:
@shared_task @shared_task
@asynctask @asynctask
async def reprocess_failed_recordings(): async def process_daily_recording(
""" meeting_id: str, recording_id: str, tracks: list[dict]
Find recordings in Whereby S3 bucket and check if they have proper transcriptions. ) -> None:
If not, requeue them for processing. """Stub processor for Daily.co recordings - writes fake transcription/diarization.
Note: Daily.co recordings are processed via webhooks, not this cron job. Handles webhook retries by checking if recording already exists.
""" Validates track structure before processing.
logger.info("Checking Whereby recordings that need processing or reprocessing")
if not settings.WHEREBY_STORAGE_AWS_BUCKET_NAME: Args:
raise ValueError( meeting_id: Meeting ID
"WHEREBY_STORAGE_AWS_BUCKET_NAME required for Whereby recording reprocessing. " recording_id: Recording ID from Daily.co webhook
"Set WHEREBY_STORAGE_AWS_BUCKET_NAME environment variable." tracks: List of track dicts from Daily.co webhook
[{type: 'audio'|'video', s3Key: str, size: int}, ...]
"""
logger.info(
"Processing Daily.co recording (STUB)",
meeting_id=meeting_id,
recording_id=recording_id,
num_tracks=len(tracks),
) )
storage = get_transcripts_storage() # Check if recording already exists (webhook retry case)
bucket_name = settings.WHEREBY_STORAGE_AWS_BUCKET_NAME existing_recording = await recordings_controller.get_by_id(recording_id)
if existing_recording:
logger.warning(
"Recording already exists, skipping processing (likely webhook retry)",
recording_id=recording_id,
)
return
meeting = await meetings_controller.get_by_id(meeting_id)
if not meeting:
raise Exception(f"Meeting {meeting_id} not found")
room = await rooms_controller.get_by_id(meeting.room_id)
# Validate bucket configuration
if not settings.AWS_DAILY_S3_BUCKET:
raise ValueError("AWS_DAILY_S3_BUCKET not configured for Daily.co processing")
# Validate and parse tracks
# Import at runtime to avoid circular dependency (daily.py imports from process.py)
from reflector.views.daily import DailyTrack # noqa: PLC0415
try:
validated_tracks = [DailyTrack(**t) for t in tracks]
except Exception as e:
logger.error(
"Invalid track structure from Daily.co webhook",
error=str(e),
tracks=tracks,
)
raise ValueError(f"Invalid track structure: {e}")
# Find first audio track for Recording entity
audio_track = next((t for t in validated_tracks if t.type == "audio"), None)
if not audio_track:
raise Exception(f"No audio tracks found in {len(tracks)} tracks")
# Create Recording entry
recording = await recordings_controller.create(
Recording(
id=recording_id,
bucket_name=settings.AWS_DAILY_S3_BUCKET,
object_key=audio_track.s3Key,
recorded_at=datetime.now(timezone.utc),
meeting_id=meeting.id,
status="completed",
)
)
logger.info(
"Created recording",
recording_id=recording.id,
s3_key=audio_track.s3Key,
)
# Create Transcript entry
transcript = await transcripts_controller.add(
"",
source_kind=SourceKind.ROOM,
source_language="en",
target_language="en",
user_id=room.user_id,
recording_id=recording.id,
share_mode="public",
meeting_id=meeting.id,
room_id=room.id,
)
logger.info("Created transcript", transcript_id=transcript.id)
# Download audio file from Daily.co S3 for playback
upload_filename = transcript.data_path / "upload.webm"
upload_filename.parent.mkdir(parents=True, exist_ok=True)
s3 = boto3.client(
"s3",
region_name=settings.TRANSCRIPT_STORAGE_AWS_REGION,
aws_access_key_id=settings.TRANSCRIPT_STORAGE_AWS_ACCESS_KEY_ID,
aws_secret_access_key=settings.TRANSCRIPT_STORAGE_AWS_SECRET_ACCESS_KEY,
)
try:
logger.info(
"Downloading audio from Daily.co S3",
bucket=settings.AWS_DAILY_S3_BUCKET,
key=audio_track.s3Key,
)
with open(upload_filename, "wb") as f:
s3.download_fileobj(settings.AWS_DAILY_S3_BUCKET, audio_track.s3Key, f)
# Validate audio file
container = av.open(upload_filename.as_posix())
try:
if not len(container.streams.audio):
raise Exception("File has no audio stream")
finally:
container.close()
logger.info("Audio file downloaded and validated", file=str(upload_filename))
except Exception as e:
logger.error(
"Failed to download or validate audio file",
error=str(e),
bucket=settings.AWS_DAILY_S3_BUCKET,
key=audio_track.s3Key,
)
# Continue with stub data even if audio download fails
pass
# Generate fake data
stub_data = get_stub_transcript_data()
# Update transcript with fake data
await transcripts_controller.update(
transcript,
{
"topics": stub_data["topics"],
"participants": stub_data["participants"],
"title": stub_data["title"],
"short_summary": stub_data["short_summary"],
"long_summary": stub_data["long_summary"],
"duration": stub_data["duration"],
"status": "uploaded" if upload_filename.exists() else "ended",
},
)
logger.info(
"Daily.co recording processed (STUB)",
transcript_id=transcript.id,
duration=stub_data["duration"],
num_topics=len(stub_data["topics"]),
has_audio=upload_filename.exists(),
)
# Convert WebM to MP3 and generate waveform without full pipeline
# (full pipeline would overwrite our stub transcription data)
if upload_filename.exists():
await convert_audio_and_waveform(transcript)
@shared_task
@asynctask
async def reprocess_failed_recordings():
"""
Find recordings in the S3 bucket and check if they have proper transcriptions.
If not, requeue them for processing.
"""
logger.info("Checking for recordings that need processing or reprocessing")
s3 = boto3.client(
"s3",
region_name=settings.TRANSCRIPT_STORAGE_AWS_REGION,
aws_access_key_id=settings.TRANSCRIPT_STORAGE_AWS_ACCESS_KEY_ID,
aws_secret_access_key=settings.TRANSCRIPT_STORAGE_AWS_SECRET_ACCESS_KEY,
)
reprocessed_count = 0 reprocessed_count = 0
try: try:
object_keys = await storage.list_objects(prefix="", bucket=bucket_name) paginator = s3.get_paginator("list_objects_v2")
bucket_name = settings.RECORDING_STORAGE_AWS_BUCKET_NAME
pages = paginator.paginate(Bucket=bucket_name)
for object_key in object_keys: for page in pages:
if not object_key.endswith(".mp4"): if "Contents" not in page:
continue
for obj in page["Contents"]:
object_key = obj["Key"]
if not (object_key.endswith(".mp4")):
continue continue
recording = await recordings_controller.get_by_object_key( recording = await recordings_controller.get_by_object_key(

View File

@@ -0,0 +1,65 @@
#!/usr/bin/env python
"""
Reprocess the Daily.co multitrack recording to fix audio mixdown
"""
import asyncio
from reflector.pipelines.main_multitrack_pipeline import (
task_pipeline_multitrack_process,
)
async def reprocess():
"""Process the multitrack recording with fixed mixdown"""
bucket_name = "reflector-dailyco-local"
track_keys = [
"monadical/daily-20251020193458/1760988935484-52f7f48b-fbab-431f-9a50-87b9abfc8255-cam-audio-1760988935922",
"monadical/daily-20251020193458/1760988935484-a37c35e3-6f8e-4274-a482-e9d0f102a732-cam-audio-1760988943823",
]
# Create a new transcript with fixed mixdown
import uuid
from reflector.db import get_database
from reflector.db.transcripts import Transcript, transcripts
db = get_database()
await db.connect()
try:
transcript_id = str(uuid.uuid4())
transcript = Transcript(
id=transcript_id,
name="Daily Multitrack - With Audio Mixdown",
source_kind="file",
source_language="en",
target_language="en",
status="idle",
events=[],
title="",
)
query = transcripts.insert().values(**transcript.model_dump())
await db.execute(query)
print(f"Created transcript: {transcript_id}")
# Process with the fixed pipeline
await task_pipeline_multitrack_process(
transcript_id=transcript_id, bucket_name=bucket_name, track_keys=track_keys
)
print(
f"Processing complete! Check: http://localhost:3000/transcripts/{transcript_id}"
)
return transcript_id
finally:
await db.disconnect()
if __name__ == "__main__":
transcript_id = asyncio.run(reprocess())
print(f"\n✅ Reprocessing complete!")
print(f"📍 View at: http://localhost:3000/transcripts/{transcript_id}")

View File

@@ -1,91 +0,0 @@
#!/usr/bin/env python3
import asyncio
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).parent.parent))
import httpx
from reflector.settings import settings
async def list_webhooks():
"""
List all Daily.co webhooks for this account.
"""
if not settings.DAILY_API_KEY:
print("Error: DAILY_API_KEY not set")
return 1
headers = {
"Authorization": f"Bearer {settings.DAILY_API_KEY}",
"Content-Type": "application/json",
}
async with httpx.AsyncClient() as client:
try:
"""
Daily.co webhook list response format:
[
{
"uuid": "0b4e4c7c-5eaf-46fe-990b-a3752f5684f5",
"url": "{{webhook_url}}",
"hmac": "NQrSA5z0FkJ44QPrFerW7uCc5kdNLv3l2FDEKDanL1U=",
"basicAuth": null,
"eventTypes": [
"recording.started",
"recording.ready-to-download"
],
"state": "ACTVIE",
"failedCount": 0,
"lastMomentPushed": "2023-08-15T18:29:52.000Z",
"domainId": "{{domain_id}}",
"createdAt": "2023-08-15T18:28:30.000Z",
"updatedAt": "2023-08-15T18:29:52.000Z"
}
]
"""
resp = await client.get(
"https://api.daily.co/v1/webhooks",
headers=headers,
)
resp.raise_for_status()
webhooks = resp.json()
if not webhooks:
print("No webhooks found")
return 0
print(f"Found {len(webhooks)} webhook(s):\n")
for webhook in webhooks:
print("=" * 80)
print(f"UUID: {webhook['uuid']}")
print(f"URL: {webhook['url']}")
print(f"State: {webhook['state']}")
print(f"Event Types: {', '.join(webhook.get('eventTypes', []))}")
print(
f"HMAC Secret: {'✓ Configured' if webhook.get('hmac') else '✗ Not set'}"
)
print()
print("=" * 80)
print(
f"\nCurrent DAILY_WEBHOOK_UUID in settings: {settings.DAILY_WEBHOOK_UUID or '(not set)'}"
)
return 0
except httpx.HTTPStatusError as e:
print(f"Error fetching webhooks: {e}")
print(f"Response: {e.response.text}")
return 1
except Exception as e:
print(f"Unexpected error: {e}")
return 1
if __name__ == "__main__":
sys.exit(asyncio.run(list_webhooks()))

View File

@@ -1,4 +1,5 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
"""Recreate Daily.co webhook (fixes circuit-breaker FAILED state)."""
import asyncio import asyncio
import sys import sys
@@ -11,11 +12,8 @@ import httpx
from reflector.settings import settings from reflector.settings import settings
async def setup_webhook(webhook_url: str): async def recreate_webhook(webhook_url: str):
""" """Delete all webhooks and create new one."""
Create or update Daily.co webhook for this environment.
Uses DAILY_WEBHOOK_UUID to identify existing webhook.
"""
if not settings.DAILY_API_KEY: if not settings.DAILY_API_KEY:
print("Error: DAILY_API_KEY not set") print("Error: DAILY_API_KEY not set")
return 1 return 1
@@ -25,6 +23,21 @@ async def setup_webhook(webhook_url: str):
"Content-Type": "application/json", "Content-Type": "application/json",
} }
async with httpx.AsyncClient() as client:
# List existing webhooks
resp = await client.get("https://api.daily.co/v1/webhooks", headers=headers)
resp.raise_for_status()
webhooks = resp.json()
# Delete all existing webhooks
for wh in webhooks:
uuid = wh["uuid"]
print(f"Deleting webhook {uuid} (state: {wh['state']})")
await client.delete(
f"https://api.daily.co/v1/webhooks/{uuid}", headers=headers
)
# Create new webhook
webhook_data = { webhook_data = {
"url": webhook_url, "url": webhook_url,
"eventTypes": [ "eventTypes": [
@@ -37,72 +50,14 @@ async def setup_webhook(webhook_url: str):
"hmac": settings.DAILY_WEBHOOK_SECRET, "hmac": settings.DAILY_WEBHOOK_SECRET,
} }
async with httpx.AsyncClient() as client:
webhook_uuid = settings.DAILY_WEBHOOK_UUID
if webhook_uuid:
# Update existing webhook
print(f"Updating existing webhook {webhook_uuid}...")
try:
resp = await client.patch(
f"https://api.daily.co/v1/webhooks/{webhook_uuid}",
headers=headers,
json=webhook_data,
)
resp.raise_for_status()
result = resp.json()
print(f"✓ Updated webhook {result['uuid']} (state: {result['state']})")
print(f" URL: {result['url']}")
return 0
except httpx.HTTPStatusError as e:
if e.response.status_code == 404:
print(f"Webhook {webhook_uuid} not found, creating new one...")
webhook_uuid = None # Fall through to creation
else:
print(f"Error updating webhook: {e}")
return 1
if not webhook_uuid:
# Create new webhook
print("Creating new webhook...")
resp = await client.post( resp = await client.post(
"https://api.daily.co/v1/webhooks", headers=headers, json=webhook_data "https://api.daily.co/v1/webhooks", headers=headers, json=webhook_data
) )
resp.raise_for_status() resp.raise_for_status()
result = resp.json() result = resp.json()
webhook_uuid = result["uuid"]
print(f"✓ Created webhook {webhook_uuid} (state: {result['state']})")
print(f" URL: {result['url']}")
print()
print("=" * 60)
print("IMPORTANT: Add this to your environment variables:")
print("=" * 60)
print(f"DAILY_WEBHOOK_UUID: {webhook_uuid}")
print("=" * 60)
print()
# Try to write UUID to .env file
env_file = Path(__file__).parent.parent / ".env"
if env_file.exists():
lines = env_file.read_text().splitlines()
updated = False
# Update existing DAILY_WEBHOOK_UUID line or add it
for i, line in enumerate(lines):
if line.startswith("DAILY_WEBHOOK_UUID="):
lines[i] = f"DAILY_WEBHOOK_UUID={webhook_uuid}"
updated = True
break
if not updated:
lines.append(f"DAILY_WEBHOOK_UUID={webhook_uuid}")
env_file.write_text("\n".join(lines) + "\n")
print(f"✓ Also saved to local .env file")
else:
print(f"⚠ Local .env file not found - please add manually")
print(f"Created webhook {result['uuid']} (state: {result['state']})")
print(f"URL: {result['url']}")
return 0 return 0
@@ -112,12 +67,6 @@ if __name__ == "__main__":
print( print(
"Example: python recreate_daily_webhook.py https://example.com/v1/daily/webhook" "Example: python recreate_daily_webhook.py https://example.com/v1/daily/webhook"
) )
print()
print("Behavior:")
print(" - If DAILY_WEBHOOK_UUID set: Updates existing webhook")
print(
" - If DAILY_WEBHOOK_UUID empty: Creates new webhook, saves UUID to .env"
)
sys.exit(1) sys.exit(1)
sys.exit(asyncio.run(setup_webhook(sys.argv[1]))) sys.exit(asyncio.run(recreate_webhook(sys.argv[1])))

View File

@@ -0,0 +1,124 @@
#!/usr/bin/env python
"""
Test script to trigger multitrack recording processing with ffmpeg padding fix
"""
import asyncio
from reflector.pipelines.main_multitrack_pipeline import PipelineMainMultitrack
async def test_processing():
"""Manually trigger multitrack processing for the test recording"""
# Initialize database connection
from reflector.db import get_database
db = get_database()
await db.connect()
try:
# The test recording with known speaker timeline
bucket_name = "monadical"
track_keys = [
"daily-20251020193458/1760988935484-52f7f48b-fbab-431f-9a50-87b9abfc8255-cam-audio-1760988935922.webm",
"daily-20251020193458/1760988935484-a37c35e3-6f8e-4274-a482-e9d0f102a732-cam-audio-1760988943823.webm",
]
# Create a new transcript ID
import uuid
transcript_id = str(uuid.uuid4())
# Create transcript directly with SQL
from reflector.db.transcripts import (
Transcript,
transcripts,
transcripts_controller,
)
pipeline = PipelineMainMultitrack(transcript_id=transcript_id)
# Create transcript model
transcript = Transcript(
id=transcript_id,
name="FFMPEG Test - Daily Multitrack Recording",
source_kind="file",
source_language="en",
target_language="en",
status="idle",
events=[],
title="",
)
# Insert into database
query = transcripts.insert().values(**transcript.model_dump())
await db.execute(query)
print(f"Created transcript: {transcript_id}")
# Process the tracks using the pipeline
print(f"Processing multitrack recording with ffmpeg padding...")
print(f"Track 0: ...935922.webm (expected to start at ~2s)")
print(f"Track 1: ...943823.webm (expected to start at ~51s)")
try:
await pipeline.set_status(transcript_id, "processing")
await pipeline.process(bucket_name, track_keys)
print(f"Processing complete!")
except Exception as e:
await pipeline.set_status(transcript_id, "error")
print(f"Error during processing: {e}")
import traceback
traceback.print_exc()
raise
# Check the results
final_transcript = await transcripts_controller.get(transcript_id)
print(f"\nTranscript status: {final_transcript.status}")
print(f"Transcript title: {final_transcript.title}")
# Extract timeline from events
if final_transcript.events:
for event in final_transcript.events:
if event.get("event") == "TRANSCRIPT":
text = event.get("data", {}).get("text", "")
# Show first 500 chars to check if speakers are properly separated
print(f"\nTranscript text (first 500 chars):")
print(text[:500])
# Show last 500 chars too to see if second speaker is at the end
print(f"\nTranscript text (last 500 chars):")
print(text[-500:])
# Count words per speaker
words = text.split()
print(f"\nTotal words in transcript: {len(words)}")
# Check if text has proper speaker separation
# Expected: First ~45% from speaker 0, then ~35% from speaker 1, then ~20% from speaker 0
first_third = " ".join(words[: len(words) // 3])
middle_third = " ".join(
words[len(words) // 3 : 2 * len(words) // 3]
)
last_third = " ".join(words[2 * len(words) // 3 :])
print(f"\nFirst third preview: {first_third[:100]}...")
print(f"Middle third preview: {middle_third[:100]}...")
print(f"Last third preview: {last_third[:100]}...")
break
return transcript_id
finally:
await db.disconnect()
if __name__ == "__main__":
transcript_id = asyncio.run(test_processing())
print(f"\n✅ Test complete! Transcript ID: {transcript_id}")
print(f"\nExpected timeline:")
print(f" Speaker 0: ~2s to ~49s (first participant speaks)")
print(f" Speaker 1: ~51s to ~70s (second participant speaks)")
print(f" Speaker 0: ~73s to end (first participant speaks again)")
print(
f"\nIf the text shows proper chronological order (not interleaved), the fix worked!"
)

View File

@@ -0,0 +1,162 @@
#!/usr/bin/env python
"""
Test script to trigger multitrack recording processing with ffmpeg padding fix
This version loads tracks from local filesystem instead of S3
"""
import asyncio
import os
from reflector.pipelines.main_multitrack_pipeline import PipelineMainMultitrack
async def test_processing():
"""Manually trigger multitrack processing for the test recording"""
# Initialize database connection
from reflector.db import get_database
db = get_database()
await db.connect()
try:
# Create a new transcript ID
import uuid
transcript_id = str(uuid.uuid4())
# Create transcript directly with SQL
from reflector.db.transcripts import (
Transcript,
transcripts,
transcripts_controller,
)
pipeline = PipelineMainMultitrack(transcript_id=transcript_id)
# Create transcript model
transcript = Transcript(
id=transcript_id,
name="FFMPEG Test - Daily Multitrack Recording",
source_kind="file",
source_language="en",
target_language="en",
status="idle",
events=[],
title="",
)
# Insert into database
query = transcripts.insert().values(**transcript.model_dump())
await db.execute(query)
print(f"Created transcript: {transcript_id}")
# Read track files from local filesystem (in the container they'll be at /app/)
tracks_dir = "/app"
track_files = [
"1760988935484-52f7f48b-fbab-431f-9a50-87b9abfc8255-cam-audio-1760988935922.webm",
"1760988935484-a37c35e3-6f8e-4274-a482-e9d0f102a732-cam-audio-1760988943823.webm",
]
# Read track data
track_datas = []
for track_file in track_files:
file_path = os.path.join(tracks_dir, track_file)
if os.path.exists(file_path):
with open(file_path, "rb") as f:
track_datas.append(f.read())
print(f"Loaded track: {track_file} ({len(track_datas[-1])} bytes)")
else:
print(f"Track file not found: {file_path}")
track_datas.append(b"")
# Process the tracks using the pipeline
print(f"\nProcessing multitrack recording with ffmpeg padding...")
print(f"Track 0: ...935922.webm (expected to start at ~2s)")
print(f"Track 1: ...943823.webm (expected to start at ~51s)")
# Call the process method directly with track data
# We'll need to mock S3 operations and directly work with the data
# Save tracks to temporary files and process them
try:
await pipeline.set_status(transcript_id, "processing")
# Create a mock bucket and keys setup
bucket_name = "test-bucket"
track_keys = ["track0.webm", "track1.webm"]
# Mock S3 client to return our local data
from unittest.mock import MagicMock, patch
mock_s3 = MagicMock()
def mock_get_object(Bucket, Key):
idx = 0 if "track0" in Key else 1
return {"Body": MagicMock(read=lambda: track_datas[idx])}
mock_s3.get_object = mock_get_object
# Patch boto3.client to return our mock
with patch("boto3.client", return_value=mock_s3):
await pipeline.process(bucket_name, track_keys)
print(f"Processing complete!")
except Exception as e:
await pipeline.set_status(transcript_id, "error")
print(f"Error during processing: {e}")
import traceback
traceback.print_exc()
raise
# Check the results
final_transcript = await transcripts_controller.get(transcript_id)
print(f"\nTranscript status: {final_transcript.status}")
print(f"Transcript title: {final_transcript.title}")
# Extract timeline from events
if final_transcript.events:
for event in final_transcript.events:
if event.get("event") == "TRANSCRIPT":
text = event.get("data", {}).get("text", "")
# Show first 500 chars to check if speakers are properly separated
print(f"\nTranscript text (first 500 chars):")
print(text[:500])
# Show last 500 chars too to see if second speaker is at the end
print(f"\nTranscript text (last 500 chars):")
print(text[-500:])
# Count words per speaker
words = text.split()
print(f"\nTotal words in transcript: {len(words)}")
# Check if text has proper speaker separation
# Expected: First ~45% from speaker 0, then ~35% from speaker 1, then ~20% from speaker 0
first_third = " ".join(words[: len(words) // 3])
middle_third = " ".join(
words[len(words) // 3 : 2 * len(words) // 3]
)
last_third = " ".join(words[2 * len(words) // 3 :])
print(f"\nFirst third preview: {first_third[:100]}...")
print(f"Middle third preview: {middle_third[:100]}...")
print(f"Last third preview: {last_third[:100]}...")
break
return transcript_id
finally:
await db.disconnect()
if __name__ == "__main__":
transcript_id = asyncio.run(test_processing())
print(f"\n✅ Test complete! Transcript ID: {transcript_id}")
print(f"\nExpected timeline:")
print(f" Speaker 0: ~2s to ~49s (first participant speaks)")
print(f" Speaker 1: ~51s to ~70s (second participant speaks)")
print(f" Speaker 0: ~73s to end (first participant speaks again)")
print(
f"\nIf the text shows proper chronological order (not interleaved), the fix worked!"
)

View File

@@ -0,0 +1,66 @@
#!/usr/bin/env python
"""
Test multitrack processing with correct S3 bucket configuration
"""
import asyncio
import uuid
from reflector.db import get_database
from reflector.db.transcripts import Transcript, transcripts
from reflector.pipelines.main_multitrack_pipeline import (
task_pipeline_multitrack_process,
)
async def create_and_process():
"""Create a new transcript and process with correct S3 bucket"""
# Correct S3 configuration
bucket_name = "reflector-dailyco-local"
track_keys = [
"monadical/daily-20251020193458/1760988935484-52f7f48b-fbab-431f-9a50-87b9abfc8255-cam-audio-1760988935922",
"monadical/daily-20251020193458/1760988935484-a37c35e3-6f8e-4274-a482-e9d0f102a732-cam-audio-1760988943823",
]
# Create a new transcript
db = get_database()
await db.connect()
try:
transcript_id = str(uuid.uuid4())
transcript = Transcript(
id=transcript_id,
name="Daily Multitrack - Correct S3 Bucket Test",
source_kind="file",
source_language="en",
target_language="en",
status="idle",
events=[],
title="",
)
query = transcripts.insert().values(**transcript.model_dump())
await db.execute(query)
print(f"Created transcript: {transcript_id}")
# Trigger processing with Celery
result = task_pipeline_multitrack_process.delay(
transcript_id=transcript_id, bucket_name=bucket_name, track_keys=track_keys
)
print(f"Task ID: {result.id}")
print(
f"Processing started! Check: http://localhost:3000/transcripts/{transcript_id}"
)
print(f"API Status: http://localhost:1250/v1/transcripts/{transcript_id}")
return transcript_id
finally:
await db.disconnect()
if __name__ == "__main__":
transcript_id = asyncio.run(create_and_process())
print(f"\n✅ Task submitted successfully!")
print(f"📍 Transcript ID: {transcript_id}")

View File

@@ -5,8 +5,6 @@ from unittest.mock import patch
import pytest import pytest
from reflector.schemas.platform import WHEREBY_PLATFORM
@pytest.fixture(scope="session", autouse=True) @pytest.fixture(scope="session", autouse=True)
def register_mock_platform(): def register_mock_platform():
@@ -14,7 +12,7 @@ def register_mock_platform():
from reflector.video_platforms.registry import register_platform from reflector.video_platforms.registry import register_platform
register_platform(WHEREBY_PLATFORM, MockPlatformClient) register_platform("whereby", MockPlatformClient)
yield yield

View File

@@ -3,11 +3,8 @@ from datetime import datetime
from typing import Any, Dict, Literal, Optional from typing import Any, Dict, Literal, Optional
from reflector.db.rooms import Room from reflector.db.rooms import Room
from reflector.utils.string import NonEmptyString
from reflector.video_platforms.base import ( from reflector.video_platforms.base import (
ROOM_PREFIX_SEPARATOR,
MeetingData, MeetingData,
SessionData,
VideoPlatformClient, VideoPlatformClient,
VideoPlatformConfig, VideoPlatformConfig,
) )
@@ -27,7 +24,7 @@ class MockPlatformClient(VideoPlatformClient):
self, room_name_prefix: str, end_date: datetime, room: Room self, room_name_prefix: str, end_date: datetime, room: Room
) -> MeetingData: ) -> MeetingData:
meeting_id = str(uuid.uuid4()) meeting_id = str(uuid.uuid4())
room_name = f"{room_name_prefix}{ROOM_PREFIX_SEPARATOR}{meeting_id[:8]}" room_name = f"{room_name_prefix}-{meeting_id[:8]}"
room_url = f"https://mock.video/{room_name}" room_url = f"https://mock.video/{room_name}"
host_room_url = f"{room_url}?host=true" host_room_url = f"{room_url}?host=true"
@@ -51,18 +48,22 @@ class MockPlatformClient(VideoPlatformClient):
extra_data={"mock": True}, extra_data={"mock": True},
) )
async def get_room_sessions(self, room_name: NonEmptyString) -> list[SessionData]: async def get_room_sessions(self, room_name: str) -> Dict[str, Any]:
if room_name not in self._rooms: if room_name not in self._rooms:
return [] return {"error": "Room not found"}
room_data = self._rooms[room_name] room_data = self._rooms[room_name]
return [ return {
SessionData( "roomName": room_name,
session_id=room_data["id"], "sessions": [
started_at=datetime.utcnow(), {
ended_at=None if room_data["is_active"] else datetime.utcnow(), "sessionId": room_data["id"],
) "startTime": datetime.utcnow().isoformat(),
] "participants": room_data["participants"],
"isActive": room_data["is_active"],
}
],
}
async def delete_room(self, room_name: str) -> bool: async def delete_room(self, room_name: str) -> bool:
if room_name in self._rooms: if room_name in self._rooms:

View File

@@ -139,8 +139,12 @@ async def test_cleanup_deletes_associated_meeting_and_recording():
mock_settings.PUBLIC_DATA_RETENTION_DAYS = 7 mock_settings.PUBLIC_DATA_RETENTION_DAYS = 7
# Mock storage deletion # Mock storage deletion
with patch("reflector.worker.cleanup.get_transcripts_storage") as mock_storage: with patch("reflector.db.transcripts.get_transcripts_storage") as mock_storage:
mock_storage.return_value.delete_file = AsyncMock() mock_storage.return_value.delete_file = AsyncMock()
with patch(
"reflector.worker.cleanup.get_recordings_storage"
) as mock_rec_storage:
mock_rec_storage.return_value.delete_file = AsyncMock()
result = await cleanup_old_public_data() result = await cleanup_old_public_data()

View File

@@ -1,330 +0,0 @@
from datetime import datetime, timezone
from unittest.mock import AsyncMock, MagicMock, patch
import pytest
from reflector.db.meetings import (
MeetingConsent,
meeting_consent_controller,
meetings_controller,
)
from reflector.db.recordings import Recording, recordings_controller
from reflector.db.rooms import rooms_controller
from reflector.db.transcripts import SourceKind, transcripts_controller
from reflector.pipelines.main_live_pipeline import cleanup_consent
@pytest.mark.asyncio
async def test_consent_cleanup_deletes_multitrack_files():
room = await rooms_controller.add(
name="Test Room",
user_id="test-user",
zulip_auto_post=False,
zulip_stream="",
zulip_topic="",
is_locked=False,
room_mode="normal",
recording_type="cloud",
recording_trigger="automatic",
is_shared=False,
platform="daily",
)
# Create meeting
meeting = await meetings_controller.create(
id="test-multitrack-meeting",
room_name="test-room-20250101120000",
room_url="https://test.daily.co/test-room",
host_room_url="https://test.daily.co/test-room",
start_date=datetime.now(timezone.utc),
end_date=datetime.now(timezone.utc),
room=room,
)
track_keys = [
"recordings/test-room-20250101120000/track-0.webm",
"recordings/test-room-20250101120000/track-1.webm",
"recordings/test-room-20250101120000/track-2.webm",
]
recording = await recordings_controller.create(
Recording(
bucket_name="test-bucket",
object_key="recordings/test-room-20250101120000", # Folder path
recorded_at=datetime.now(timezone.utc),
meeting_id=meeting.id,
track_keys=track_keys,
)
)
# Create transcript
transcript = await transcripts_controller.add(
name="Test Multitrack Transcript",
source_kind=SourceKind.ROOM,
recording_id=recording.id,
meeting_id=meeting.id,
)
# Add consent denial
await meeting_consent_controller.upsert(
MeetingConsent(
meeting_id=meeting.id,
user_id="test-user",
consent_given=False,
consent_timestamp=datetime.now(timezone.utc),
)
)
# Mock get_transcripts_storage (master credentials with bucket override)
with patch(
"reflector.pipelines.main_live_pipeline.get_transcripts_storage"
) as mock_get_transcripts_storage:
mock_master_storage = MagicMock()
mock_master_storage.delete_file = AsyncMock()
mock_get_transcripts_storage.return_value = mock_master_storage
await cleanup_consent(transcript_id=transcript.id)
# Verify master storage was used with bucket override for all track keys
assert mock_master_storage.delete_file.call_count == 3
deleted_keys = []
for call_args in mock_master_storage.delete_file.call_args_list:
key = call_args[0][0]
bucket_kwarg = call_args[1].get("bucket")
deleted_keys.append(key)
assert bucket_kwarg == "test-bucket" # Verify bucket override!
assert set(deleted_keys) == set(track_keys)
updated_transcript = await transcripts_controller.get_by_id(transcript.id)
assert updated_transcript.audio_deleted is True
@pytest.mark.asyncio
async def test_consent_cleanup_handles_missing_track_keys():
room = await rooms_controller.add(
name="Test Room 2",
user_id="test-user",
zulip_auto_post=False,
zulip_stream="",
zulip_topic="",
is_locked=False,
room_mode="normal",
recording_type="cloud",
recording_trigger="automatic",
is_shared=False,
platform="daily",
)
# Create meeting
meeting = await meetings_controller.create(
id="test-multitrack-meeting-2",
room_name="test-room-20250101120001",
room_url="https://test.daily.co/test-room-2",
host_room_url="https://test.daily.co/test-room-2",
start_date=datetime.now(timezone.utc),
end_date=datetime.now(timezone.utc),
room=room,
)
recording = await recordings_controller.create(
Recording(
bucket_name="test-bucket",
object_key="recordings/old-style-recording.mp4",
recorded_at=datetime.now(timezone.utc),
meeting_id=meeting.id,
track_keys=None,
)
)
transcript = await transcripts_controller.add(
name="Test Old-Style Transcript",
source_kind=SourceKind.ROOM,
recording_id=recording.id,
meeting_id=meeting.id,
)
# Add consent denial
await meeting_consent_controller.upsert(
MeetingConsent(
meeting_id=meeting.id,
user_id="test-user-2",
consent_given=False,
consent_timestamp=datetime.now(timezone.utc),
)
)
# Mock get_transcripts_storage (master credentials with bucket override)
with patch(
"reflector.pipelines.main_live_pipeline.get_transcripts_storage"
) as mock_get_transcripts_storage:
mock_master_storage = MagicMock()
mock_master_storage.delete_file = AsyncMock()
mock_get_transcripts_storage.return_value = mock_master_storage
await cleanup_consent(transcript_id=transcript.id)
# Verify master storage was used with bucket override
assert mock_master_storage.delete_file.call_count == 1
call_args = mock_master_storage.delete_file.call_args
assert call_args[0][0] == recording.object_key
assert call_args[1].get("bucket") == "test-bucket" # Verify bucket override!
@pytest.mark.asyncio
async def test_consent_cleanup_empty_track_keys_falls_back():
room = await rooms_controller.add(
name="Test Room 3",
user_id="test-user",
zulip_auto_post=False,
zulip_stream="",
zulip_topic="",
is_locked=False,
room_mode="normal",
recording_type="cloud",
recording_trigger="automatic",
is_shared=False,
platform="daily",
)
# Create meeting
meeting = await meetings_controller.create(
id="test-multitrack-meeting-3",
room_name="test-room-20250101120002",
room_url="https://test.daily.co/test-room-3",
host_room_url="https://test.daily.co/test-room-3",
start_date=datetime.now(timezone.utc),
end_date=datetime.now(timezone.utc),
room=room,
)
recording = await recordings_controller.create(
Recording(
bucket_name="test-bucket",
object_key="recordings/fallback-recording.mp4",
recorded_at=datetime.now(timezone.utc),
meeting_id=meeting.id,
track_keys=[],
)
)
transcript = await transcripts_controller.add(
name="Test Empty Track Keys Transcript",
source_kind=SourceKind.ROOM,
recording_id=recording.id,
meeting_id=meeting.id,
)
# Add consent denial
await meeting_consent_controller.upsert(
MeetingConsent(
meeting_id=meeting.id,
user_id="test-user-3",
consent_given=False,
consent_timestamp=datetime.now(timezone.utc),
)
)
# Mock get_transcripts_storage (master credentials with bucket override)
with patch(
"reflector.pipelines.main_live_pipeline.get_transcripts_storage"
) as mock_get_transcripts_storage:
mock_master_storage = MagicMock()
mock_master_storage.delete_file = AsyncMock()
mock_get_transcripts_storage.return_value = mock_master_storage
# Run cleanup
await cleanup_consent(transcript_id=transcript.id)
# Verify master storage was used with bucket override
assert mock_master_storage.delete_file.call_count == 1
call_args = mock_master_storage.delete_file.call_args
assert call_args[0][0] == recording.object_key
assert call_args[1].get("bucket") == "test-bucket" # Verify bucket override!
@pytest.mark.asyncio
async def test_consent_cleanup_partial_failure_doesnt_mark_deleted():
room = await rooms_controller.add(
name="Test Room 4",
user_id="test-user",
zulip_auto_post=False,
zulip_stream="",
zulip_topic="",
is_locked=False,
room_mode="normal",
recording_type="cloud",
recording_trigger="automatic",
is_shared=False,
platform="daily",
)
# Create meeting
meeting = await meetings_controller.create(
id="test-multitrack-meeting-4",
room_name="test-room-20250101120003",
room_url="https://test.daily.co/test-room-4",
host_room_url="https://test.daily.co/test-room-4",
start_date=datetime.now(timezone.utc),
end_date=datetime.now(timezone.utc),
room=room,
)
track_keys = [
"recordings/test-room-20250101120003/track-0.webm",
"recordings/test-room-20250101120003/track-1.webm",
"recordings/test-room-20250101120003/track-2.webm",
]
recording = await recordings_controller.create(
Recording(
bucket_name="test-bucket",
object_key="recordings/test-room-20250101120003",
recorded_at=datetime.now(timezone.utc),
meeting_id=meeting.id,
track_keys=track_keys,
)
)
# Create transcript
transcript = await transcripts_controller.add(
name="Test Partial Failure Transcript",
source_kind=SourceKind.ROOM,
recording_id=recording.id,
meeting_id=meeting.id,
)
# Add consent denial
await meeting_consent_controller.upsert(
MeetingConsent(
meeting_id=meeting.id,
user_id="test-user-4",
consent_given=False,
consent_timestamp=datetime.now(timezone.utc),
)
)
# Mock get_transcripts_storage (master credentials with bucket override) with partial failure
with patch(
"reflector.pipelines.main_live_pipeline.get_transcripts_storage"
) as mock_get_transcripts_storage:
mock_master_storage = MagicMock()
call_count = 0
async def delete_side_effect(key, bucket=None):
nonlocal call_count
call_count += 1
if call_count == 2:
raise Exception("S3 deletion failed")
mock_master_storage.delete_file = AsyncMock(side_effect=delete_side_effect)
mock_get_transcripts_storage.return_value = mock_master_storage
await cleanup_consent(transcript_id=transcript.id)
# Verify master storage was called with bucket override
assert mock_master_storage.delete_file.call_count == 3
updated_transcript = await transcripts_controller.get_by_id(transcript.id)
assert (
updated_transcript.audio_deleted is None
or updated_transcript.audio_deleted is False
)

View File

@@ -127,27 +127,18 @@ async def mock_storage():
from reflector.storage.base import Storage from reflector.storage.base import Storage
class TestStorage(Storage): class TestStorage(Storage):
async def _put_file(self, path, data, bucket=None): async def _put_file(self, path, data):
return None return None
async def _get_file_url( async def _get_file_url(self, path):
self,
path,
operation: str = "get_object",
expires_in: int = 3600,
bucket=None,
):
return f"http://test-storage/{path}" return f"http://test-storage/{path}"
async def _get_file(self, path, bucket=None): async def _get_file(self, path):
return b"test_audio_data" return b"test_audio_data"
async def _delete_file(self, path, bucket=None): async def _delete_file(self, path):
return None return None
async def _stream_to_fileobj(self, path, fileobj, bucket=None):
fileobj.write(b"test_audio_data")
storage = TestStorage() storage = TestStorage()
# Add mock tracking for verification # Add mock tracking for verification
storage._put_file = AsyncMock(side_effect=storage._put_file) storage._put_file = AsyncMock(side_effect=storage._put_file)
@@ -190,7 +181,7 @@ async def mock_waveform_processor():
async def mock_topic_detector(): async def mock_topic_detector():
"""Mock TranscriptTopicDetectorProcessor""" """Mock TranscriptTopicDetectorProcessor"""
with patch( with patch(
"reflector.pipelines.topic_processing.TranscriptTopicDetectorProcessor" "reflector.pipelines.main_file_pipeline.TranscriptTopicDetectorProcessor"
) as mock_topic_class: ) as mock_topic_class:
mock_topic = AsyncMock() mock_topic = AsyncMock()
mock_topic.set_pipeline = MagicMock() mock_topic.set_pipeline = MagicMock()
@@ -227,7 +218,7 @@ async def mock_topic_detector():
async def mock_title_processor(): async def mock_title_processor():
"""Mock TranscriptFinalTitleProcessor""" """Mock TranscriptFinalTitleProcessor"""
with patch( with patch(
"reflector.pipelines.topic_processing.TranscriptFinalTitleProcessor" "reflector.pipelines.main_file_pipeline.TranscriptFinalTitleProcessor"
) as mock_title_class: ) as mock_title_class:
mock_title = AsyncMock() mock_title = AsyncMock()
mock_title.set_pipeline = MagicMock() mock_title.set_pipeline = MagicMock()
@@ -256,7 +247,7 @@ async def mock_title_processor():
async def mock_summary_processor(): async def mock_summary_processor():
"""Mock TranscriptFinalSummaryProcessor""" """Mock TranscriptFinalSummaryProcessor"""
with patch( with patch(
"reflector.pipelines.topic_processing.TranscriptFinalSummaryProcessor" "reflector.pipelines.main_file_pipeline.TranscriptFinalSummaryProcessor"
) as mock_summary_class: ) as mock_summary_class:
mock_summary = AsyncMock() mock_summary = AsyncMock()
mock_summary.set_pipeline = MagicMock() mock_summary.set_pipeline = MagicMock()

View File

@@ -48,7 +48,6 @@ async def test_create_room_with_ics_fields(authenticated_client):
"ics_url": "https://calendar.example.com/test.ics", "ics_url": "https://calendar.example.com/test.ics",
"ics_fetch_interval": 600, "ics_fetch_interval": 600,
"ics_enabled": True, "ics_enabled": True,
"platform": "daily",
}, },
) )
assert response.status_code == 200 assert response.status_code == 200
@@ -76,7 +75,6 @@ async def test_update_room_ics_configuration(authenticated_client):
"is_shared": False, "is_shared": False,
"webhook_url": "", "webhook_url": "",
"webhook_secret": "", "webhook_secret": "",
"platform": "daily",
}, },
) )
assert response.status_code == 200 assert response.status_code == 200
@@ -113,7 +111,6 @@ async def test_trigger_ics_sync(authenticated_client):
is_shared=False, is_shared=False,
ics_url="https://calendar.example.com/api.ics", ics_url="https://calendar.example.com/api.ics",
ics_enabled=True, ics_enabled=True,
platform="daily",
) )
cal = Calendar() cal = Calendar()
@@ -157,7 +154,6 @@ async def test_trigger_ics_sync_unauthorized(client):
is_shared=False, is_shared=False,
ics_url="https://calendar.example.com/api.ics", ics_url="https://calendar.example.com/api.ics",
ics_enabled=True, ics_enabled=True,
platform="daily",
) )
response = await client.post(f"/rooms/{room.name}/ics/sync") response = await client.post(f"/rooms/{room.name}/ics/sync")
@@ -180,7 +176,6 @@ async def test_trigger_ics_sync_not_configured(authenticated_client):
recording_trigger="automatic-2nd-participant", recording_trigger="automatic-2nd-participant",
is_shared=False, is_shared=False,
ics_enabled=False, ics_enabled=False,
platform="daily",
) )
response = await client.post(f"/rooms/{room.name}/ics/sync") response = await client.post(f"/rooms/{room.name}/ics/sync")
@@ -205,7 +200,6 @@ async def test_get_ics_status(authenticated_client):
ics_url="https://calendar.example.com/status.ics", ics_url="https://calendar.example.com/status.ics",
ics_enabled=True, ics_enabled=True,
ics_fetch_interval=300, ics_fetch_interval=300,
platform="daily",
) )
now = datetime.now(timezone.utc) now = datetime.now(timezone.utc)
@@ -237,7 +231,6 @@ async def test_get_ics_status_unauthorized(client):
is_shared=False, is_shared=False,
ics_url="https://calendar.example.com/status.ics", ics_url="https://calendar.example.com/status.ics",
ics_enabled=True, ics_enabled=True,
platform="daily",
) )
response = await client.get(f"/rooms/{room.name}/ics/status") response = await client.get(f"/rooms/{room.name}/ics/status")
@@ -259,7 +252,6 @@ async def test_list_room_meetings(authenticated_client):
recording_type="cloud", recording_type="cloud",
recording_trigger="automatic-2nd-participant", recording_trigger="automatic-2nd-participant",
is_shared=False, is_shared=False,
platform="daily",
) )
now = datetime.now(timezone.utc) now = datetime.now(timezone.utc)
@@ -306,7 +298,6 @@ async def test_list_room_meetings_non_owner(client):
recording_type="cloud", recording_type="cloud",
recording_trigger="automatic-2nd-participant", recording_trigger="automatic-2nd-participant",
is_shared=False, is_shared=False,
platform="daily",
) )
event = CalendarEvent( event = CalendarEvent(
@@ -343,7 +334,6 @@ async def test_list_upcoming_meetings(authenticated_client):
recording_type="cloud", recording_type="cloud",
recording_trigger="automatic-2nd-participant", recording_trigger="automatic-2nd-participant",
is_shared=False, is_shared=False,
platform="daily",
) )
now = datetime.now(timezone.utc) now = datetime.now(timezone.utc)

View File

@@ -1,256 +0,0 @@
from datetime import datetime, timedelta, timezone
import pytest
from reflector.db import get_database
from reflector.db.search import SearchParameters, search_controller
from reflector.db.transcripts import SourceKind, transcripts
@pytest.mark.asyncio
class TestDateRangeIntegration:
async def setup_test_transcripts(self):
# Use a test user_id that will match in our search parameters
test_user_id = "test-user-123"
test_data = [
{
"id": "test-before-range",
"created_at": datetime(2024, 1, 15, tzinfo=timezone.utc),
"title": "Before Range Transcript",
"user_id": test_user_id,
},
{
"id": "test-start-boundary",
"created_at": datetime(2024, 6, 1, tzinfo=timezone.utc),
"title": "Start Boundary Transcript",
"user_id": test_user_id,
},
{
"id": "test-middle-range",
"created_at": datetime(2024, 6, 15, tzinfo=timezone.utc),
"title": "Middle Range Transcript",
"user_id": test_user_id,
},
{
"id": "test-end-boundary",
"created_at": datetime(2024, 6, 30, 23, 59, 59, tzinfo=timezone.utc),
"title": "End Boundary Transcript",
"user_id": test_user_id,
},
{
"id": "test-after-range",
"created_at": datetime(2024, 12, 31, tzinfo=timezone.utc),
"title": "After Range Transcript",
"user_id": test_user_id,
},
]
for data in test_data:
full_data = {
"id": data["id"],
"name": data["id"],
"status": "ended",
"locked": False,
"duration": 60.0,
"created_at": data["created_at"],
"title": data["title"],
"short_summary": "Test summary",
"long_summary": "Test long summary",
"share_mode": "public",
"source_kind": SourceKind.FILE,
"audio_deleted": False,
"reviewed": False,
"user_id": data["user_id"],
}
await get_database().execute(transcripts.insert().values(**full_data))
return test_data
async def cleanup_test_transcripts(self, test_data):
"""Clean up test transcripts."""
for data in test_data:
await get_database().execute(
transcripts.delete().where(transcripts.c.id == data["id"])
)
@pytest.mark.asyncio
async def test_filter_with_from_datetime_only(self):
"""Test filtering with only from_datetime parameter."""
test_data = await self.setup_test_transcripts()
test_user_id = "test-user-123"
try:
params = SearchParameters(
query_text=None,
from_datetime=datetime(2024, 6, 1, tzinfo=timezone.utc),
to_datetime=None,
user_id=test_user_id,
)
results, total = await search_controller.search_transcripts(params)
# Should include: start_boundary, middle, end_boundary, after
result_ids = [r.id for r in results]
assert "test-before-range" not in result_ids
assert "test-start-boundary" in result_ids
assert "test-middle-range" in result_ids
assert "test-end-boundary" in result_ids
assert "test-after-range" in result_ids
finally:
await self.cleanup_test_transcripts(test_data)
@pytest.mark.asyncio
async def test_filter_with_to_datetime_only(self):
"""Test filtering with only to_datetime parameter."""
test_data = await self.setup_test_transcripts()
test_user_id = "test-user-123"
try:
params = SearchParameters(
query_text=None,
from_datetime=None,
to_datetime=datetime(2024, 6, 30, tzinfo=timezone.utc),
user_id=test_user_id,
)
results, total = await search_controller.search_transcripts(params)
result_ids = [r.id for r in results]
assert "test-before-range" in result_ids
assert "test-start-boundary" in result_ids
assert "test-middle-range" in result_ids
assert "test-end-boundary" not in result_ids
assert "test-after-range" not in result_ids
finally:
await self.cleanup_test_transcripts(test_data)
@pytest.mark.asyncio
async def test_filter_with_both_datetimes(self):
test_data = await self.setup_test_transcripts()
test_user_id = "test-user-123"
try:
params = SearchParameters(
query_text=None,
from_datetime=datetime(2024, 6, 1, tzinfo=timezone.utc),
to_datetime=datetime(
2024, 7, 1, tzinfo=timezone.utc
), # Inclusive of 6/30
user_id=test_user_id,
)
results, total = await search_controller.search_transcripts(params)
result_ids = [r.id for r in results]
assert "test-before-range" not in result_ids
assert "test-start-boundary" in result_ids
assert "test-middle-range" in result_ids
assert "test-end-boundary" in result_ids
assert "test-after-range" not in result_ids
finally:
await self.cleanup_test_transcripts(test_data)
@pytest.mark.asyncio
async def test_date_filter_with_room_and_source_kind(self):
test_data = await self.setup_test_transcripts()
test_user_id = "test-user-123"
try:
params = SearchParameters(
query_text=None,
from_datetime=datetime(2024, 6, 1, tzinfo=timezone.utc),
to_datetime=datetime(2024, 7, 1, tzinfo=timezone.utc),
source_kind=SourceKind.FILE,
room_id=None,
user_id=test_user_id,
)
results, total = await search_controller.search_transcripts(params)
for result in results:
assert result.source_kind == SourceKind.FILE
assert result.created_at >= datetime(2024, 6, 1, tzinfo=timezone.utc)
assert result.created_at <= datetime(2024, 7, 1, tzinfo=timezone.utc)
finally:
await self.cleanup_test_transcripts(test_data)
@pytest.mark.asyncio
async def test_empty_results_for_future_dates(self):
test_data = await self.setup_test_transcripts()
test_user_id = "test-user-123"
try:
params = SearchParameters(
query_text=None,
from_datetime=datetime(2099, 1, 1, tzinfo=timezone.utc),
to_datetime=datetime(2099, 12, 31, tzinfo=timezone.utc),
user_id=test_user_id,
)
results, total = await search_controller.search_transcripts(params)
assert results == []
assert total == 0
finally:
await self.cleanup_test_transcripts(test_data)
@pytest.mark.asyncio
async def test_date_only_input_handling(self):
test_data = await self.setup_test_transcripts()
test_user_id = "test-user-123"
try:
# Pydantic will parse date-only strings to datetime at midnight
from_dt = datetime(2024, 6, 15, 0, 0, 0, tzinfo=timezone.utc)
to_dt = datetime(2024, 6, 16, 0, 0, 0, tzinfo=timezone.utc)
params = SearchParameters(
query_text=None,
from_datetime=from_dt,
to_datetime=to_dt,
user_id=test_user_id,
)
results, total = await search_controller.search_transcripts(params)
result_ids = [r.id for r in results]
assert "test-middle-range" in result_ids
assert "test-before-range" not in result_ids
assert "test-after-range" not in result_ids
finally:
await self.cleanup_test_transcripts(test_data)
class TestDateValidationEdgeCases:
"""Edge case tests for datetime validation."""
def test_timezone_aware_comparison(self):
"""Test that timezone-aware comparisons work correctly."""
# PST time (UTC-8)
pst = timezone(timedelta(hours=-8))
pst_dt = datetime(2024, 6, 15, 8, 0, 0, tzinfo=pst)
# UTC time equivalent (8AM PST = 4PM UTC)
utc_dt = datetime(2024, 6, 15, 16, 0, 0, tzinfo=timezone.utc)
assert pst_dt == utc_dt
def test_mixed_timezone_input(self):
"""Test handling mixed timezone inputs."""
pst = timezone(timedelta(hours=-8))
ist = timezone(timedelta(hours=5, minutes=30))
from_date = datetime(2024, 6, 15, 0, 0, 0, tzinfo=pst) # PST midnight
to_date = datetime(2024, 6, 15, 23, 59, 59, tzinfo=ist) # IST end of day
assert from_date.tzinfo is not None
assert to_date.tzinfo is not None
assert from_date < to_date

View File

@@ -1,321 +0,0 @@
"""Tests for storage abstraction layer."""
import io
from unittest.mock import AsyncMock, MagicMock, patch
import pytest
from botocore.exceptions import ClientError
from reflector.storage.base import StoragePermissionError
from reflector.storage.storage_aws import AwsStorage
@pytest.mark.asyncio
async def test_aws_storage_stream_to_fileobj():
"""Test that AWS storage can stream directly to a file object without loading into memory."""
# Setup
storage = AwsStorage(
aws_bucket_name="test-bucket",
aws_region="us-east-1",
aws_access_key_id="test-key",
aws_secret_access_key="test-secret",
)
# Mock download_fileobj to write data
async def mock_download(Bucket, Key, Fileobj, **kwargs):
Fileobj.write(b"chunk1chunk2")
mock_client = AsyncMock()
mock_client.download_fileobj = AsyncMock(side_effect=mock_download)
mock_client.__aenter__ = AsyncMock(return_value=mock_client)
mock_client.__aexit__ = AsyncMock(return_value=None)
# Patch the session client
with patch.object(storage.session, "client", return_value=mock_client):
# Create a file-like object to stream to
output = io.BytesIO()
# Act - stream to file object
await storage.stream_to_fileobj("test-file.mp4", output, bucket="test-bucket")
# Assert
mock_client.download_fileobj.assert_called_once_with(
Bucket="test-bucket", Key="test-file.mp4", Fileobj=output
)
# Check that data was written to output
output.seek(0)
assert output.read() == b"chunk1chunk2"
@pytest.mark.asyncio
async def test_aws_storage_stream_to_fileobj_with_folder():
"""Test streaming with folder prefix in bucket name."""
storage = AwsStorage(
aws_bucket_name="test-bucket/recordings",
aws_region="us-east-1",
aws_access_key_id="test-key",
aws_secret_access_key="test-secret",
)
async def mock_download(Bucket, Key, Fileobj, **kwargs):
Fileobj.write(b"data")
mock_client = AsyncMock()
mock_client.download_fileobj = AsyncMock(side_effect=mock_download)
mock_client.__aenter__ = AsyncMock(return_value=mock_client)
mock_client.__aexit__ = AsyncMock(return_value=None)
with patch.object(storage.session, "client", return_value=mock_client):
output = io.BytesIO()
await storage.stream_to_fileobj("file.mp4", output, bucket="other-bucket")
# Should use folder prefix from instance config
mock_client.download_fileobj.assert_called_once_with(
Bucket="other-bucket", Key="recordings/file.mp4", Fileobj=output
)
@pytest.mark.asyncio
async def test_storage_base_class_stream_to_fileobj():
"""Test that base Storage class has stream_to_fileobj method."""
from reflector.storage.base import Storage
# Verify method exists in base class
assert hasattr(Storage, "stream_to_fileobj")
# Create a mock storage instance
storage = MagicMock(spec=Storage)
storage.stream_to_fileobj = AsyncMock()
# Should be callable
await storage.stream_to_fileobj("file.mp4", io.BytesIO())
storage.stream_to_fileobj.assert_called_once()
@pytest.mark.asyncio
async def test_aws_storage_stream_uses_download_fileobj():
"""Test that download_fileobj is called correctly."""
storage = AwsStorage(
aws_bucket_name="test-bucket",
aws_region="us-east-1",
aws_access_key_id="test-key",
aws_secret_access_key="test-secret",
)
async def mock_download(Bucket, Key, Fileobj, **kwargs):
Fileobj.write(b"data")
mock_client = AsyncMock()
mock_client.download_fileobj = AsyncMock(side_effect=mock_download)
mock_client.__aenter__ = AsyncMock(return_value=mock_client)
mock_client.__aexit__ = AsyncMock(return_value=None)
with patch.object(storage.session, "client", return_value=mock_client):
output = io.BytesIO()
await storage.stream_to_fileobj("test.mp4", output)
# Verify download_fileobj was called with correct parameters
mock_client.download_fileobj.assert_called_once_with(
Bucket="test-bucket", Key="test.mp4", Fileobj=output
)
@pytest.mark.asyncio
async def test_aws_storage_handles_access_denied_error():
"""Test that AccessDenied errors are caught and wrapped in StoragePermissionError."""
storage = AwsStorage(
aws_bucket_name="test-bucket",
aws_region="us-east-1",
aws_access_key_id="test-key",
aws_secret_access_key="test-secret",
)
# Mock ClientError with AccessDenied
error_response = {"Error": {"Code": "AccessDenied", "Message": "Access Denied"}}
mock_client = AsyncMock()
mock_client.put_object = AsyncMock(
side_effect=ClientError(error_response, "PutObject")
)
mock_client.__aenter__ = AsyncMock(return_value=mock_client)
mock_client.__aexit__ = AsyncMock(return_value=None)
with patch.object(storage.session, "client", return_value=mock_client):
with pytest.raises(StoragePermissionError) as exc_info:
await storage.put_file("test.txt", b"data")
# Verify error message contains expected information
error_msg = str(exc_info.value)
assert "AccessDenied" in error_msg
assert "default bucket 'test-bucket'" in error_msg
assert "S3 upload failed" in error_msg
@pytest.mark.asyncio
async def test_aws_storage_handles_no_such_bucket_error():
"""Test that NoSuchBucket errors are caught and wrapped in StoragePermissionError."""
storage = AwsStorage(
aws_bucket_name="test-bucket",
aws_region="us-east-1",
aws_access_key_id="test-key",
aws_secret_access_key="test-secret",
)
# Mock ClientError with NoSuchBucket
error_response = {
"Error": {
"Code": "NoSuchBucket",
"Message": "The specified bucket does not exist",
}
}
mock_client = AsyncMock()
mock_client.delete_object = AsyncMock(
side_effect=ClientError(error_response, "DeleteObject")
)
mock_client.__aenter__ = AsyncMock(return_value=mock_client)
mock_client.__aexit__ = AsyncMock(return_value=None)
with patch.object(storage.session, "client", return_value=mock_client):
with pytest.raises(StoragePermissionError) as exc_info:
await storage.delete_file("test.txt")
# Verify error message contains expected information
error_msg = str(exc_info.value)
assert "NoSuchBucket" in error_msg
assert "default bucket 'test-bucket'" in error_msg
assert "S3 delete failed" in error_msg
@pytest.mark.asyncio
async def test_aws_storage_error_message_with_bucket_override():
"""Test that error messages correctly show overridden bucket."""
storage = AwsStorage(
aws_bucket_name="default-bucket",
aws_region="us-east-1",
aws_access_key_id="test-key",
aws_secret_access_key="test-secret",
)
# Mock ClientError with AccessDenied
error_response = {"Error": {"Code": "AccessDenied", "Message": "Access Denied"}}
mock_client = AsyncMock()
mock_client.get_object = AsyncMock(
side_effect=ClientError(error_response, "GetObject")
)
mock_client.__aenter__ = AsyncMock(return_value=mock_client)
mock_client.__aexit__ = AsyncMock(return_value=None)
with patch.object(storage.session, "client", return_value=mock_client):
with pytest.raises(StoragePermissionError) as exc_info:
await storage.get_file("test.txt", bucket="override-bucket")
# Verify error message shows overridden bucket, not default
error_msg = str(exc_info.value)
assert "overridden bucket 'override-bucket'" in error_msg
assert "default-bucket" not in error_msg
assert "S3 download failed" in error_msg
@pytest.mark.asyncio
async def test_aws_storage_reraises_non_handled_errors():
"""Test that non-AccessDenied/NoSuchBucket errors are re-raised as-is."""
storage = AwsStorage(
aws_bucket_name="test-bucket",
aws_region="us-east-1",
aws_access_key_id="test-key",
aws_secret_access_key="test-secret",
)
# Mock ClientError with different error code
error_response = {
"Error": {"Code": "InternalError", "Message": "Internal Server Error"}
}
mock_client = AsyncMock()
mock_client.put_object = AsyncMock(
side_effect=ClientError(error_response, "PutObject")
)
mock_client.__aenter__ = AsyncMock(return_value=mock_client)
mock_client.__aexit__ = AsyncMock(return_value=None)
with patch.object(storage.session, "client", return_value=mock_client):
# Should raise ClientError, not StoragePermissionError
with pytest.raises(ClientError) as exc_info:
await storage.put_file("test.txt", b"data")
# Verify it's the original ClientError
assert exc_info.value.response["Error"]["Code"] == "InternalError"
@pytest.mark.asyncio
async def test_aws_storage_presign_url_handles_errors():
"""Test that presigned URL generation handles permission errors."""
storage = AwsStorage(
aws_bucket_name="test-bucket",
aws_region="us-east-1",
aws_access_key_id="test-key",
aws_secret_access_key="test-secret",
)
# Mock ClientError with AccessDenied during presign operation
error_response = {"Error": {"Code": "AccessDenied", "Message": "Access Denied"}}
mock_client = AsyncMock()
mock_client.generate_presigned_url = AsyncMock(
side_effect=ClientError(error_response, "GeneratePresignedUrl")
)
mock_client.__aenter__ = AsyncMock(return_value=mock_client)
mock_client.__aexit__ = AsyncMock(return_value=None)
with patch.object(storage.session, "client", return_value=mock_client):
with pytest.raises(StoragePermissionError) as exc_info:
await storage.get_file_url("test.txt")
# Verify error message
error_msg = str(exc_info.value)
assert "S3 presign failed" in error_msg
assert "AccessDenied" in error_msg
@pytest.mark.asyncio
async def test_aws_storage_list_objects_handles_errors():
"""Test that list_objects handles permission errors."""
storage = AwsStorage(
aws_bucket_name="test-bucket",
aws_region="us-east-1",
aws_access_key_id="test-key",
aws_secret_access_key="test-secret",
)
# Mock ClientError during list operation
error_response = {"Error": {"Code": "AccessDenied", "Message": "Access Denied"}}
mock_paginator = MagicMock()
async def mock_paginate(*args, **kwargs):
raise ClientError(error_response, "ListObjectsV2")
yield # Make it an async generator
mock_paginator.paginate = mock_paginate
mock_client = AsyncMock()
mock_client.get_paginator = MagicMock(return_value=mock_paginator)
mock_client.__aenter__ = AsyncMock(return_value=mock_client)
mock_client.__aexit__ = AsyncMock(return_value=None)
with patch.object(storage.session, "client", return_value=mock_client):
with pytest.raises(StoragePermissionError) as exc_info:
await storage.list_objects(prefix="test/")
error_msg = str(exc_info.value)
assert "S3 list_objects failed" in error_msg
assert "AccessDenied" in error_msg
def test_aws_storage_constructor_rejects_mixed_auth():
"""Test that constructor rejects both role_arn and access keys."""
with pytest.raises(ValueError, match="cannot use both.*role_arn.*access keys"):
AwsStorage(
aws_bucket_name="test-bucket",
aws_region="us-east-1",
aws_access_key_id="test-key",
aws_secret_access_key="test-secret",
aws_role_arn="arn:aws:iam::123456789012:role/test-role",
)

View File

@@ -1,6 +1,5 @@
import asyncio import asyncio
import time import time
from unittest.mock import patch
import pytest import pytest
from httpx import ASGITransport, AsyncClient from httpx import ASGITransport, AsyncClient
@@ -102,113 +101,3 @@ async def test_transcript_process(
assert response.status_code == 200 assert response.status_code == 200
assert len(response.json()) == 1 assert len(response.json()) == 1
assert "Hello world. How are you today?" in response.json()[0]["transcript"] assert "Hello world. How are you today?" in response.json()[0]["transcript"]
@pytest.mark.usefixtures("setup_database")
@pytest.mark.asyncio
async def test_whereby_recording_uses_file_pipeline(client):
"""Test that Whereby recordings (bucket_name but no track_keys) use file pipeline"""
from datetime import datetime, timezone
from reflector.db.recordings import Recording, recordings_controller
from reflector.db.transcripts import transcripts_controller
# Create transcript with Whereby recording (has bucket_name, no track_keys)
transcript = await transcripts_controller.add(
"",
source_kind="room",
source_language="en",
target_language="en",
user_id="test-user",
share_mode="public",
)
recording = await recordings_controller.create(
Recording(
bucket_name="whereby-bucket",
object_key="test-recording.mp4", # gitleaks:allow
meeting_id="test-meeting",
recorded_at=datetime.now(timezone.utc),
track_keys=None, # Whereby recordings have no track_keys
)
)
await transcripts_controller.update(
transcript, {"recording_id": recording.id, "status": "uploaded"}
)
with (
patch(
"reflector.views.transcripts_process.task_pipeline_file_process"
) as mock_file_pipeline,
patch(
"reflector.views.transcripts_process.task_pipeline_multitrack_process"
) as mock_multitrack_pipeline,
):
response = await client.post(f"/transcripts/{transcript.id}/process")
assert response.status_code == 200
assert response.json()["status"] == "ok"
# Whereby recordings should use file pipeline
mock_file_pipeline.delay.assert_called_once_with(transcript_id=transcript.id)
mock_multitrack_pipeline.delay.assert_not_called()
@pytest.mark.usefixtures("setup_database")
@pytest.mark.asyncio
async def test_dailyco_recording_uses_multitrack_pipeline(client):
"""Test that Daily.co recordings (bucket_name + track_keys) use multitrack pipeline"""
from datetime import datetime, timezone
from reflector.db.recordings import Recording, recordings_controller
from reflector.db.transcripts import transcripts_controller
# Create transcript with Daily.co multitrack recording
transcript = await transcripts_controller.add(
"",
source_kind="room",
source_language="en",
target_language="en",
user_id="test-user",
share_mode="public",
)
track_keys = [
"recordings/test-room/track1.webm",
"recordings/test-room/track2.webm",
]
recording = await recordings_controller.create(
Recording(
bucket_name="daily-bucket",
object_key="recordings/test-room",
meeting_id="test-meeting",
track_keys=track_keys,
recorded_at=datetime.now(timezone.utc),
)
)
await transcripts_controller.update(
transcript, {"recording_id": recording.id, "status": "uploaded"}
)
with (
patch(
"reflector.views.transcripts_process.task_pipeline_file_process"
) as mock_file_pipeline,
patch(
"reflector.views.transcripts_process.task_pipeline_multitrack_process"
) as mock_multitrack_pipeline,
):
response = await client.post(f"/transcripts/{transcript.id}/process")
assert response.status_code == 200
assert response.json()["status"] == "ok"
# Daily.co multitrack recordings should use multitrack pipeline
mock_multitrack_pipeline.delay.assert_called_once_with(
transcript_id=transcript.id,
bucket_name="daily-bucket",
track_keys=track_keys,
)
mock_file_pipeline.delay.assert_not_called()

View File

@@ -22,16 +22,13 @@ async def test_recording_deleted_with_transcript():
recording_id=recording.id, recording_id=recording.id,
) )
with patch("reflector.db.transcripts.get_transcripts_storage") as mock_get_storage: with patch("reflector.db.transcripts.get_recordings_storage") as mock_get_storage:
storage_instance = mock_get_storage.return_value storage_instance = mock_get_storage.return_value
storage_instance.delete_file = AsyncMock() storage_instance.delete_file = AsyncMock()
await transcripts_controller.remove_by_id(transcript.id) await transcripts_controller.remove_by_id(transcript.id)
# Should be called with bucket override storage_instance.delete_file.assert_awaited_once_with(recording.object_key)
storage_instance.delete_file.assert_awaited_once_with(
recording.object_key, bucket=recording.bucket_name
)
assert await recordings_controller.get_by_id(recording.id) is None assert await recordings_controller.get_by_id(recording.id) is None
assert await transcripts_controller.get_by_id(transcript.id) is None assert await transcripts_controller.get_by_id(transcript.id) is None

View File

@@ -1,70 +0,0 @@
import pytest
from reflector.db.user_api_keys import user_api_keys_controller
@pytest.mark.asyncio
async def test_api_key_creation_and_verification():
api_key_model, plaintext = await user_api_keys_controller.create_key(
user_id="test_user",
name="Test API Key",
)
verified = await user_api_keys_controller.verify_key(plaintext)
assert verified is not None
assert verified.user_id == "test_user"
assert verified.name == "Test API Key"
invalid = await user_api_keys_controller.verify_key("fake_key")
assert invalid is None
@pytest.mark.asyncio
async def test_api_key_hashing():
_, plaintext = await user_api_keys_controller.create_key(
user_id="test_user_2",
)
api_keys = await user_api_keys_controller.list_by_user_id("test_user_2")
assert len(api_keys) == 1
assert api_keys[0].key_hash != plaintext
@pytest.mark.asyncio
async def test_generate_api_key_uniqueness():
key1 = user_api_keys_controller.generate_key()
key2 = user_api_keys_controller.generate_key()
assert key1 != key2
@pytest.mark.asyncio
async def test_hash_api_key_deterministic():
key = "test_key_123"
hash1 = user_api_keys_controller.hash_key(key)
hash2 = user_api_keys_controller.hash_key(key)
assert hash1 == hash2
@pytest.mark.asyncio
async def test_get_by_user_id_empty():
api_keys = await user_api_keys_controller.list_by_user_id("nonexistent_user")
assert api_keys == []
@pytest.mark.asyncio
async def test_get_by_user_id_multiple():
user_id = "multi_key_user"
_, plaintext1 = await user_api_keys_controller.create_key(
user_id=user_id,
name="API Key 1",
)
_, plaintext2 = await user_api_keys_controller.create_key(
user_id=user_id,
name="API Key 2",
)
api_keys = await user_api_keys_controller.list_by_user_id(user_id)
assert len(api_keys) == 2
names = {k.name for k in api_keys}
assert names == {"API Key 1", "API Key 2"}

View File

@@ -1,17 +0,0 @@
import pytest
from reflector.utils.daily import extract_base_room_name
@pytest.mark.parametrize(
"daily_room_name,expected",
[
("daily-20251020193458", "daily"),
("daily-2-20251020193458", "daily-2"),
("my-room-name-20251020193458", "my-room-name"),
("room-with-numbers-123-20251020193458", "room-with-numbers-123"),
("x-20251020193458", "x"),
],
)
def test_extract_base_room_name(daily_room_name, expected):
assert extract_base_room_name(daily_room_name) == expected

View File

@@ -1,63 +0,0 @@
"""Tests for URL utility functions."""
from reflector.utils.url import add_query_param
class TestAddQueryParam:
"""Test the add_query_param function."""
def test_add_param_to_url_without_query(self):
"""Should add query param with ? to URL without existing params."""
url = "https://example.com/room"
result = add_query_param(url, "t", "token123")
assert result == "https://example.com/room?t=token123"
def test_add_param_to_url_with_existing_query(self):
"""Should add query param with & to URL with existing params."""
url = "https://example.com/room?existing=param"
result = add_query_param(url, "t", "token123")
assert result == "https://example.com/room?existing=param&t=token123"
def test_add_param_to_url_with_multiple_existing_params(self):
"""Should add query param to URL with multiple existing params."""
url = "https://example.com/room?param1=value1&param2=value2"
result = add_query_param(url, "t", "token123")
assert (
result == "https://example.com/room?param1=value1&param2=value2&t=token123"
)
def test_add_param_with_special_characters(self):
"""Should properly encode special characters in param value."""
url = "https://example.com/room"
result = add_query_param(url, "name", "hello world")
assert result == "https://example.com/room?name=hello+world"
def test_add_param_to_url_with_fragment(self):
"""Should preserve URL fragment when adding query param."""
url = "https://example.com/room#section"
result = add_query_param(url, "t", "token123")
assert result == "https://example.com/room?t=token123#section"
def test_add_param_to_url_with_query_and_fragment(self):
"""Should preserve fragment when adding param to URL with existing query."""
url = "https://example.com/room?existing=param#section"
result = add_query_param(url, "t", "token123")
assert result == "https://example.com/room?existing=param&t=token123#section"
def test_add_param_overwrites_existing_param(self):
"""Should overwrite existing param with same name."""
url = "https://example.com/room?t=oldtoken"
result = add_query_param(url, "t", "newtoken")
assert result == "https://example.com/room?t=newtoken"
def test_url_without_scheme(self):
"""Should handle URLs without scheme (relative URLs)."""
url = "/room/path"
result = add_query_param(url, "t", "token123")
assert result == "/room/path?t=token123"
def test_empty_url(self):
"""Should handle empty URL."""
url = ""
result = add_query_param(url, "t", "token123")
assert result == "?t=token123"

View File

@@ -1,58 +0,0 @@
"""Tests for video_platforms.factory module."""
from unittest.mock import patch
from reflector.video_platforms.factory import get_platform
class TestGetPlatformF:
"""Test suite for get_platform function."""
@patch("reflector.video_platforms.factory.settings")
def test_with_room_platform(self, mock_settings):
"""When room_platform provided, should return room_platform."""
mock_settings.DEFAULT_VIDEO_PLATFORM = "whereby"
# Should return the room's platform when provided
assert get_platform(room_platform="daily") == "daily"
assert get_platform(room_platform="whereby") == "whereby"
@patch("reflector.video_platforms.factory.settings")
def test_without_room_platform_uses_default(self, mock_settings):
"""When no room_platform, should return DEFAULT_VIDEO_PLATFORM."""
mock_settings.DEFAULT_VIDEO_PLATFORM = "whereby"
# Should return default when room_platform is None
assert get_platform(room_platform=None) == "whereby"
@patch("reflector.video_platforms.factory.settings")
def test_with_daily_default(self, mock_settings):
"""When DEFAULT_VIDEO_PLATFORM is 'daily', should return 'daily' when no room_platform."""
mock_settings.DEFAULT_VIDEO_PLATFORM = "daily"
# Should return default 'daily' when room_platform is None
assert get_platform(room_platform=None) == "daily"
@patch("reflector.video_platforms.factory.settings")
def test_no_room_id_provided(self, mock_settings):
"""Should work correctly even when room_id is not provided."""
mock_settings.DEFAULT_VIDEO_PLATFORM = "whereby"
# Should use room_platform when provided
assert get_platform(room_platform="daily") == "daily"
# Should use default when room_platform not provided
assert get_platform(room_platform=None) == "whereby"
@patch("reflector.video_platforms.factory.settings")
def test_room_platform_always_takes_precedence(self, mock_settings):
"""room_platform should always be used when provided."""
mock_settings.DEFAULT_VIDEO_PLATFORM = "whereby"
# room_platform should take precedence over default
assert get_platform(room_platform="daily") == "daily"
assert get_platform(room_platform="whereby") == "whereby"
# Different default shouldn't matter when room_platform provided
mock_settings.DEFAULT_VIDEO_PLATFORM = "daily"
assert get_platform(room_platform="whereby") == "whereby"

View File

@@ -0,0 +1,23 @@
#!/usr/bin/env python
"""
Trigger reprocessing of Daily.co multitrack recording via Celery
"""
from reflector.pipelines.main_multitrack_pipeline import (
task_pipeline_multitrack_process,
)
# Trigger the Celery task
result = task_pipeline_multitrack_process.delay(
transcript_id="32fad706-f8cf-434c-94c8-1ee69f7be081", # The ID that was created
bucket_name="reflector-dailyco-local",
track_keys=[
"monadical/daily-20251020193458/1760988935484-52f7f48b-fbab-431f-9a50-87b9abfc8255-cam-audio-1760988935922",
"monadical/daily-20251020193458/1760988935484-a37c35e3-6f8e-4274-a482-e9d0f102a732-cam-audio-1760988943823",
],
)
print(f"Task ID: {result.id}")
print(
f"Processing started! Check: http://localhost:3000/transcripts/32fad706-f8cf-434c-94c8-1ee69f7be081"
)

View File

@@ -1,5 +1,5 @@
"use client"; "use client";
import React, { useState, useEffect, useMemo } from "react"; import React, { useState, useEffect } from "react";
import { import {
Flex, Flex,
Spinner, Spinner,
@@ -235,26 +235,15 @@ export default function TranscriptBrowser() {
const pageSize = 20; const pageSize = 20;
// must be json-able
const searchFilters = useMemo(
() => ({
q: urlSearchQuery,
extras: {
room_id: urlRoomId || undefined,
source_kind: urlSourceKind || undefined,
},
}),
[urlSearchQuery, urlRoomId, urlSourceKind],
);
const { const {
data: searchData, data: searchData,
isLoading: searchLoading, isLoading: searchLoading,
refetch: reloadSearch, refetch: reloadSearch,
} = useTranscriptsSearch(searchFilters.q, { } = useTranscriptsSearch(urlSearchQuery, {
limit: pageSize, limit: pageSize,
offset: paginationPageTo0Based(page) * pageSize, offset: paginationPageTo0Based(page) * pageSize,
...searchFilters.extras, room_id: urlRoomId || undefined,
source_kind: urlSourceKind || undefined,
}); });
const results = searchData?.results || []; const results = searchData?.results || [];
@@ -266,12 +255,6 @@ export default function TranscriptBrowser() {
const totalPages = getTotalPages(totalResults, pageSize); const totalPages = getTotalPages(totalResults, pageSize);
// reset pagination when search results change (detected by total change; good enough approximation)
useEffect(() => {
// operation is idempotent
setPage(FIRST_PAGE).then(() => {});
}, [JSON.stringify(searchFilters)]);
const userName = useUserName(); const userName = useUserName();
const [deletionLoading, setDeletionLoading] = useState(false); const [deletionLoading, setDeletionLoading] = useState(false);
const cancelRef = React.useRef(null); const cancelRef = React.useRef(null);

View File

@@ -78,14 +78,6 @@ export default async function AppLayout({
)} )}
{featureEnabled("requireLogin") ? ( {featureEnabled("requireLogin") ? (
<> <>
&nbsp;·&nbsp;
<Link
href="/settings/api-keys"
as={NextLink}
className="font-light px-2"
>
Settings
</Link>
&nbsp;·&nbsp; &nbsp;·&nbsp;
<UserInfo /> <UserInfo />
</> </>

View File

@@ -1,341 +0,0 @@
"use client";
import React, { useState, useRef } from "react";
import {
Box,
Button,
Heading,
Stack,
Text,
Input,
Table,
Flex,
IconButton,
Code,
Dialog,
} from "@chakra-ui/react";
import { LuTrash2, LuCopy, LuPlus } from "react-icons/lu";
import { useQueryClient } from "@tanstack/react-query";
import { $api } from "../../../lib/apiClient";
import { toaster } from "../../../components/ui/toaster";
interface CreateApiKeyResponse {
id: string;
user_id: string;
name: string | null;
created_at: string;
key: string;
}
export default function ApiKeysPage() {
const [newKeyName, setNewKeyName] = useState("");
const [isCreating, setIsCreating] = useState(false);
const [createdKey, setCreatedKey] = useState<CreateApiKeyResponse | null>(
null,
);
const [keyToDelete, setKeyToDelete] = useState<string | null>(null);
const queryClient = useQueryClient();
const cancelRef = useRef<HTMLButtonElement>(null);
const { data: apiKeys, isLoading } = $api.useQuery(
"get",
"/v1/user/api-keys",
);
const createKeyMutation = $api.useMutation("post", "/v1/user/api-keys", {
onSuccess: (data) => {
setCreatedKey(data);
setNewKeyName("");
setIsCreating(false);
queryClient.invalidateQueries({ queryKey: ["get", "/v1/user/api-keys"] });
toaster.create({
duration: 5000,
render: () => (
<Box bg="green.500" color="white" px={4} py={3} borderRadius="md">
<Text fontWeight="bold">API key created</Text>
<Text fontSize="sm">
Make sure to copy it now - you won't see it again!
</Text>
</Box>
),
});
setTimeout(() => {
const keyElement = document.querySelector(".api-key-code");
if (keyElement) {
const range = document.createRange();
range.selectNodeContents(keyElement);
const selection = window.getSelection();
selection?.removeAllRanges();
selection?.addRange(range);
}
}, 100);
},
onError: () => {
toaster.create({
duration: 3000,
render: () => (
<Box bg="red.500" color="white" px={4} py={3} borderRadius="md">
<Text fontWeight="bold">Error</Text>
<Text fontSize="sm">Failed to create API key</Text>
</Box>
),
});
},
});
const deleteKeyMutation = $api.useMutation(
"delete",
"/v1/user/api-keys/{key_id}",
{
onSuccess: () => {
queryClient.invalidateQueries({
queryKey: ["get", "/v1/user/api-keys"],
});
toaster.create({
duration: 3000,
render: () => (
<Box bg="green.500" color="white" px={4} py={3} borderRadius="md">
<Text fontWeight="bold">API key deleted</Text>
</Box>
),
});
},
onError: () => {
toaster.create({
duration: 3000,
render: () => (
<Box bg="red.500" color="white" px={4} py={3} borderRadius="md">
<Text fontWeight="bold">Error</Text>
<Text fontSize="sm">Failed to delete API key</Text>
</Box>
),
});
},
},
);
const handleCreateKey = () => {
createKeyMutation.mutate({
body: { name: newKeyName || null },
});
};
const handleCopyKey = (key: string) => {
navigator.clipboard.writeText(key);
toaster.create({
duration: 2000,
render: () => (
<Box bg="green.500" color="white" px={4} py={3} borderRadius="md">
<Text fontWeight="bold">Copied to clipboard</Text>
</Box>
),
});
};
const handleDeleteRequest = (keyId: string) => {
setKeyToDelete(keyId);
};
const confirmDelete = () => {
if (keyToDelete) {
deleteKeyMutation.mutate({
params: { path: { key_id: keyToDelete } },
});
setKeyToDelete(null);
}
};
const formatDate = (dateString: string) => {
return new Date(dateString).toLocaleDateString("en-US", {
year: "numeric",
month: "short",
day: "numeric",
hour: "2-digit",
minute: "2-digit",
});
};
return (
<Box maxW="800px" w="100%" mx="auto" p={8}>
<Heading mb={2}>API Keys</Heading>
<Text color="gray.600" mb={6}>
Manage your API keys for programmatic access to Reflector
</Text>
{/* Show newly created key */}
{createdKey && (
<Box
mb={6}
p={4}
bg="green.50"
borderWidth={1}
borderColor="green.200"
borderRadius="md"
>
<Flex justify="space-between" align="start" mb={2}>
<Heading size="sm" color="green.800">
API Key Created
</Heading>
<Button
size="sm"
variant="ghost"
onClick={() => setCreatedKey(null)}
>
×
</Button>
</Flex>
<Text mb={2} fontSize="sm" color="green.700">
Make sure to copy your API key now. You won't be able to see it
again!
</Text>
<Flex gap={2} align="center">
<Code
p={2}
flex={1}
fontSize="sm"
bg="white"
className="api-key-code"
>
{createdKey.key}
</Code>
<IconButton
aria-label="Copy API key"
size="sm"
onClick={() => handleCopyKey(createdKey.key)}
>
<LuCopy />
</IconButton>
</Flex>
</Box>
)}
{/* Create new key */}
<Box mb={8} p={6} borderWidth={1} borderRadius="md">
<Heading size="md" mb={4}>
Create New API Key
</Heading>
{!isCreating ? (
<Button onClick={() => setIsCreating(true)} colorPalette="blue">
<LuPlus /> Create API Key
</Button>
) : (
<Stack gap={4}>
<Box>
<Text mb={2}>Name (optional)</Text>
<Input
placeholder="e.g., Production API Key"
value={newKeyName}
onChange={(e) => setNewKeyName(e.target.value)}
/>
</Box>
<Flex gap={2}>
<Button
onClick={handleCreateKey}
colorPalette="blue"
loading={createKeyMutation.isPending}
>
Create
</Button>
<Button
onClick={() => {
setIsCreating(false);
setNewKeyName("");
}}
variant="outline"
>
Cancel
</Button>
</Flex>
</Stack>
)}
</Box>
{/* List of API keys */}
<Box>
<Heading size="md" mb={4}>
Your API Keys
</Heading>
{isLoading ? (
<Text>Loading...</Text>
) : !apiKeys || apiKeys.length === 0 ? (
<Text color="gray.600">
No API keys yet. Create one to get started.
</Text>
) : (
<Table.Root>
<Table.Header>
<Table.Row>
<Table.ColumnHeader>Name</Table.ColumnHeader>
<Table.ColumnHeader>Created</Table.ColumnHeader>
<Table.ColumnHeader>Actions</Table.ColumnHeader>
</Table.Row>
</Table.Header>
<Table.Body>
{apiKeys.map((key) => (
<Table.Row key={key.id}>
<Table.Cell>
{key.name || <Text color="gray.500">Unnamed</Text>}
</Table.Cell>
<Table.Cell>{formatDate(key.created_at)}</Table.Cell>
<Table.Cell>
<IconButton
aria-label="Delete API key"
size="sm"
colorPalette="red"
variant="ghost"
onClick={() => handleDeleteRequest(key.id)}
loading={
deleteKeyMutation.isPending &&
deleteKeyMutation.variables?.params?.path?.key_id ===
key.id
}
>
<LuTrash2 />
</IconButton>
</Table.Cell>
</Table.Row>
))}
</Table.Body>
</Table.Root>
)}
</Box>
{/* Delete confirmation dialog */}
<Dialog.Root
open={!!keyToDelete}
onOpenChange={(e) => {
if (!e.open) setKeyToDelete(null);
}}
initialFocusEl={() => cancelRef.current}
>
<Dialog.Backdrop />
<Dialog.Positioner>
<Dialog.Content>
<Dialog.Header fontSize="lg" fontWeight="bold">
Delete API Key
</Dialog.Header>
<Dialog.Body>
<Text>
Are you sure you want to delete this API key? This action cannot
be undone.
</Text>
</Dialog.Body>
<Dialog.Footer>
<Button
ref={cancelRef}
onClick={() => setKeyToDelete(null)}
variant="outline"
colorPalette="gray"
>
Cancel
</Button>
<Button colorPalette="red" onClick={confirmDelete} ml={3}>
Delete
</Button>
</Dialog.Footer>
</Dialog.Content>
</Dialog.Positioner>
</Dialog.Root>
</Box>
);
}

View File

@@ -1,4 +1,4 @@
import { useEffect, useState } from "react"; import { useEffect, useRef, useState } from "react";
import React from "react"; import React from "react";
import Markdown from "react-markdown"; import Markdown from "react-markdown";
import "../../../styles/markdown.css"; import "../../../styles/markdown.css";
@@ -16,15 +16,17 @@ import {
} from "@chakra-ui/react"; } from "@chakra-ui/react";
import { LuPen } from "react-icons/lu"; import { LuPen } from "react-icons/lu";
import { useError } from "../../../(errors)/errorContext"; import { useError } from "../../../(errors)/errorContext";
import ShareAndPrivacy from "../shareAndPrivacy";
type FinalSummaryProps = { type FinalSummaryProps = {
transcript: GetTranscript; transcriptResponse: GetTranscript;
topics: GetTranscriptTopic[]; topicsResponse: GetTranscriptTopic[];
onUpdate: (newSummary: string) => void; onUpdate?: (newSummary) => void;
finalSummaryRef: React.Dispatch<React.SetStateAction<HTMLDivElement | null>>;
}; };
export default function FinalSummary(props: FinalSummaryProps) { export default function FinalSummary(props: FinalSummaryProps) {
const finalSummaryRef = useRef<HTMLParagraphElement>(null);
const [isEditMode, setIsEditMode] = useState(false); const [isEditMode, setIsEditMode] = useState(false);
const [preEditSummary, setPreEditSummary] = useState(""); const [preEditSummary, setPreEditSummary] = useState("");
const [editedSummary, setEditedSummary] = useState(""); const [editedSummary, setEditedSummary] = useState("");
@@ -33,10 +35,10 @@ export default function FinalSummary(props: FinalSummaryProps) {
const updateTranscriptMutation = useTranscriptUpdate(); const updateTranscriptMutation = useTranscriptUpdate();
useEffect(() => { useEffect(() => {
setEditedSummary(props.transcript?.long_summary || ""); setEditedSummary(props.transcriptResponse?.long_summary || "");
}, [props.transcript?.long_summary]); }, [props.transcriptResponse?.long_summary]);
if (!props.topics || !props.transcript) { if (!props.topicsResponse || !props.transcriptResponse) {
return null; return null;
} }
@@ -52,7 +54,9 @@ export default function FinalSummary(props: FinalSummaryProps) {
long_summary: newSummary, long_summary: newSummary,
}, },
}); });
if (props.onUpdate) {
props.onUpdate(newSummary); props.onUpdate(newSummary);
}
console.log("Updated long summary:", updatedTranscript); console.log("Updated long summary:", updatedTranscript);
} catch (err) { } catch (err) {
console.error("Failed to update long summary:", err); console.error("Failed to update long summary:", err);
@@ -71,7 +75,7 @@ export default function FinalSummary(props: FinalSummaryProps) {
}; };
const onSaveClick = () => { const onSaveClick = () => {
updateSummary(editedSummary, props.transcript.id); updateSummary(editedSummary, props.transcriptResponse.id);
setIsEditMode(false); setIsEditMode(false);
}; };
@@ -129,6 +133,11 @@ export default function FinalSummary(props: FinalSummaryProps) {
> >
<LuPen /> <LuPen />
</IconButton> </IconButton>
<ShareAndPrivacy
finalSummaryRef={finalSummaryRef}
transcriptResponse={props.transcriptResponse}
topicsResponse={props.topicsResponse}
/>
</> </>
)} )}
</Flex> </Flex>
@@ -144,7 +153,7 @@ export default function FinalSummary(props: FinalSummaryProps) {
mt={2} mt={2}
/> />
) : ( ) : (
<div ref={props.finalSummaryRef} className="markdown"> <div ref={finalSummaryRef} className="markdown">
<Markdown>{editedSummary}</Markdown> <Markdown>{editedSummary}</Markdown>
</div> </div>
)} )}

View File

@@ -10,15 +10,7 @@ import FinalSummary from "./finalSummary";
import TranscriptTitle from "../transcriptTitle"; import TranscriptTitle from "../transcriptTitle";
import Player from "../player"; import Player from "../player";
import { useRouter } from "next/navigation"; import { useRouter } from "next/navigation";
import { import { Box, Flex, Grid, GridItem, Skeleton, Text } from "@chakra-ui/react";
Box,
Flex,
Grid,
GridItem,
Skeleton,
Text,
Spinner,
} from "@chakra-ui/react";
import { useTranscriptGet } from "../../../lib/apiHooks"; import { useTranscriptGet } from "../../../lib/apiHooks";
import { TranscriptStatus } from "../../../lib/transcript"; import { TranscriptStatus } from "../../../lib/transcript";
@@ -36,7 +28,6 @@ export default function TranscriptDetails(details: TranscriptDetails) {
"idle", "idle",
"recording", "recording",
"processing", "processing",
"uploaded",
] satisfies TranscriptStatus[] as TranscriptStatus[]; ] satisfies TranscriptStatus[] as TranscriptStatus[];
const transcript = useTranscriptGet(transcriptId); const transcript = useTranscriptGet(transcriptId);
@@ -50,59 +41,17 @@ export default function TranscriptDetails(details: TranscriptDetails) {
waiting || mp3.audioDeleted === true, waiting || mp3.audioDeleted === true,
); );
const useActiveTopic = useState<Topic | null>(null); const useActiveTopic = useState<Topic | null>(null);
const [finalSummaryElement, setFinalSummaryElement] =
useState<HTMLDivElement | null>(null);
useEffect(() => { useEffect(() => {
if (!waiting || !transcript.data) return; if (waiting) {
const newUrl = "/transcripts/" + params.transcriptId + "/record";
const status = transcript.data.status;
let newUrl: string | null = null;
if (status === "processing" || status === "uploaded") {
newUrl = `/transcripts/${params.transcriptId}/processing`;
} else if (status === "recording") {
newUrl = `/transcripts/${params.transcriptId}/record`;
} else if (status === "idle") {
newUrl =
transcript.data.source_kind === "file"
? `/transcripts/${params.transcriptId}/upload`
: `/transcripts/${params.transcriptId}/record`;
}
if (newUrl) {
// Shallow redirection does not work on NextJS 13 // Shallow redirection does not work on NextJS 13
// https://github.com/vercel/next.js/discussions/48110 // https://github.com/vercel/next.js/discussions/48110
// https://github.com/vercel/next.js/discussions/49540 // https://github.com/vercel/next.js/discussions/49540
router.replace(newUrl); router.replace(newUrl);
// history.replaceState({}, "", newUrl);
} }
}, [waiting, transcript.data?.status, transcript.data?.source_kind]); }, [waiting]);
if (waiting) {
return (
<Box>
<Box
w="full"
background="gray.bg"
border={"2px solid"}
borderColor={"gray.bg"}
borderRadius={8}
p={6}
minH="100%"
display="flex"
alignItems="center"
justifyContent="center"
>
<Flex direction="column" align="center" gap={3}>
<Spinner size="xl" color="blue.500" />
<Text color="gray.600" textAlign="center">
Loading transcript...
</Text>
</Flex>
</Box>
</Box>
);
}
if (transcript.error || topics?.error) { if (transcript.error || topics?.error) {
return ( return (
@@ -175,12 +124,9 @@ export default function TranscriptDetails(details: TranscriptDetails) {
<TranscriptTitle <TranscriptTitle
title={transcript.data?.title || "Unnamed Transcript"} title={transcript.data?.title || "Unnamed Transcript"}
transcriptId={transcriptId} transcriptId={transcriptId}
onUpdate={() => { onUpdate={(newTitle) => {
transcript.refetch().then(() => {}); transcript.refetch().then(() => {});
}} }}
transcript={transcript.data || null}
topics={topics.topics}
finalSummaryElement={finalSummaryElement}
/> />
</Flex> </Flex>
{mp3.audioDeleted && ( {mp3.audioDeleted && (
@@ -202,12 +148,11 @@ export default function TranscriptDetails(details: TranscriptDetails) {
{transcript.data && topics.topics ? ( {transcript.data && topics.topics ? (
<> <>
<FinalSummary <FinalSummary
transcript={transcript.data} transcriptResponse={transcript.data}
topics={topics.topics} topicsResponse={topics.topics}
onUpdate={() => { onUpdate={() => {
transcript.refetch().then(() => {}); transcript.refetch();
}} }}
finalSummaryRef={setFinalSummaryElement}
/> />
</> </>
) : ( ) : (

View File

@@ -1,97 +0,0 @@
"use client";
import { useEffect, use } from "react";
import {
Heading,
Text,
VStack,
Spinner,
Button,
Center,
} from "@chakra-ui/react";
import { useRouter } from "next/navigation";
import { useTranscriptGet } from "../../../../lib/apiHooks";
type TranscriptProcessing = {
params: Promise<{
transcriptId: string;
}>;
};
export default function TranscriptProcessing(details: TranscriptProcessing) {
const params = use(details.params);
const transcriptId = params.transcriptId;
const router = useRouter();
const transcript = useTranscriptGet(transcriptId);
useEffect(() => {
const status = transcript.data?.status;
if (!status) return;
if (status === "ended" || status === "error") {
router.replace(`/transcripts/${transcriptId}`);
} else if (status === "recording") {
router.replace(`/transcripts/${transcriptId}/record`);
} else if (status === "idle") {
const dest =
transcript.data?.source_kind === "file"
? `/transcripts/${transcriptId}/upload`
: `/transcripts/${transcriptId}/record`;
router.replace(dest);
}
}, [
transcript.data?.status,
transcript.data?.source_kind,
router,
transcriptId,
]);
if (transcript.isLoading) {
return (
<VStack align="center" py={8}>
<Heading size="lg">Loading transcript...</Heading>
</VStack>
);
}
if (transcript.error) {
return (
<VStack align="center" py={8}>
<Heading size="lg">Transcript not found</Heading>
<Text>We couldn't load this transcript.</Text>
</VStack>
);
}
return (
<>
<VStack
align={"left"}
minH="100vh"
pt={4}
mx="auto"
w={{ base: "full", md: "container.xl" }}
>
<Center h={"full"} w="full">
<VStack gap={10} bg="gray.100" p={10} borderRadius="md" maxW="500px">
<Spinner size="xl" color="blue.500" />
<Heading size={"md"} textAlign="center">
Processing recording
</Heading>
<Text color="gray.600" textAlign="center">
You can safely return to the library while your recording is being
processed.
</Text>
<Button
onClick={() => {
router.push("/browse");
}}
>
Browse
</Button>
</VStack>
</Center>
</VStack>
</>
);
}

View File

@@ -4,7 +4,7 @@ import { useWebSockets } from "../../useWebSockets";
import { lockWakeState, releaseWakeState } from "../../../../lib/wakeLock"; import { lockWakeState, releaseWakeState } from "../../../../lib/wakeLock";
import { useRouter } from "next/navigation"; import { useRouter } from "next/navigation";
import useMp3 from "../../useMp3"; import useMp3 from "../../useMp3";
import { Center, VStack, Text, Heading } from "@chakra-ui/react"; import { Center, VStack, Text, Heading, Button } from "@chakra-ui/react";
import FileUploadButton from "../../fileUploadButton"; import FileUploadButton from "../../fileUploadButton";
import { useTranscriptGet } from "../../../../lib/apiHooks"; import { useTranscriptGet } from "../../../../lib/apiHooks";
@@ -53,12 +53,6 @@ const TranscriptUpload = (details: TranscriptUpload) => {
const newUrl = "/transcripts/" + params.transcriptId; const newUrl = "/transcripts/" + params.transcriptId;
router.replace(newUrl); router.replace(newUrl);
} else if (
newStatus &&
(newStatus == "uploaded" || newStatus == "processing")
) {
// After upload finishes (or if already processing), redirect to the unified processing page
router.replace(`/transcripts/${params.transcriptId}/processing`);
} }
}, [webSockets.status?.value, transcript.data?.status]); }, [webSockets.status?.value, transcript.data?.status]);
@@ -77,7 +71,7 @@ const TranscriptUpload = (details: TranscriptUpload) => {
<> <>
<VStack <VStack
align={"left"} align={"left"}
minH="100vh" h="full"
pt={4} pt={4}
mx="auto" mx="auto"
w={{ base: "full", md: "container.xl" }} w={{ base: "full", md: "container.xl" }}
@@ -85,16 +79,34 @@ const TranscriptUpload = (details: TranscriptUpload) => {
<Heading size={"lg"}>Upload meeting</Heading> <Heading size={"lg"}>Upload meeting</Heading>
<Center h={"full"} w="full"> <Center h={"full"} w="full">
<VStack gap={10} bg="gray.100" p={10} borderRadius="md" maxW="500px"> <VStack gap={10} bg="gray.100" p={10} borderRadius="md" maxW="500px">
{status && status == "idle" && (
<>
<Text> <Text>
Please select the file, supported formats: .mp3, m4a, .wav, .mp4, Please select the file, supported formats: .mp3, m4a, .wav,
.mov or .webm .mp4, .mov or .webm
</Text> </Text>
<FileUploadButton <FileUploadButton transcriptId={params.transcriptId} />
transcriptId={params.transcriptId} </>
onUploadComplete={() => )}
router.replace(`/transcripts/${params.transcriptId}/processing`) {status && status == "uploaded" && (
} <Text>File is uploaded, processing...</Text>
/> )}
{(status == "recording" || status == "processing") && (
<>
<Heading size={"lg"}>Processing your recording...</Heading>
<Text>
You can safely return to the library while your file is being
processed.
</Text>
<Button
onClick={() => {
router.push("/browse");
}}
>
Browse
</Button>
</>
)}
</VStack> </VStack>
</Center> </Center>
</VStack> </VStack>

View File

@@ -1,60 +0,0 @@
import type { components } from "../../reflector-api";
import { formatTime } from "../../lib/time";
type GetTranscriptTopic = components["schemas"]["GetTranscriptTopic"];
type Participant = components["schemas"]["Participant"];
function getSpeakerName(
speakerNumber: number,
participants?: Participant[] | null,
): string {
const name = participants?.find((p) => p.speaker === speakerNumber)?.name;
return name && name.trim().length > 0 ? name : `Speaker ${speakerNumber}`;
}
export function buildTranscriptWithTopics(
topics: GetTranscriptTopic[],
participants?: Participant[] | null,
transcriptTitle?: string | null,
): string {
const blocks: string[] = [];
if (transcriptTitle && transcriptTitle.trim()) {
blocks.push(`# ${transcriptTitle.trim()}`);
blocks.push("");
}
for (const topic of topics) {
// Topic header
const topicTime = formatTime(Math.floor(topic.timestamp || 0));
const title = topic.title?.trim() || "Untitled Topic";
blocks.push(`## ${title} [${topicTime}]`);
if (topic.segments && topic.segments.length > 0) {
for (const seg of topic.segments) {
const ts = formatTime(Math.floor(seg.start || 0));
const speaker = getSpeakerName(seg.speaker as number, participants);
const text = (seg.text || "").replace(/\s+/g, " ").trim();
if (text) {
blocks.push(`[${ts}] ${speaker}: ${text}`);
}
}
} else if (topic.transcript) {
// Fallback: plain transcript when segments are not present
const text = topic.transcript.replace(/\s+/g, " ").trim();
if (text) {
blocks.push(text);
}
}
// Blank line between topics
blocks.push("");
}
// Trim trailing blank line
while (blocks.length > 0 && blocks[blocks.length - 1] === "") {
blocks.pop();
}
return blocks.join("\n");
}

View File

@@ -5,7 +5,6 @@ import { useError } from "../../(errors)/errorContext";
type FileUploadButton = { type FileUploadButton = {
transcriptId: string; transcriptId: string;
onUploadComplete?: () => void;
}; };
export default function FileUploadButton(props: FileUploadButton) { export default function FileUploadButton(props: FileUploadButton) {
@@ -32,7 +31,6 @@ export default function FileUploadButton(props: FileUploadButton) {
const uploadNextChunk = async () => { const uploadNextChunk = async () => {
if (chunkNumber == totalChunks) { if (chunkNumber == totalChunks) {
setProgress(0); setProgress(0);
props.onUploadComplete?.();
return; return;
} }

View File

@@ -26,9 +26,9 @@ import { useAuth } from "../../lib/AuthProvider";
import { featureEnabled } from "../../lib/features"; import { featureEnabled } from "../../lib/features";
type ShareAndPrivacyProps = { type ShareAndPrivacyProps = {
finalSummaryElement: HTMLDivElement | null; finalSummaryRef: any;
transcript: GetTranscript; transcriptResponse: GetTranscript;
topics: GetTranscriptTopic[]; topicsResponse: GetTranscriptTopic[];
}; };
type ShareOption = { value: ShareMode; label: string }; type ShareOption = { value: ShareMode; label: string };
@@ -48,7 +48,7 @@ export default function ShareAndPrivacy(props: ShareAndPrivacyProps) {
const [isOwner, setIsOwner] = useState(false); const [isOwner, setIsOwner] = useState(false);
const [shareMode, setShareMode] = useState<ShareOption>( const [shareMode, setShareMode] = useState<ShareOption>(
shareOptionsData.find( shareOptionsData.find(
(option) => option.value === props.transcript.share_mode, (option) => option.value === props.transcriptResponse.share_mode,
) || shareOptionsData[0], ) || shareOptionsData[0],
); );
const [shareLoading, setShareLoading] = useState(false); const [shareLoading, setShareLoading] = useState(false);
@@ -70,7 +70,7 @@ export default function ShareAndPrivacy(props: ShareAndPrivacyProps) {
try { try {
const updatedTranscript = await updateTranscriptMutation.mutateAsync({ const updatedTranscript = await updateTranscriptMutation.mutateAsync({
params: { params: {
path: { transcript_id: props.transcript.id }, path: { transcript_id: props.transcriptResponse.id },
}, },
body: requestBody, body: requestBody,
}); });
@@ -90,8 +90,8 @@ export default function ShareAndPrivacy(props: ShareAndPrivacyProps) {
const userId = auth.status === "authenticated" ? auth.user?.id : null; const userId = auth.status === "authenticated" ? auth.user?.id : null;
useEffect(() => { useEffect(() => {
setIsOwner(!!(requireLogin && userId === props.transcript.user_id)); setIsOwner(!!(requireLogin && userId === props.transcriptResponse.user_id));
}, [userId, props.transcript.user_id]); }, [userId, props.transcriptResponse.user_id]);
return ( return (
<> <>
@@ -171,19 +171,19 @@ export default function ShareAndPrivacy(props: ShareAndPrivacyProps) {
<Flex gap={2} mb={2}> <Flex gap={2} mb={2}>
{requireLogin && ( {requireLogin && (
<ShareZulip <ShareZulip
transcript={props.transcript} transcriptResponse={props.transcriptResponse}
topics={props.topics} topicsResponse={props.topicsResponse}
disabled={toShareMode(shareMode.value) === "private"} disabled={toShareMode(shareMode.value) === "private"}
/> />
)} )}
<ShareCopy <ShareCopy
finalSummaryElement={props.finalSummaryElement} finalSummaryRef={props.finalSummaryRef}
transcript={props.transcript} transcriptResponse={props.transcriptResponse}
topics={props.topics} topicsResponse={props.topicsResponse}
/> />
</Flex> </Flex>
<ShareLink transcriptId={props.transcript.id} /> <ShareLink transcriptId={props.transcriptResponse.id} />
</Dialog.Body> </Dialog.Body>
</Dialog.Content> </Dialog.Content>
</Dialog.Positioner> </Dialog.Positioner>

View File

@@ -3,44 +3,40 @@ import type { components } from "../../reflector-api";
type GetTranscript = components["schemas"]["GetTranscript"]; type GetTranscript = components["schemas"]["GetTranscript"];
type GetTranscriptTopic = components["schemas"]["GetTranscriptTopic"]; type GetTranscriptTopic = components["schemas"]["GetTranscriptTopic"];
import { Button, BoxProps, Box } from "@chakra-ui/react"; import { Button, BoxProps, Box } from "@chakra-ui/react";
import { buildTranscriptWithTopics } from "./buildTranscriptWithTopics";
import { useTranscriptParticipants } from "../../lib/apiHooks";
type ShareCopyProps = { type ShareCopyProps = {
finalSummaryElement: HTMLDivElement | null; finalSummaryRef: any;
transcript: GetTranscript; transcriptResponse: GetTranscript;
topics: GetTranscriptTopic[]; topicsResponse: GetTranscriptTopic[];
}; };
export default function ShareCopy({ export default function ShareCopy({
finalSummaryElement, finalSummaryRef,
transcript, transcriptResponse,
topics, topicsResponse,
...boxProps ...boxProps
}: ShareCopyProps & BoxProps) { }: ShareCopyProps & BoxProps) {
const [isCopiedSummary, setIsCopiedSummary] = useState(false); const [isCopiedSummary, setIsCopiedSummary] = useState(false);
const [isCopiedTranscript, setIsCopiedTranscript] = useState(false); const [isCopiedTranscript, setIsCopiedTranscript] = useState(false);
const participantsQuery = useTranscriptParticipants(transcript?.id || null);
const onCopySummaryClick = () => { const onCopySummaryClick = () => {
const text_to_copy = finalSummaryElement?.innerText; let text_to_copy = finalSummaryRef.current?.innerText;
if (text_to_copy) { text_to_copy &&
navigator.clipboard.writeText(text_to_copy).then(() => { navigator.clipboard.writeText(text_to_copy).then(() => {
setIsCopiedSummary(true); setIsCopiedSummary(true);
// Reset the copied state after 2 seconds // Reset the copied state after 2 seconds
setTimeout(() => setIsCopiedSummary(false), 2000); setTimeout(() => setIsCopiedSummary(false), 2000);
}); });
}
}; };
const onCopyTranscriptClick = () => { const onCopyTranscriptClick = () => {
const text_to_copy = let text_to_copy =
buildTranscriptWithTopics( topicsResponse
topics || [], ?.map((topic) => topic.transcript)
participantsQuery?.data || null, .join("\n\n")
transcript?.title || null, .replace(/ +/g, " ")
) || ""; .trim() || "";
text_to_copy && text_to_copy &&
navigator.clipboard.writeText(text_to_copy).then(() => { navigator.clipboard.writeText(text_to_copy).then(() => {

View File

@@ -26,8 +26,8 @@ import {
import { featureEnabled } from "../../lib/features"; import { featureEnabled } from "../../lib/features";
type ShareZulipProps = { type ShareZulipProps = {
transcript: GetTranscript; transcriptResponse: GetTranscript;
topics: GetTranscriptTopic[]; topicsResponse: GetTranscriptTopic[];
disabled: boolean; disabled: boolean;
}; };
@@ -88,14 +88,14 @@ export default function ShareZulip(props: ShareZulipProps & BoxProps) {
}, [stream, streams]); }, [stream, streams]);
const handleSendToZulip = async () => { const handleSendToZulip = async () => {
if (!props.transcript) return; if (!props.transcriptResponse) return;
if (stream && topic) { if (stream && topic) {
try { try {
await postToZulipMutation.mutateAsync({ await postToZulipMutation.mutateAsync({
params: { params: {
path: { path: {
transcript_id: props.transcript.id, transcript_id: props.transcriptResponse.id,
}, },
query: { query: {
stream, stream,

View File

@@ -2,27 +2,14 @@ import { useState } from "react";
import type { components } from "../../reflector-api"; import type { components } from "../../reflector-api";
type UpdateTranscript = components["schemas"]["UpdateTranscript"]; type UpdateTranscript = components["schemas"]["UpdateTranscript"];
type GetTranscript = components["schemas"]["GetTranscript"]; import { useTranscriptUpdate } from "../../lib/apiHooks";
type GetTranscriptTopic = components["schemas"]["GetTranscriptTopic"];
import {
useTranscriptUpdate,
useTranscriptParticipants,
} from "../../lib/apiHooks";
import { Heading, IconButton, Input, Flex, Spacer } from "@chakra-ui/react"; import { Heading, IconButton, Input, Flex, Spacer } from "@chakra-ui/react";
import { LuPen, LuCopy, LuCheck } from "react-icons/lu"; import { LuPen } from "react-icons/lu";
import ShareAndPrivacy from "./shareAndPrivacy";
import { buildTranscriptWithTopics } from "./buildTranscriptWithTopics";
import { toaster } from "../../components/ui/toaster";
type TranscriptTitle = { type TranscriptTitle = {
title: string; title: string;
transcriptId: string; transcriptId: string;
onUpdate: (newTitle: string) => void; onUpdate?: (newTitle: string) => void;
// share props
transcript: GetTranscript | null;
topics: GetTranscriptTopic[] | null;
finalSummaryElement: HTMLDivElement | null;
}; };
const TranscriptTitle = (props: TranscriptTitle) => { const TranscriptTitle = (props: TranscriptTitle) => {
@@ -30,9 +17,6 @@ const TranscriptTitle = (props: TranscriptTitle) => {
const [preEditTitle, setPreEditTitle] = useState(props.title); const [preEditTitle, setPreEditTitle] = useState(props.title);
const [isEditing, setIsEditing] = useState(false); const [isEditing, setIsEditing] = useState(false);
const updateTranscriptMutation = useTranscriptUpdate(); const updateTranscriptMutation = useTranscriptUpdate();
const participantsQuery = useTranscriptParticipants(
props.transcript?.id || null,
);
const updateTitle = async (newTitle: string, transcriptId: string) => { const updateTitle = async (newTitle: string, transcriptId: string) => {
try { try {
@@ -45,7 +29,9 @@ const TranscriptTitle = (props: TranscriptTitle) => {
}, },
body: requestBody, body: requestBody,
}); });
if (props.onUpdate) {
props.onUpdate(newTitle); props.onUpdate(newTitle);
}
console.log("Updated transcript title:", newTitle); console.log("Updated transcript title:", newTitle);
} catch (err) { } catch (err) {
console.error("Failed to update transcript:", err); console.error("Failed to update transcript:", err);
@@ -76,11 +62,11 @@ const TranscriptTitle = (props: TranscriptTitle) => {
} }
setIsEditing(false); setIsEditing(false);
}; };
const handleChange = (e: React.ChangeEvent<HTMLInputElement>) => { const handleChange = (e) => {
setDisplayedTitle(e.target.value); setDisplayedTitle(e.target.value);
}; };
const handleKeyDown = (e: React.KeyboardEvent<HTMLInputElement>) => { const handleKeyDown = (e) => {
if (e.key === "Enter") { if (e.key === "Enter") {
updateTitle(displayedTitle, props.transcriptId); updateTitle(displayedTitle, props.transcriptId);
setIsEditing(false); setIsEditing(false);
@@ -125,59 +111,6 @@ const TranscriptTitle = (props: TranscriptTitle) => {
> >
<LuPen /> <LuPen />
</IconButton> </IconButton>
{props.transcript && props.topics && (
<>
<IconButton
aria-label="Copy Transcript"
size="sm"
variant="subtle"
onClick={() => {
const text = buildTranscriptWithTopics(
props.topics || [],
participantsQuery?.data || null,
props.transcript?.title || null,
);
if (!text) return;
navigator.clipboard
.writeText(text)
.then(() => {
toaster
.create({
placement: "top",
duration: 2500,
render: () => (
<div className="chakra-ui-light">
<div
style={{
background: "#38A169",
color: "white",
padding: "8px 12px",
borderRadius: 6,
display: "flex",
alignItems: "center",
gap: 8,
boxShadow: "rgba(0,0,0,0.25) 0px 4px 12px",
}}
>
<LuCheck /> Transcript copied
</div>
</div>
),
})
.then(() => {});
})
.catch(() => {});
}}
>
<LuCopy />
</IconButton>
<ShareAndPrivacy
finalSummaryElement={props.finalSummaryElement}
transcript={props.transcript}
topics={props.topics}
/>
</>
)}
</Flex> </Flex>
)} )}
</> </>

Some files were not shown because too many files have changed in this diff Show More