mirror of
https://github.com/Monadical-SAS/reflector.git
synced 2025-12-20 20:29:06 +00:00
Compare commits
12 Commits
igor/daili
...
v0.17.0
| Author | SHA1 | Date | |
|---|---|---|---|
| 34a3f5618c | |||
|
1473fd82dc | ||
| 372202b0e1 | |||
|
|
d20aac66c4 | ||
| dc4b737daa | |||
|
|
0baff7abf7 | ||
|
|
962c40e2b6 | ||
|
|
3c4b9f2103 | ||
|
|
c6c035aacf | ||
| c086b91445 | |||
|
|
9a258abc02 | ||
| af86c47f1d |
2
.github/workflows/deploy.yml
vendored
2
.github/workflows/deploy.yml
vendored
@@ -1,4 +1,4 @@
|
||||
name: Deploy to Amazon ECS
|
||||
name: Build container/push to container registry
|
||||
|
||||
on: [workflow_dispatch]
|
||||
|
||||
|
||||
22
CHANGELOG.md
22
CHANGELOG.md
@@ -1,5 +1,27 @@
|
||||
# Changelog
|
||||
|
||||
## [0.17.0](https://github.com/Monadical-SAS/reflector/compare/v0.16.0...v0.17.0) (2025-11-13)
|
||||
|
||||
|
||||
### Features
|
||||
|
||||
* add API key management UI ([#716](https://github.com/Monadical-SAS/reflector/issues/716)) ([372202b](https://github.com/Monadical-SAS/reflector/commit/372202b0e1a86823900b0aa77be1bfbc2893d8a1))
|
||||
* daily.co support as alternative to whereby ([#691](https://github.com/Monadical-SAS/reflector/issues/691)) ([1473fd8](https://github.com/Monadical-SAS/reflector/commit/1473fd82dc472c394cbaa2987212ad662a74bcac))
|
||||
|
||||
## [0.16.0](https://github.com/Monadical-SAS/reflector/compare/v0.15.0...v0.16.0) (2025-10-24)
|
||||
|
||||
|
||||
### Features
|
||||
|
||||
* search date filter ([#710](https://github.com/Monadical-SAS/reflector/issues/710)) ([962c40e](https://github.com/Monadical-SAS/reflector/commit/962c40e2b6428ac42fd10aea926782d7a6f3f902))
|
||||
|
||||
## [0.15.0](https://github.com/Monadical-SAS/reflector/compare/v0.14.0...v0.15.0) (2025-10-20)
|
||||
|
||||
|
||||
### Features
|
||||
|
||||
* api tokens ([#705](https://github.com/Monadical-SAS/reflector/issues/705)) ([9a258ab](https://github.com/Monadical-SAS/reflector/commit/9a258abc0209b0ac3799532a507ea6a9125d703a))
|
||||
|
||||
## [0.14.0](https://github.com/Monadical-SAS/reflector/compare/v0.13.1...v0.14.0) (2025-10-08)
|
||||
|
||||
|
||||
|
||||
@@ -1,345 +0,0 @@
|
||||
# Multi-Provider Video Platform Implementation - Coder Briefing
|
||||
|
||||
## Your Mission
|
||||
|
||||
Implement multi-provider video platform support in Reflector, allowing the system to work with both Whereby and Daily.co video conferencing providers. The goal is to abstract the current Whereby-only implementation and add Daily.co as a second provider, with the ability to switch between them via environment variables.
|
||||
|
||||
**Branch:** `igor/dailico-2` (you're already on it)
|
||||
|
||||
**Estimated Time:** 12-16 hours (senior engineer)
|
||||
|
||||
**Complexity:** Medium-High (requires careful integration with existing codebase)
|
||||
|
||||
---
|
||||
|
||||
## What You Have
|
||||
|
||||
### 1. **PLAN.md** - Your Technical Specification (2,452 lines)
|
||||
- Complete step-by-step implementation guide
|
||||
- All code examples you need
|
||||
- Architecture diagrams and design rationale
|
||||
- Testing strategy and success metrics
|
||||
- **Read this first** to understand the overall approach
|
||||
|
||||
### 2. **IMPLEMENTATION_GUIDE.md** - Your Practical Guide
|
||||
- What to copy vs. adapt vs. rewrite
|
||||
- Common pitfalls and how to avoid them
|
||||
- Verification checklists for each phase
|
||||
- Decision trees for implementation choices
|
||||
- **Use this as your day-to-day reference**
|
||||
|
||||
### 3. **Reference Implementation** - `./reflector-dailyco-reference/`
|
||||
- Working implementation from 2.5 months ago
|
||||
- Good architecture and patterns
|
||||
- **BUT:** 91 commits behind current main, DO NOT merge directly
|
||||
- Use for inspiration and code patterns only
|
||||
|
||||
---
|
||||
|
||||
## Critical Context: Why Not Just Merge?
|
||||
|
||||
The reference branch (`origin/igor/feat-dailyco`) was started on August 1, 2025 and is now severely diverged from main:
|
||||
|
||||
- **91 commits behind main**
|
||||
- Main has 12x more changes (45,840 insertions vs 3,689)
|
||||
- Main added: calendar integration, webhooks, full-text search, React Query migration, security fixes
|
||||
- Reference removed: features that main still has and needs
|
||||
|
||||
**Merging would be a disaster.** We're implementing fresh on current main, using the reference for validated patterns.
|
||||
|
||||
---
|
||||
|
||||
## High-Level Approach
|
||||
|
||||
### Phase 1: Analysis (2 hours)
|
||||
- Study current Whereby integration
|
||||
- Define abstraction requirements
|
||||
- Create standard data models
|
||||
|
||||
### Phase 2: Abstraction Layer (4-5 hours)
|
||||
- Build platform abstraction (base class, registry, factory)
|
||||
- Extract Whereby into the abstraction
|
||||
- Update database schema (add `platform` field)
|
||||
- Integrate into rooms.py **without breaking calendar/webhooks**
|
||||
|
||||
### Phase 3: Daily.co Implementation (4-5 hours)
|
||||
- Implement Daily.co client
|
||||
- Add webhook handler
|
||||
- Create frontend components (rewrite API calls for React Query)
|
||||
- Add recording processing
|
||||
|
||||
### Phase 4: Testing (2-3 hours)
|
||||
- Unit tests for platform abstraction
|
||||
- Integration tests for webhooks
|
||||
- Manual testing with both providers
|
||||
|
||||
---
|
||||
|
||||
## Key Files You'll Touch
|
||||
|
||||
### Backend (New)
|
||||
```
|
||||
server/reflector/video_platforms/
|
||||
├── __init__.py
|
||||
├── base.py ← Abstract base class
|
||||
├── models.py ← Platform, MeetingData, VideoPlatformConfig
|
||||
├── registry.py ← Platform registration system
|
||||
├── factory.py ← Client creation and config
|
||||
├── whereby.py ← Whereby client wrapper
|
||||
├── daily.py ← Daily.co client
|
||||
└── mock.py ← Mock client for testing
|
||||
|
||||
server/reflector/views/daily.py ← Daily.co webhooks
|
||||
server/tests/test_video_platforms.py ← Platform tests
|
||||
server/tests/test_daily_webhook.py ← Webhook tests
|
||||
```
|
||||
|
||||
### Backend (Modified - Careful!)
|
||||
```
|
||||
server/reflector/settings.py ← Add Daily.co settings
|
||||
server/reflector/db/rooms.py ← Add platform field, PRESERVE calendar fields
|
||||
server/reflector/db/meetings.py ← Add platform field
|
||||
server/reflector/views/rooms.py ← Integrate abstraction, PRESERVE calendar/webhooks
|
||||
server/reflector/worker/process.py ← Add process_recording_from_url task
|
||||
server/reflector/app.py ← Register daily router
|
||||
server/env.example ← Document new env vars
|
||||
```
|
||||
|
||||
### Frontend (New)
|
||||
```
|
||||
www/app/[roomName]/components/
|
||||
├── RoomContainer.tsx ← Platform router
|
||||
├── DailyRoom.tsx ← Daily.co component (rewrite API calls!)
|
||||
└── WherebyRoom.tsx ← Extract existing logic
|
||||
```
|
||||
|
||||
### Frontend (Modified)
|
||||
```
|
||||
www/app/[roomName]/page.tsx ← Use RoomContainer
|
||||
www/package.json ← Add @daily-co/daily-js
|
||||
```
|
||||
|
||||
### Database
|
||||
```
|
||||
server/migrations/versions/XXXXXX_add_platform_support.py ← Generate fresh migration
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Critical Warnings ⚠️
|
||||
|
||||
### 1. **DO NOT Copy Database Migrations**
|
||||
The reference migration has the wrong `down_revision` and is based on old schema.
|
||||
```bash
|
||||
# Instead:
|
||||
cd server
|
||||
uv run alembic revision -m "add_platform_support"
|
||||
# Then edit the generated file
|
||||
```
|
||||
|
||||
### 2. **DO NOT Remove Main's Features**
|
||||
Main has calendar integration, webhooks, ICS sync that reference doesn't have.
|
||||
When modifying `rooms.py`, only change meeting creation logic, preserve everything else.
|
||||
|
||||
### 3. **DO NOT Copy Frontend API Calls**
|
||||
Reference uses old OpenAPI client. Main uses React Query.
|
||||
Check how main currently makes API calls and replicate that pattern.
|
||||
|
||||
### 4. **DO NOT Copy package.json/migrations**
|
||||
These files are severely outdated in reference.
|
||||
|
||||
### 5. **Preserve Type Safety**
|
||||
Use `TYPE_CHECKING` imports to avoid circular dependencies:
|
||||
```python
|
||||
from typing import TYPE_CHECKING
|
||||
if TYPE_CHECKING:
|
||||
from reflector.db.rooms import Room
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## How to Start
|
||||
|
||||
### Day 1 Morning: Setup & Understanding (2-3 hours)
|
||||
```bash
|
||||
# 1. Verify you're on the right branch
|
||||
git branch
|
||||
# Should show: igor/dailico-2
|
||||
|
||||
# 2. Read the docs (in order)
|
||||
# - PLAN.md (skim to understand scope, read Phase 1 carefully)
|
||||
# - IMPLEMENTATION_GUIDE.md (read fully, bookmark it)
|
||||
|
||||
# 3. Study current Whereby integration
|
||||
cat server/reflector/views/rooms.py | grep -A 20 "whereby"
|
||||
cat www/app/[roomName]/page.tsx
|
||||
|
||||
# 4. Check reference implementation structure
|
||||
ls -la reflector-dailyco-reference/server/reflector/video_platforms/
|
||||
```
|
||||
|
||||
### Day 1 Afternoon: Phase 1 Execution (2-3 hours)
|
||||
```bash
|
||||
# 5. Copy video_platforms directory from reference
|
||||
cp -r reflector-dailyco-reference/server/reflector/video_platforms/ \
|
||||
server/reflector/
|
||||
|
||||
# 6. Review and fix imports
|
||||
cd server
|
||||
uv run ruff check reflector/video_platforms/
|
||||
|
||||
# 7. Add settings to settings.py (see PLAN.md Phase 2.7)
|
||||
|
||||
# 8. Test imports work
|
||||
uv run python -c "from reflector.video_platforms import create_platform_client; print('OK')"
|
||||
```
|
||||
|
||||
### Day 2: Phase 2 - Database & Integration (4-5 hours)
|
||||
```bash
|
||||
# 9. Generate migration
|
||||
uv run alembic revision -m "add_platform_support"
|
||||
# Edit the file following PLAN.md Phase 2.8
|
||||
|
||||
# 10. Update Room/Meeting models
|
||||
# Add platform field, PRESERVE all existing fields
|
||||
|
||||
# 11. Integrate into rooms.py
|
||||
# Carefully modify meeting creation, preserve calendar/webhooks
|
||||
|
||||
# 12. Add Daily.co webhook handler
|
||||
cp reflector-dailyco-reference/server/reflector/views/daily.py \
|
||||
server/reflector/views/
|
||||
# Register in app.py
|
||||
```
|
||||
|
||||
### Day 3: Phase 3 - Frontend & Testing (4-5 hours)
|
||||
```bash
|
||||
# 13. Create frontend components
|
||||
mkdir -p www/app/[roomName]/components
|
||||
|
||||
# 14. Add Daily.co dependency
|
||||
cd www
|
||||
pnpm add @daily-co/daily-js@^0.81.0
|
||||
|
||||
# 15. Create RoomContainer, DailyRoom, WherebyRoom
|
||||
# IMPORTANT: Rewrite API calls using React Query patterns
|
||||
|
||||
# 16. Regenerate types
|
||||
pnpm openapi
|
||||
|
||||
# 17. Copy and adapt tests
|
||||
cp reflector-dailyco-reference/server/tests/test_*.py server/tests/
|
||||
|
||||
# 18. Run tests
|
||||
cd server
|
||||
REDIS_HOST=localhost \
|
||||
CELERY_BROKER_URL=redis://localhost:6379/1 \
|
||||
uv run pytest tests/test_video_platforms.py -v
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Verification Checklist
|
||||
|
||||
After implementation, all of these must pass:
|
||||
|
||||
**Backend:**
|
||||
- [ ] `cd server && uv run ruff check .` passes
|
||||
- [ ] `uv run alembic upgrade head` works cleanly
|
||||
- [ ] `uv run pytest tests/test_video_platforms.py` passes
|
||||
- [ ] Can import: `from reflector.video_platforms import create_platform_client`
|
||||
- [ ] Settings has all Daily.co variables
|
||||
|
||||
**Frontend:**
|
||||
- [ ] `cd www && pnpm lint` passes
|
||||
- [ ] No TypeScript errors
|
||||
- [ ] `pnpm openapi` generates platform field
|
||||
- [ ] No `@ts-ignore` for platform field
|
||||
|
||||
**Integration:**
|
||||
- [ ] Whereby meetings still work (existing flow unchanged)
|
||||
- [ ] Calendar/webhook features still work in rooms.py
|
||||
- [ ] env.example documents all new variables
|
||||
|
||||
---
|
||||
|
||||
## When You're Stuck
|
||||
|
||||
### Check These Resources:
|
||||
1. **PLAN.md** - Detailed code examples for your exact scenario
|
||||
2. **IMPLEMENTATION_GUIDE.md** - Common pitfalls section
|
||||
3. **Reference code** - See how it was solved before
|
||||
4. **Git diff** - Compare reference to your implementation
|
||||
|
||||
### Compare Files:
|
||||
```bash
|
||||
# See what reference did
|
||||
diff reflector-dailyco-reference/server/reflector/views/rooms.py \
|
||||
server/reflector/views/rooms.py
|
||||
|
||||
# See what changed in main since reference branch
|
||||
git log --oneline --since="2025-08-01" -- server/reflector/views/rooms.py
|
||||
```
|
||||
|
||||
### Common Issues:
|
||||
- **Circular imports:** Use `TYPE_CHECKING` pattern
|
||||
- **Tests fail with postgres error:** Use `REDIS_HOST=localhost` env vars
|
||||
- **Frontend API calls broken:** Check current React Query patterns in main
|
||||
- **Migrations fail:** Ensure you generated fresh, not copied
|
||||
|
||||
---
|
||||
|
||||
## Success Looks Like
|
||||
|
||||
When you're done:
|
||||
- ✅ All tests pass
|
||||
- ✅ Linting passes
|
||||
- ✅ Can create Whereby meetings (unchanged behavior)
|
||||
- ✅ Can create Daily.co meetings (with env vars)
|
||||
- ✅ Calendar/webhooks still work
|
||||
- ✅ Frontend has no TypeScript errors
|
||||
- ✅ Platform selection via environment variables works
|
||||
|
||||
---
|
||||
|
||||
## Communication
|
||||
|
||||
If you need clarification on requirements, have questions about architecture decisions, or find issues with the spec, document them clearly with:
|
||||
- What you expected
|
||||
- What you found
|
||||
- Your proposed solution
|
||||
|
||||
The PLAN.md document is comprehensive but you may find edge cases. Use your engineering judgment and document decisions.
|
||||
|
||||
---
|
||||
|
||||
## Final Notes
|
||||
|
||||
**This is not a simple copy-paste job.** You're doing careful integration work where you need to:
|
||||
- Understand the abstraction pattern (PLAN.md)
|
||||
- Preserve all of main's features
|
||||
- Adapt reference code to current patterns
|
||||
- Think about edge cases and testing
|
||||
|
||||
Take your time with Phase 2 (rooms.py integration) - that's where most bugs will come from if you accidentally break calendar/webhook features.
|
||||
|
||||
**Good luck! You've got comprehensive specs, working reference code, and a clean starting point. You can do this.**
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference
|
||||
|
||||
```bash
|
||||
# Your workspace
|
||||
├── PLAN.md ← Complete technical spec (read first)
|
||||
├── IMPLEMENTATION_GUIDE.md ← Practical guide (bookmark this)
|
||||
├── CODER_BRIEFING.md ← This file
|
||||
└── reflector-dailyco-reference/ ← Reference implementation (inspiration only)
|
||||
|
||||
# Key commands
|
||||
cd server && uv run ruff check . # Lint backend
|
||||
cd www && pnpm lint # Lint frontend
|
||||
cd server && uv run alembic revision -m "..." # Create migration
|
||||
cd www && pnpm openapi # Regenerate types
|
||||
cd server && uv run pytest -v # Run tests
|
||||
```
|
||||
@@ -1,489 +0,0 @@
|
||||
# Daily.co Implementation Guide
|
||||
|
||||
## Overview
|
||||
Implement multi-provider video platform support (Whereby + Daily.co) following PLAN.md.
|
||||
|
||||
## Reference Code Location
|
||||
- **Reference branch:** `origin/igor/feat-dailyco` (on remote)
|
||||
- **Worktree location:** `./reflector-dailyco-reference/`
|
||||
- **Status:** Reference only - DO NOT merge or copy directly
|
||||
|
||||
## What Exists in Reference Branch (For Inspiration)
|
||||
|
||||
### ✅ Can Use As Reference (Well-Implemented)
|
||||
```
|
||||
server/reflector/video_platforms/
|
||||
├── base.py ← Platform abstraction (good design, copy-safe)
|
||||
├── models.py ← Data models (copy-safe)
|
||||
├── registry.py ← Registry pattern (copy-safe)
|
||||
├── factory.py ← Factory pattern (needs settings updates)
|
||||
├── whereby.py ← Whereby client (needs adaptation)
|
||||
├── daily.py ← Daily.co client (needs adaptation)
|
||||
└── mock.py ← Mock client (copy-safe for tests)
|
||||
|
||||
server/reflector/views/daily.py ← Webhook handler (needs adaptation)
|
||||
server/tests/test_video_platforms.py ← Tests (good reference)
|
||||
server/tests/test_daily_webhook.py ← Tests (good reference)
|
||||
|
||||
www/app/[roomName]/components/
|
||||
├── RoomContainer.tsx ← Platform router (needs React Query)
|
||||
├── DailyRoom.tsx ← Daily component (needs React Query)
|
||||
└── WherebyRoom.tsx ← Whereby extraction (needs React Query)
|
||||
```
|
||||
|
||||
### ⚠️ Needs Significant Changes (Use Logic Only)
|
||||
- `server/reflector/db/rooms.py` - Reference removed calendar/webhook fields that main has
|
||||
- `server/reflector/db/meetings.py` - Same issue (missing user_id handling differences)
|
||||
- `server/reflector/views/rooms.py` - Main has calendar integration, webhooks, ICS sync
|
||||
- `server/reflector/worker/process.py` - Main has different recording flow
|
||||
- Migration files - Must regenerate against current main schema
|
||||
|
||||
### ❌ Do NOT Use (Outdated/Incompatible)
|
||||
- `package.json`/`pnpm-lock.yaml` - Main uses different dependency versions
|
||||
- Frontend API client calls - Main uses React Query (reference uses old OpenAPI client)
|
||||
- Database migrations - Must create new ones from scratch
|
||||
- Any files that delete features present in main (search, calendar, webhooks)
|
||||
|
||||
## Key Differences: Reference vs Current Main
|
||||
|
||||
| Aspect | Reference Branch | Current Main | Action Required |
|
||||
|--------|------------------|--------------|-----------------|
|
||||
| **API client** | Old OpenAPI generated | React Query hooks | Rewrite all API calls |
|
||||
| **Database schema** | Simplified (removed features) | Has calendar, webhooks, full-text search | Merge carefully, preserve main features |
|
||||
| **Settings** | Aug 2025 structure | Current structure | Adapt carefully |
|
||||
| **Migrations** | Branched from Aug 1 | Current main (91+ commits ahead) | Regenerate from scratch |
|
||||
| **Frontend deps** | `@daily-co/daily-js@0.81.0` | Check current versions | Update to compatible versions |
|
||||
| **Package manager** | yarn | pnpm (maybe both?) | Use what main uses |
|
||||
|
||||
## Branch Divergence Analysis
|
||||
|
||||
**The reference branch is 91 commits behind main and severely diverged:**
|
||||
- Reference: 8 commits, 3,689 insertions, 425 deletions
|
||||
- Main since divergence: 320 files changed, 45,840 insertions, 16,827 deletions
|
||||
- **Main has 12x more changes**
|
||||
|
||||
**Major features in main that reference lacks:**
|
||||
1. Calendar integration (ICS sync with rooms)
|
||||
2. Self-hosted GPU API infrastructure
|
||||
3. Frontend OpenAPI React Query migration
|
||||
4. Full-text search (backend + frontend)
|
||||
5. Webhook system for room events
|
||||
6. Environment variable migration
|
||||
7. Security fixes and auth improvements
|
||||
8. Docker production frontend
|
||||
9. Meeting user ID removal (schema change)
|
||||
10. NextJS version upgrades
|
||||
|
||||
**High conflict risk files:**
|
||||
- `server/reflector/views/rooms.py` - 12x more changes in main
|
||||
- `server/reflector/db/rooms.py` - Main added 7+ fields
|
||||
- `www/package.json` - NextJS major version bump
|
||||
- Database migrations - 20+ new migrations in main
|
||||
|
||||
## Implementation Approach
|
||||
|
||||
### Phase 1: Copy Clean Abstractions (1-2 hours)
|
||||
|
||||
**Files to copy directly from reference:**
|
||||
```bash
|
||||
# Core abstraction (review but mostly safe to copy)
|
||||
cp -r reflector-dailyco-reference/server/reflector/video_platforms/ \
|
||||
server/reflector/
|
||||
|
||||
# BUT review each file for:
|
||||
# - Import paths (make sure they match current main)
|
||||
# - Settings references (adapt to current settings.py)
|
||||
# - Type imports (ensure no circular dependencies)
|
||||
```
|
||||
|
||||
**After copying, immediately:**
|
||||
```bash
|
||||
cd server
|
||||
# Check for issues
|
||||
uv run ruff check reflector/video_platforms/
|
||||
# Fix any import errors or type issues
|
||||
```
|
||||
|
||||
### Phase 2: Adapt to Current Main (2-3 hours)
|
||||
|
||||
**2.1 Settings Integration**
|
||||
|
||||
File: `server/reflector/settings.py`
|
||||
|
||||
Add at the appropriate location (near existing Whereby settings):
|
||||
|
||||
```python
|
||||
# Daily.co API Integration (NEW)
|
||||
DAILY_API_KEY: str | None = None
|
||||
DAILY_WEBHOOK_SECRET: str | None = None
|
||||
DAILY_SUBDOMAIN: str | None = None
|
||||
AWS_DAILY_S3_BUCKET: str | None = None
|
||||
AWS_DAILY_S3_REGION: str = "us-west-2"
|
||||
AWS_DAILY_ROLE_ARN: str | None = None
|
||||
|
||||
# Platform Migration Feature Flags (NEW)
|
||||
DAILY_MIGRATION_ENABLED: bool = False # Conservative default
|
||||
DAILY_MIGRATION_ROOM_IDS: list[str] = []
|
||||
DEFAULT_VIDEO_PLATFORM: Literal["whereby", "daily"] = "whereby"
|
||||
```
|
||||
|
||||
**2.2 Database Migration**
|
||||
|
||||
⚠️ **CRITICAL: Do NOT copy migration from reference**
|
||||
|
||||
Generate new migration:
|
||||
```bash
|
||||
cd server
|
||||
uv run alembic revision -m "add_platform_support"
|
||||
```
|
||||
|
||||
Edit the generated migration file to add `platform` column:
|
||||
```python
|
||||
def upgrade():
|
||||
with op.batch_alter_table("room", schema=None) as batch_op:
|
||||
batch_op.add_column(
|
||||
sa.Column("platform", sa.String(), nullable=False, server_default="whereby")
|
||||
)
|
||||
|
||||
with op.batch_alter_table("meeting", schema=None) as batch_op:
|
||||
batch_op.add_column(
|
||||
sa.Column("platform", sa.String(), nullable=False, server_default="whereby")
|
||||
)
|
||||
```
|
||||
|
||||
**2.3 Update Database Models**
|
||||
|
||||
File: `server/reflector/db/rooms.py`
|
||||
|
||||
Add platform field (preserve all existing fields from main):
|
||||
```python
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from reflector.video_platforms.models import Platform
|
||||
|
||||
class Room:
|
||||
# ... ALL existing fields from main (calendar, webhooks, etc.) ...
|
||||
|
||||
# NEW: Platform field
|
||||
platform: "Platform" = sqlalchemy.Column(
|
||||
sqlalchemy.String,
|
||||
nullable=False,
|
||||
server_default="whereby",
|
||||
)
|
||||
```
|
||||
|
||||
File: `server/reflector/db/meetings.py`
|
||||
|
||||
Same approach - add platform field, preserve everything from main.
|
||||
|
||||
**2.4 Integrate Platform Abstraction into rooms.py**
|
||||
|
||||
⚠️ **This is the most delicate part - main has calendar/webhook features**
|
||||
|
||||
File: `server/reflector/views/rooms.py`
|
||||
|
||||
Strategy:
|
||||
1. Add imports at top
|
||||
2. Modify meeting creation logic only
|
||||
3. Preserve all calendar/webhook/ICS logic from main
|
||||
|
||||
```python
|
||||
# Add imports
|
||||
from reflector.video_platforms import (
|
||||
create_platform_client,
|
||||
get_platform_for_room,
|
||||
)
|
||||
|
||||
# In create_meeting endpoint:
|
||||
# OLD: Direct Whereby API calls
|
||||
# NEW: Platform abstraction
|
||||
|
||||
# Find the meeting creation section and replace:
|
||||
platform = get_platform_for_room(room.id)
|
||||
client = create_platform_client(platform)
|
||||
|
||||
meeting_data = await client.create_meeting(
|
||||
room_name_prefix=room.name,
|
||||
end_date=meeting_data.end_date,
|
||||
room=room,
|
||||
)
|
||||
|
||||
# Then create Meeting record with meeting_data.platform, meeting_data.meeting_id, etc.
|
||||
```
|
||||
|
||||
**2.5 Add Daily.co Webhook Handler**
|
||||
|
||||
Copy from reference, minimal changes needed:
|
||||
```bash
|
||||
cp reflector-dailyco-reference/server/reflector/views/daily.py \
|
||||
server/reflector/views/
|
||||
```
|
||||
|
||||
Register in `server/reflector/app.py`:
|
||||
```python
|
||||
from reflector.views import daily
|
||||
|
||||
app.include_router(daily.router, prefix="/v1/daily", tags=["daily"])
|
||||
```
|
||||
|
||||
**2.6 Add Recording Processing Task**
|
||||
|
||||
File: `server/reflector/worker/process.py`
|
||||
|
||||
Add the `process_recording_from_url` task from reference (copy the function).
|
||||
|
||||
### Phase 3: Frontend Adaptation (3-4 hours)
|
||||
|
||||
**3.1 Determine Current API Client Pattern**
|
||||
|
||||
First, check how main currently makes API calls:
|
||||
```bash
|
||||
cd www
|
||||
grep -r "api\." app/ | head -20
|
||||
# Look for patterns like: api.v1Something()
|
||||
```
|
||||
|
||||
**3.2 Create Components**
|
||||
|
||||
Copy component structure from reference but **rewrite all API calls**:
|
||||
|
||||
```bash
|
||||
mkdir -p www/app/[roomName]/components
|
||||
```
|
||||
|
||||
Files to create:
|
||||
- `RoomContainer.tsx` - Platform router (mostly copy-safe, just fix imports)
|
||||
- `DailyRoom.tsx` - Needs React Query API calls
|
||||
- `WherebyRoom.tsx` - Extract current room page logic
|
||||
|
||||
**Example React Query pattern** (adapt to your actual API):
|
||||
```typescript
|
||||
import { api } from '@/app/api/client'
|
||||
|
||||
// In DailyRoom.tsx
|
||||
const handleConsent = async () => {
|
||||
try {
|
||||
await api.v1MeetingAudioConsent({
|
||||
path: { meeting_id: meeting.id },
|
||||
body: { consent: true },
|
||||
})
|
||||
// ...
|
||||
} catch (error) {
|
||||
// ...
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**3.3 Add Daily.co Dependency**
|
||||
|
||||
Check current package manager:
|
||||
```bash
|
||||
cd www
|
||||
ls package-lock.json yarn.lock pnpm-lock.yaml
|
||||
```
|
||||
|
||||
Then install:
|
||||
```bash
|
||||
# If using pnpm
|
||||
pnpm add @daily-co/daily-js@^0.81.0
|
||||
|
||||
# If using yarn
|
||||
yarn add @daily-co/daily-js@^0.81.0
|
||||
```
|
||||
|
||||
**3.4 Update TypeScript Types**
|
||||
|
||||
After backend changes, regenerate types:
|
||||
```bash
|
||||
cd www
|
||||
pnpm openapi # or yarn openapi
|
||||
```
|
||||
|
||||
This should pick up the new `platform` field on Meeting type.
|
||||
|
||||
### Phase 4: Testing (2-3 hours)
|
||||
|
||||
**4.1 Copy Test Structure**
|
||||
|
||||
```bash
|
||||
cp reflector-dailyco-reference/server/tests/test_video_platforms.py \
|
||||
server/tests/
|
||||
|
||||
cp reflector-dailyco-reference/server/tests/test_daily_webhook.py \
|
||||
server/tests/
|
||||
```
|
||||
|
||||
**4.2 Fix Test Imports and Fixtures**
|
||||
|
||||
Update imports to match current test infrastructure:
|
||||
- Check `server/tests/conftest.py` for fixture patterns
|
||||
- Update database access patterns if changed
|
||||
- Fix any import errors
|
||||
|
||||
**4.3 Run Tests**
|
||||
|
||||
```bash
|
||||
cd server
|
||||
# Run with environment variables for Mac
|
||||
REDIS_HOST=localhost \
|
||||
CELERY_BROKER_URL=redis://localhost:6379/1 \
|
||||
CELERY_RESULT_BACKEND=redis://localhost:6379/1 \
|
||||
uv run pytest tests/test_video_platforms.py -v
|
||||
```
|
||||
|
||||
### Phase 5: Environment Configuration
|
||||
|
||||
**Update `server/env.example`:**
|
||||
|
||||
Add at the end:
|
||||
```bash
|
||||
# Daily.co API Integration
|
||||
DAILY_API_KEY=your-daily-api-key
|
||||
DAILY_WEBHOOK_SECRET=your-daily-webhook-secret
|
||||
DAILY_SUBDOMAIN=your-subdomain
|
||||
AWS_DAILY_S3_BUCKET=your-daily-bucket
|
||||
AWS_DAILY_S3_REGION=us-west-2
|
||||
AWS_DAILY_ROLE_ARN=arn:aws:iam::ACCOUNT:role/DailyRecording
|
||||
|
||||
# Platform Selection
|
||||
DAILY_MIGRATION_ENABLED=false # Master switch
|
||||
DAILY_MIGRATION_ROOM_IDS=[] # Specific room IDs
|
||||
DEFAULT_VIDEO_PLATFORM=whereby # Default platform
|
||||
```
|
||||
|
||||
## Decision Tree: Copy vs Adapt vs Rewrite
|
||||
|
||||
```
|
||||
┌─ Is it pure abstraction logic? (base.py, registry.py, models.py)
|
||||
│ YES → Copy directly, review imports
|
||||
│ NO → Continue ↓
|
||||
│
|
||||
├─ Does it touch database models?
|
||||
│ YES → Adapt carefully, preserve main's fields
|
||||
│ NO → Continue ↓
|
||||
│
|
||||
├─ Does it make API calls on frontend?
|
||||
│ YES → Rewrite using React Query
|
||||
│ NO → Continue ↓
|
||||
│
|
||||
├─ Is it a database migration?
|
||||
│ YES → Generate fresh from current schema
|
||||
│ NO → Continue ↓
|
||||
│
|
||||
└─ Does it touch rooms.py or core business logic?
|
||||
YES → Merge carefully, preserve calendar/webhooks
|
||||
NO → Safe to adapt from reference
|
||||
```
|
||||
|
||||
## Verification Checklist
|
||||
|
||||
After each phase, verify:
|
||||
|
||||
**Phase 1 (Abstraction Layer):**
|
||||
- [ ] `uv run ruff check server/reflector/video_platforms/` passes
|
||||
- [ ] No circular import errors
|
||||
- [ ] Can import `from reflector.video_platforms import create_platform_client`
|
||||
|
||||
**Phase 2 (Backend Integration):**
|
||||
- [ ] `uv run ruff check server/` passes
|
||||
- [ ] Migration file generated (not copied)
|
||||
- [ ] Room and Meeting models have platform field
|
||||
- [ ] rooms.py still has calendar/webhook features
|
||||
|
||||
**Phase 3 (Frontend):**
|
||||
- [ ] `pnpm lint` passes
|
||||
- [ ] No TypeScript errors
|
||||
- [ ] No `@ts-ignore` for platform field
|
||||
- [ ] API calls use React Query patterns
|
||||
|
||||
**Phase 4 (Testing):**
|
||||
- [ ] Tests can be collected: `pytest tests/test_video_platforms.py --collect-only`
|
||||
- [ ] Database fixtures work
|
||||
- [ ] Mock platform works
|
||||
|
||||
**Phase 5 (Config):**
|
||||
- [ ] env.example has Daily.co variables
|
||||
- [ ] settings.py has all new variables
|
||||
- [ ] No duplicate variable definitions
|
||||
|
||||
## Common Pitfalls
|
||||
|
||||
### 1. Database Schema Conflicts
|
||||
**Problem:** Reference removed fields that main has (calendar, webhooks)
|
||||
**Solution:** Always preserve main's fields, only add platform field
|
||||
|
||||
### 2. Migration Conflicts
|
||||
**Problem:** Reference migration has wrong `down_revision`
|
||||
**Solution:** Always generate fresh migration from current main
|
||||
|
||||
### 3. Frontend API Calls
|
||||
**Problem:** Reference uses old API client patterns
|
||||
**Solution:** Check current main's API usage, replicate that pattern
|
||||
|
||||
### 4. Import Errors
|
||||
**Problem:** Circular imports with TYPE_CHECKING
|
||||
**Solution:** Use `if TYPE_CHECKING:` for Room/Meeting imports in video_platforms
|
||||
|
||||
### 5. Test Database Issues
|
||||
**Problem:** Tests fail with "could not translate host name 'postgres'"
|
||||
**Solution:** Use environment variables: `REDIS_HOST=localhost DATABASE_URL=...`
|
||||
|
||||
### 6. Preserved Features Broken
|
||||
**Problem:** Calendar/webhook features stop working
|
||||
**Solution:** Carefully review rooms.py diff, only change meeting creation, not calendar logic
|
||||
|
||||
## File Modification Summary
|
||||
|
||||
**New files (can copy):**
|
||||
- `server/reflector/video_platforms/*.py` (entire directory)
|
||||
- `server/reflector/views/daily.py`
|
||||
- `server/tests/test_video_platforms.py`
|
||||
- `server/tests/test_daily_webhook.py`
|
||||
- `www/app/[roomName]/components/RoomContainer.tsx`
|
||||
- `www/app/[roomName]/components/DailyRoom.tsx`
|
||||
- `www/app/[roomName]/components/WherebyRoom.tsx`
|
||||
|
||||
**Modified files (careful merging):**
|
||||
- `server/reflector/settings.py` - Add Daily.co settings
|
||||
- `server/reflector/db/rooms.py` - Add platform field
|
||||
- `server/reflector/db/meetings.py` - Add platform field
|
||||
- `server/reflector/views/rooms.py` - Integrate platform abstraction
|
||||
- `server/reflector/worker/process.py` - Add process_recording_from_url
|
||||
- `server/reflector/app.py` - Register daily router
|
||||
- `server/env.example` - Add Daily.co variables
|
||||
- `www/app/[roomName]/page.tsx` - Use RoomContainer
|
||||
- `www/package.json` - Add @daily-co/daily-js
|
||||
|
||||
**Generated files (do not copy):**
|
||||
- `server/migrations/versions/XXXXXX_add_platform_support.py` - Generate fresh
|
||||
|
||||
## Success Metrics
|
||||
|
||||
Implementation is complete when:
|
||||
- [ ] All tests pass (including new platform tests)
|
||||
- [ ] Linting passes (ruff, pnpm lint)
|
||||
- [ ] Migration applies cleanly: `uv run alembic upgrade head`
|
||||
- [ ] Can create Whereby meeting (existing flow unchanged)
|
||||
- [ ] Can create Daily.co meeting (with env vars set)
|
||||
- [ ] Frontend loads without TypeScript errors
|
||||
- [ ] No features from main were accidentally removed
|
||||
|
||||
## Getting Help
|
||||
|
||||
**Reference documentation locations:**
|
||||
- Implementation plan: `PLAN.md`
|
||||
- Reference implementation: `./reflector-dailyco-reference/`
|
||||
- Current main codebase: `./ ` (current directory)
|
||||
|
||||
**Compare implementations:**
|
||||
```bash
|
||||
# Compare specific files
|
||||
diff reflector-dailyco-reference/server/reflector/video_platforms/base.py \
|
||||
server/reflector/video_platforms/base.py
|
||||
|
||||
# See what changed in rooms.py between reference branch point and now
|
||||
git log --oneline --since="2025-08-01" -- server/reflector/views/rooms.py
|
||||
```
|
||||
|
||||
**Key insight:** The reference branch validates the approach and provides working code patterns, but you're implementing fresh against current main to avoid merge conflicts and preserve all new features.
|
||||
@@ -1,613 +0,0 @@
|
||||
# Daily.co Integration Test Plan
|
||||
|
||||
## ✅ IMPLEMENTATION STATUS: Real Transcription Active
|
||||
|
||||
**This test validates Daily.co multitrack recording integration with REAL transcription/diarization.**
|
||||
|
||||
The implementation includes complete audio processing pipeline:
|
||||
- **Multitrack recordings** from Daily.co S3 (separate audio stream per participant)
|
||||
- **PyAV-based audio mixdown** with PTS-based track alignment
|
||||
- **Real transcription** via Modal GPU backend (Whisper)
|
||||
- **Real diarization** via Modal GPU backend (speaker identification)
|
||||
- **Per-track transcription** with timestamp synchronization
|
||||
- **Complete database entities** (recording, transcript, topics, participants, words)
|
||||
|
||||
**Processing pipeline** (`PipelineMainMultitrack`):
|
||||
1. Download all audio tracks from Daily.co S3
|
||||
2. Align tracks by PTS (presentation timestamp) to handle late joiners
|
||||
3. Mix tracks into single audio file for unified playback
|
||||
4. Transcribe each track individually with proper offset handling
|
||||
5. Perform diarization on mixed audio
|
||||
6. Generate topics, summaries, and word-level timestamps
|
||||
7. Convert audio to MP3 and generate waveform visualization
|
||||
|
||||
**Note:** A stub processor (`process_daily_recording`) exists for testing webhook flow without GPU costs, but the production code path uses `process_multitrack_recording` with full ML pipeline.
|
||||
|
||||
---
|
||||
|
||||
## Prerequisites
|
||||
|
||||
**1. Environment Variables** (check in `.env.development.local`):
|
||||
```bash
|
||||
# Daily.co API Configuration
|
||||
DAILY_API_KEY=<key>
|
||||
DAILY_SUBDOMAIN=monadical
|
||||
DAILY_WEBHOOK_SECRET=<base64-encoded-secret>
|
||||
AWS_DAILY_S3_BUCKET=reflector-dailyco-local
|
||||
AWS_DAILY_S3_REGION=us-east-1
|
||||
AWS_DAILY_ROLE_ARN=arn:aws:iam::950402358378:role/DailyCo
|
||||
DAILY_MIGRATION_ENABLED=true
|
||||
DAILY_MIGRATION_ROOM_IDS=["552640fd-16f2-4162-9526-8cf40cd2357e"]
|
||||
|
||||
# Transcription/Diarization Backend (Required for real processing)
|
||||
DIARIZATION_BACKEND=modal
|
||||
DIARIZATION_MODAL_API_KEY=<modal-api-key>
|
||||
# TRANSCRIPTION_BACKEND is not explicitly set (uses default/modal)
|
||||
```
|
||||
|
||||
**2. Services Running:**
|
||||
```bash
|
||||
docker compose ps # server, postgres, redis, worker, beat should be UP
|
||||
```
|
||||
|
||||
**IMPORTANT:** Worker and beat services MUST be running for transcription processing:
|
||||
```bash
|
||||
docker compose up -d worker beat
|
||||
```
|
||||
|
||||
**3. ngrok Tunnel for Webhooks:**
|
||||
```bash
|
||||
# Start ngrok (if not already running)
|
||||
ngrok http 1250 --log=stdout > /tmp/ngrok.log 2>&1 &
|
||||
|
||||
# Get public URL
|
||||
curl -s http://localhost:4040/api/tunnels | python3 -c "import sys, json; data=json.load(sys.stdin); print(data['tunnels'][0]['public_url'])"
|
||||
```
|
||||
|
||||
**Current ngrok URL:** `https://0503947384a3.ngrok-free.app` (as of last registration)
|
||||
|
||||
**4. Webhook Created:**
|
||||
```bash
|
||||
cd server
|
||||
uv run python scripts/recreate_daily_webhook.py https://0503947384a3.ngrok-free.app/v1/daily/webhook
|
||||
# Verify: "Created webhook <uuid> (state: ACTIVE)"
|
||||
```
|
||||
|
||||
**Current webhook status:** ✅ ACTIVE (webhook ID: dad5ad16-ceca-488e-8fc5-dae8650b51d0)
|
||||
|
||||
---
|
||||
|
||||
## Test 1: Database Configuration
|
||||
|
||||
**Check room platform:**
|
||||
```bash
|
||||
docker-compose exec -T postgres psql -U reflector -d reflector -c \
|
||||
"SELECT id, name, platform, recording_type FROM room WHERE name = 'test2';"
|
||||
```
|
||||
|
||||
**Expected:**
|
||||
```
|
||||
id: 552640fd-16f2-4162-9526-8cf40cd2357e
|
||||
name: test2
|
||||
platform: whereby # DB value (overridden by env var DAILY_MIGRATION_ROOM_IDS)
|
||||
recording_type: cloud
|
||||
```
|
||||
|
||||
**Clear old meetings:**
|
||||
```bash
|
||||
docker-compose exec -T postgres psql -U reflector -d reflector -c \
|
||||
"UPDATE meeting SET is_active = false WHERE room_id = '552640fd-16f2-4162-9526-8cf40cd2357e';"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Test 2: Meeting Creation with Auto-Recording
|
||||
|
||||
**Create meeting:**
|
||||
```bash
|
||||
curl -s -X POST http://localhost:1250/v1/rooms/test2/meeting \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"allow_duplicated":false}' | python3 -m json.tool
|
||||
```
|
||||
|
||||
**Expected Response:**
|
||||
```json
|
||||
{
|
||||
"room_name": "test2-YYYYMMDDHHMMSS", // Includes "test2" prefix!
|
||||
"room_url": "https://monadical.daily.co/test2-...?t=<JWT_TOKEN>", // Has token!
|
||||
"platform": "daily",
|
||||
"recording_type": "cloud" // DB value (Whereby-specific)
|
||||
}
|
||||
```
|
||||
|
||||
**Decode token to verify auto-recording:**
|
||||
```bash
|
||||
# Extract token from room_url, decode JWT payload
|
||||
echo "<token>" | python3 -c "
|
||||
import sys, json, base64
|
||||
token = sys.stdin.read().strip()
|
||||
payload = token.split('.')[1] + '=' * (4 - len(token.split('.')[1]) % 4)
|
||||
print(json.dumps(json.loads(base64.b64decode(payload)), indent=2))
|
||||
"
|
||||
```
|
||||
|
||||
**Expected token payload:**
|
||||
```json
|
||||
{
|
||||
"r": "test2-YYYYMMDDHHMMSS", // Room name
|
||||
"sr": true, // start_recording: true ✅
|
||||
"d": "...", // Domain ID
|
||||
"iat": 1234567890
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Test 3: Daily.co API Verification
|
||||
|
||||
**Check room configuration:**
|
||||
```bash
|
||||
ROOM_NAME="<from previous step>"
|
||||
curl -s -X GET "https://api.daily.co/v1/rooms/$ROOM_NAME" \
|
||||
-H "Authorization: Bearer $DAILY_API_KEY" | python3 -m json.tool
|
||||
```
|
||||
|
||||
**Expected config:**
|
||||
```json
|
||||
{
|
||||
"config": {
|
||||
"enable_recording": "raw-tracks", // ✅
|
||||
"recordings_bucket": {
|
||||
"bucket_name": "reflector-dailyco-local",
|
||||
"bucket_region": "us-east-1",
|
||||
"assume_role_arn": "arn:aws:iam::950402358378:role/DailyCo"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Test 4: Browser UI Test (Playwright MCP)
|
||||
|
||||
**Using Claude Code MCP tools:**
|
||||
|
||||
**Load room:**
|
||||
```
|
||||
Use: mcp__playwright__browser_navigate
|
||||
Input: {"url": "http://localhost:3000/test2"}
|
||||
|
||||
Then wait 12 seconds for iframe to load
|
||||
```
|
||||
|
||||
**Verify Daily.co iframe loaded:**
|
||||
```
|
||||
Use: mcp__playwright__browser_snapshot
|
||||
|
||||
Expected in snapshot:
|
||||
- iframe element with src containing "monadical.daily.co"
|
||||
- Daily.co pre-call UI visible
|
||||
```
|
||||
|
||||
**Take screenshot:**
|
||||
```
|
||||
Use: mcp__playwright__browser_take_screenshot
|
||||
Input: {"filename": "test2-before-join.png"}
|
||||
|
||||
Expected: Daily.co pre-call UI with "Join" button visible
|
||||
```
|
||||
|
||||
**Join meeting:**
|
||||
```
|
||||
Note: Daily.co iframe interaction requires clicking inside iframe.
|
||||
Use: mcp__playwright__browser_click
|
||||
Input: {"element": "Join button in Daily.co iframe", "ref": "<ref-from-snapshot>"}
|
||||
|
||||
Then wait 5 seconds for call to connect
|
||||
```
|
||||
|
||||
**Verify in-call:**
|
||||
```
|
||||
Use: mcp__playwright__browser_take_screenshot
|
||||
Input: {"filename": "test2-in-call.png"}
|
||||
|
||||
Expected: "Waiting for others to join" or participant video visible
|
||||
```
|
||||
|
||||
**Leave meeting:**
|
||||
```
|
||||
Use: mcp__playwright__browser_click
|
||||
Input: {"element": "Leave button in Daily.co iframe", "ref": "<ref-from-snapshot>"}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
**Alternative: JavaScript snippets (for manual testing):**
|
||||
|
||||
```javascript
|
||||
await page.goto('http://localhost:3000/test2');
|
||||
await new Promise(f => setTimeout(f, 12000)); // Wait for load
|
||||
|
||||
// Verify iframe
|
||||
const iframes = document.querySelectorAll('iframe');
|
||||
// Expected: 1 iframe with src containing "monadical.daily.co"
|
||||
|
||||
// Screenshot
|
||||
await page.screenshot({ path: 'test2-before-join.png' });
|
||||
|
||||
// Join
|
||||
await page.locator('iframe').contentFrame().getByRole('button', { name: 'Join' }).click();
|
||||
await new Promise(f => setTimeout(f, 5000));
|
||||
|
||||
// In-call screenshot
|
||||
await page.screenshot({ path: 'test2-in-call.png' });
|
||||
|
||||
// Leave
|
||||
await page.locator('iframe').contentFrame().getByRole('button', { name: 'Leave' }).click();
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Test 5: Webhook Verification
|
||||
|
||||
**Check server logs for webhooks:**
|
||||
```bash
|
||||
docker-compose logs --since 15m server 2>&1 | grep -i "participant joined\|recording started"
|
||||
```
|
||||
|
||||
**Expected logs:**
|
||||
```
|
||||
[info] Participant joined | meeting_id=... | num_clients=1 | recording_type=cloud | recording_trigger=automatic-2nd-participant
|
||||
[info] Recording started | meeting_id=... | recording_id=... | platform=daily
|
||||
```
|
||||
|
||||
**Check Daily.co webhook delivery logs:**
|
||||
```bash
|
||||
curl -s -X GET "https://api.daily.co/v1/logs/webhooks?limit=20" \
|
||||
-H "Authorization: Bearer $DAILY_API_KEY" | python3 -c "
|
||||
import sys, json
|
||||
logs = json.load(sys.stdin)
|
||||
for log in logs[:10]:
|
||||
req = json.loads(log['request'])
|
||||
room = req.get('payload', {}).get('room') or req.get('payload', {}).get('room_name', 'N/A')
|
||||
print(f\"{req['type']:30s} | room: {room:30s} | status: {log['status']}\")
|
||||
"
|
||||
```
|
||||
|
||||
**Expected output:**
|
||||
```
|
||||
participant.joined | room: test2-YYYYMMDDHHMMSS | status: 200
|
||||
recording.started | room: test2-YYYYMMDDHHMMSS | status: 200
|
||||
participant.left | room: test2-YYYYMMDDHHMMSS | status: 200
|
||||
recording.ready-to-download | room: test2-YYYYMMDDHHMMSS | status: 200
|
||||
```
|
||||
|
||||
**Check database updated:**
|
||||
```bash
|
||||
docker-compose exec -T postgres psql -U reflector -d reflector -c \
|
||||
"SELECT room_name, num_clients FROM meeting WHERE room_name LIKE 'test2-%' ORDER BY end_date DESC LIMIT 1;"
|
||||
```
|
||||
|
||||
**Expected:**
|
||||
```
|
||||
room_name: test2-YYYYMMDDHHMMSS
|
||||
num_clients: 0 // After participant left
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Test 6: Recording in S3
|
||||
|
||||
**List recent recordings:**
|
||||
```bash
|
||||
curl -s -X GET "https://api.daily.co/v1/recordings" \
|
||||
-H "Authorization: Bearer $DAILY_API_KEY" | python3 -c "
|
||||
import sys, json
|
||||
data = json.load(sys.stdin)
|
||||
for rec in data.get('data', [])[:5]:
|
||||
if 'test2-' in rec.get('room_name', ''):
|
||||
print(f\"Room: {rec['room_name']}\")
|
||||
print(f\"Status: {rec['status']}\")
|
||||
print(f\"Duration: {rec.get('duration', 0)}s\")
|
||||
print(f\"S3 key: {rec.get('s3key', 'N/A')}\")
|
||||
print(f\"Tracks: {len(rec.get('tracks', []))} files\")
|
||||
for track in rec.get('tracks', []):
|
||||
print(f\" - {track['type']}: {track['s3Key'].split('/')[-1]} ({track['size']} bytes)\")
|
||||
print()
|
||||
"
|
||||
```
|
||||
|
||||
**Expected output:**
|
||||
```
|
||||
Room: test2-20251009192341
|
||||
Status: finished
|
||||
Duration: ~30-120s
|
||||
S3 key: monadical/test2-20251009192341/1760037914930
|
||||
Tracks: 2 files
|
||||
- audio: 1760037914930-<uuid>-cam-audio-1760037915265 (~400 KB)
|
||||
- video: 1760037914930-<uuid>-cam-video-1760037915269 (~10-30 MB)
|
||||
```
|
||||
|
||||
**Verify S3 path structure:**
|
||||
- `monadical/` - Daily.co subdomain
|
||||
- `test2-20251009192341/` - Reflector room name + timestamp
|
||||
- `<timestamp>-<participant-uuid>-<media-type>-<track-start>.webm` - Individual track files
|
||||
|
||||
---
|
||||
|
||||
## Test 7: Database Check - Recording and Transcript
|
||||
|
||||
**Check recording created:**
|
||||
```bash
|
||||
docker-compose exec -T postgres psql -U reflector -d reflector -c \
|
||||
"SELECT id, bucket_name, object_key, status, meeting_id, recorded_at
|
||||
FROM recording
|
||||
ORDER BY recorded_at DESC LIMIT 1;"
|
||||
```
|
||||
|
||||
**Expected:**
|
||||
```
|
||||
id: <recording-id-from-webhook>
|
||||
bucket_name: reflector-dailyco-local
|
||||
object_key: monadical/test2-<timestamp>/<recording-timestamp>-<uuid>-cam-audio-<track-start>.webm
|
||||
status: completed
|
||||
meeting_id: <meeting-id>
|
||||
recorded_at: <recent-timestamp>
|
||||
```
|
||||
|
||||
**Check transcript created:**
|
||||
```bash
|
||||
docker compose exec -T postgres psql -U reflector -d reflector -c \
|
||||
"SELECT id, title, status, duration, recording_id, meeting_id, room_id
|
||||
FROM transcript
|
||||
ORDER BY created_at DESC LIMIT 1;"
|
||||
```
|
||||
|
||||
**Expected (REAL transcription):**
|
||||
```
|
||||
id: <transcript-id>
|
||||
title: <AI-generated title based on actual conversation content>
|
||||
status: uploaded (audio file processed and available)
|
||||
duration: <actual meeting duration in seconds>
|
||||
recording_id: <same-as-recording-id-above>
|
||||
meeting_id: <meeting-id>
|
||||
room_id: 552640fd-16f2-4162-9526-8cf40cd2357e
|
||||
```
|
||||
|
||||
**Note:** Title and content will reflect the ACTUAL conversation, not mock data. Processing time depends on recording length and GPU backend availability (Modal).
|
||||
|
||||
**Verify audio file exists:**
|
||||
```bash
|
||||
ls -lh data/<transcript-id>/upload.webm
|
||||
```
|
||||
|
||||
**Expected:**
|
||||
```
|
||||
-rw-r--r-- 1 user staff ~100-200K Oct 10 18:48 upload.webm
|
||||
```
|
||||
|
||||
**Check transcript topics (REAL transcription):**
|
||||
```bash
|
||||
TRANSCRIPT_ID=$(docker compose exec -T postgres psql -U reflector -d reflector -t -c \
|
||||
"SELECT id FROM transcript ORDER BY created_at DESC LIMIT 1;")
|
||||
|
||||
docker compose exec -T postgres psql -U reflector -d reflector -c \
|
||||
"SELECT
|
||||
jsonb_array_length(topics) as num_topics,
|
||||
jsonb_array_length(participants) as num_participants,
|
||||
short_summary,
|
||||
title
|
||||
FROM transcript
|
||||
WHERE id = '$TRANSCRIPT_ID';"
|
||||
```
|
||||
|
||||
**Expected (REAL data):**
|
||||
```
|
||||
num_topics: <varies based on conversation>
|
||||
num_participants: <actual number of participants who spoke>
|
||||
short_summary: <AI-generated summary of actual conversation>
|
||||
title: <AI-generated title based on content>
|
||||
```
|
||||
|
||||
**Check topics contain actual transcription:**
|
||||
```bash
|
||||
docker compose exec -T postgres psql -U reflector -d reflector -c \
|
||||
"SELECT topics->0->'title', topics->0->'summary', topics->0->'transcript'
|
||||
FROM transcript
|
||||
ORDER BY created_at DESC LIMIT 1;" | head -20
|
||||
```
|
||||
|
||||
**Expected output:** Will contain the ACTUAL transcribed conversation from the Daily.co meeting, not mock data.
|
||||
|
||||
**Check participants:**
|
||||
```bash
|
||||
docker compose exec -T postgres psql -U reflector -d reflector -c \
|
||||
"SELECT participants FROM transcript ORDER BY created_at DESC LIMIT 1;" \
|
||||
| python3 -c "import sys, json; data=json.loads(sys.stdin.read()); print(json.dumps(data, indent=2))"
|
||||
```
|
||||
|
||||
**Expected (REAL diarization):**
|
||||
```json
|
||||
[
|
||||
{
|
||||
"id": "<uuid>",
|
||||
"speaker": 0,
|
||||
"name": "Speaker 1"
|
||||
},
|
||||
{
|
||||
"id": "<uuid>",
|
||||
"speaker": 1,
|
||||
"name": "Speaker 2"
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
**Note:** Speaker names will be generic ("Speaker 1", "Speaker 2", etc.) as determined by the diarization backend. Number of participants depends on how many actually spoke during the meeting.
|
||||
|
||||
**Check word-level data:**
|
||||
```bash
|
||||
docker compose exec -T postgres psql -U reflector -d reflector -c \
|
||||
"SELECT jsonb_array_length(topics->0->'words') as num_words_first_topic
|
||||
FROM transcript
|
||||
ORDER BY created_at DESC LIMIT 1;"
|
||||
```
|
||||
|
||||
**Expected:**
|
||||
```
|
||||
num_words_first_topic: <varies based on actual conversation length and topic chunking>
|
||||
```
|
||||
|
||||
**Verify speaker diarization in words:**
|
||||
```bash
|
||||
docker compose exec -T postgres psql -U reflector -d reflector -c \
|
||||
"SELECT
|
||||
topics->0->'words'->0->>'text' as first_word,
|
||||
topics->0->'words'->0->>'speaker' as speaker,
|
||||
topics->0->'words'->0->>'start' as start_time,
|
||||
topics->0->'words'->0->>'end' as end_time
|
||||
FROM transcript
|
||||
ORDER BY created_at DESC LIMIT 1;"
|
||||
```
|
||||
|
||||
**Expected (REAL transcription):**
|
||||
```
|
||||
first_word: <actual first word from transcription>
|
||||
speaker: 0, 1, 2, ... (actual speaker ID from diarization)
|
||||
start_time: <actual timestamp in seconds>
|
||||
end_time: <actual end timestamp>
|
||||
```
|
||||
|
||||
**Note:** All timestamps and speaker IDs are from real transcription/diarization, synchronized across tracks.
|
||||
|
||||
---
|
||||
|
||||
## Test 8: Recording Type Verification
|
||||
|
||||
**Check what Daily.co received:**
|
||||
```bash
|
||||
curl -s -X GET "https://api.daily.co/v1/rooms/test2-<timestamp>" \
|
||||
-H "Authorization: Bearer $DAILY_API_KEY" | python3 -m json.tool | grep "enable_recording"
|
||||
```
|
||||
|
||||
**Expected:**
|
||||
```json
|
||||
"enable_recording": "raw-tracks"
|
||||
```
|
||||
|
||||
**NOT:** `"enable_recording": "cloud"` (that would be wrong - we want raw tracks)
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Issue: No webhooks received
|
||||
|
||||
**Check webhook state:**
|
||||
```bash
|
||||
curl -s -X GET "https://api.daily.co/v1/webhooks" \
|
||||
-H "Authorization: Bearer $DAILY_API_KEY" | python3 -m json.tool
|
||||
```
|
||||
|
||||
**If state is FAILED:**
|
||||
```bash
|
||||
cd server
|
||||
uv run python scripts/recreate_daily_webhook.py https://<ngrok-url>/v1/daily/webhook
|
||||
```
|
||||
|
||||
### Issue: Webhooks return 422
|
||||
|
||||
**Check server logs:**
|
||||
```bash
|
||||
docker-compose logs --tail=50 server | grep "Failed to parse webhook event"
|
||||
```
|
||||
|
||||
**Common cause:** Event structure mismatch. Daily.co events use:
|
||||
```json
|
||||
{
|
||||
"version": "1.0.0",
|
||||
"type": "participant.joined",
|
||||
"payload": {...}, // NOT "data"
|
||||
"event_ts": 123.456 // NOT "ts"
|
||||
}
|
||||
```
|
||||
|
||||
### Issue: Recording not starting
|
||||
|
||||
1. **Check token has `sr: true`:**
|
||||
- Decode JWT token from room_url query param
|
||||
- Should contain `"sr": true`
|
||||
|
||||
2. **Check Daily.co room config:**
|
||||
- `enable_recording` must be set (not false)
|
||||
- For raw-tracks: must be exactly `"raw-tracks"`
|
||||
|
||||
3. **Check participant actually joined:**
|
||||
- Logs should show "Participant joined"
|
||||
- Must click "Join" button, not just pre-call screen
|
||||
|
||||
### Issue: Recording in S3 but wrong format
|
||||
|
||||
**Daily.co recording types:**
|
||||
- `"cloud"` → Single MP4 file (`download_link` in webhook)
|
||||
- `"raw-tracks"` → Multiple WebM files (`tracks` array in webhook)
|
||||
- `"raw-tracks-audio-only"` → Only audio WebM files
|
||||
|
||||
**Current implementation:** Always uses `"raw-tracks"` (better for transcription)
|
||||
|
||||
---
|
||||
|
||||
## Quick Validation Commands
|
||||
|
||||
**One-liner to verify everything:**
|
||||
```bash
|
||||
# 1. Check room exists
|
||||
docker-compose exec -T postgres psql -U reflector -d reflector -c \
|
||||
"SELECT name, platform FROM room WHERE name = 'test2';" && \
|
||||
|
||||
# 2. Create meeting
|
||||
MEETING=$(curl -s -X POST http://localhost:1250/v1/rooms/test2/meeting \
|
||||
-H "Content-Type: application/json" -d '{"allow_duplicated":false}') && \
|
||||
echo "$MEETING" | python3 -c "import sys,json; m=json.load(sys.stdin); print(f'Room: {m[\"room_name\"]}\nURL: {m[\"room_url\"][:80]}...')" && \
|
||||
|
||||
# 3. Check Daily.co config
|
||||
ROOM_NAME=$(echo "$MEETING" | python3 -c "import sys,json; print(json.load(sys.stdin)['room_name'])") && \
|
||||
curl -s -X GET "https://api.daily.co/v1/rooms/$ROOM_NAME" \
|
||||
-H "Authorization: Bearer $DAILY_API_KEY" | python3 -c "import sys,json; print(f'Recording: {json.load(sys.stdin)[\"config\"][\"enable_recording\"]}')"
|
||||
```
|
||||
|
||||
**Expected output:**
|
||||
```
|
||||
name: test2, platform: whereby
|
||||
Room: test2-20251009192341
|
||||
URL: https://monadical.daily.co/test2-20251009192341?t=eyJhbGc...
|
||||
Recording: raw-tracks
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Success Criteria Checklist
|
||||
|
||||
- [x] Room name includes Reflector room prefix (`test2-...`)
|
||||
- [x] Meeting URL contains JWT token (`?t=...`)
|
||||
- [x] Token has `sr: true` (auto-recording enabled)
|
||||
- [x] Daily.co room config: `enable_recording: "raw-tracks"`
|
||||
- [x] Browser loads Daily.co interface (not Whereby)
|
||||
- [x] Recording auto-starts when participant joins
|
||||
- [x] Webhooks received: participant.joined, recording.started, participant.left, recording.ready-to-download
|
||||
- [x] Recording status: `finished`
|
||||
- [x] S3 contains 2 files: audio (.webm) and video (.webm)
|
||||
- [x] S3 path: `monadical/test2-{timestamp}/{recording-start-ts}-{participant-uuid}-cam-{audio|video}-{track-start-ts}`
|
||||
- [x] Database `num_clients` increments/decrements correctly
|
||||
- [x] **Database recording entry created** with correct S3 path and status `completed`
|
||||
- [ ] **Database transcript entry created** with status `uploaded`
|
||||
- [ ] **Audio file downloaded** to `data/{transcript_id}/upload.webm`
|
||||
- [ ] **Transcript has REAL data**: AI-generated title based on conversation
|
||||
- [ ] **Transcript has topics** generated from actual content
|
||||
- [ ] **Transcript has participants** with proper speaker diarization
|
||||
- [ ] **Topics contain word-level data** with accurate timestamps and speaker IDs
|
||||
- [ ] **Total duration** matches actual meeting length
|
||||
- [ ] **MP3 and waveform files generated** by file processing pipeline
|
||||
- [ ] **Frontend transcript page loads** without "Failed to load audio" error
|
||||
- [ ] **Audio player functional** with working playback and waveform visualization
|
||||
- [ ] **Multitrack processing completed** without errors in worker logs
|
||||
- [ ] **Modal GPU backends accessible** (transcription and diarization)
|
||||
@@ -6,7 +6,7 @@ ENV PYTHONUNBUFFERED=1 \
|
||||
|
||||
# builder install base dependencies
|
||||
WORKDIR /tmp
|
||||
RUN apt-get update && apt-get install -y curl ffmpeg && apt-get clean
|
||||
RUN apt-get update && apt-get install -y curl && apt-get clean
|
||||
ADD https://astral.sh/uv/install.sh /uv-installer.sh
|
||||
RUN sh /uv-installer.sh && rm /uv-installer.sh
|
||||
ENV PATH="/root/.local/bin/:$PATH"
|
||||
|
||||
@@ -1,3 +1,29 @@
|
||||
## API Key Management
|
||||
|
||||
### Finding Your User ID
|
||||
|
||||
```bash
|
||||
# Get your OAuth sub (user ID) - requires authentication
|
||||
curl -H "Authorization: Bearer <your_jwt>" http://localhost:1250/v1/me
|
||||
# Returns: {"sub": "your-oauth-sub-here", "email": "...", ...}
|
||||
```
|
||||
|
||||
### Creating API Keys
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:1250/v1/user/api-keys \
|
||||
-H "Authorization: Bearer <your_jwt>" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"name": "My API Key"}'
|
||||
```
|
||||
|
||||
### Using API Keys
|
||||
|
||||
```bash
|
||||
# Use X-API-Key header instead of Authorization
|
||||
curl -H "X-API-Key: <your_api_key>" http://localhost:1250/v1/transcripts
|
||||
```
|
||||
|
||||
## AWS S3/SQS usage clarification
|
||||
|
||||
Whereby.com uploads recordings directly to our S3 bucket when meetings end.
|
||||
|
||||
234
server/docs/video-platforms/README.md
Normal file
234
server/docs/video-platforms/README.md
Normal file
@@ -0,0 +1,234 @@
|
||||
# Reflector Architecture: Whereby + Daily.co Recording Storage
|
||||
|
||||
## System Overview
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph "Actors"
|
||||
APP[Our App<br/>Reflector]
|
||||
WHEREBY[Whereby Service<br/>External]
|
||||
DAILY[Daily.co Service<br/>External]
|
||||
end
|
||||
|
||||
subgraph "AWS S3 Buckets"
|
||||
TRANSCRIPT_BUCKET[Transcript Bucket<br/>reflector-transcripts<br/>Output: Processed MP3s]
|
||||
WHEREBY_BUCKET[Whereby Bucket<br/>reflector-whereby-recordings<br/>Input: Raw MP4s]
|
||||
DAILY_BUCKET[Daily.co Bucket<br/>reflector-dailyco-recordings<br/>Input: Raw WebM tracks]
|
||||
end
|
||||
|
||||
subgraph "AWS Infrastructure"
|
||||
SQS[SQS Queue<br/>Whereby notifications]
|
||||
end
|
||||
|
||||
subgraph "Database"
|
||||
DB[(PostgreSQL<br/>Recordings, Transcripts, Meetings)]
|
||||
end
|
||||
|
||||
APP -->|Write processed| TRANSCRIPT_BUCKET
|
||||
APP -->|Read/Delete| WHEREBY_BUCKET
|
||||
APP -->|Read/Delete| DAILY_BUCKET
|
||||
APP -->|Poll| SQS
|
||||
APP -->|Store metadata| DB
|
||||
|
||||
WHEREBY -->|Write recordings| WHEREBY_BUCKET
|
||||
WHEREBY_BUCKET -->|S3 Event| SQS
|
||||
WHEREBY -->|Participant webhooks<br/>room.client.joined/left| APP
|
||||
|
||||
DAILY -->|Write recordings| DAILY_BUCKET
|
||||
DAILY -->|Recording webhook<br/>recording.ready-to-download| APP
|
||||
```
|
||||
|
||||
**Note on Webhook vs S3 Event for Recording Processing:**
|
||||
- **Whereby**: Uses S3 Events → SQS for recording availability (S3 as source of truth, no race conditions)
|
||||
- **Daily.co**: Uses webhooks for recording availability (more immediate, built-in reliability)
|
||||
- **Both**: Use webhooks for participant tracking (real-time updates)
|
||||
|
||||
## Credentials & Permissions
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
subgraph "Master Credentials"
|
||||
MASTER[TRANSCRIPT_STORAGE_AWS_*<br/>Access Key ID + Secret]
|
||||
end
|
||||
|
||||
subgraph "Whereby Upload Credentials"
|
||||
WHEREBY_CREDS[AWS_WHEREBY_ACCESS_KEY_*<br/>Access Key ID + Secret]
|
||||
end
|
||||
|
||||
subgraph "Daily.co Upload Role"
|
||||
DAILY_ROLE[DAILY_STORAGE_AWS_ROLE_ARN<br/>IAM Role ARN]
|
||||
end
|
||||
|
||||
subgraph "Our App Uses"
|
||||
MASTER -->|Read/Write/Delete| TRANSCRIPT_BUCKET[Transcript Bucket]
|
||||
MASTER -->|Read/Delete| WHEREBY_BUCKET[Whereby Bucket]
|
||||
MASTER -->|Read/Delete| DAILY_BUCKET[Daily.co Bucket]
|
||||
MASTER -->|Poll/Delete| SQS[SQS Queue]
|
||||
end
|
||||
|
||||
subgraph "We Give To Services"
|
||||
WHEREBY_CREDS -->|Passed in API call| WHEREBY_SERVICE[Whereby Service]
|
||||
WHEREBY_SERVICE -->|Write Only| WHEREBY_BUCKET
|
||||
|
||||
DAILY_ROLE -->|Passed in API call| DAILY_SERVICE[Daily.co Service]
|
||||
DAILY_SERVICE -->|Assume Role| DAILY_ROLE
|
||||
DAILY_SERVICE -->|Write Only| DAILY_BUCKET
|
||||
end
|
||||
```
|
||||
|
||||
# Video Platform Recording Integration
|
||||
|
||||
This document explains how Reflector receives and identifies multitrack audio recordings from different video platforms.
|
||||
|
||||
## Platform Comparison
|
||||
|
||||
| Platform | Delivery Method | Track Identification |
|
||||
|----------|----------------|---------------------|
|
||||
| **Daily.co** | Webhook | Explicit track list in payload |
|
||||
| **Whereby** | SQS (S3 notifications) | Single file per notification |
|
||||
|
||||
---
|
||||
|
||||
## Daily.co (Webhook-based)
|
||||
|
||||
Daily.co uses **webhooks** to notify Reflector when recordings are ready.
|
||||
|
||||
### How It Works
|
||||
|
||||
1. **Daily.co sends webhook** when recording is ready
|
||||
- Event type: `recording.ready-to-download`
|
||||
- Endpoint: `/v1/daily/webhook` (`reflector/views/daily.py:46-102`)
|
||||
|
||||
2. **Webhook payload explicitly includes track list**:
|
||||
```json
|
||||
{
|
||||
"recording_id": "7443ee0a-dab1-40eb-b316-33d6c0d5ff88",
|
||||
"room_name": "daily-20251020193458",
|
||||
"tracks": [
|
||||
{
|
||||
"type": "audio",
|
||||
"s3Key": "monadical/daily-20251020193458/1760988935484-52f7f48b-fbab-431f-9a50-87b9abfc8255-cam-audio-1760988935922",
|
||||
"size": 831843
|
||||
},
|
||||
{
|
||||
"type": "audio",
|
||||
"s3Key": "monadical/daily-20251020193458/1760988935484-a37c35e3-6f8e-4274-a482-e9d0f102a732-cam-audio-1760988943823",
|
||||
"size": 408438
|
||||
},
|
||||
{
|
||||
"type": "video",
|
||||
"s3Key": "monadical/daily-20251020193458/...-video.webm",
|
||||
"size": 30000000
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
3. **System extracts audio tracks** (`daily.py:211`):
|
||||
```python
|
||||
track_keys = [t.s3Key for t in tracks if t.type == "audio"]
|
||||
```
|
||||
|
||||
4. **Triggers multitrack processing** (`daily.py:213-218`):
|
||||
```python
|
||||
process_multitrack_recording.delay(
|
||||
bucket_name=bucket_name, # reflector-dailyco-local
|
||||
room_name=room_name, # daily-20251020193458
|
||||
recording_id=recording_id, # 7443ee0a-dab1-40eb-b316-33d6c0d5ff88
|
||||
track_keys=track_keys # Only audio s3Keys
|
||||
)
|
||||
```
|
||||
|
||||
### Key Advantage: No Ambiguity
|
||||
|
||||
Even though multiple meetings may share the same S3 bucket/folder (`monadical/`), **there's no ambiguity** because:
|
||||
- Each webhook payload contains the exact `s3Key` list for that specific `recording_id`
|
||||
- No need to scan folders or guess which files belong together
|
||||
- Each track's s3Key includes the room timestamp subfolder (e.g., `daily-20251020193458/`)
|
||||
|
||||
The room name includes timestamp (`daily-20251020193458`) to keep recordings organized, but **the webhook's explicit track list is what prevents mixing files from different meetings**.
|
||||
|
||||
### Track Timeline Extraction
|
||||
|
||||
Daily.co provides timing information in two places:
|
||||
|
||||
**1. PyAV WebM Metadata (current approach)**:
|
||||
```python
|
||||
# Read from WebM container stream metadata
|
||||
stream.start_time = 8.130s # Meeting-relative timing
|
||||
```
|
||||
|
||||
**2. Filename Timestamps (alternative approach, commit 3bae9076)**:
|
||||
```
|
||||
Filename format: {recording_start_ts}-{uuid}-cam-audio-{track_start_ts}.webm
|
||||
Example: 1760988935484-52f7f48b-fbab-431f-9a50-87b9abfc8255-cam-audio-1760988935922.webm
|
||||
|
||||
Parse timestamps:
|
||||
- recording_start_ts: 1760988935484 (Unix ms)
|
||||
- track_start_ts: 1760988935922 (Unix ms)
|
||||
- offset: (1760988935922 - 1760988935484) / 1000 = 0.438s
|
||||
```
|
||||
|
||||
**Time Difference (PyAV vs Filename)**:
|
||||
```
|
||||
Track 0:
|
||||
Filename offset: 438ms
|
||||
PyAV metadata: 229ms
|
||||
Difference: 209ms
|
||||
|
||||
Track 1:
|
||||
Filename offset: 8339ms
|
||||
PyAV metadata: 8130ms
|
||||
Difference: 209ms
|
||||
```
|
||||
|
||||
**Consistent 209ms delta** suggests network/encoding delay between file upload initiation (filename) and actual audio stream start (metadata).
|
||||
|
||||
**Current implementation uses PyAV metadata** because:
|
||||
- More accurate (represents when audio actually started)
|
||||
- Padding BEFORE transcription produces correct Whisper timestamps automatically
|
||||
- No manual offset adjustment needed during transcript merge
|
||||
|
||||
### Why Re-encoding During Padding
|
||||
|
||||
Padding coincidentally involves re-encoding, which is important for Daily.co + Whisper:
|
||||
|
||||
**Problem:** Daily.co skips frames in recordings when microphone is muted or paused
|
||||
- WebM containers have gaps where audio frames should be
|
||||
- Whisper doesn't understand these gaps and produces incorrect timestamps
|
||||
- Example: 5s of audio with 2s muted → file has frames only for 3s, Whisper thinks duration is 3s
|
||||
|
||||
**Solution:** Re-encoding via PyAV filter graph (`adelay` + `aresample`)
|
||||
- Restores missing frames as silence
|
||||
- Produces continuous audio stream without gaps
|
||||
- Whisper now sees correct duration and produces accurate timestamps
|
||||
|
||||
**Why combined with padding:**
|
||||
- Already re-encoding for padding (adding initial silence)
|
||||
- More performant to do both operations in single PyAV pipeline
|
||||
- Padded values needed for mixdown anyway (creating final MP3)
|
||||
|
||||
Implementation: `main_multitrack_pipeline.py:_apply_audio_padding_streaming()`
|
||||
|
||||
---
|
||||
|
||||
## Whereby (SQS-based)
|
||||
|
||||
Whereby uses **AWS SQS** (via S3 notifications) to notify Reflector when files are uploaded.
|
||||
|
||||
### How It Works
|
||||
|
||||
1. **Whereby uploads recording** to S3
|
||||
2. **S3 sends notification** to SQS queue (one notification per file)
|
||||
3. **Reflector polls SQS queue** (`worker/process.py:process_messages()`)
|
||||
4. **System processes single file** (`worker/process.py:process_recording()`)
|
||||
|
||||
### Key Difference from Daily.co
|
||||
|
||||
**Whereby (SQS):** System receives S3 notification "file X was created" - only knows about one file at a time, would need to scan folder to find related files
|
||||
|
||||
**Daily.co (Webhook):** Daily explicitly tells system which files belong together in the webhook payload
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -79,19 +79,22 @@ DIARIZATION_URL=https://monadical-sas--reflector-diarizer-web.modal.run
|
||||
## Whereby
|
||||
#WHEREBY_API_KEY=your-whereby-api-key
|
||||
#WHEREBY_WEBHOOK_SECRET=your-whereby-webhook-secret
|
||||
#AWS_WHEREBY_ACCESS_KEY_ID=your-aws-key
|
||||
#AWS_WHEREBY_ACCESS_KEY_SECRET=your-aws-secret
|
||||
#WHEREBY_STORAGE_AWS_ACCESS_KEY_ID=your-aws-key
|
||||
#WHEREBY_STORAGE_AWS_SECRET_ACCESS_KEY=your-aws-secret
|
||||
#AWS_PROCESS_RECORDING_QUEUE_URL=https://sqs.us-west-2.amazonaws.com/...
|
||||
|
||||
## Daily.co
|
||||
#DAILY_API_KEY=your-daily-api-key
|
||||
#DAILY_WEBHOOK_SECRET=your-daily-webhook-secret
|
||||
#DAILY_SUBDOMAIN=your-subdomain
|
||||
#AWS_DAILY_S3_BUCKET=your-daily-bucket
|
||||
#AWS_DAILY_S3_REGION=us-west-2
|
||||
#AWS_DAILY_ROLE_ARN=arn:aws:iam::ACCOUNT:role/DailyRecording
|
||||
#DAILY_WEBHOOK_UUID= # Auto-populated by recreate_daily_webhook.py script
|
||||
#DAILYCO_STORAGE_AWS_ROLE_ARN=... # IAM role ARN for Daily.co S3 access
|
||||
#DAILYCO_STORAGE_AWS_BUCKET_NAME=reflector-dailyco
|
||||
#DAILYCO_STORAGE_AWS_REGION=us-west-2
|
||||
|
||||
## Platform Selection
|
||||
#DAILY_MIGRATION_ENABLED=false # Enable Daily.co support
|
||||
#DAILY_MIGRATION_ROOM_IDS=[] # Specific rooms to use Daily
|
||||
## Whereby (optional separate bucket)
|
||||
#WHEREBY_STORAGE_AWS_BUCKET_NAME=reflector-whereby
|
||||
#WHEREBY_STORAGE_AWS_REGION=us-east-1
|
||||
|
||||
## Platform Configuration
|
||||
#DEFAULT_VIDEO_PLATFORM=whereby # Default platform for new rooms
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
"""add_platform_support
|
||||
|
||||
Revision ID: 1e49625677e4
|
||||
Revises: dc035ff72fd5
|
||||
Revises: 9e3f7b2a4c8e
|
||||
Create Date: 2025-10-08 13:17:29.943612
|
||||
|
||||
"""
|
||||
@@ -13,7 +13,7 @@ from alembic import op
|
||||
|
||||
# revision identifiers, used by Alembic.
|
||||
revision: str = "1e49625677e4"
|
||||
down_revision: Union[str, None] = "dc035ff72fd5"
|
||||
down_revision: Union[str, None] = "9e3f7b2a4c8e"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
@@ -25,8 +25,8 @@ def upgrade() -> None:
|
||||
sa.Column(
|
||||
"platform",
|
||||
sa.String(),
|
||||
nullable=False,
|
||||
server_default="whereby",
|
||||
nullable=True,
|
||||
server_default=None,
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
38
server/migrations/versions/9e3f7b2a4c8e_add_user_api_keys.py
Normal file
38
server/migrations/versions/9e3f7b2a4c8e_add_user_api_keys.py
Normal file
@@ -0,0 +1,38 @@
|
||||
"""add user api keys
|
||||
|
||||
Revision ID: 9e3f7b2a4c8e
|
||||
Revises: dc035ff72fd5
|
||||
Create Date: 2025-10-17 00:00:00.000000
|
||||
|
||||
"""
|
||||
|
||||
from typing import Sequence, Union
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
|
||||
# revision identifiers, used by Alembic.
|
||||
revision: str = "9e3f7b2a4c8e"
|
||||
down_revision: Union[str, None] = "dc035ff72fd5"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.create_table(
|
||||
"user_api_key",
|
||||
sa.Column("id", sa.String(), nullable=False),
|
||||
sa.Column("user_id", sa.String(), nullable=False),
|
||||
sa.Column("key_hash", sa.String(), nullable=False),
|
||||
sa.Column("name", sa.String(), nullable=True),
|
||||
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.PrimaryKeyConstraint("id"),
|
||||
)
|
||||
|
||||
with op.batch_alter_table("user_api_key", schema=None) as batch_op:
|
||||
batch_op.create_index("idx_user_api_key_hash", ["key_hash"], unique=True)
|
||||
batch_op.create_index("idx_user_api_key_user_id", ["user_id"], unique=False)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_table("user_api_key")
|
||||
28
server/migrations/versions/f8294b31f022_add_track_keys.py
Normal file
28
server/migrations/versions/f8294b31f022_add_track_keys.py
Normal file
@@ -0,0 +1,28 @@
|
||||
"""add_track_keys
|
||||
|
||||
Revision ID: f8294b31f022
|
||||
Revises: 1e49625677e4
|
||||
Create Date: 2025-10-27 18:52:17.589167
|
||||
|
||||
"""
|
||||
|
||||
from typing import Sequence, Union
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
|
||||
# revision identifiers, used by Alembic.
|
||||
revision: str = "f8294b31f022"
|
||||
down_revision: Union[str, None] = "1e49625677e4"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
with op.batch_alter_table("recording", schema=None) as batch_op:
|
||||
batch_op.add_column(sa.Column("track_keys", sa.JSON(), nullable=True))
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
with op.batch_alter_table("recording", schema=None) as batch_op:
|
||||
batch_op.drop_column("track_keys")
|
||||
@@ -27,6 +27,7 @@ from reflector.views.transcripts_upload import router as transcripts_upload_rout
|
||||
from reflector.views.transcripts_webrtc import router as transcripts_webrtc_router
|
||||
from reflector.views.transcripts_websocket import router as transcripts_websocket_router
|
||||
from reflector.views.user import router as user_router
|
||||
from reflector.views.user_api_keys import router as user_api_keys_router
|
||||
from reflector.views.user_websocket import router as user_ws_router
|
||||
from reflector.views.whereby import router as whereby_router
|
||||
from reflector.views.zulip import router as zulip_router
|
||||
@@ -92,6 +93,7 @@ app.include_router(transcripts_websocket_router, prefix="/v1")
|
||||
app.include_router(transcripts_webrtc_router, prefix="/v1")
|
||||
app.include_router(transcripts_process_router, prefix="/v1")
|
||||
app.include_router(user_router, prefix="/v1")
|
||||
app.include_router(user_api_keys_router, prefix="/v1")
|
||||
app.include_router(user_ws_router, prefix="/v1")
|
||||
app.include_router(zulip_router, prefix="/v1")
|
||||
app.include_router(whereby_router, prefix="/v1")
|
||||
|
||||
@@ -1,14 +1,16 @@
|
||||
from typing import Annotated, Optional
|
||||
from typing import Annotated, List, Optional
|
||||
|
||||
from fastapi import Depends, HTTPException
|
||||
from fastapi.security import OAuth2PasswordBearer
|
||||
from fastapi.security import APIKeyHeader, OAuth2PasswordBearer
|
||||
from jose import JWTError, jwt
|
||||
from pydantic import BaseModel
|
||||
|
||||
from reflector.db.user_api_keys import user_api_keys_controller
|
||||
from reflector.logger import logger
|
||||
from reflector.settings import settings
|
||||
|
||||
oauth2_scheme = OAuth2PasswordBearer(tokenUrl="token", auto_error=False)
|
||||
api_key_header = APIKeyHeader(name="X-API-Key", auto_error=False)
|
||||
|
||||
jwt_public_key = open(f"reflector/auth/jwt/keys/{settings.AUTH_JWT_PUBLIC_KEY}").read()
|
||||
jwt_algorithm = settings.AUTH_JWT_ALGORITHM
|
||||
@@ -26,7 +28,7 @@ class JWTException(Exception):
|
||||
|
||||
class UserInfo(BaseModel):
|
||||
sub: str
|
||||
email: str
|
||||
email: Optional[str] = None
|
||||
|
||||
def __getitem__(self, key):
|
||||
return getattr(self, key)
|
||||
@@ -58,34 +60,53 @@ def authenticated(token: Annotated[str, Depends(oauth2_scheme)]):
|
||||
return None
|
||||
|
||||
|
||||
def current_user(
|
||||
token: Annotated[Optional[str], Depends(oauth2_scheme)],
|
||||
jwtauth: JWTAuth = Depends(),
|
||||
):
|
||||
if token is None:
|
||||
raise HTTPException(status_code=401, detail="Not authenticated")
|
||||
try:
|
||||
payload = jwtauth.verify_token(token)
|
||||
sub = payload["sub"]
|
||||
email = payload["email"]
|
||||
return UserInfo(sub=sub, email=email)
|
||||
except JWTError as e:
|
||||
logger.error(f"JWT error: {e}")
|
||||
raise HTTPException(status_code=401, detail="Invalid authentication")
|
||||
async def _authenticate_user(
|
||||
jwt_token: Optional[str],
|
||||
api_key: Optional[str],
|
||||
jwtauth: JWTAuth,
|
||||
) -> UserInfo | None:
|
||||
user_infos: List[UserInfo] = []
|
||||
if api_key:
|
||||
user_api_key = await user_api_keys_controller.verify_key(api_key)
|
||||
if user_api_key:
|
||||
user_infos.append(UserInfo(sub=user_api_key.user_id, email=None))
|
||||
|
||||
if jwt_token:
|
||||
try:
|
||||
payload = jwtauth.verify_token(jwt_token)
|
||||
sub = payload["sub"]
|
||||
email = payload["email"]
|
||||
user_infos.append(UserInfo(sub=sub, email=email))
|
||||
except JWTError as e:
|
||||
logger.error(f"JWT error: {e}")
|
||||
raise HTTPException(status_code=401, detail="Invalid authentication")
|
||||
|
||||
def current_user_optional(
|
||||
token: Annotated[Optional[str], Depends(oauth2_scheme)],
|
||||
jwtauth: JWTAuth = Depends(),
|
||||
):
|
||||
# we accept no token, but if one is provided, it must be a valid one.
|
||||
if token is None:
|
||||
if len(user_infos) == 0:
|
||||
return None
|
||||
try:
|
||||
payload = jwtauth.verify_token(token)
|
||||
sub = payload["sub"]
|
||||
email = payload["email"]
|
||||
return UserInfo(sub=sub, email=email)
|
||||
except JWTError as e:
|
||||
logger.error(f"JWT error: {e}")
|
||||
raise HTTPException(status_code=401, detail="Invalid authentication")
|
||||
|
||||
if len(set([x.sub for x in user_infos])) > 1:
|
||||
raise JWTException(
|
||||
status_code=401,
|
||||
detail="Invalid authentication: more than one user provided",
|
||||
)
|
||||
|
||||
return user_infos[0]
|
||||
|
||||
|
||||
async def current_user(
|
||||
jwt_token: Annotated[Optional[str], Depends(oauth2_scheme)],
|
||||
api_key: Annotated[Optional[str], Depends(api_key_header)],
|
||||
jwtauth: JWTAuth = Depends(),
|
||||
):
|
||||
user = await _authenticate_user(jwt_token, api_key, jwtauth)
|
||||
if user is None:
|
||||
raise HTTPException(status_code=401, detail="Not authenticated")
|
||||
return user
|
||||
|
||||
|
||||
async def current_user_optional(
|
||||
jwt_token: Annotated[Optional[str], Depends(oauth2_scheme)],
|
||||
api_key: Annotated[Optional[str], Depends(api_key_header)],
|
||||
jwtauth: JWTAuth = Depends(),
|
||||
):
|
||||
return await _authenticate_user(jwt_token, api_key, jwtauth)
|
||||
|
||||
@@ -29,6 +29,7 @@ import reflector.db.meetings # noqa
|
||||
import reflector.db.recordings # noqa
|
||||
import reflector.db.rooms # noqa
|
||||
import reflector.db.transcripts # noqa
|
||||
import reflector.db.user_api_keys # noqa
|
||||
|
||||
kwargs = {}
|
||||
if "postgres" not in settings.DATABASE_URL:
|
||||
|
||||
@@ -7,8 +7,10 @@ from sqlalchemy.dialects.postgresql import JSONB
|
||||
|
||||
from reflector.db import get_database, metadata
|
||||
from reflector.db.rooms import Room
|
||||
from reflector.platform_types import Platform
|
||||
from reflector.schemas.platform import WHEREBY_PLATFORM, Platform
|
||||
from reflector.utils import generate_uuid4
|
||||
from reflector.utils.string import assert_equal
|
||||
from reflector.video_platforms.factory import get_platform
|
||||
|
||||
meetings = sa.Table(
|
||||
"meeting",
|
||||
@@ -60,7 +62,7 @@ meetings = sa.Table(
|
||||
"platform",
|
||||
sa.String,
|
||||
nullable=False,
|
||||
server_default="whereby",
|
||||
server_default=assert_equal(WHEREBY_PLATFORM, "whereby"),
|
||||
),
|
||||
sa.Index("idx_meeting_room_id", "room_id"),
|
||||
sa.Index("idx_meeting_calendar_event", "calendar_event_id"),
|
||||
@@ -108,7 +110,7 @@ class Meeting(BaseModel):
|
||||
is_active: bool = True
|
||||
calendar_event_id: str | None = None
|
||||
calendar_metadata: dict[str, Any] | None = None
|
||||
platform: Platform = "whereby"
|
||||
platform: Platform = WHEREBY_PLATFORM
|
||||
|
||||
|
||||
class MeetingController:
|
||||
@@ -123,7 +125,6 @@ class MeetingController:
|
||||
room: Room,
|
||||
calendar_event_id: str | None = None,
|
||||
calendar_metadata: dict[str, Any] | None = None,
|
||||
platform: Platform = "whereby",
|
||||
):
|
||||
meeting = Meeting(
|
||||
id=id,
|
||||
@@ -139,7 +140,7 @@ class MeetingController:
|
||||
recording_trigger=room.recording_trigger,
|
||||
calendar_event_id=calendar_event_id,
|
||||
calendar_metadata=calendar_metadata,
|
||||
platform=platform,
|
||||
platform=get_platform(room.platform),
|
||||
)
|
||||
query = meetings.insert().values(**meeting.model_dump())
|
||||
await get_database().execute(query)
|
||||
@@ -147,7 +148,8 @@ class MeetingController:
|
||||
|
||||
async def get_all_active(self) -> list[Meeting]:
|
||||
query = meetings.select().where(meetings.c.is_active)
|
||||
return await get_database().fetch_all(query)
|
||||
results = await get_database().fetch_all(query)
|
||||
return [Meeting(**result) for result in results]
|
||||
|
||||
async def get_by_room_name(
|
||||
self,
|
||||
@@ -157,16 +159,14 @@ class MeetingController:
|
||||
Get a meeting by room name.
|
||||
For backward compatibility, returns the most recent meeting.
|
||||
"""
|
||||
end_date = getattr(meetings.c, "end_date")
|
||||
query = (
|
||||
meetings.select()
|
||||
.where(meetings.c.room_name == room_name)
|
||||
.order_by(end_date.desc())
|
||||
.order_by(meetings.c.end_date.desc())
|
||||
)
|
||||
result = await get_database().fetch_one(query)
|
||||
if not result:
|
||||
return None
|
||||
|
||||
return Meeting(**result)
|
||||
|
||||
async def get_active(self, room: Room, current_time: datetime) -> Meeting | None:
|
||||
@@ -189,7 +189,6 @@ class MeetingController:
|
||||
result = await get_database().fetch_one(query)
|
||||
if not result:
|
||||
return None
|
||||
|
||||
return Meeting(**result)
|
||||
|
||||
async def get_all_active_for_room(
|
||||
@@ -229,17 +228,27 @@ class MeetingController:
|
||||
return None
|
||||
return Meeting(**result)
|
||||
|
||||
async def get_by_id(self, meeting_id: str, **kwargs) -> Meeting | None:
|
||||
async def get_by_id(
|
||||
self, meeting_id: str, room: Room | None = None
|
||||
) -> Meeting | None:
|
||||
query = meetings.select().where(meetings.c.id == meeting_id)
|
||||
|
||||
if room:
|
||||
query = query.where(meetings.c.room_id == room.id)
|
||||
|
||||
result = await get_database().fetch_one(query)
|
||||
if not result:
|
||||
return None
|
||||
return Meeting(**result)
|
||||
|
||||
async def get_by_calendar_event(self, calendar_event_id: str) -> Meeting | None:
|
||||
async def get_by_calendar_event(
|
||||
self, calendar_event_id: str, room: Room
|
||||
) -> Meeting | None:
|
||||
query = meetings.select().where(
|
||||
meetings.c.calendar_event_id == calendar_event_id
|
||||
)
|
||||
if room:
|
||||
query = query.where(meetings.c.room_id == room.id)
|
||||
result = await get_database().fetch_one(query)
|
||||
if not result:
|
||||
return None
|
||||
@@ -249,7 +258,7 @@ class MeetingController:
|
||||
query = meetings.update().where(meetings.c.id == meeting_id).values(**kwargs)
|
||||
await get_database().execute(query)
|
||||
|
||||
async def increment_num_clients(self, meeting_id: str):
|
||||
async def increment_num_clients(self, meeting_id: str) -> None:
|
||||
"""Atomically increment participant count."""
|
||||
query = (
|
||||
meetings.update()
|
||||
@@ -258,7 +267,7 @@ class MeetingController:
|
||||
)
|
||||
await get_database().execute(query)
|
||||
|
||||
async def decrement_num_clients(self, meeting_id: str):
|
||||
async def decrement_num_clients(self, meeting_id: str) -> None:
|
||||
"""Atomically decrement participant count (min 0)."""
|
||||
query = (
|
||||
meetings.update()
|
||||
|
||||
@@ -21,6 +21,7 @@ recordings = sa.Table(
|
||||
server_default="pending",
|
||||
),
|
||||
sa.Column("meeting_id", sa.String),
|
||||
sa.Column("track_keys", sa.JSON, nullable=True),
|
||||
sa.Index("idx_recording_meeting_id", "meeting_id"),
|
||||
)
|
||||
|
||||
@@ -28,10 +29,13 @@ recordings = sa.Table(
|
||||
class Recording(BaseModel):
|
||||
id: str = Field(default_factory=generate_uuid4)
|
||||
bucket_name: str
|
||||
# for single-track
|
||||
object_key: str
|
||||
recorded_at: datetime
|
||||
status: Literal["pending", "processing", "completed", "failed"] = "pending"
|
||||
meeting_id: str | None = None
|
||||
# for multitrack reprocessing
|
||||
track_keys: list[str] | None = None
|
||||
|
||||
|
||||
class RecordingController:
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import secrets
|
||||
from datetime import datetime, timezone
|
||||
from sqlite3 import IntegrityError
|
||||
from typing import Literal, Optional
|
||||
from typing import Literal
|
||||
|
||||
import sqlalchemy
|
||||
from fastapi import HTTPException
|
||||
@@ -9,7 +9,7 @@ from pydantic import BaseModel, Field
|
||||
from sqlalchemy.sql import false, or_
|
||||
|
||||
from reflector.db import get_database, metadata
|
||||
from reflector.platform_types import Platform
|
||||
from reflector.schemas.platform import Platform
|
||||
from reflector.utils import generate_uuid4
|
||||
|
||||
rooms = sqlalchemy.Table(
|
||||
@@ -54,8 +54,8 @@ rooms = sqlalchemy.Table(
|
||||
sqlalchemy.Column(
|
||||
"platform",
|
||||
sqlalchemy.String,
|
||||
nullable=False,
|
||||
server_default="whereby",
|
||||
nullable=True,
|
||||
server_default=None,
|
||||
),
|
||||
sqlalchemy.Index("idx_room_is_shared", "is_shared"),
|
||||
sqlalchemy.Index("idx_room_ics_enabled", "ics_enabled"),
|
||||
@@ -84,7 +84,7 @@ class Room(BaseModel):
|
||||
ics_enabled: bool = False
|
||||
ics_last_sync: datetime | None = None
|
||||
ics_last_etag: str | None = None
|
||||
platform: Platform = "whereby"
|
||||
platform: Platform | None = None
|
||||
|
||||
|
||||
class RoomController:
|
||||
@@ -138,7 +138,7 @@ class RoomController:
|
||||
ics_url: str | None = None,
|
||||
ics_fetch_interval: int = 300,
|
||||
ics_enabled: bool = False,
|
||||
platform: Optional[Platform] = None,
|
||||
platform: Platform | None = None,
|
||||
):
|
||||
"""
|
||||
Add a new room
|
||||
@@ -162,7 +162,7 @@ class RoomController:
|
||||
ics_url=ics_url,
|
||||
ics_fetch_interval=ics_fetch_interval,
|
||||
ics_enabled=ics_enabled,
|
||||
platform=platform or "whereby",
|
||||
platform=platform,
|
||||
)
|
||||
query = rooms.insert().values(**room.model_dump())
|
||||
try:
|
||||
|
||||
@@ -135,6 +135,8 @@ class SearchParameters(BaseModel):
|
||||
user_id: str | None = None
|
||||
room_id: str | None = None
|
||||
source_kind: SourceKind | None = None
|
||||
from_datetime: datetime | None = None
|
||||
to_datetime: datetime | None = None
|
||||
|
||||
|
||||
class SearchResultDB(BaseModel):
|
||||
@@ -402,6 +404,14 @@ class SearchController:
|
||||
base_query = base_query.where(
|
||||
transcripts.c.source_kind == params.source_kind
|
||||
)
|
||||
if params.from_datetime:
|
||||
base_query = base_query.where(
|
||||
transcripts.c.created_at >= params.from_datetime
|
||||
)
|
||||
if params.to_datetime:
|
||||
base_query = base_query.where(
|
||||
transcripts.c.created_at <= params.to_datetime
|
||||
)
|
||||
|
||||
if params.query_text is not None:
|
||||
order_by = sqlalchemy.desc(sqlalchemy.text("rank"))
|
||||
|
||||
@@ -21,7 +21,7 @@ from reflector.db.utils import is_postgresql
|
||||
from reflector.logger import logger
|
||||
from reflector.processors.types import Word as ProcessorWord
|
||||
from reflector.settings import settings
|
||||
from reflector.storage import get_recordings_storage, get_transcripts_storage
|
||||
from reflector.storage import get_transcripts_storage
|
||||
from reflector.utils import generate_uuid4
|
||||
from reflector.utils.webvtt import topics_to_webvtt
|
||||
|
||||
@@ -186,6 +186,7 @@ class TranscriptParticipant(BaseModel):
|
||||
id: str = Field(default_factory=generate_uuid4)
|
||||
speaker: int | None
|
||||
name: str
|
||||
user_id: str | None = None
|
||||
|
||||
|
||||
class Transcript(BaseModel):
|
||||
@@ -623,7 +624,9 @@ class TranscriptController:
|
||||
)
|
||||
if recording:
|
||||
try:
|
||||
await get_recordings_storage().delete_file(recording.object_key)
|
||||
await get_transcripts_storage().delete_file(
|
||||
recording.object_key, bucket=recording.bucket_name
|
||||
)
|
||||
except Exception as e:
|
||||
logger.warning(
|
||||
"Failed to delete recording object from S3",
|
||||
@@ -725,11 +728,13 @@ class TranscriptController:
|
||||
"""
|
||||
Download audio from storage
|
||||
"""
|
||||
transcript.audio_mp3_filename.write_bytes(
|
||||
await get_transcripts_storage().get_file(
|
||||
transcript.storage_audio_path,
|
||||
)
|
||||
)
|
||||
storage = get_transcripts_storage()
|
||||
try:
|
||||
with open(transcript.audio_mp3_filename, "wb") as f:
|
||||
await storage.stream_to_fileobj(transcript.storage_audio_path, f)
|
||||
except Exception:
|
||||
transcript.audio_mp3_filename.unlink(missing_ok=True)
|
||||
raise
|
||||
|
||||
async def upsert_participant(
|
||||
self,
|
||||
|
||||
91
server/reflector/db/user_api_keys.py
Normal file
91
server/reflector/db/user_api_keys.py
Normal file
@@ -0,0 +1,91 @@
|
||||
import hmac
|
||||
import secrets
|
||||
from datetime import datetime, timezone
|
||||
from hashlib import sha256
|
||||
|
||||
import sqlalchemy
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from reflector.db import get_database, metadata
|
||||
from reflector.settings import settings
|
||||
from reflector.utils import generate_uuid4
|
||||
from reflector.utils.string import NonEmptyString
|
||||
|
||||
user_api_keys = sqlalchemy.Table(
|
||||
"user_api_key",
|
||||
metadata,
|
||||
sqlalchemy.Column("id", sqlalchemy.String, primary_key=True),
|
||||
sqlalchemy.Column("user_id", sqlalchemy.String, nullable=False),
|
||||
sqlalchemy.Column("key_hash", sqlalchemy.String, nullable=False),
|
||||
sqlalchemy.Column("name", sqlalchemy.String, nullable=True),
|
||||
sqlalchemy.Column("created_at", sqlalchemy.DateTime(timezone=True), nullable=False),
|
||||
sqlalchemy.Index("idx_user_api_key_hash", "key_hash", unique=True),
|
||||
sqlalchemy.Index("idx_user_api_key_user_id", "user_id"),
|
||||
)
|
||||
|
||||
|
||||
class UserApiKey(BaseModel):
|
||||
id: NonEmptyString = Field(default_factory=generate_uuid4)
|
||||
user_id: NonEmptyString
|
||||
key_hash: NonEmptyString
|
||||
name: NonEmptyString | None = None
|
||||
created_at: datetime = Field(default_factory=lambda: datetime.now(timezone.utc))
|
||||
|
||||
|
||||
class UserApiKeyController:
|
||||
@staticmethod
|
||||
def generate_key() -> NonEmptyString:
|
||||
return secrets.token_urlsafe(48)
|
||||
|
||||
@staticmethod
|
||||
def hash_key(key: NonEmptyString) -> str:
|
||||
return hmac.new(
|
||||
settings.SECRET_KEY.encode(), key.encode(), digestmod=sha256
|
||||
).hexdigest()
|
||||
|
||||
@classmethod
|
||||
async def create_key(
|
||||
cls,
|
||||
user_id: NonEmptyString,
|
||||
name: NonEmptyString | None = None,
|
||||
) -> tuple[UserApiKey, NonEmptyString]:
|
||||
plaintext = cls.generate_key()
|
||||
api_key = UserApiKey(
|
||||
user_id=user_id,
|
||||
key_hash=cls.hash_key(plaintext),
|
||||
name=name,
|
||||
)
|
||||
query = user_api_keys.insert().values(**api_key.model_dump())
|
||||
await get_database().execute(query)
|
||||
return api_key, plaintext
|
||||
|
||||
@classmethod
|
||||
async def verify_key(cls, plaintext_key: NonEmptyString) -> UserApiKey | None:
|
||||
key_hash = cls.hash_key(plaintext_key)
|
||||
query = user_api_keys.select().where(
|
||||
user_api_keys.c.key_hash == key_hash,
|
||||
)
|
||||
result = await get_database().fetch_one(query)
|
||||
return UserApiKey(**result) if result else None
|
||||
|
||||
@staticmethod
|
||||
async def list_by_user_id(user_id: NonEmptyString) -> list[UserApiKey]:
|
||||
query = (
|
||||
user_api_keys.select()
|
||||
.where(user_api_keys.c.user_id == user_id)
|
||||
.order_by(user_api_keys.c.created_at.desc())
|
||||
)
|
||||
results = await get_database().fetch_all(query)
|
||||
return [UserApiKey(**r) for r in results]
|
||||
|
||||
@staticmethod
|
||||
async def delete_key(key_id: NonEmptyString, user_id: NonEmptyString) -> bool:
|
||||
query = user_api_keys.delete().where(
|
||||
(user_api_keys.c.id == key_id) & (user_api_keys.c.user_id == user_id)
|
||||
)
|
||||
result = await get_database().execute(query)
|
||||
# asyncpg returns None for DELETE, consider it success if no exception
|
||||
return result is None or result > 0
|
||||
|
||||
|
||||
user_api_keys_controller = UserApiKeyController()
|
||||
@@ -1,84 +0,0 @@
|
||||
# Multitrack Pipeline Fix Summary
|
||||
|
||||
## Problem
|
||||
Whisper timestamps were incorrect because it ignores leading silence in audio files. Daily.co tracks can have arbitrary amounts of silence before speech starts.
|
||||
|
||||
## Solution
|
||||
**Pad tracks BEFORE transcription using stream metadata `start_time`**
|
||||
|
||||
This makes Whisper timestamps automatically correct relative to recording start.
|
||||
|
||||
## Key Changes in `main_multitrack_pipeline_fixed.py`
|
||||
|
||||
### 1. Added `pad_track_for_transcription()` method (lines 55-172)
|
||||
|
||||
```python
|
||||
async def pad_track_for_transcription(
|
||||
self,
|
||||
track_data: bytes,
|
||||
track_idx: int,
|
||||
storage,
|
||||
) -> tuple[bytes, str]:
|
||||
```
|
||||
|
||||
- Extracts stream metadata `start_time` using PyAV
|
||||
- Creates PyAV filter graph with `adelay` filter to add padding
|
||||
- Stores padded track to S3 and returns URL
|
||||
- Uses same audio processing library (PyAV) already in the pipeline
|
||||
|
||||
### 2. Modified `process()` method
|
||||
|
||||
#### REMOVED (lines 255-302):
|
||||
- Entire filename parsing for offsets - NOT NEEDED ANYMORE
|
||||
- The complex regex parsing of Daily.co filenames
|
||||
- Offset adjustment after transcription
|
||||
|
||||
#### ADDED (lines 371-382):
|
||||
- Padding step BEFORE transcription:
|
||||
```python
|
||||
# PAD TRACKS BEFORE TRANSCRIPTION - THIS IS THE KEY FIX!
|
||||
padded_track_urls: list[str] = []
|
||||
for idx, data in enumerate(track_datas):
|
||||
if not data:
|
||||
padded_track_urls.append("")
|
||||
continue
|
||||
|
||||
_, padded_url = await self.pad_track_for_transcription(
|
||||
data, idx, storage
|
||||
)
|
||||
padded_track_urls.append(padded_url)
|
||||
```
|
||||
|
||||
#### MODIFIED (lines 385-435):
|
||||
- Transcribe PADDED tracks instead of raw tracks
|
||||
- Removed all timestamp offset adjustment code
|
||||
- Just set speaker ID - timestamps already correct!
|
||||
|
||||
```python
|
||||
# NO OFFSET ADJUSTMENT NEEDED!
|
||||
# Timestamps are already correct because we transcribed padded tracks
|
||||
# Just set speaker ID
|
||||
for w in t.words:
|
||||
w.speaker = idx
|
||||
```
|
||||
|
||||
## Why This Works
|
||||
|
||||
1. **Stream metadata is authoritative**: Daily.co sets `start_time` in the WebM container
|
||||
2. **PyAV respects metadata**: `audio_stream.start_time * audio_stream.time_base` gives seconds
|
||||
3. **Padding before transcription**: Whisper sees continuous audio from time 0
|
||||
4. **Automatic alignment**: Word at 51s in padded track = 51s in recording
|
||||
|
||||
## Testing
|
||||
|
||||
Process the test recording (daily-20251020193458) and verify:
|
||||
- Participant 0 words appear at ~2s
|
||||
- Participant 1 words appear at ~51s
|
||||
- No word interleaving
|
||||
- Correct chronological order
|
||||
|
||||
## Files
|
||||
|
||||
- **Original**: `main_multitrack_pipeline.py`
|
||||
- **Fixed**: `main_multitrack_pipeline_fixed.py`
|
||||
- **Test data**: `/Users/firfi/work/clients/monadical/reflector/1760988935484-*.webm`
|
||||
1
server/reflector/pipelines/__init__.py
Normal file
1
server/reflector/pipelines/__init__.py
Normal file
@@ -0,0 +1 @@
|
||||
"""Pipeline modules for audio processing."""
|
||||
@@ -23,23 +23,18 @@ from reflector.db.transcripts import (
|
||||
transcripts_controller,
|
||||
)
|
||||
from reflector.logger import logger
|
||||
from reflector.pipelines import topic_processing
|
||||
from reflector.pipelines.main_live_pipeline import (
|
||||
PipelineMainBase,
|
||||
broadcast_to_sockets,
|
||||
task_cleanup_consent,
|
||||
task_pipeline_post_to_zulip,
|
||||
)
|
||||
from reflector.processors import (
|
||||
AudioFileWriterProcessor,
|
||||
TranscriptFinalSummaryProcessor,
|
||||
TranscriptFinalTitleProcessor,
|
||||
TranscriptTopicDetectorProcessor,
|
||||
)
|
||||
from reflector.pipelines.transcription_helpers import transcribe_file_with_processor
|
||||
from reflector.processors import AudioFileWriterProcessor
|
||||
from reflector.processors.audio_waveform_processor import AudioWaveformProcessor
|
||||
from reflector.processors.file_diarization import FileDiarizationInput
|
||||
from reflector.processors.file_diarization_auto import FileDiarizationAutoProcessor
|
||||
from reflector.processors.file_transcript import FileTranscriptInput
|
||||
from reflector.processors.file_transcript_auto import FileTranscriptAutoProcessor
|
||||
from reflector.processors.transcript_diarization_assembler import (
|
||||
TranscriptDiarizationAssemblerInput,
|
||||
TranscriptDiarizationAssemblerProcessor,
|
||||
@@ -56,19 +51,6 @@ from reflector.storage import get_transcripts_storage
|
||||
from reflector.worker.webhook import send_transcript_webhook
|
||||
|
||||
|
||||
class EmptyPipeline:
|
||||
"""Empty pipeline for processors that need a pipeline reference"""
|
||||
|
||||
def __init__(self, logger: structlog.BoundLogger):
|
||||
self.logger = logger
|
||||
|
||||
def get_pref(self, k, d=None):
|
||||
return d
|
||||
|
||||
async def emit(self, event):
|
||||
pass
|
||||
|
||||
|
||||
class PipelineMainFile(PipelineMainBase):
|
||||
"""
|
||||
Optimized file processing pipeline.
|
||||
@@ -81,7 +63,7 @@ class PipelineMainFile(PipelineMainBase):
|
||||
def __init__(self, transcript_id: str):
|
||||
super().__init__(transcript_id=transcript_id)
|
||||
self.logger = logger.bind(transcript_id=self.transcript_id)
|
||||
self.empty_pipeline = EmptyPipeline(logger=self.logger)
|
||||
self.empty_pipeline = topic_processing.EmptyPipeline(logger=self.logger)
|
||||
|
||||
def _handle_gather_exceptions(self, results: list, operation: str) -> None:
|
||||
"""Handle exceptions from asyncio.gather with return_exceptions=True"""
|
||||
@@ -262,24 +244,7 @@ class PipelineMainFile(PipelineMainBase):
|
||||
|
||||
async def transcribe_file(self, audio_url: str, language: str) -> TranscriptType:
|
||||
"""Transcribe complete file"""
|
||||
processor = FileTranscriptAutoProcessor()
|
||||
input_data = FileTranscriptInput(audio_url=audio_url, language=language)
|
||||
|
||||
# Store result for retrieval
|
||||
result: TranscriptType | None = None
|
||||
|
||||
async def capture_result(transcript):
|
||||
nonlocal result
|
||||
result = transcript
|
||||
|
||||
processor.on(capture_result)
|
||||
await processor.push(input_data)
|
||||
await processor.flush()
|
||||
|
||||
if not result:
|
||||
raise ValueError("No transcript captured")
|
||||
|
||||
return result
|
||||
return await transcribe_file_with_processor(audio_url, language)
|
||||
|
||||
async def diarize_file(self, audio_url: str) -> list[DiarizationSegment] | None:
|
||||
"""Get diarization for file"""
|
||||
@@ -322,63 +287,31 @@ class PipelineMainFile(PipelineMainBase):
|
||||
async def detect_topics(
|
||||
self, transcript: TranscriptType, target_language: str
|
||||
) -> list[TitleSummary]:
|
||||
"""Detect topics from complete transcript"""
|
||||
chunk_size = 300
|
||||
topics: list[TitleSummary] = []
|
||||
|
||||
async def on_topic(topic: TitleSummary):
|
||||
topics.append(topic)
|
||||
return await self.on_topic(topic)
|
||||
|
||||
topic_detector = TranscriptTopicDetectorProcessor(callback=on_topic)
|
||||
topic_detector.set_pipeline(self.empty_pipeline)
|
||||
|
||||
for i in range(0, len(transcript.words), chunk_size):
|
||||
chunk_words = transcript.words[i : i + chunk_size]
|
||||
if not chunk_words:
|
||||
continue
|
||||
|
||||
chunk_transcript = TranscriptType(
|
||||
words=chunk_words, translation=transcript.translation
|
||||
)
|
||||
|
||||
await topic_detector.push(chunk_transcript)
|
||||
|
||||
await topic_detector.flush()
|
||||
return topics
|
||||
return await topic_processing.detect_topics(
|
||||
transcript,
|
||||
target_language,
|
||||
on_topic_callback=self.on_topic,
|
||||
empty_pipeline=self.empty_pipeline,
|
||||
)
|
||||
|
||||
async def generate_title(self, topics: list[TitleSummary]):
|
||||
"""Generate title from topics"""
|
||||
if not topics:
|
||||
self.logger.warning("No topics for title generation")
|
||||
return
|
||||
|
||||
processor = TranscriptFinalTitleProcessor(callback=self.on_title)
|
||||
processor.set_pipeline(self.empty_pipeline)
|
||||
|
||||
for topic in topics:
|
||||
await processor.push(topic)
|
||||
|
||||
await processor.flush()
|
||||
return await topic_processing.generate_title(
|
||||
topics,
|
||||
on_title_callback=self.on_title,
|
||||
empty_pipeline=self.empty_pipeline,
|
||||
logger=self.logger,
|
||||
)
|
||||
|
||||
async def generate_summaries(self, topics: list[TitleSummary]):
|
||||
"""Generate long and short summaries from topics"""
|
||||
if not topics:
|
||||
self.logger.warning("No topics for summary generation")
|
||||
return
|
||||
|
||||
transcript = await self.get_transcript()
|
||||
processor = TranscriptFinalSummaryProcessor(
|
||||
transcript=transcript,
|
||||
callback=self.on_long_summary,
|
||||
on_short_summary=self.on_short_summary,
|
||||
return await topic_processing.generate_summaries(
|
||||
topics,
|
||||
transcript,
|
||||
on_long_summary_callback=self.on_long_summary,
|
||||
on_short_summary_callback=self.on_short_summary,
|
||||
empty_pipeline=self.empty_pipeline,
|
||||
logger=self.logger,
|
||||
)
|
||||
processor.set_pipeline(self.empty_pipeline)
|
||||
|
||||
for topic in topics:
|
||||
await processor.push(topic)
|
||||
|
||||
await processor.flush()
|
||||
|
||||
|
||||
@shared_task
|
||||
@@ -426,7 +359,12 @@ async def task_pipeline_file_process(*, transcript_id: str):
|
||||
|
||||
await pipeline.process(audio_file)
|
||||
|
||||
except Exception:
|
||||
except Exception as e:
|
||||
logger.error(
|
||||
f"File pipeline failed for transcript {transcript_id}: {type(e).__name__}: {str(e)}",
|
||||
exc_info=True,
|
||||
transcript_id=transcript_id,
|
||||
)
|
||||
await pipeline.set_status(transcript_id, "error")
|
||||
raise
|
||||
|
||||
|
||||
@@ -17,7 +17,6 @@ from contextlib import asynccontextmanager
|
||||
from typing import Generic
|
||||
|
||||
import av
|
||||
import boto3
|
||||
from celery import chord, current_task, group, shared_task
|
||||
from pydantic import BaseModel
|
||||
from structlog import BoundLogger as Logger
|
||||
@@ -584,6 +583,7 @@ async def cleanup_consent(transcript: Transcript, logger: Logger):
|
||||
|
||||
consent_denied = False
|
||||
recording = None
|
||||
meeting = None
|
||||
try:
|
||||
if transcript.recording_id:
|
||||
recording = await recordings_controller.get_by_id(transcript.recording_id)
|
||||
@@ -594,8 +594,8 @@ async def cleanup_consent(transcript: Transcript, logger: Logger):
|
||||
meeting.id
|
||||
)
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to get fetch consent: {e}", exc_info=e)
|
||||
consent_denied = True
|
||||
logger.error(f"Failed to fetch consent: {e}", exc_info=e)
|
||||
raise
|
||||
|
||||
if not consent_denied:
|
||||
logger.info("Consent approved, keeping all files")
|
||||
@@ -603,25 +603,24 @@ async def cleanup_consent(transcript: Transcript, logger: Logger):
|
||||
|
||||
logger.info("Consent denied, cleaning up all related audio files")
|
||||
|
||||
if recording and recording.bucket_name and recording.object_key:
|
||||
s3_whereby = boto3.client(
|
||||
"s3",
|
||||
aws_access_key_id=settings.AWS_WHEREBY_ACCESS_KEY_ID,
|
||||
aws_secret_access_key=settings.AWS_WHEREBY_ACCESS_KEY_SECRET,
|
||||
)
|
||||
try:
|
||||
s3_whereby.delete_object(
|
||||
Bucket=recording.bucket_name, Key=recording.object_key
|
||||
)
|
||||
logger.info(
|
||||
f"Deleted original Whereby recording: {recording.bucket_name}/{recording.object_key}"
|
||||
)
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to delete Whereby recording: {e}", exc_info=e)
|
||||
deletion_errors = []
|
||||
if recording and recording.bucket_name:
|
||||
keys_to_delete = []
|
||||
if recording.track_keys:
|
||||
keys_to_delete = recording.track_keys
|
||||
elif recording.object_key:
|
||||
keys_to_delete = [recording.object_key]
|
||||
|
||||
master_storage = get_transcripts_storage()
|
||||
for key in keys_to_delete:
|
||||
try:
|
||||
await master_storage.delete_file(key, bucket=recording.bucket_name)
|
||||
logger.info(f"Deleted recording file: {recording.bucket_name}/{key}")
|
||||
except Exception as e:
|
||||
error_msg = f"Failed to delete {key}: {e}"
|
||||
logger.error(error_msg, exc_info=e)
|
||||
deletion_errors.append(error_msg)
|
||||
|
||||
# non-transactional, files marked for deletion not actually deleted is possible
|
||||
await transcripts_controller.update(transcript, {"audio_deleted": True})
|
||||
# 2. Delete processed audio from transcript storage S3 bucket
|
||||
if transcript.audio_location == "storage":
|
||||
storage = get_transcripts_storage()
|
||||
try:
|
||||
@@ -630,18 +629,28 @@ async def cleanup_consent(transcript: Transcript, logger: Logger):
|
||||
f"Deleted processed audio from storage: {transcript.storage_audio_path}"
|
||||
)
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to delete processed audio: {e}", exc_info=e)
|
||||
error_msg = f"Failed to delete processed audio: {e}"
|
||||
logger.error(error_msg, exc_info=e)
|
||||
deletion_errors.append(error_msg)
|
||||
|
||||
# 3. Delete local audio files
|
||||
try:
|
||||
if hasattr(transcript, "audio_mp3_filename") and transcript.audio_mp3_filename:
|
||||
transcript.audio_mp3_filename.unlink(missing_ok=True)
|
||||
if hasattr(transcript, "audio_wav_filename") and transcript.audio_wav_filename:
|
||||
transcript.audio_wav_filename.unlink(missing_ok=True)
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to delete local audio files: {e}", exc_info=e)
|
||||
error_msg = f"Failed to delete local audio files: {e}"
|
||||
logger.error(error_msg, exc_info=e)
|
||||
deletion_errors.append(error_msg)
|
||||
|
||||
logger.info("Consent cleanup done")
|
||||
if deletion_errors:
|
||||
logger.warning(
|
||||
f"Consent cleanup completed with {len(deletion_errors)} errors",
|
||||
errors=deletion_errors,
|
||||
)
|
||||
else:
|
||||
await transcripts_controller.update(transcript, {"audio_deleted": True})
|
||||
logger.info("Consent cleanup done - all audio deleted")
|
||||
|
||||
|
||||
@get_transcript
|
||||
|
||||
@@ -1,510 +0,0 @@
|
||||
import asyncio
|
||||
import io
|
||||
from fractions import Fraction
|
||||
|
||||
import av
|
||||
import boto3
|
||||
import structlog
|
||||
from av.audio.resampler import AudioResampler
|
||||
from celery import chain, shared_task
|
||||
|
||||
from reflector.asynctask import asynctask
|
||||
from reflector.db.transcripts import (
|
||||
TranscriptStatus,
|
||||
TranscriptText,
|
||||
transcripts_controller,
|
||||
)
|
||||
from reflector.logger import logger
|
||||
from reflector.pipelines.main_file_pipeline import task_send_webhook_if_needed
|
||||
from reflector.pipelines.main_live_pipeline import (
|
||||
PipelineMainBase,
|
||||
task_cleanup_consent,
|
||||
task_pipeline_post_to_zulip,
|
||||
)
|
||||
from reflector.processors import (
|
||||
AudioFileWriterProcessor,
|
||||
TranscriptFinalSummaryProcessor,
|
||||
TranscriptFinalTitleProcessor,
|
||||
TranscriptTopicDetectorProcessor,
|
||||
)
|
||||
from reflector.processors.file_transcript import FileTranscriptInput
|
||||
from reflector.processors.file_transcript_auto import FileTranscriptAutoProcessor
|
||||
from reflector.processors.types import TitleSummary
|
||||
from reflector.processors.types import (
|
||||
Transcript as TranscriptType,
|
||||
)
|
||||
from reflector.settings import settings
|
||||
from reflector.storage import get_transcripts_storage
|
||||
|
||||
|
||||
class EmptyPipeline:
|
||||
def __init__(self, logger: structlog.BoundLogger):
|
||||
self.logger = logger
|
||||
|
||||
def get_pref(self, k, d=None):
|
||||
return d
|
||||
|
||||
async def emit(self, event):
|
||||
pass
|
||||
|
||||
|
||||
class PipelineMainMultitrack(PipelineMainBase):
|
||||
"""Process multiple participant tracks for a transcript without mixing audio."""
|
||||
|
||||
def __init__(self, transcript_id: str):
|
||||
super().__init__(transcript_id=transcript_id)
|
||||
self.logger = logger.bind(transcript_id=self.transcript_id)
|
||||
self.empty_pipeline = EmptyPipeline(logger=self.logger)
|
||||
|
||||
async def mixdown_tracks(
|
||||
self,
|
||||
track_datas: list[bytes],
|
||||
writer: AudioFileWriterProcessor,
|
||||
offsets_seconds: list[float] | None = None,
|
||||
) -> None:
|
||||
"""
|
||||
Minimal multi-track mixdown using a PyAV filter graph (amix), no resampling.
|
||||
"""
|
||||
|
||||
# Discover target sample rate from first decodable frame
|
||||
target_sample_rate: int | None = None
|
||||
for data in track_datas:
|
||||
if not data:
|
||||
continue
|
||||
try:
|
||||
container = av.open(io.BytesIO(data))
|
||||
try:
|
||||
for frame in container.decode(audio=0):
|
||||
target_sample_rate = frame.sample_rate
|
||||
break
|
||||
finally:
|
||||
container.close()
|
||||
except Exception:
|
||||
continue
|
||||
if target_sample_rate:
|
||||
break
|
||||
|
||||
if not target_sample_rate:
|
||||
self.logger.warning("Mixdown skipped - no decodable audio frames found")
|
||||
return
|
||||
|
||||
# Build PyAV filter graph:
|
||||
# N abuffer (s32/stereo)
|
||||
# -> optional adelay per input (for alignment)
|
||||
# -> amix (s32)
|
||||
# -> aformat(s16)
|
||||
# -> sink
|
||||
graph = av.filter.Graph()
|
||||
inputs = []
|
||||
valid_track_datas = [d for d in track_datas if d]
|
||||
# Align offsets list with the filtered inputs (skip empties)
|
||||
input_offsets_seconds = None
|
||||
if offsets_seconds is not None:
|
||||
input_offsets_seconds = [
|
||||
offsets_seconds[i] for i, d in enumerate(track_datas) if d
|
||||
]
|
||||
for idx, data in enumerate(valid_track_datas):
|
||||
args = (
|
||||
f"time_base=1/{target_sample_rate}:"
|
||||
f"sample_rate={target_sample_rate}:"
|
||||
f"sample_fmt=s32:"
|
||||
f"channel_layout=stereo"
|
||||
)
|
||||
in_ctx = graph.add("abuffer", args=args, name=f"in{idx}")
|
||||
inputs.append(in_ctx)
|
||||
|
||||
if not inputs:
|
||||
self.logger.warning("Mixdown skipped - no valid inputs for graph")
|
||||
return
|
||||
|
||||
mixer = graph.add("amix", args=f"inputs={len(inputs)}:normalize=0", name="mix")
|
||||
|
||||
fmt = graph.add(
|
||||
"aformat",
|
||||
args=(
|
||||
f"sample_fmts=s32:channel_layouts=stereo:sample_rates={target_sample_rate}"
|
||||
),
|
||||
name="fmt",
|
||||
)
|
||||
|
||||
sink = graph.add("abuffersink", name="out")
|
||||
|
||||
# Optional per-input delay before mixing
|
||||
delays_ms: list[int] = []
|
||||
if input_offsets_seconds is not None:
|
||||
base = min(input_offsets_seconds) if input_offsets_seconds else 0.0
|
||||
delays_ms = [
|
||||
max(0, int(round((o - base) * 1000))) for o in input_offsets_seconds
|
||||
]
|
||||
else:
|
||||
delays_ms = [0 for _ in inputs]
|
||||
|
||||
for idx, in_ctx in enumerate(inputs):
|
||||
delay_ms = delays_ms[idx] if idx < len(delays_ms) else 0
|
||||
if delay_ms > 0:
|
||||
# adelay requires one value per channel; use same for stereo
|
||||
adelay = graph.add(
|
||||
"adelay",
|
||||
args=f"delays={delay_ms}|{delay_ms}:all=1",
|
||||
name=f"delay{idx}",
|
||||
)
|
||||
in_ctx.link_to(adelay)
|
||||
adelay.link_to(mixer, 0, idx)
|
||||
else:
|
||||
in_ctx.link_to(mixer, 0, idx)
|
||||
mixer.link_to(fmt)
|
||||
fmt.link_to(sink)
|
||||
graph.configure()
|
||||
|
||||
# Open containers for decoding
|
||||
containers = []
|
||||
for i, d in enumerate(valid_track_datas):
|
||||
try:
|
||||
c = av.open(io.BytesIO(d))
|
||||
containers.append(c)
|
||||
except Exception as e:
|
||||
self.logger.warning(
|
||||
"Mixdown: failed to open container", input=i, error=str(e)
|
||||
)
|
||||
containers.append(None)
|
||||
# Filter out Nones for decoders
|
||||
containers = [c for c in containers if c is not None]
|
||||
decoders = [c.decode(audio=0) for c in containers]
|
||||
active = [True] * len(decoders)
|
||||
# Per-input resamplers to enforce s32/stereo at the same rate (no resample of rate)
|
||||
resamplers = [
|
||||
AudioResampler(format="s32", layout="stereo", rate=target_sample_rate)
|
||||
for _ in decoders
|
||||
]
|
||||
|
||||
try:
|
||||
# Round-robin feed frames into graph, pull mixed frames as they become available
|
||||
while any(active):
|
||||
for i, (dec, is_active) in enumerate(zip(decoders, active)):
|
||||
if not is_active:
|
||||
continue
|
||||
try:
|
||||
frame = next(dec)
|
||||
except StopIteration:
|
||||
active[i] = False
|
||||
continue
|
||||
|
||||
# Enforce same sample rate; convert format/layout to s16/stereo (no resample)
|
||||
if frame.sample_rate != target_sample_rate:
|
||||
# Skip frames with differing rate
|
||||
continue
|
||||
out_frames = resamplers[i].resample(frame) or []
|
||||
for rf in out_frames:
|
||||
rf.sample_rate = target_sample_rate
|
||||
rf.time_base = Fraction(1, target_sample_rate)
|
||||
inputs[i].push(rf)
|
||||
|
||||
# Drain available mixed frames
|
||||
while True:
|
||||
try:
|
||||
mixed = sink.pull()
|
||||
except Exception:
|
||||
break
|
||||
mixed.sample_rate = target_sample_rate
|
||||
mixed.time_base = Fraction(1, target_sample_rate)
|
||||
await writer.push(mixed)
|
||||
|
||||
# Signal EOF to inputs and drain remaining
|
||||
for in_ctx in inputs:
|
||||
in_ctx.push(None)
|
||||
while True:
|
||||
try:
|
||||
mixed = sink.pull()
|
||||
except Exception:
|
||||
break
|
||||
mixed.sample_rate = target_sample_rate
|
||||
mixed.time_base = Fraction(1, target_sample_rate)
|
||||
await writer.push(mixed)
|
||||
finally:
|
||||
for c in containers:
|
||||
c.close()
|
||||
|
||||
async def set_status(self, transcript_id: str, status: TranscriptStatus):
|
||||
async with self.lock_transaction():
|
||||
return await transcripts_controller.set_status(transcript_id, status)
|
||||
|
||||
async def process(self, bucket_name: str, track_keys: list[str]):
|
||||
transcript = await self.get_transcript()
|
||||
|
||||
s3 = boto3.client(
|
||||
"s3",
|
||||
region_name=settings.RECORDING_STORAGE_AWS_REGION,
|
||||
aws_access_key_id=settings.RECORDING_STORAGE_AWS_ACCESS_KEY_ID,
|
||||
aws_secret_access_key=settings.RECORDING_STORAGE_AWS_SECRET_ACCESS_KEY,
|
||||
)
|
||||
|
||||
storage = get_transcripts_storage()
|
||||
|
||||
# Pre-download bytes for all tracks for mixing and transcription
|
||||
track_datas: list[bytes] = []
|
||||
for key in track_keys:
|
||||
try:
|
||||
obj = s3.get_object(Bucket=bucket_name, Key=key)
|
||||
track_datas.append(obj["Body"].read())
|
||||
except Exception as e:
|
||||
self.logger.warning(
|
||||
"Skipping track - cannot read S3 object", key=key, error=str(e)
|
||||
)
|
||||
track_datas.append(b"")
|
||||
|
||||
# Extract offsets from Daily.co filename timestamps
|
||||
# Format: {rec_start_ts}-{uuid}-{media_type}-{track_start_ts}.{ext}
|
||||
# Example: 1760988935484-uuid-cam-audio-1760988935922
|
||||
import re
|
||||
|
||||
offsets_seconds: list[float] = []
|
||||
recording_start_ts: int | None = None
|
||||
|
||||
for key in track_keys:
|
||||
# Parse Daily.co raw-tracks filename pattern
|
||||
match = re.search(r"(\d+)-([0-9a-f-]{36})-(cam-audio)-(\d+)", key)
|
||||
if not match:
|
||||
self.logger.warning(
|
||||
"Track key doesn't match Daily.co pattern, using 0.0 offset",
|
||||
key=key,
|
||||
)
|
||||
offsets_seconds.append(0.0)
|
||||
continue
|
||||
|
||||
rec_start_ts = int(match.group(1))
|
||||
track_start_ts = int(match.group(4))
|
||||
|
||||
# Validate all tracks belong to same recording
|
||||
if recording_start_ts is None:
|
||||
recording_start_ts = rec_start_ts
|
||||
elif rec_start_ts != recording_start_ts:
|
||||
self.logger.error(
|
||||
"Track belongs to different recording",
|
||||
key=key,
|
||||
expected_start=recording_start_ts,
|
||||
got_start=rec_start_ts,
|
||||
)
|
||||
offsets_seconds.append(0.0)
|
||||
continue
|
||||
|
||||
# Calculate offset in seconds
|
||||
offset_ms = track_start_ts - rec_start_ts
|
||||
offset_s = offset_ms / 1000.0
|
||||
|
||||
self.logger.info(
|
||||
"Parsed track offset from filename",
|
||||
key=key,
|
||||
recording_start=rec_start_ts,
|
||||
track_start=track_start_ts,
|
||||
offset_seconds=offset_s,
|
||||
)
|
||||
|
||||
offsets_seconds.append(max(0.0, offset_s))
|
||||
|
||||
# Mixdown all available tracks into transcript.audio_mp3_filename, preserving sample rate
|
||||
try:
|
||||
mp3_writer = AudioFileWriterProcessor(
|
||||
path=str(transcript.audio_mp3_filename)
|
||||
)
|
||||
await self.mixdown_tracks(track_datas, mp3_writer, offsets_seconds)
|
||||
await mp3_writer.flush()
|
||||
except Exception as e:
|
||||
self.logger.error("Mixdown failed", error=str(e))
|
||||
|
||||
speaker_transcripts: list[TranscriptType] = []
|
||||
for idx, key in enumerate(track_keys):
|
||||
ext = ".mp4"
|
||||
|
||||
try:
|
||||
obj = s3.get_object(Bucket=bucket_name, Key=key)
|
||||
data = obj["Body"].read()
|
||||
except Exception as e:
|
||||
self.logger.error(
|
||||
"Skipping track - cannot read S3 object", key=key, error=str(e)
|
||||
)
|
||||
continue
|
||||
|
||||
storage_path = f"file_pipeline/{transcript.id}/tracks/track_{idx}{ext}"
|
||||
try:
|
||||
await storage.put_file(storage_path, data)
|
||||
audio_url = await storage.get_file_url(storage_path)
|
||||
except Exception as e:
|
||||
self.logger.error(
|
||||
"Skipping track - cannot upload to storage", key=key, error=str(e)
|
||||
)
|
||||
continue
|
||||
|
||||
try:
|
||||
t = await self.transcribe_file(audio_url, transcript.source_language)
|
||||
except Exception as e:
|
||||
self.logger.error(
|
||||
"Transcription via default backend failed, trying local whisper",
|
||||
key=key,
|
||||
url=audio_url,
|
||||
error=str(e),
|
||||
)
|
||||
try:
|
||||
fallback = FileTranscriptAutoProcessor(name="whisper")
|
||||
result = None
|
||||
|
||||
async def capture_result(r):
|
||||
nonlocal result
|
||||
result = r
|
||||
|
||||
fallback.on(capture_result)
|
||||
await fallback.push(
|
||||
FileTranscriptInput(
|
||||
audio_url=audio_url, language=transcript.source_language
|
||||
)
|
||||
)
|
||||
await fallback.flush()
|
||||
if not result:
|
||||
raise Exception("No transcript captured in fallback")
|
||||
t = result
|
||||
except Exception as e2:
|
||||
self.logger.error(
|
||||
"Skipping track - transcription failed after fallback",
|
||||
key=key,
|
||||
url=audio_url,
|
||||
error=str(e2),
|
||||
)
|
||||
continue
|
||||
|
||||
if not t.words:
|
||||
continue
|
||||
# Shift word timestamps by the track's offset so all are relative to 00:00
|
||||
track_offset = offsets_seconds[idx] if idx < len(offsets_seconds) else 0.0
|
||||
for w in t.words:
|
||||
try:
|
||||
if hasattr(w, "start") and w.start is not None:
|
||||
w.start = float(w.start) + track_offset
|
||||
if hasattr(w, "end") and w.end is not None:
|
||||
w.end = float(w.end) + track_offset
|
||||
except Exception:
|
||||
pass
|
||||
w.speaker = idx
|
||||
speaker_transcripts.append(t)
|
||||
|
||||
if not speaker_transcripts:
|
||||
raise Exception("No valid track transcriptions")
|
||||
|
||||
merged_words = []
|
||||
for t in speaker_transcripts:
|
||||
merged_words.extend(t.words)
|
||||
merged_words.sort(key=lambda w: w.start)
|
||||
|
||||
merged_transcript = TranscriptType(words=merged_words, translation=None)
|
||||
|
||||
await transcripts_controller.append_event(
|
||||
transcript,
|
||||
event="TRANSCRIPT",
|
||||
data=TranscriptText(
|
||||
text=merged_transcript.text, translation=merged_transcript.translation
|
||||
),
|
||||
)
|
||||
|
||||
topics = await self.detect_topics(merged_transcript, transcript.target_language)
|
||||
await asyncio.gather(
|
||||
self.generate_title(topics),
|
||||
self.generate_summaries(topics),
|
||||
return_exceptions=False,
|
||||
)
|
||||
|
||||
await self.set_status(transcript.id, "ended")
|
||||
|
||||
async def transcribe_file(self, audio_url: str, language: str) -> TranscriptType:
|
||||
processor = FileTranscriptAutoProcessor()
|
||||
input_data = FileTranscriptInput(audio_url=audio_url, language=language)
|
||||
|
||||
result: TranscriptType | None = None
|
||||
|
||||
async def capture_result(transcript):
|
||||
nonlocal result
|
||||
result = transcript
|
||||
|
||||
processor.on(capture_result)
|
||||
await processor.push(input_data)
|
||||
await processor.flush()
|
||||
|
||||
if not result:
|
||||
raise ValueError("No transcript captured")
|
||||
|
||||
return result
|
||||
|
||||
async def detect_topics(
|
||||
self, transcript: TranscriptType, target_language: str
|
||||
) -> list[TitleSummary]:
|
||||
chunk_size = 300
|
||||
topics: list[TitleSummary] = []
|
||||
|
||||
async def on_topic(topic: TitleSummary):
|
||||
topics.append(topic)
|
||||
return await self.on_topic(topic)
|
||||
|
||||
topic_detector = TranscriptTopicDetectorProcessor(callback=on_topic)
|
||||
topic_detector.set_pipeline(self.empty_pipeline)
|
||||
|
||||
for i in range(0, len(transcript.words), chunk_size):
|
||||
chunk_words = transcript.words[i : i + chunk_size]
|
||||
if not chunk_words:
|
||||
continue
|
||||
|
||||
chunk_transcript = TranscriptType(
|
||||
words=chunk_words, translation=transcript.translation
|
||||
)
|
||||
await topic_detector.push(chunk_transcript)
|
||||
|
||||
await topic_detector.flush()
|
||||
return topics
|
||||
|
||||
async def generate_title(self, topics: list[TitleSummary]):
|
||||
if not topics:
|
||||
self.logger.warning("No topics for title generation")
|
||||
return
|
||||
|
||||
processor = TranscriptFinalTitleProcessor(callback=self.on_title)
|
||||
processor.set_pipeline(self.empty_pipeline)
|
||||
|
||||
for topic in topics:
|
||||
await processor.push(topic)
|
||||
|
||||
await processor.flush()
|
||||
|
||||
async def generate_summaries(self, topics: list[TitleSummary]):
|
||||
if not topics:
|
||||
self.logger.warning("No topics for summary generation")
|
||||
return
|
||||
|
||||
transcript = await self.get_transcript()
|
||||
processor = TranscriptFinalSummaryProcessor(
|
||||
transcript=transcript,
|
||||
callback=self.on_long_summary,
|
||||
on_short_summary=self.on_short_summary,
|
||||
)
|
||||
processor.set_pipeline(self.empty_pipeline)
|
||||
|
||||
for topic in topics:
|
||||
await processor.push(topic)
|
||||
|
||||
await processor.flush()
|
||||
|
||||
|
||||
@shared_task
|
||||
@asynctask
|
||||
async def task_pipeline_multitrack_process(
|
||||
*, transcript_id: str, bucket_name: str, track_keys: list[str]
|
||||
):
|
||||
pipeline = PipelineMainMultitrack(transcript_id=transcript_id)
|
||||
try:
|
||||
await pipeline.set_status(transcript_id, "processing")
|
||||
await pipeline.process(bucket_name, track_keys)
|
||||
except Exception:
|
||||
await pipeline.set_status(transcript_id, "error")
|
||||
raise
|
||||
|
||||
post_chain = chain(
|
||||
task_cleanup_consent.si(transcript_id=transcript_id),
|
||||
task_pipeline_post_to_zulip.si(transcript_id=transcript_id),
|
||||
task_send_webhook_if_needed.si(transcript_id=transcript_id),
|
||||
)
|
||||
post_chain.delay()
|
||||
@@ -1,10 +1,10 @@
|
||||
import asyncio
|
||||
import io
|
||||
import math
|
||||
import tempfile
|
||||
from fractions import Fraction
|
||||
from pathlib import Path
|
||||
|
||||
import av
|
||||
import boto3
|
||||
import structlog
|
||||
from av.audio.resampler import AudioResampler
|
||||
from celery import chain, shared_task
|
||||
|
||||
@@ -15,6 +15,7 @@ from reflector.db.transcripts import (
|
||||
transcripts_controller,
|
||||
)
|
||||
from reflector.logger import logger
|
||||
from reflector.pipelines import topic_processing
|
||||
from reflector.pipelines.main_file_pipeline import task_send_webhook_if_needed
|
||||
from reflector.pipelines.main_live_pipeline import (
|
||||
PipelineMainBase,
|
||||
@@ -22,213 +23,325 @@ from reflector.pipelines.main_live_pipeline import (
|
||||
task_cleanup_consent,
|
||||
task_pipeline_post_to_zulip,
|
||||
)
|
||||
from reflector.processors import (
|
||||
AudioFileWriterProcessor,
|
||||
TranscriptFinalSummaryProcessor,
|
||||
TranscriptFinalTitleProcessor,
|
||||
TranscriptTopicDetectorProcessor,
|
||||
)
|
||||
from reflector.pipelines.transcription_helpers import transcribe_file_with_processor
|
||||
from reflector.processors import AudioFileWriterProcessor
|
||||
from reflector.processors.audio_waveform_processor import AudioWaveformProcessor
|
||||
from reflector.processors.file_transcript import FileTranscriptInput
|
||||
from reflector.processors.file_transcript_auto import FileTranscriptAutoProcessor
|
||||
from reflector.processors.types import TitleSummary
|
||||
from reflector.processors.types import (
|
||||
Transcript as TranscriptType,
|
||||
)
|
||||
from reflector.settings import settings
|
||||
from reflector.storage import get_transcripts_storage
|
||||
from reflector.processors.types import Transcript as TranscriptType
|
||||
from reflector.storage import Storage, get_transcripts_storage
|
||||
from reflector.utils.string import NonEmptyString
|
||||
|
||||
# Audio encoding constants
|
||||
OPUS_STANDARD_SAMPLE_RATE = 48000
|
||||
OPUS_DEFAULT_BIT_RATE = 128000
|
||||
|
||||
class EmptyPipeline:
|
||||
def __init__(self, logger: structlog.BoundLogger):
|
||||
self.logger = logger
|
||||
|
||||
def get_pref(self, k, d=None):
|
||||
return d
|
||||
|
||||
async def emit(self, event):
|
||||
pass
|
||||
# Storage operation constants
|
||||
PRESIGNED_URL_EXPIRATION_SECONDS = 7200 # 2 hours
|
||||
|
||||
|
||||
class PipelineMainMultitrack(PipelineMainBase):
|
||||
"""Process multiple participant tracks for a transcript without mixing audio."""
|
||||
|
||||
def __init__(self, transcript_id: str):
|
||||
super().__init__(transcript_id=transcript_id)
|
||||
self.logger = logger.bind(transcript_id=self.transcript_id)
|
||||
self.empty_pipeline = EmptyPipeline(logger=self.logger)
|
||||
self.empty_pipeline = topic_processing.EmptyPipeline(logger=self.logger)
|
||||
|
||||
async def pad_track_for_transcription(
|
||||
self,
|
||||
track_data: bytes,
|
||||
track_url: NonEmptyString,
|
||||
track_idx: int,
|
||||
storage,
|
||||
) -> tuple[bytes, str]:
|
||||
storage: Storage,
|
||||
) -> NonEmptyString:
|
||||
"""
|
||||
Pad a single track with silence based on stream metadata start_time.
|
||||
This ensures Whisper timestamps will be relative to recording start.
|
||||
Uses ffmpeg subprocess approach proven to work with python-raw-tracks-align.
|
||||
Downloads from S3 presigned URL, processes via PyAV using tempfile, uploads to S3.
|
||||
Returns presigned URL of padded track (or original URL if no padding needed).
|
||||
|
||||
Returns: (padded_data, storage_url)
|
||||
Memory usage:
|
||||
- Pattern: fixed_overhead(2-5MB) for PyAV codec/filters
|
||||
- PyAV streams input efficiently (no full download, verified)
|
||||
- Output written to tempfile (disk-based, not memory)
|
||||
- Upload streams from file handle (boto3 chunks, typically 5-10MB)
|
||||
|
||||
Daily.co raw-tracks timing - Two approaches:
|
||||
|
||||
CURRENT APPROACH (PyAV metadata):
|
||||
The WebM stream.start_time field encodes MEETING-RELATIVE timing:
|
||||
- t=0: When Daily.co recording started (first participant joined)
|
||||
- start_time=8.13s: This participant's track began 8.13s after recording started
|
||||
- Purpose: Enables track alignment without external manifest files
|
||||
|
||||
This is NOT:
|
||||
- Stream-internal offset (first packet timestamp relative to stream start)
|
||||
- Absolute/wall-clock time
|
||||
- Recording duration
|
||||
|
||||
ALTERNATIVE APPROACH (filename parsing):
|
||||
Daily.co filenames contain Unix timestamps (milliseconds):
|
||||
Format: {recording_start_ts}-{participant_id}-cam-audio-{track_start_ts}.webm
|
||||
Example: 1760988935484-52f7f48b-fbab-431f-9a50-87b9abfc8255-cam-audio-1760988935922.webm
|
||||
|
||||
Can calculate offset: (track_start_ts - recording_start_ts) / 1000
|
||||
- Track 0: (1760988935922 - 1760988935484) / 1000 = 0.438s
|
||||
- Track 1: (1760988943823 - 1760988935484) / 1000 = 8.339s
|
||||
|
||||
TIME DIFFERENCE: PyAV metadata vs filename timestamps differ by ~209ms:
|
||||
- Track 0: filename=438ms, metadata=229ms (diff: 209ms)
|
||||
- Track 1: filename=8339ms, metadata=8130ms (diff: 209ms)
|
||||
|
||||
Consistent delta suggests network/encoding delay. PyAV metadata is ground truth
|
||||
(represents when audio stream actually started vs when file upload initiated).
|
||||
|
||||
Example with 2 participants:
|
||||
Track A: start_time=0.2s → Joined 200ms after recording began
|
||||
Track B: start_time=8.1s → Joined 8.1 seconds later
|
||||
|
||||
After padding:
|
||||
Track A: [0.2s silence] + [speech...]
|
||||
Track B: [8.1s silence] + [speech...]
|
||||
|
||||
Whisper transcription timestamps are now synchronized:
|
||||
Track A word at 5.0s → happened at meeting t=5.0s
|
||||
Track B word at 10.0s → happened at meeting t=10.0s
|
||||
|
||||
Merging just sorts by timestamp - no offset calculation needed.
|
||||
|
||||
Padding coincidentally involves re-encoding. It's important when we work with Daily.co + Whisper.
|
||||
This is because Daily.co returns recordings with skipped frames e.g. when microphone muted.
|
||||
Daily.co doesn't understand those frames and ignores them, causing timestamp issues in transcription.
|
||||
Re-encoding restores those frames. We do padding and re-encoding together just because it's convenient and more performant:
|
||||
we need padded values for mix mp3 anyways
|
||||
"""
|
||||
import json
|
||||
import math
|
||||
import subprocess
|
||||
import tempfile
|
||||
|
||||
if not track_data:
|
||||
return b"", ""
|
||||
|
||||
transcript = await self.get_transcript()
|
||||
|
||||
# Create temp files for ffmpeg processing
|
||||
with tempfile.NamedTemporaryFile(suffix=".webm", delete=False) as input_file:
|
||||
input_file.write(track_data)
|
||||
input_file_path = input_file.name
|
||||
try:
|
||||
# PyAV streams input from S3 URL efficiently (2-5MB fixed overhead for codec/filters)
|
||||
with av.open(track_url) as in_container:
|
||||
start_time_seconds = self._extract_stream_start_time_from_container(
|
||||
in_container, track_idx
|
||||
)
|
||||
|
||||
output_file_path = input_file_path.replace(".webm", "_padded.webm")
|
||||
if start_time_seconds <= 0:
|
||||
self.logger.info(
|
||||
f"Track {track_idx} requires no padding (start_time={start_time_seconds}s)",
|
||||
track_idx=track_idx,
|
||||
)
|
||||
return track_url
|
||||
|
||||
# Use tempfile instead of BytesIO for better memory efficiency
|
||||
# Reduces peak memory usage during encoding/upload
|
||||
with tempfile.NamedTemporaryFile(
|
||||
suffix=".webm", delete=False
|
||||
) as temp_file:
|
||||
temp_path = temp_file.name
|
||||
|
||||
try:
|
||||
self._apply_audio_padding_to_file(
|
||||
in_container, temp_path, start_time_seconds, track_idx
|
||||
)
|
||||
|
||||
storage_path = (
|
||||
f"file_pipeline/{transcript.id}/tracks/padded_{track_idx}.webm"
|
||||
)
|
||||
|
||||
# Upload using file handle for streaming
|
||||
with open(temp_path, "rb") as padded_file:
|
||||
await storage.put_file(storage_path, padded_file)
|
||||
finally:
|
||||
# Clean up temp file
|
||||
Path(temp_path).unlink(missing_ok=True)
|
||||
|
||||
padded_url = await storage.get_file_url(
|
||||
storage_path,
|
||||
operation="get_object",
|
||||
expires_in=PRESIGNED_URL_EXPIRATION_SECONDS,
|
||||
)
|
||||
|
||||
self.logger.info(
|
||||
f"Successfully padded track {track_idx}",
|
||||
track_idx=track_idx,
|
||||
start_time_seconds=start_time_seconds,
|
||||
padded_url=padded_url,
|
||||
)
|
||||
|
||||
return padded_url
|
||||
|
||||
except Exception as e:
|
||||
self.logger.error(
|
||||
f"Failed to process track {track_idx}",
|
||||
track_idx=track_idx,
|
||||
url=track_url,
|
||||
error=str(e),
|
||||
exc_info=True,
|
||||
)
|
||||
raise Exception(
|
||||
f"Track {track_idx} padding failed - transcript would have incorrect timestamps"
|
||||
) from e
|
||||
|
||||
def _extract_stream_start_time_from_container(
|
||||
self, container, track_idx: int
|
||||
) -> float:
|
||||
"""
|
||||
Extract meeting-relative start time from WebM stream metadata.
|
||||
Uses PyAV to read stream.start_time from WebM container.
|
||||
More accurate than filename timestamps by ~209ms due to network/encoding delays.
|
||||
"""
|
||||
start_time_seconds = 0.0
|
||||
try:
|
||||
audio_streams = [s for s in container.streams if s.type == "audio"]
|
||||
stream = audio_streams[0] if audio_streams else container.streams[0]
|
||||
|
||||
# 1) Try stream-level start_time (most reliable for Daily.co tracks)
|
||||
if stream.start_time is not None and stream.time_base is not None:
|
||||
start_time_seconds = float(stream.start_time * stream.time_base)
|
||||
|
||||
# 2) Fallback to container-level start_time (in av.time_base units)
|
||||
if (start_time_seconds <= 0) and (container.start_time is not None):
|
||||
start_time_seconds = float(container.start_time * av.time_base)
|
||||
|
||||
# 3) Fallback to first packet DTS in stream.time_base
|
||||
if start_time_seconds <= 0:
|
||||
for packet in container.demux(stream):
|
||||
if packet.dts is not None:
|
||||
start_time_seconds = float(packet.dts * stream.time_base)
|
||||
break
|
||||
except Exception as e:
|
||||
self.logger.warning(
|
||||
"PyAV metadata read failed; assuming 0 start_time",
|
||||
track_idx=track_idx,
|
||||
error=str(e),
|
||||
)
|
||||
start_time_seconds = 0.0
|
||||
|
||||
self.logger.info(
|
||||
f"Track {track_idx} stream metadata: start_time={start_time_seconds:.3f}s",
|
||||
track_idx=track_idx,
|
||||
)
|
||||
return start_time_seconds
|
||||
|
||||
def _apply_audio_padding_to_file(
|
||||
self,
|
||||
in_container,
|
||||
output_path: str,
|
||||
start_time_seconds: float,
|
||||
track_idx: int,
|
||||
) -> None:
|
||||
"""Apply silence padding to audio track using PyAV filter graph, writing to file"""
|
||||
delay_ms = math.floor(start_time_seconds * 1000)
|
||||
|
||||
self.logger.info(
|
||||
f"Padding track {track_idx} with {delay_ms}ms delay using PyAV",
|
||||
track_idx=track_idx,
|
||||
delay_ms=delay_ms,
|
||||
)
|
||||
|
||||
try:
|
||||
# Get stream metadata using ffprobe
|
||||
ffprobe_cmd = [
|
||||
"ffprobe",
|
||||
"-v",
|
||||
"error",
|
||||
"-show_entries",
|
||||
"stream=start_time",
|
||||
"-of",
|
||||
"json",
|
||||
input_file_path,
|
||||
]
|
||||
|
||||
result = subprocess.run(
|
||||
ffprobe_cmd, capture_output=True, text=True, check=True
|
||||
)
|
||||
metadata = json.loads(result.stdout)
|
||||
|
||||
# Extract start_time from stream metadata
|
||||
start_time_seconds = 0.0
|
||||
if metadata.get("streams") and len(metadata["streams"]) > 0:
|
||||
start_time_str = metadata["streams"][0].get("start_time", "0")
|
||||
start_time_seconds = float(start_time_str)
|
||||
|
||||
self.logger.info(
|
||||
f"Track {track_idx} stream metadata: start_time={start_time_seconds:.3f}s",
|
||||
track_idx=track_idx,
|
||||
)
|
||||
|
||||
# If no padding needed, use original
|
||||
if start_time_seconds <= 0:
|
||||
storage_path = f"file_pipeline/{transcript.id}/tracks/original_track_{track_idx}.webm"
|
||||
await storage.put_file(storage_path, track_data)
|
||||
url = await storage.get_file_url(storage_path)
|
||||
return track_data, url
|
||||
|
||||
# Calculate delay in milliseconds
|
||||
delay_ms = math.floor(start_time_seconds * 1000)
|
||||
|
||||
# Run ffmpeg to pad the audio while maintaining WebM/Opus format for Modal compatibility
|
||||
# ffmpeg quirk: aresample needs to come before adelay in the filter chain
|
||||
ffmpeg_cmd = [
|
||||
"ffmpeg",
|
||||
"-hide_banner",
|
||||
"-loglevel",
|
||||
"error",
|
||||
"-y", # overwrite output
|
||||
"-i",
|
||||
input_file_path,
|
||||
"-af",
|
||||
f"aresample=async=1,adelay={delay_ms}:all=true",
|
||||
"-c:a",
|
||||
"libopus", # Keep Opus codec for Modal compatibility
|
||||
"-b:a",
|
||||
"128k", # Standard bitrate for Opus
|
||||
output_file_path,
|
||||
]
|
||||
|
||||
self.logger.info(
|
||||
f"Padding track {track_idx} with {delay_ms}ms delay using ffmpeg",
|
||||
track_idx=track_idx,
|
||||
delay_ms=delay_ms,
|
||||
command=" ".join(ffmpeg_cmd),
|
||||
)
|
||||
|
||||
result = subprocess.run(ffmpeg_cmd, capture_output=True, text=True)
|
||||
if result.returncode != 0:
|
||||
self.logger.error(
|
||||
f"ffmpeg padding failed for track {track_idx}",
|
||||
track_idx=track_idx,
|
||||
stderr=result.stderr,
|
||||
returncode=result.returncode,
|
||||
with av.open(output_path, "w", format="webm") as out_container:
|
||||
in_stream = next(
|
||||
(s for s in in_container.streams if s.type == "audio"), None
|
||||
)
|
||||
raise Exception(f"ffmpeg padding failed: {result.stderr}")
|
||||
if in_stream is None:
|
||||
raise Exception("No audio stream in input")
|
||||
|
||||
# Read the padded output
|
||||
with open(output_file_path, "rb") as f:
|
||||
padded_data = f.read()
|
||||
out_stream = out_container.add_stream(
|
||||
"libopus", rate=OPUS_STANDARD_SAMPLE_RATE
|
||||
)
|
||||
out_stream.bit_rate = OPUS_DEFAULT_BIT_RATE
|
||||
graph = av.filter.Graph()
|
||||
|
||||
# Store padded track
|
||||
storage_path = (
|
||||
f"file_pipeline/{transcript.id}/tracks/padded_track_{track_idx}.webm"
|
||||
)
|
||||
await storage.put_file(storage_path, padded_data)
|
||||
padded_url = await storage.get_file_url(storage_path)
|
||||
abuf_args = (
|
||||
f"time_base=1/{OPUS_STANDARD_SAMPLE_RATE}:"
|
||||
f"sample_rate={OPUS_STANDARD_SAMPLE_RATE}:"
|
||||
f"sample_fmt=s16:"
|
||||
f"channel_layout=stereo"
|
||||
)
|
||||
src = graph.add("abuffer", args=abuf_args, name="src")
|
||||
aresample_f = graph.add("aresample", args="async=1", name="ares")
|
||||
# adelay requires one delay value per channel separated by '|'
|
||||
delays_arg = f"{delay_ms}|{delay_ms}"
|
||||
adelay_f = graph.add(
|
||||
"adelay", args=f"delays={delays_arg}:all=1", name="delay"
|
||||
)
|
||||
sink = graph.add("abuffersink", name="sink")
|
||||
|
||||
self.logger.info(
|
||||
f"Successfully padded track {track_idx} with {start_time_seconds:.3f}s offset, stored at {storage_path}",
|
||||
src.link_to(aresample_f)
|
||||
aresample_f.link_to(adelay_f)
|
||||
adelay_f.link_to(sink)
|
||||
graph.configure()
|
||||
|
||||
resampler = AudioResampler(
|
||||
format="s16", layout="stereo", rate=OPUS_STANDARD_SAMPLE_RATE
|
||||
)
|
||||
# Decode -> resample -> push through graph -> encode Opus
|
||||
for frame in in_container.decode(in_stream):
|
||||
out_frames = resampler.resample(frame) or []
|
||||
for rframe in out_frames:
|
||||
rframe.sample_rate = OPUS_STANDARD_SAMPLE_RATE
|
||||
rframe.time_base = Fraction(1, OPUS_STANDARD_SAMPLE_RATE)
|
||||
src.push(rframe)
|
||||
|
||||
while True:
|
||||
try:
|
||||
f_out = sink.pull()
|
||||
except Exception:
|
||||
break
|
||||
f_out.sample_rate = OPUS_STANDARD_SAMPLE_RATE
|
||||
f_out.time_base = Fraction(1, OPUS_STANDARD_SAMPLE_RATE)
|
||||
for packet in out_stream.encode(f_out):
|
||||
out_container.mux(packet)
|
||||
|
||||
src.push(None)
|
||||
while True:
|
||||
try:
|
||||
f_out = sink.pull()
|
||||
except Exception:
|
||||
break
|
||||
f_out.sample_rate = OPUS_STANDARD_SAMPLE_RATE
|
||||
f_out.time_base = Fraction(1, OPUS_STANDARD_SAMPLE_RATE)
|
||||
for packet in out_stream.encode(f_out):
|
||||
out_container.mux(packet)
|
||||
|
||||
for packet in out_stream.encode(None):
|
||||
out_container.mux(packet)
|
||||
except Exception as e:
|
||||
self.logger.error(
|
||||
"PyAV padding failed for track",
|
||||
track_idx=track_idx,
|
||||
delay_ms=delay_ms,
|
||||
padded_url=padded_url,
|
||||
padded_size=len(padded_data),
|
||||
error=str(e),
|
||||
exc_info=True,
|
||||
)
|
||||
|
||||
return padded_data, padded_url
|
||||
|
||||
finally:
|
||||
# Clean up temp files
|
||||
import os
|
||||
|
||||
try:
|
||||
os.unlink(input_file_path)
|
||||
except:
|
||||
pass
|
||||
try:
|
||||
os.unlink(output_file_path)
|
||||
except:
|
||||
pass
|
||||
raise
|
||||
|
||||
async def mixdown_tracks(
|
||||
self,
|
||||
track_datas: list[bytes],
|
||||
track_urls: list[str],
|
||||
writer: AudioFileWriterProcessor,
|
||||
offsets_seconds: list[float] | None = None,
|
||||
) -> None:
|
||||
"""
|
||||
Minimal multi-track mixdown using a PyAV filter graph (amix), no resampling.
|
||||
"""
|
||||
"""Multi-track mixdown using PyAV filter graph (amix), reading from S3 presigned URLs"""
|
||||
|
||||
# Discover target sample rate from first decodable frame
|
||||
target_sample_rate: int | None = None
|
||||
for data in track_datas:
|
||||
if not data:
|
||||
for url in track_urls:
|
||||
if not url:
|
||||
continue
|
||||
container = None
|
||||
try:
|
||||
container = av.open(io.BytesIO(data))
|
||||
try:
|
||||
for frame in container.decode(audio=0):
|
||||
target_sample_rate = frame.sample_rate
|
||||
break
|
||||
finally:
|
||||
container.close()
|
||||
container = av.open(url)
|
||||
for frame in container.decode(audio=0):
|
||||
target_sample_rate = frame.sample_rate
|
||||
break
|
||||
except Exception:
|
||||
continue
|
||||
finally:
|
||||
if container is not None:
|
||||
container.close()
|
||||
if target_sample_rate:
|
||||
break
|
||||
|
||||
if not target_sample_rate:
|
||||
self.logger.warning("Mixdown skipped - no decodable audio frames found")
|
||||
return
|
||||
|
||||
self.logger.error("Mixdown failed - no decodable audio frames found")
|
||||
raise Exception("Mixdown failed: No decodable audio frames in any track")
|
||||
# Build PyAV filter graph:
|
||||
# N abuffer (s32/stereo)
|
||||
# -> optional adelay per input (for alignment)
|
||||
@@ -237,14 +350,13 @@ class PipelineMainMultitrack(PipelineMainBase):
|
||||
# -> sink
|
||||
graph = av.filter.Graph()
|
||||
inputs = []
|
||||
valid_track_datas = [d for d in track_datas if d]
|
||||
# Align offsets list with the filtered inputs (skip empties)
|
||||
valid_track_urls = [url for url in track_urls if url]
|
||||
input_offsets_seconds = None
|
||||
if offsets_seconds is not None:
|
||||
input_offsets_seconds = [
|
||||
offsets_seconds[i] for i, d in enumerate(track_datas) if d
|
||||
offsets_seconds[i] for i, url in enumerate(track_urls) if url
|
||||
]
|
||||
for idx, data in enumerate(valid_track_datas):
|
||||
for idx, url in enumerate(valid_track_urls):
|
||||
args = (
|
||||
f"time_base=1/{target_sample_rate}:"
|
||||
f"sample_rate={target_sample_rate}:"
|
||||
@@ -255,8 +367,8 @@ class PipelineMainMultitrack(PipelineMainBase):
|
||||
inputs.append(in_ctx)
|
||||
|
||||
if not inputs:
|
||||
self.logger.warning("Mixdown skipped - no valid inputs for graph")
|
||||
return
|
||||
self.logger.error("Mixdown failed - no valid inputs for graph")
|
||||
raise Exception("Mixdown failed: No valid inputs for filter graph")
|
||||
|
||||
mixer = graph.add("amix", args=f"inputs={len(inputs)}:normalize=0", name="mix")
|
||||
|
||||
@@ -297,29 +409,32 @@ class PipelineMainMultitrack(PipelineMainBase):
|
||||
fmt.link_to(sink)
|
||||
graph.configure()
|
||||
|
||||
# Open containers for decoding
|
||||
containers = []
|
||||
for i, d in enumerate(valid_track_datas):
|
||||
try:
|
||||
c = av.open(io.BytesIO(d))
|
||||
containers.append(c)
|
||||
except Exception as e:
|
||||
self.logger.warning(
|
||||
"Mixdown: failed to open container", input=i, error=str(e)
|
||||
)
|
||||
containers.append(None)
|
||||
# Filter out Nones for decoders
|
||||
containers = [c for c in containers if c is not None]
|
||||
decoders = [c.decode(audio=0) for c in containers]
|
||||
active = [True] * len(decoders)
|
||||
# Per-input resamplers to enforce s32/stereo at the same rate (no resample of rate)
|
||||
resamplers = [
|
||||
AudioResampler(format="s32", layout="stereo", rate=target_sample_rate)
|
||||
for _ in decoders
|
||||
]
|
||||
|
||||
try:
|
||||
# Round-robin feed frames into graph, pull mixed frames as they become available
|
||||
# Open all containers with cleanup guaranteed
|
||||
for i, url in enumerate(valid_track_urls):
|
||||
try:
|
||||
c = av.open(url)
|
||||
containers.append(c)
|
||||
except Exception as e:
|
||||
self.logger.warning(
|
||||
"Mixdown: failed to open container from URL",
|
||||
input=i,
|
||||
url=url,
|
||||
error=str(e),
|
||||
)
|
||||
|
||||
if not containers:
|
||||
self.logger.error("Mixdown failed - no valid containers opened")
|
||||
raise Exception("Mixdown failed: Could not open any track containers")
|
||||
|
||||
decoders = [c.decode(audio=0) for c in containers]
|
||||
active = [True] * len(decoders)
|
||||
resamplers = [
|
||||
AudioResampler(format="s32", layout="stereo", rate=target_sample_rate)
|
||||
for _ in decoders
|
||||
]
|
||||
|
||||
while any(active):
|
||||
for i, (dec, is_active) in enumerate(zip(decoders, active)):
|
||||
if not is_active:
|
||||
@@ -330,9 +445,7 @@ class PipelineMainMultitrack(PipelineMainBase):
|
||||
active[i] = False
|
||||
continue
|
||||
|
||||
# Enforce same sample rate; convert format/layout to s16/stereo (no resample)
|
||||
if frame.sample_rate != target_sample_rate:
|
||||
# Skip frames with differing rate
|
||||
continue
|
||||
out_frames = resamplers[i].resample(frame) or []
|
||||
for rf in out_frames:
|
||||
@@ -340,7 +453,6 @@ class PipelineMainMultitrack(PipelineMainBase):
|
||||
rf.time_base = Fraction(1, target_sample_rate)
|
||||
inputs[i].push(rf)
|
||||
|
||||
# Drain available mixed frames
|
||||
while True:
|
||||
try:
|
||||
mixed = sink.pull()
|
||||
@@ -350,7 +462,6 @@ class PipelineMainMultitrack(PipelineMainBase):
|
||||
mixed.time_base = Fraction(1, target_sample_rate)
|
||||
await writer.push(mixed)
|
||||
|
||||
# Signal EOF to inputs and drain remaining
|
||||
for in_ctx in inputs:
|
||||
in_ctx.push(None)
|
||||
while True:
|
||||
@@ -362,8 +473,13 @@ class PipelineMainMultitrack(PipelineMainBase):
|
||||
mixed.time_base = Fraction(1, target_sample_rate)
|
||||
await writer.push(mixed)
|
||||
finally:
|
||||
# Cleanup all containers, even if processing failed
|
||||
for c in containers:
|
||||
c.close()
|
||||
if c is not None:
|
||||
try:
|
||||
c.close()
|
||||
except Exception:
|
||||
pass # Best effort cleanup
|
||||
|
||||
@broadcast_to_sockets
|
||||
async def set_status(self, transcript_id: str, status: TranscriptStatus):
|
||||
@@ -380,148 +496,94 @@ class PipelineMainMultitrack(PipelineMainBase):
|
||||
|
||||
async def process(self, bucket_name: str, track_keys: list[str]):
|
||||
transcript = await self.get_transcript()
|
||||
async with self.transaction():
|
||||
await transcripts_controller.update(
|
||||
transcript,
|
||||
{
|
||||
"events": [],
|
||||
"topics": [],
|
||||
},
|
||||
)
|
||||
|
||||
s3 = boto3.client(
|
||||
"s3",
|
||||
region_name=settings.RECORDING_STORAGE_AWS_REGION,
|
||||
aws_access_key_id=settings.RECORDING_STORAGE_AWS_ACCESS_KEY_ID,
|
||||
aws_secret_access_key=settings.RECORDING_STORAGE_AWS_SECRET_ACCESS_KEY,
|
||||
source_storage = get_transcripts_storage()
|
||||
transcript_storage = source_storage
|
||||
|
||||
track_urls: list[str] = []
|
||||
for key in track_keys:
|
||||
url = await source_storage.get_file_url(
|
||||
key,
|
||||
operation="get_object",
|
||||
expires_in=PRESIGNED_URL_EXPIRATION_SECONDS,
|
||||
bucket=bucket_name,
|
||||
)
|
||||
track_urls.append(url)
|
||||
self.logger.info(
|
||||
f"Generated presigned URL for track from {bucket_name}",
|
||||
key=key,
|
||||
)
|
||||
|
||||
created_padded_files = set()
|
||||
padded_track_urls: list[str] = []
|
||||
for idx, url in enumerate(track_urls):
|
||||
padded_url = await self.pad_track_for_transcription(
|
||||
url, idx, transcript_storage
|
||||
)
|
||||
padded_track_urls.append(padded_url)
|
||||
if padded_url != url:
|
||||
storage_path = f"file_pipeline/{transcript.id}/tracks/padded_{idx}.webm"
|
||||
created_padded_files.add(storage_path)
|
||||
self.logger.info(f"Track {idx} processed, padded URL: {padded_url}")
|
||||
|
||||
transcript.data_path.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
mp3_writer = AudioFileWriterProcessor(
|
||||
path=str(transcript.audio_mp3_filename),
|
||||
on_duration=self.on_duration,
|
||||
)
|
||||
await self.mixdown_tracks(padded_track_urls, mp3_writer, offsets_seconds=None)
|
||||
await mp3_writer.flush()
|
||||
|
||||
if not transcript.audio_mp3_filename.exists():
|
||||
raise Exception(
|
||||
"Mixdown failed - no MP3 file generated. Cannot proceed without playable audio."
|
||||
)
|
||||
|
||||
storage_path = f"{transcript.id}/audio.mp3"
|
||||
# Use file handle streaming to avoid loading entire MP3 into memory
|
||||
mp3_size = transcript.audio_mp3_filename.stat().st_size
|
||||
with open(transcript.audio_mp3_filename, "rb") as mp3_file:
|
||||
await transcript_storage.put_file(storage_path, mp3_file)
|
||||
mp3_url = await transcript_storage.get_file_url(storage_path)
|
||||
|
||||
await transcripts_controller.update(transcript, {"audio_location": "storage"})
|
||||
|
||||
self.logger.info(
|
||||
f"Uploaded mixed audio to storage",
|
||||
storage_path=storage_path,
|
||||
size=mp3_size,
|
||||
url=mp3_url,
|
||||
)
|
||||
|
||||
storage = get_transcripts_storage()
|
||||
self.logger.info("Generating waveform from mixed audio")
|
||||
waveform_processor = AudioWaveformProcessor(
|
||||
audio_path=transcript.audio_mp3_filename,
|
||||
waveform_path=transcript.audio_waveform_filename,
|
||||
on_waveform=self.on_waveform,
|
||||
)
|
||||
waveform_processor.set_pipeline(self.empty_pipeline)
|
||||
await waveform_processor.flush()
|
||||
self.logger.info("Waveform generated successfully")
|
||||
|
||||
# Pre-download bytes for all tracks for mixing and transcription
|
||||
track_datas: list[bytes] = []
|
||||
for key in track_keys:
|
||||
try:
|
||||
obj = s3.get_object(Bucket=bucket_name, Key=key)
|
||||
track_datas.append(obj["Body"].read())
|
||||
except Exception as e:
|
||||
self.logger.warning(
|
||||
"Skipping track - cannot read S3 object", key=key, error=str(e)
|
||||
)
|
||||
track_datas.append(b"")
|
||||
|
||||
# PAD TRACKS FIRST - this creates full-length tracks with correct timeline
|
||||
padded_track_datas: list[bytes] = []
|
||||
padded_track_urls: list[str] = []
|
||||
for idx, data in enumerate(track_datas):
|
||||
if not data:
|
||||
padded_track_datas.append(b"")
|
||||
padded_track_urls.append("")
|
||||
continue
|
||||
|
||||
padded_data, padded_url = await self.pad_track_for_transcription(
|
||||
data, idx, storage
|
||||
)
|
||||
padded_track_datas.append(padded_data)
|
||||
padded_track_urls.append(padded_url)
|
||||
self.logger.info(f"Padded track {idx} for transcription: {padded_url}")
|
||||
|
||||
# Mixdown PADDED tracks (already aligned with timeline) into transcript.audio_mp3_filename
|
||||
try:
|
||||
# Ensure data directory exists
|
||||
transcript.data_path.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
mp3_writer = AudioFileWriterProcessor(
|
||||
path=str(transcript.audio_mp3_filename),
|
||||
on_duration=self.on_duration,
|
||||
)
|
||||
# Use PADDED tracks with NO additional offsets (already aligned by padding)
|
||||
await self.mixdown_tracks(
|
||||
padded_track_datas, mp3_writer, offsets_seconds=None
|
||||
)
|
||||
await mp3_writer.flush()
|
||||
|
||||
# Upload the mixed audio to S3 for web playback
|
||||
if transcript.audio_mp3_filename.exists():
|
||||
mp3_data = transcript.audio_mp3_filename.read_bytes()
|
||||
storage_path = f"{transcript.id}/audio.mp3"
|
||||
await storage.put_file(storage_path, mp3_data)
|
||||
mp3_url = await storage.get_file_url(storage_path)
|
||||
|
||||
# Update transcript to indicate audio is in storage
|
||||
await transcripts_controller.update(
|
||||
transcript, {"audio_location": "storage"}
|
||||
)
|
||||
|
||||
self.logger.info(
|
||||
f"Uploaded mixed audio to storage",
|
||||
storage_path=storage_path,
|
||||
size=len(mp3_data),
|
||||
url=mp3_url,
|
||||
)
|
||||
else:
|
||||
self.logger.warning("Mixdown file does not exist after processing")
|
||||
except Exception as e:
|
||||
self.logger.error("Mixdown failed", error=str(e), exc_info=True)
|
||||
|
||||
# Generate waveform from the mixed audio file
|
||||
if transcript.audio_mp3_filename.exists():
|
||||
try:
|
||||
self.logger.info("Generating waveform from mixed audio")
|
||||
waveform_processor = AudioWaveformProcessor(
|
||||
audio_path=transcript.audio_mp3_filename,
|
||||
waveform_path=transcript.audio_waveform_filename,
|
||||
on_waveform=self.on_waveform,
|
||||
)
|
||||
waveform_processor.set_pipeline(self.empty_pipeline)
|
||||
await waveform_processor.flush()
|
||||
self.logger.info("Waveform generated successfully")
|
||||
except Exception as e:
|
||||
self.logger.error(
|
||||
"Waveform generation failed", error=str(e), exc_info=True
|
||||
)
|
||||
|
||||
# Transcribe PADDED tracks - timestamps will be automatically correct!
|
||||
speaker_transcripts: list[TranscriptType] = []
|
||||
for idx, padded_url in enumerate(padded_track_urls):
|
||||
if not padded_url:
|
||||
continue
|
||||
|
||||
try:
|
||||
# Transcribe the PADDED track
|
||||
t = await self.transcribe_file(padded_url, transcript.source_language)
|
||||
except Exception as e:
|
||||
self.logger.error(
|
||||
"Transcription via default backend failed, trying local whisper",
|
||||
track_idx=idx,
|
||||
url=padded_url,
|
||||
error=str(e),
|
||||
)
|
||||
try:
|
||||
fallback = FileTranscriptAutoProcessor(name="whisper")
|
||||
result = None
|
||||
|
||||
async def capture_result(r):
|
||||
nonlocal result
|
||||
result = r
|
||||
|
||||
fallback.on(capture_result)
|
||||
await fallback.push(
|
||||
FileTranscriptInput(
|
||||
audio_url=padded_url, language=transcript.source_language
|
||||
)
|
||||
)
|
||||
await fallback.flush()
|
||||
if not result:
|
||||
raise Exception("No transcript captured in fallback")
|
||||
t = result
|
||||
except Exception as e2:
|
||||
self.logger.error(
|
||||
"Skipping track - transcription failed after fallback",
|
||||
track_idx=idx,
|
||||
url=padded_url,
|
||||
error=str(e2),
|
||||
)
|
||||
continue
|
||||
t = await self.transcribe_file(padded_url, transcript.source_language)
|
||||
|
||||
if not t.words:
|
||||
continue
|
||||
|
||||
# NO OFFSET ADJUSTMENT NEEDED!
|
||||
# Timestamps are already correct because we transcribed padded tracks
|
||||
# Just set speaker ID
|
||||
for w in t.words:
|
||||
w.speaker = idx
|
||||
|
||||
@@ -531,10 +593,33 @@ class PipelineMainMultitrack(PipelineMainBase):
|
||||
track_idx=idx,
|
||||
)
|
||||
|
||||
valid_track_count = len([url for url in padded_track_urls if url])
|
||||
if valid_track_count > 0 and len(speaker_transcripts) != valid_track_count:
|
||||
raise Exception(
|
||||
f"Only {len(speaker_transcripts)}/{valid_track_count} tracks transcribed successfully. "
|
||||
f"All tracks must succeed to avoid incomplete transcripts."
|
||||
)
|
||||
|
||||
if not speaker_transcripts:
|
||||
raise Exception("No valid track transcriptions")
|
||||
|
||||
# Merge all words and sort by timestamp
|
||||
self.logger.info(f"Cleaning up {len(created_padded_files)} temporary S3 files")
|
||||
cleanup_tasks = []
|
||||
for storage_path in created_padded_files:
|
||||
cleanup_tasks.append(transcript_storage.delete_file(storage_path))
|
||||
|
||||
if cleanup_tasks:
|
||||
cleanup_results = await asyncio.gather(
|
||||
*cleanup_tasks, return_exceptions=True
|
||||
)
|
||||
for storage_path, result in zip(created_padded_files, cleanup_results):
|
||||
if isinstance(result, Exception):
|
||||
self.logger.warning(
|
||||
"Failed to cleanup temporary padded track",
|
||||
storage_path=storage_path,
|
||||
error=str(result),
|
||||
)
|
||||
|
||||
merged_words = []
|
||||
for t in speaker_transcripts:
|
||||
merged_words.extend(t.words)
|
||||
@@ -544,7 +629,6 @@ class PipelineMainMultitrack(PipelineMainBase):
|
||||
|
||||
merged_transcript = TranscriptType(words=merged_words, translation=None)
|
||||
|
||||
# Emit TRANSCRIPT event through the shared handler (persists and broadcasts)
|
||||
await self.on_transcript(merged_transcript)
|
||||
|
||||
topics = await self.detect_topics(merged_transcript, transcript.target_language)
|
||||
@@ -557,80 +641,36 @@ class PipelineMainMultitrack(PipelineMainBase):
|
||||
await self.set_status(transcript.id, "ended")
|
||||
|
||||
async def transcribe_file(self, audio_url: str, language: str) -> TranscriptType:
|
||||
processor = FileTranscriptAutoProcessor()
|
||||
input_data = FileTranscriptInput(audio_url=audio_url, language=language)
|
||||
|
||||
result: TranscriptType | None = None
|
||||
|
||||
async def capture_result(transcript):
|
||||
nonlocal result
|
||||
result = transcript
|
||||
|
||||
processor.on(capture_result)
|
||||
await processor.push(input_data)
|
||||
await processor.flush()
|
||||
|
||||
if not result:
|
||||
raise ValueError("No transcript captured")
|
||||
|
||||
return result
|
||||
return await transcribe_file_with_processor(audio_url, language)
|
||||
|
||||
async def detect_topics(
|
||||
self, transcript: TranscriptType, target_language: str
|
||||
) -> list[TitleSummary]:
|
||||
chunk_size = 300
|
||||
topics: list[TitleSummary] = []
|
||||
|
||||
async def on_topic(topic: TitleSummary):
|
||||
topics.append(topic)
|
||||
return await self.on_topic(topic)
|
||||
|
||||
topic_detector = TranscriptTopicDetectorProcessor(callback=on_topic)
|
||||
topic_detector.set_pipeline(self.empty_pipeline)
|
||||
|
||||
for i in range(0, len(transcript.words), chunk_size):
|
||||
chunk_words = transcript.words[i : i + chunk_size]
|
||||
if not chunk_words:
|
||||
continue
|
||||
|
||||
chunk_transcript = TranscriptType(
|
||||
words=chunk_words, translation=transcript.translation
|
||||
)
|
||||
await topic_detector.push(chunk_transcript)
|
||||
|
||||
await topic_detector.flush()
|
||||
return topics
|
||||
return await topic_processing.detect_topics(
|
||||
transcript,
|
||||
target_language,
|
||||
on_topic_callback=self.on_topic,
|
||||
empty_pipeline=self.empty_pipeline,
|
||||
)
|
||||
|
||||
async def generate_title(self, topics: list[TitleSummary]):
|
||||
if not topics:
|
||||
self.logger.warning("No topics for title generation")
|
||||
return
|
||||
|
||||
processor = TranscriptFinalTitleProcessor(callback=self.on_title)
|
||||
processor.set_pipeline(self.empty_pipeline)
|
||||
|
||||
for topic in topics:
|
||||
await processor.push(topic)
|
||||
|
||||
await processor.flush()
|
||||
return await topic_processing.generate_title(
|
||||
topics,
|
||||
on_title_callback=self.on_title,
|
||||
empty_pipeline=self.empty_pipeline,
|
||||
logger=self.logger,
|
||||
)
|
||||
|
||||
async def generate_summaries(self, topics: list[TitleSummary]):
|
||||
if not topics:
|
||||
self.logger.warning("No topics for summary generation")
|
||||
return
|
||||
|
||||
transcript = await self.get_transcript()
|
||||
processor = TranscriptFinalSummaryProcessor(
|
||||
transcript=transcript,
|
||||
callback=self.on_long_summary,
|
||||
on_short_summary=self.on_short_summary,
|
||||
return await topic_processing.generate_summaries(
|
||||
topics,
|
||||
transcript,
|
||||
on_long_summary_callback=self.on_long_summary,
|
||||
on_short_summary_callback=self.on_short_summary,
|
||||
empty_pipeline=self.empty_pipeline,
|
||||
logger=self.logger,
|
||||
)
|
||||
processor.set_pipeline(self.empty_pipeline)
|
||||
|
||||
for topic in topics:
|
||||
await processor.push(topic)
|
||||
|
||||
await processor.flush()
|
||||
|
||||
|
||||
@shared_task
|
||||
|
||||
@@ -1,629 +0,0 @@
|
||||
import asyncio
|
||||
import io
|
||||
from fractions import Fraction
|
||||
|
||||
import av
|
||||
import boto3
|
||||
import structlog
|
||||
from av.audio.resampler import AudioResampler
|
||||
from celery import chain, shared_task
|
||||
|
||||
from reflector.asynctask import asynctask
|
||||
from reflector.db.transcripts import (
|
||||
TranscriptStatus,
|
||||
TranscriptText,
|
||||
transcripts_controller,
|
||||
)
|
||||
from reflector.logger import logger
|
||||
from reflector.pipelines.main_file_pipeline import task_send_webhook_if_needed
|
||||
from reflector.pipelines.main_live_pipeline import (
|
||||
PipelineMainBase,
|
||||
task_cleanup_consent,
|
||||
task_pipeline_post_to_zulip,
|
||||
)
|
||||
from reflector.processors import (
|
||||
AudioFileWriterProcessor,
|
||||
TranscriptFinalSummaryProcessor,
|
||||
TranscriptFinalTitleProcessor,
|
||||
TranscriptTopicDetectorProcessor,
|
||||
)
|
||||
from reflector.processors.file_transcript import FileTranscriptInput
|
||||
from reflector.processors.file_transcript_auto import FileTranscriptAutoProcessor
|
||||
from reflector.processors.types import TitleSummary
|
||||
from reflector.processors.types import (
|
||||
Transcript as TranscriptType,
|
||||
)
|
||||
from reflector.settings import settings
|
||||
from reflector.storage import get_transcripts_storage
|
||||
|
||||
|
||||
class EmptyPipeline:
|
||||
def __init__(self, logger: structlog.BoundLogger):
|
||||
self.logger = logger
|
||||
|
||||
def get_pref(self, k, d=None):
|
||||
return d
|
||||
|
||||
async def emit(self, event):
|
||||
pass
|
||||
|
||||
|
||||
class PipelineMainMultitrack(PipelineMainBase):
|
||||
"""Process multiple participant tracks for a transcript without mixing audio."""
|
||||
|
||||
def __init__(self, transcript_id: str):
|
||||
super().__init__(transcript_id=transcript_id)
|
||||
self.logger = logger.bind(transcript_id=self.transcript_id)
|
||||
self.empty_pipeline = EmptyPipeline(logger=self.logger)
|
||||
|
||||
async def pad_track_for_transcription(
|
||||
self,
|
||||
track_data: bytes,
|
||||
track_idx: int,
|
||||
storage,
|
||||
) -> tuple[bytes, str]:
|
||||
"""
|
||||
Pad a single track with silence based on stream metadata start_time.
|
||||
This ensures Whisper timestamps will be relative to recording start.
|
||||
|
||||
Returns: (padded_data, storage_url)
|
||||
"""
|
||||
if not track_data:
|
||||
return b"", ""
|
||||
|
||||
transcript = await self.get_transcript()
|
||||
|
||||
# Get stream metadata start_time using PyAV
|
||||
container = av.open(io.BytesIO(track_data))
|
||||
try:
|
||||
audio_stream = container.streams.audio[0]
|
||||
|
||||
# Extract start_time from stream metadata
|
||||
if (
|
||||
audio_stream.start_time is not None
|
||||
and audio_stream.time_base is not None
|
||||
):
|
||||
start_time_seconds = float(
|
||||
audio_stream.start_time * audio_stream.time_base
|
||||
)
|
||||
else:
|
||||
start_time_seconds = 0.0
|
||||
|
||||
sample_rate = audio_stream.sample_rate
|
||||
codec_name = audio_stream.codec.name
|
||||
finally:
|
||||
container.close()
|
||||
|
||||
self.logger.info(
|
||||
f"Track {track_idx} stream metadata: start_time={start_time_seconds:.3f}s, sample_rate={sample_rate}",
|
||||
track_idx=track_idx,
|
||||
)
|
||||
|
||||
# If no padding needed, use original
|
||||
if start_time_seconds <= 0:
|
||||
storage_path = (
|
||||
f"file_pipeline/{transcript.id}/tracks/original_track_{track_idx}.webm"
|
||||
)
|
||||
await storage.put_file(storage_path, track_data)
|
||||
url = await storage.get_file_url(storage_path)
|
||||
return track_data, url
|
||||
|
||||
# Create PyAV filter graph for padding
|
||||
graph = av.filter.Graph()
|
||||
|
||||
# Input buffer
|
||||
in_args = (
|
||||
f"time_base=1/{sample_rate}:"
|
||||
f"sample_rate={sample_rate}:"
|
||||
f"sample_fmt=s16:"
|
||||
f"channel_layout=stereo"
|
||||
)
|
||||
input_buffer = graph.add("abuffer", args=in_args, name="in")
|
||||
|
||||
# Add delay filter for padding
|
||||
delay_ms = int(start_time_seconds * 1000)
|
||||
delay_filter = graph.add(
|
||||
"adelay", args=f"delays={delay_ms}|{delay_ms}:all=1", name="delay"
|
||||
)
|
||||
|
||||
# Output sink
|
||||
sink = graph.add("abuffersink", name="out")
|
||||
|
||||
# Link filters
|
||||
input_buffer.link_to(delay_filter)
|
||||
delay_filter.link_to(sink)
|
||||
|
||||
graph.configure()
|
||||
|
||||
# Process audio through filter
|
||||
output_bytes = io.BytesIO()
|
||||
output_container = av.open(output_bytes, "w", format="webm")
|
||||
output_stream = output_container.add_stream("libopus", rate=sample_rate)
|
||||
output_stream.channels = 2
|
||||
|
||||
# Reopen input for processing
|
||||
input_container = av.open(io.BytesIO(track_data))
|
||||
resampler = AudioResampler(format="s16", layout="stereo", rate=sample_rate)
|
||||
|
||||
try:
|
||||
# Process frames
|
||||
for frame in input_container.decode(audio=0):
|
||||
# Resample to match filter requirements
|
||||
resampled_frames = resampler.resample(frame)
|
||||
for resampled_frame in resampled_frames:
|
||||
resampled_frame.pts = frame.pts
|
||||
resampled_frame.time_base = Fraction(1, sample_rate)
|
||||
input_buffer.push(resampled_frame)
|
||||
|
||||
# Pull from filter and encode
|
||||
while True:
|
||||
try:
|
||||
out_frame = sink.pull()
|
||||
out_frame.pts = out_frame.pts if out_frame.pts else 0
|
||||
out_frame.time_base = Fraction(1, sample_rate)
|
||||
for packet in output_stream.encode(out_frame):
|
||||
output_container.mux(packet)
|
||||
except av.BlockingIOError:
|
||||
break
|
||||
|
||||
# Flush
|
||||
input_buffer.push(None)
|
||||
while True:
|
||||
try:
|
||||
out_frame = sink.pull()
|
||||
for packet in output_stream.encode(out_frame):
|
||||
output_container.mux(packet)
|
||||
except (av.BlockingIOError, av.EOFError):
|
||||
break
|
||||
|
||||
# Flush encoder
|
||||
for packet in output_stream.encode(None):
|
||||
output_container.mux(packet)
|
||||
|
||||
finally:
|
||||
input_container.close()
|
||||
output_container.close()
|
||||
|
||||
padded_data = output_bytes.getvalue()
|
||||
|
||||
# Store padded track
|
||||
storage_path = (
|
||||
f"file_pipeline/{transcript.id}/tracks/padded_track_{track_idx}.webm"
|
||||
)
|
||||
await storage.put_file(storage_path, padded_data)
|
||||
padded_url = await storage.get_file_url(storage_path)
|
||||
|
||||
self.logger.info(
|
||||
f"Padded track {track_idx} with {start_time_seconds:.3f}s offset, stored at {storage_path}",
|
||||
track_idx=track_idx,
|
||||
delay_ms=delay_ms,
|
||||
padded_url=padded_url,
|
||||
)
|
||||
|
||||
return padded_data, padded_url
|
||||
|
||||
async def mixdown_tracks(
|
||||
self,
|
||||
track_datas: list[bytes],
|
||||
writer: AudioFileWriterProcessor,
|
||||
offsets_seconds: list[float] | None = None,
|
||||
) -> None:
|
||||
"""
|
||||
Minimal multi-track mixdown using a PyAV filter graph (amix), no resampling.
|
||||
"""
|
||||
|
||||
# Discover target sample rate from first decodable frame
|
||||
target_sample_rate: int | None = None
|
||||
for data in track_datas:
|
||||
if not data:
|
||||
continue
|
||||
try:
|
||||
container = av.open(io.BytesIO(data))
|
||||
try:
|
||||
for frame in container.decode(audio=0):
|
||||
target_sample_rate = frame.sample_rate
|
||||
break
|
||||
finally:
|
||||
container.close()
|
||||
except Exception:
|
||||
continue
|
||||
if target_sample_rate:
|
||||
break
|
||||
|
||||
if not target_sample_rate:
|
||||
self.logger.warning("Mixdown skipped - no decodable audio frames found")
|
||||
return
|
||||
|
||||
# Build PyAV filter graph:
|
||||
# N abuffer (s32/stereo)
|
||||
# -> optional adelay per input (for alignment)
|
||||
# -> amix (s32)
|
||||
# -> aformat(s16)
|
||||
# -> sink
|
||||
graph = av.filter.Graph()
|
||||
inputs = []
|
||||
valid_track_datas = [d for d in track_datas if d]
|
||||
# Align offsets list with the filtered inputs (skip empties)
|
||||
input_offsets_seconds = None
|
||||
if offsets_seconds is not None:
|
||||
input_offsets_seconds = [
|
||||
offsets_seconds[i] for i, d in enumerate(track_datas) if d
|
||||
]
|
||||
for idx, data in enumerate(valid_track_datas):
|
||||
args = (
|
||||
f"time_base=1/{target_sample_rate}:"
|
||||
f"sample_rate={target_sample_rate}:"
|
||||
f"sample_fmt=s32:"
|
||||
f"channel_layout=stereo"
|
||||
)
|
||||
in_ctx = graph.add("abuffer", args=args, name=f"in{idx}")
|
||||
inputs.append(in_ctx)
|
||||
|
||||
if not inputs:
|
||||
self.logger.warning("Mixdown skipped - no valid inputs for graph")
|
||||
return
|
||||
|
||||
mixer = graph.add("amix", args=f"inputs={len(inputs)}:normalize=0", name="mix")
|
||||
|
||||
fmt = graph.add(
|
||||
"aformat",
|
||||
args=(
|
||||
f"sample_fmts=s32:channel_layouts=stereo:sample_rates={target_sample_rate}"
|
||||
),
|
||||
name="fmt",
|
||||
)
|
||||
|
||||
sink = graph.add("abuffersink", name="out")
|
||||
|
||||
# Optional per-input delay before mixing
|
||||
delays_ms: list[int] = []
|
||||
if input_offsets_seconds is not None:
|
||||
base = min(input_offsets_seconds) if input_offsets_seconds else 0.0
|
||||
delays_ms = [
|
||||
max(0, int(round((o - base) * 1000))) for o in input_offsets_seconds
|
||||
]
|
||||
else:
|
||||
delays_ms = [0 for _ in inputs]
|
||||
|
||||
for idx, in_ctx in enumerate(inputs):
|
||||
delay_ms = delays_ms[idx] if idx < len(delays_ms) else 0
|
||||
if delay_ms > 0:
|
||||
# adelay requires one value per channel; use same for stereo
|
||||
adelay = graph.add(
|
||||
"adelay",
|
||||
args=f"delays={delay_ms}|{delay_ms}:all=1",
|
||||
name=f"delay{idx}",
|
||||
)
|
||||
in_ctx.link_to(adelay)
|
||||
adelay.link_to(mixer, 0, idx)
|
||||
else:
|
||||
in_ctx.link_to(mixer, 0, idx)
|
||||
mixer.link_to(fmt)
|
||||
fmt.link_to(sink)
|
||||
graph.configure()
|
||||
|
||||
# Open containers for decoding
|
||||
containers = []
|
||||
for i, d in enumerate(valid_track_datas):
|
||||
try:
|
||||
c = av.open(io.BytesIO(d))
|
||||
containers.append(c)
|
||||
except Exception as e:
|
||||
self.logger.warning(
|
||||
"Mixdown: failed to open container", input=i, error=str(e)
|
||||
)
|
||||
containers.append(None)
|
||||
# Filter out Nones for decoders
|
||||
containers = [c for c in containers if c is not None]
|
||||
decoders = [c.decode(audio=0) for c in containers]
|
||||
active = [True] * len(decoders)
|
||||
# Per-input resamplers to enforce s32/stereo at the same rate (no resample of rate)
|
||||
resamplers = [
|
||||
AudioResampler(format="s32", layout="stereo", rate=target_sample_rate)
|
||||
for _ in decoders
|
||||
]
|
||||
|
||||
try:
|
||||
# Round-robin feed frames into graph, pull mixed frames as they become available
|
||||
while any(active):
|
||||
for i, (dec, is_active) in enumerate(zip(decoders, active)):
|
||||
if not is_active:
|
||||
continue
|
||||
try:
|
||||
frame = next(dec)
|
||||
except StopIteration:
|
||||
active[i] = False
|
||||
continue
|
||||
|
||||
# Enforce same sample rate; convert format/layout to s16/stereo (no resample)
|
||||
if frame.sample_rate != target_sample_rate:
|
||||
# Skip frames with differing rate
|
||||
continue
|
||||
out_frames = resamplers[i].resample(frame) or []
|
||||
for rf in out_frames:
|
||||
rf.sample_rate = target_sample_rate
|
||||
rf.time_base = Fraction(1, target_sample_rate)
|
||||
inputs[i].push(rf)
|
||||
|
||||
# Drain available mixed frames
|
||||
while True:
|
||||
try:
|
||||
mixed = sink.pull()
|
||||
except Exception:
|
||||
break
|
||||
mixed.sample_rate = target_sample_rate
|
||||
mixed.time_base = Fraction(1, target_sample_rate)
|
||||
await writer.push(mixed)
|
||||
|
||||
# Signal EOF to inputs and drain remaining
|
||||
for in_ctx in inputs:
|
||||
in_ctx.push(None)
|
||||
while True:
|
||||
try:
|
||||
mixed = sink.pull()
|
||||
except Exception:
|
||||
break
|
||||
mixed.sample_rate = target_sample_rate
|
||||
mixed.time_base = Fraction(1, target_sample_rate)
|
||||
await writer.push(mixed)
|
||||
finally:
|
||||
for c in containers:
|
||||
c.close()
|
||||
|
||||
async def set_status(self, transcript_id: str, status: TranscriptStatus):
|
||||
async with self.lock_transaction():
|
||||
return await transcripts_controller.set_status(transcript_id, status)
|
||||
|
||||
async def process(self, bucket_name: str, track_keys: list[str]):
|
||||
transcript = await self.get_transcript()
|
||||
|
||||
s3 = boto3.client(
|
||||
"s3",
|
||||
region_name=settings.RECORDING_STORAGE_AWS_REGION,
|
||||
aws_access_key_id=settings.RECORDING_STORAGE_AWS_ACCESS_KEY_ID,
|
||||
aws_secret_access_key=settings.RECORDING_STORAGE_AWS_SECRET_ACCESS_KEY,
|
||||
)
|
||||
|
||||
storage = get_transcripts_storage()
|
||||
|
||||
# Pre-download bytes for all tracks for mixing and transcription
|
||||
track_datas: list[bytes] = []
|
||||
for key in track_keys:
|
||||
try:
|
||||
obj = s3.get_object(Bucket=bucket_name, Key=key)
|
||||
track_datas.append(obj["Body"].read())
|
||||
except Exception as e:
|
||||
self.logger.warning(
|
||||
"Skipping track - cannot read S3 object", key=key, error=str(e)
|
||||
)
|
||||
track_datas.append(b"")
|
||||
|
||||
# REMOVED: Filename offset extraction - not needed anymore!
|
||||
# We use stream metadata start_time for padding instead
|
||||
|
||||
# Get stream metadata start_times for mixing (still useful for mixdown)
|
||||
stream_start_times: list[float] = []
|
||||
for data in track_datas:
|
||||
if not data:
|
||||
stream_start_times.append(0.0)
|
||||
continue
|
||||
|
||||
container = av.open(io.BytesIO(data))
|
||||
try:
|
||||
audio_stream = container.streams.audio[0]
|
||||
if (
|
||||
audio_stream.start_time is not None
|
||||
and audio_stream.time_base is not None
|
||||
):
|
||||
start_time = float(audio_stream.start_time * audio_stream.time_base)
|
||||
else:
|
||||
start_time = 0.0
|
||||
stream_start_times.append(start_time)
|
||||
finally:
|
||||
container.close()
|
||||
|
||||
# Mixdown all available tracks into transcript.audio_mp3_filename, using stream metadata offsets
|
||||
try:
|
||||
mp3_writer = AudioFileWriterProcessor(
|
||||
path=str(transcript.audio_mp3_filename)
|
||||
)
|
||||
await self.mixdown_tracks(track_datas, mp3_writer, stream_start_times)
|
||||
await mp3_writer.flush()
|
||||
except Exception as e:
|
||||
self.logger.error("Mixdown failed", error=str(e))
|
||||
|
||||
# PAD TRACKS BEFORE TRANSCRIPTION - THIS IS THE KEY FIX!
|
||||
padded_track_urls: list[str] = []
|
||||
for idx, data in enumerate(track_datas):
|
||||
if not data:
|
||||
padded_track_urls.append("")
|
||||
continue
|
||||
|
||||
_, padded_url = await self.pad_track_for_transcription(data, idx, storage)
|
||||
padded_track_urls.append(padded_url)
|
||||
self.logger.info(f"Padded track {idx} for transcription: {padded_url}")
|
||||
|
||||
# Transcribe PADDED tracks - timestamps will be automatically correct!
|
||||
speaker_transcripts: list[TranscriptType] = []
|
||||
for idx, padded_url in enumerate(padded_track_urls):
|
||||
if not padded_url:
|
||||
continue
|
||||
|
||||
try:
|
||||
# Transcribe the PADDED track
|
||||
t = await self.transcribe_file(padded_url, transcript.source_language)
|
||||
except Exception as e:
|
||||
self.logger.error(
|
||||
"Transcription via default backend failed, trying local whisper",
|
||||
track_idx=idx,
|
||||
url=padded_url,
|
||||
error=str(e),
|
||||
)
|
||||
try:
|
||||
fallback = FileTranscriptAutoProcessor(name="whisper")
|
||||
result = None
|
||||
|
||||
async def capture_result(r):
|
||||
nonlocal result
|
||||
result = r
|
||||
|
||||
fallback.on(capture_result)
|
||||
await fallback.push(
|
||||
FileTranscriptInput(
|
||||
audio_url=padded_url, language=transcript.source_language
|
||||
)
|
||||
)
|
||||
await fallback.flush()
|
||||
if not result:
|
||||
raise Exception("No transcript captured in fallback")
|
||||
t = result
|
||||
except Exception as e2:
|
||||
self.logger.error(
|
||||
"Skipping track - transcription failed after fallback",
|
||||
track_idx=idx,
|
||||
url=padded_url,
|
||||
error=str(e2),
|
||||
)
|
||||
continue
|
||||
|
||||
if not t.words:
|
||||
continue
|
||||
|
||||
# NO OFFSET ADJUSTMENT NEEDED!
|
||||
# Timestamps are already correct because we transcribed padded tracks
|
||||
# Just set speaker ID
|
||||
for w in t.words:
|
||||
w.speaker = idx
|
||||
|
||||
speaker_transcripts.append(t)
|
||||
self.logger.info(
|
||||
f"Track {idx} transcribed successfully with {len(t.words)} words",
|
||||
track_idx=idx,
|
||||
)
|
||||
|
||||
if not speaker_transcripts:
|
||||
raise Exception("No valid track transcriptions")
|
||||
|
||||
# Merge all words and sort by timestamp
|
||||
merged_words = []
|
||||
for t in speaker_transcripts:
|
||||
merged_words.extend(t.words)
|
||||
merged_words.sort(
|
||||
key=lambda w: w.start if hasattr(w, "start") and w.start is not None else 0
|
||||
)
|
||||
|
||||
merged_transcript = TranscriptType(words=merged_words, translation=None)
|
||||
|
||||
await transcripts_controller.append_event(
|
||||
transcript,
|
||||
event="TRANSCRIPT",
|
||||
data=TranscriptText(
|
||||
text=merged_transcript.text, translation=merged_transcript.translation
|
||||
),
|
||||
)
|
||||
|
||||
topics = await self.detect_topics(merged_transcript, transcript.target_language)
|
||||
await asyncio.gather(
|
||||
self.generate_title(topics),
|
||||
self.generate_summaries(topics),
|
||||
return_exceptions=False,
|
||||
)
|
||||
|
||||
await self.set_status(transcript.id, "ended")
|
||||
|
||||
async def transcribe_file(self, audio_url: str, language: str) -> TranscriptType:
|
||||
processor = FileTranscriptAutoProcessor()
|
||||
input_data = FileTranscriptInput(audio_url=audio_url, language=language)
|
||||
|
||||
result: TranscriptType | None = None
|
||||
|
||||
async def capture_result(transcript):
|
||||
nonlocal result
|
||||
result = transcript
|
||||
|
||||
processor.on(capture_result)
|
||||
await processor.push(input_data)
|
||||
await processor.flush()
|
||||
|
||||
if not result:
|
||||
raise ValueError("No transcript captured")
|
||||
|
||||
return result
|
||||
|
||||
async def detect_topics(
|
||||
self, transcript: TranscriptType, target_language: str
|
||||
) -> list[TitleSummary]:
|
||||
chunk_size = 300
|
||||
topics: list[TitleSummary] = []
|
||||
|
||||
async def on_topic(topic: TitleSummary):
|
||||
topics.append(topic)
|
||||
return await self.on_topic(topic)
|
||||
|
||||
topic_detector = TranscriptTopicDetectorProcessor(callback=on_topic)
|
||||
topic_detector.set_pipeline(self.empty_pipeline)
|
||||
|
||||
for i in range(0, len(transcript.words), chunk_size):
|
||||
chunk_words = transcript.words[i : i + chunk_size]
|
||||
if not chunk_words:
|
||||
continue
|
||||
|
||||
chunk_transcript = TranscriptType(
|
||||
words=chunk_words, translation=transcript.translation
|
||||
)
|
||||
await topic_detector.push(chunk_transcript)
|
||||
|
||||
await topic_detector.flush()
|
||||
return topics
|
||||
|
||||
async def generate_title(self, topics: list[TitleSummary]):
|
||||
if not topics:
|
||||
self.logger.warning("No topics for title generation")
|
||||
return
|
||||
|
||||
processor = TranscriptFinalTitleProcessor(callback=self.on_title)
|
||||
processor.set_pipeline(self.empty_pipeline)
|
||||
|
||||
for topic in topics:
|
||||
await processor.push(topic)
|
||||
|
||||
await processor.flush()
|
||||
|
||||
async def generate_summaries(self, topics: list[TitleSummary]):
|
||||
if not topics:
|
||||
self.logger.warning("No topics for summary generation")
|
||||
return
|
||||
|
||||
transcript = await self.get_transcript()
|
||||
processor = TranscriptFinalSummaryProcessor(
|
||||
transcript=transcript,
|
||||
callback=self.on_long_summary,
|
||||
on_short_summary=self.on_short_summary,
|
||||
)
|
||||
processor.set_pipeline(self.empty_pipeline)
|
||||
|
||||
for topic in topics:
|
||||
await processor.push(topic)
|
||||
|
||||
await processor.flush()
|
||||
|
||||
|
||||
@shared_task
|
||||
@asynctask
|
||||
async def task_pipeline_multitrack_process(
|
||||
*, transcript_id: str, bucket_name: str, track_keys: list[str]
|
||||
):
|
||||
pipeline = PipelineMainMultitrack(transcript_id=transcript_id)
|
||||
try:
|
||||
await pipeline.set_status(transcript_id, "processing")
|
||||
await pipeline.process(bucket_name, track_keys)
|
||||
except Exception:
|
||||
await pipeline.set_status(transcript_id, "error")
|
||||
raise
|
||||
|
||||
post_chain = chain(
|
||||
task_cleanup_consent.si(transcript_id=transcript_id),
|
||||
task_pipeline_post_to_zulip.si(transcript_id=transcript_id),
|
||||
task_send_webhook_if_needed.si(transcript_id=transcript_id),
|
||||
)
|
||||
post_chain.delay()
|
||||
109
server/reflector/pipelines/topic_processing.py
Normal file
109
server/reflector/pipelines/topic_processing.py
Normal file
@@ -0,0 +1,109 @@
|
||||
"""
|
||||
Topic processing utilities
|
||||
==========================
|
||||
|
||||
Shared topic detection, title generation, and summarization logic
|
||||
used across file and multitrack pipelines.
|
||||
"""
|
||||
|
||||
from typing import Callable
|
||||
|
||||
import structlog
|
||||
|
||||
from reflector.db.transcripts import Transcript
|
||||
from reflector.processors import (
|
||||
TranscriptFinalSummaryProcessor,
|
||||
TranscriptFinalTitleProcessor,
|
||||
TranscriptTopicDetectorProcessor,
|
||||
)
|
||||
from reflector.processors.types import TitleSummary
|
||||
from reflector.processors.types import Transcript as TranscriptType
|
||||
|
||||
|
||||
class EmptyPipeline:
|
||||
def __init__(self, logger: structlog.BoundLogger):
|
||||
self.logger = logger
|
||||
|
||||
def get_pref(self, k, d=None):
|
||||
return d
|
||||
|
||||
async def emit(self, event):
|
||||
pass
|
||||
|
||||
|
||||
async def detect_topics(
|
||||
transcript: TranscriptType,
|
||||
target_language: str,
|
||||
*,
|
||||
on_topic_callback: Callable,
|
||||
empty_pipeline: EmptyPipeline,
|
||||
) -> list[TitleSummary]:
|
||||
chunk_size = 300
|
||||
topics: list[TitleSummary] = []
|
||||
|
||||
async def on_topic(topic: TitleSummary):
|
||||
topics.append(topic)
|
||||
return await on_topic_callback(topic)
|
||||
|
||||
topic_detector = TranscriptTopicDetectorProcessor(callback=on_topic)
|
||||
topic_detector.set_pipeline(empty_pipeline)
|
||||
|
||||
for i in range(0, len(transcript.words), chunk_size):
|
||||
chunk_words = transcript.words[i : i + chunk_size]
|
||||
if not chunk_words:
|
||||
continue
|
||||
|
||||
chunk_transcript = TranscriptType(
|
||||
words=chunk_words, translation=transcript.translation
|
||||
)
|
||||
|
||||
await topic_detector.push(chunk_transcript)
|
||||
|
||||
await topic_detector.flush()
|
||||
return topics
|
||||
|
||||
|
||||
async def generate_title(
|
||||
topics: list[TitleSummary],
|
||||
*,
|
||||
on_title_callback: Callable,
|
||||
empty_pipeline: EmptyPipeline,
|
||||
logger: structlog.BoundLogger,
|
||||
):
|
||||
if not topics:
|
||||
logger.warning("No topics for title generation")
|
||||
return
|
||||
|
||||
processor = TranscriptFinalTitleProcessor(callback=on_title_callback)
|
||||
processor.set_pipeline(empty_pipeline)
|
||||
|
||||
for topic in topics:
|
||||
await processor.push(topic)
|
||||
|
||||
await processor.flush()
|
||||
|
||||
|
||||
async def generate_summaries(
|
||||
topics: list[TitleSummary],
|
||||
transcript: Transcript,
|
||||
*,
|
||||
on_long_summary_callback: Callable,
|
||||
on_short_summary_callback: Callable,
|
||||
empty_pipeline: EmptyPipeline,
|
||||
logger: structlog.BoundLogger,
|
||||
):
|
||||
if not topics:
|
||||
logger.warning("No topics for summary generation")
|
||||
return
|
||||
|
||||
processor = TranscriptFinalSummaryProcessor(
|
||||
transcript=transcript,
|
||||
callback=on_long_summary_callback,
|
||||
on_short_summary=on_short_summary_callback,
|
||||
)
|
||||
processor.set_pipeline(empty_pipeline)
|
||||
|
||||
for topic in topics:
|
||||
await processor.push(topic)
|
||||
|
||||
await processor.flush()
|
||||
34
server/reflector/pipelines/transcription_helpers.py
Normal file
34
server/reflector/pipelines/transcription_helpers.py
Normal file
@@ -0,0 +1,34 @@
|
||||
from reflector.processors.file_transcript import FileTranscriptInput
|
||||
from reflector.processors.file_transcript_auto import FileTranscriptAutoProcessor
|
||||
from reflector.processors.types import Transcript as TranscriptType
|
||||
|
||||
|
||||
async def transcribe_file_with_processor(
|
||||
audio_url: str,
|
||||
language: str,
|
||||
processor_name: str | None = None,
|
||||
) -> TranscriptType:
|
||||
processor = (
|
||||
FileTranscriptAutoProcessor(name=processor_name)
|
||||
if processor_name
|
||||
else FileTranscriptAutoProcessor()
|
||||
)
|
||||
input_data = FileTranscriptInput(audio_url=audio_url, language=language)
|
||||
|
||||
result: TranscriptType | None = None
|
||||
|
||||
async def capture_result(transcript):
|
||||
nonlocal result
|
||||
result = transcript
|
||||
|
||||
processor.on(capture_result)
|
||||
await processor.push(input_data)
|
||||
await processor.flush()
|
||||
|
||||
if not result:
|
||||
processor_label = processor_name or "default"
|
||||
raise ValueError(
|
||||
f"No transcript captured from {processor_label} processor for audio: {audio_url}"
|
||||
)
|
||||
|
||||
return result
|
||||
@@ -1,9 +0,0 @@
|
||||
"""Platform type definitions.
|
||||
|
||||
This module exists solely to define the Platform literal type without any imports,
|
||||
preventing circular import issues when used across the codebase.
|
||||
"""
|
||||
|
||||
from typing import Literal
|
||||
|
||||
Platform = Literal["whereby", "daily"]
|
||||
@@ -56,6 +56,16 @@ class FileTranscriptModalProcessor(FileTranscriptProcessor):
|
||||
},
|
||||
follow_redirects=True,
|
||||
)
|
||||
|
||||
if response.status_code != 200:
|
||||
error_body = response.text
|
||||
self.logger.error(
|
||||
"Modal API error",
|
||||
audio_url=data.audio_url,
|
||||
status_code=response.status_code,
|
||||
error_body=error_body,
|
||||
)
|
||||
|
||||
response.raise_for_status()
|
||||
result = response.json()
|
||||
|
||||
|
||||
@@ -165,6 +165,7 @@ class SummaryBuilder:
|
||||
self.llm: LLM = llm
|
||||
self.model_name: str = llm.model_name
|
||||
self.logger = logger or structlog.get_logger()
|
||||
self.participant_instructions: str | None = None
|
||||
if filename:
|
||||
self.read_transcript_from_file(filename)
|
||||
|
||||
@@ -191,14 +192,61 @@ class SummaryBuilder:
|
||||
self, prompt: str, output_cls: Type[T], tone_name: str | None = None
|
||||
) -> T:
|
||||
"""Generic function to get structured output from LLM for non-function-calling models."""
|
||||
# Add participant instructions to the prompt if available
|
||||
enhanced_prompt = self._enhance_prompt_with_participants(prompt)
|
||||
return await self.llm.get_structured_response(
|
||||
prompt, [self.transcript], output_cls, tone_name=tone_name
|
||||
enhanced_prompt, [self.transcript], output_cls, tone_name=tone_name
|
||||
)
|
||||
|
||||
async def _get_response(
|
||||
self, prompt: str, texts: list[str], tone_name: str | None = None
|
||||
) -> str:
|
||||
"""Get text response with automatic participant instructions injection."""
|
||||
enhanced_prompt = self._enhance_prompt_with_participants(prompt)
|
||||
return await self.llm.get_response(enhanced_prompt, texts, tone_name=tone_name)
|
||||
|
||||
def _enhance_prompt_with_participants(self, prompt: str) -> str:
|
||||
"""Add participant instructions to any prompt if participants are known."""
|
||||
if self.participant_instructions:
|
||||
self.logger.debug("Adding participant instructions to prompt")
|
||||
return f"{prompt}\n\n{self.participant_instructions}"
|
||||
return prompt
|
||||
|
||||
# ----------------------------------------------------------------------------
|
||||
# Participants
|
||||
# ----------------------------------------------------------------------------
|
||||
|
||||
def set_known_participants(self, participants: list[str]) -> None:
|
||||
"""
|
||||
Set known participants directly without LLM identification.
|
||||
This is used when participants are already identified and stored.
|
||||
They are appended at the end of the transcript, providing more context for the assistant.
|
||||
"""
|
||||
if not participants:
|
||||
self.logger.warning("No participants provided")
|
||||
return
|
||||
|
||||
self.logger.info(
|
||||
"Using known participants",
|
||||
participants=participants,
|
||||
)
|
||||
|
||||
participants_md = self.format_list_md(participants)
|
||||
self.transcript += f"\n\n# Participants\n\n{participants_md}"
|
||||
|
||||
# Set instructions that will be automatically added to all prompts
|
||||
participants_list = ", ".join(participants)
|
||||
self.participant_instructions = dedent(
|
||||
f"""
|
||||
# IMPORTANT: Participant Names
|
||||
The following participants are identified in this conversation: {participants_list}
|
||||
|
||||
You MUST use these specific participant names when referring to people in your response.
|
||||
Do NOT use generic terms like "a participant", "someone", "attendee", "Speaker 1", "Speaker 2", etc.
|
||||
Always refer to people by their actual names (e.g., "John suggested..." not "A participant suggested...").
|
||||
"""
|
||||
).strip()
|
||||
|
||||
async def identify_participants(self) -> None:
|
||||
"""
|
||||
From a transcript, try to identify the participants using TreeSummarize with structured output.
|
||||
@@ -232,6 +280,19 @@ class SummaryBuilder:
|
||||
if unique_participants:
|
||||
participants_md = self.format_list_md(unique_participants)
|
||||
self.transcript += f"\n\n# Participants\n\n{participants_md}"
|
||||
|
||||
# Set instructions that will be automatically added to all prompts
|
||||
participants_list = ", ".join(unique_participants)
|
||||
self.participant_instructions = dedent(
|
||||
f"""
|
||||
# IMPORTANT: Participant Names
|
||||
The following participants are identified in this conversation: {participants_list}
|
||||
|
||||
You MUST use these specific participant names when referring to people in your response.
|
||||
Do NOT use generic terms like "a participant", "someone", "attendee", "Speaker 1", "Speaker 2", etc.
|
||||
Always refer to people by their actual names (e.g., "John suggested..." not "A participant suggested...").
|
||||
"""
|
||||
).strip()
|
||||
else:
|
||||
self.logger.warning("No participants identified in the transcript")
|
||||
|
||||
@@ -318,13 +379,13 @@ class SummaryBuilder:
|
||||
for subject in self.subjects:
|
||||
detailed_prompt = DETAILED_SUBJECT_PROMPT_TEMPLATE.format(subject=subject)
|
||||
|
||||
detailed_response = await self.llm.get_response(
|
||||
detailed_response = await self._get_response(
|
||||
detailed_prompt, [self.transcript], tone_name="Topic assistant"
|
||||
)
|
||||
|
||||
paragraph_prompt = PARAGRAPH_SUMMARY_PROMPT
|
||||
|
||||
paragraph_response = await self.llm.get_response(
|
||||
paragraph_response = await self._get_response(
|
||||
paragraph_prompt, [str(detailed_response)], tone_name="Topic summarizer"
|
||||
)
|
||||
|
||||
@@ -345,7 +406,7 @@ class SummaryBuilder:
|
||||
|
||||
recap_prompt = RECAP_PROMPT
|
||||
|
||||
recap_response = await self.llm.get_response(
|
||||
recap_response = await self._get_response(
|
||||
recap_prompt, [summaries_text], tone_name="Recap summarizer"
|
||||
)
|
||||
|
||||
|
||||
@@ -26,7 +26,25 @@ class TranscriptFinalSummaryProcessor(Processor):
|
||||
async def get_summary_builder(self, text) -> SummaryBuilder:
|
||||
builder = SummaryBuilder(self.llm, logger=self.logger)
|
||||
builder.set_transcript(text)
|
||||
await builder.identify_participants()
|
||||
|
||||
# Use known participants if available, otherwise identify them
|
||||
if self.transcript and self.transcript.participants:
|
||||
# Extract participant names from the stored participants
|
||||
participant_names = [p.name for p in self.transcript.participants if p.name]
|
||||
if participant_names:
|
||||
self.logger.info(
|
||||
f"Using {len(participant_names)} known participants from transcript"
|
||||
)
|
||||
builder.set_known_participants(participant_names)
|
||||
else:
|
||||
self.logger.info(
|
||||
"Participants field exists but is empty, identifying participants"
|
||||
)
|
||||
await builder.identify_participants()
|
||||
else:
|
||||
self.logger.info("No participants stored, identifying participants")
|
||||
await builder.identify_participants()
|
||||
|
||||
await builder.generate_summary()
|
||||
return builder
|
||||
|
||||
@@ -49,18 +67,30 @@ class TranscriptFinalSummaryProcessor(Processor):
|
||||
speakermap = {}
|
||||
if self.transcript:
|
||||
speakermap = {
|
||||
participant["speaker"]: participant["name"]
|
||||
for participant in self.transcript.participants
|
||||
p.speaker: p.name
|
||||
for p in (self.transcript.participants or [])
|
||||
if p.speaker is not None and p.name
|
||||
}
|
||||
self.logger.info(
|
||||
f"Built speaker map with {len(speakermap)} participants",
|
||||
speakermap=speakermap,
|
||||
)
|
||||
|
||||
# build the transcript as a single string
|
||||
# XXX: unsure if the participants name as replaced directly in speaker ?
|
||||
# Replace speaker IDs with actual participant names if available
|
||||
text_transcript = []
|
||||
unique_speakers = set()
|
||||
for topic in self.chunks:
|
||||
for segment in topic.transcript.as_segments():
|
||||
name = speakermap.get(segment.speaker, f"Speaker {segment.speaker}")
|
||||
unique_speakers.add((segment.speaker, name))
|
||||
text_transcript.append(f"{name}: {segment.text}")
|
||||
|
||||
self.logger.info(
|
||||
f"Built transcript with {len(unique_speakers)} unique speakers",
|
||||
speakers=list(unique_speakers),
|
||||
)
|
||||
|
||||
text_transcript = "\n".join(text_transcript)
|
||||
|
||||
last_chunk = self.chunks[-1]
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
from textwrap import dedent
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field
|
||||
from pydantic import AliasChoices, BaseModel, Field
|
||||
|
||||
from reflector.llm import LLM
|
||||
from reflector.processors.base import Processor
|
||||
@@ -34,13 +34,13 @@ TOPIC_PROMPT = dedent(
|
||||
class TopicResponse(BaseModel):
|
||||
"""Structured response for topic detection"""
|
||||
|
||||
model_config = ConfigDict(populate_by_name=True)
|
||||
|
||||
title: str = Field(
|
||||
description="A descriptive title for the topic being discussed", alias="Title"
|
||||
description="A descriptive title for the topic being discussed",
|
||||
validation_alias=AliasChoices("title", "Title"),
|
||||
)
|
||||
summary: str = Field(
|
||||
description="A concise 1-2 sentence summary of the discussion", alias="Summary"
|
||||
description="A concise 1-2 sentence summary of the discussion",
|
||||
validation_alias=AliasChoices("summary", "Summary"),
|
||||
)
|
||||
|
||||
|
||||
|
||||
5
server/reflector/schemas/platform.py
Normal file
5
server/reflector/schemas/platform.py
Normal file
@@ -0,0 +1,5 @@
|
||||
from typing import Literal
|
||||
|
||||
Platform = Literal["whereby", "daily"]
|
||||
WHEREBY_PLATFORM: Platform = "whereby"
|
||||
DAILY_PLATFORM: Platform = "daily"
|
||||
@@ -1,7 +1,7 @@
|
||||
from pydantic.types import PositiveInt
|
||||
from pydantic_settings import BaseSettings, SettingsConfigDict
|
||||
|
||||
from reflector.platform_types import Platform
|
||||
from reflector.schemas.platform import WHEREBY_PLATFORM, Platform
|
||||
from reflector.utils.string import NonEmptyString
|
||||
|
||||
|
||||
@@ -48,14 +48,17 @@ class Settings(BaseSettings):
|
||||
TRANSCRIPT_STORAGE_AWS_ACCESS_KEY_ID: str | None = None
|
||||
TRANSCRIPT_STORAGE_AWS_SECRET_ACCESS_KEY: str | None = None
|
||||
|
||||
# Recording storage
|
||||
RECORDING_STORAGE_BACKEND: str | None = None
|
||||
# Platform-specific recording storage (follows {PREFIX}_STORAGE_AWS_{CREDENTIAL} pattern)
|
||||
# Whereby storage configuration
|
||||
WHEREBY_STORAGE_AWS_BUCKET_NAME: str | None = None
|
||||
WHEREBY_STORAGE_AWS_REGION: str | None = None
|
||||
WHEREBY_STORAGE_AWS_ACCESS_KEY_ID: str | None = None
|
||||
WHEREBY_STORAGE_AWS_SECRET_ACCESS_KEY: str | None = None
|
||||
|
||||
# Recording storage configuration for AWS
|
||||
RECORDING_STORAGE_AWS_BUCKET_NAME: str = "recording-bucket"
|
||||
RECORDING_STORAGE_AWS_REGION: str = "us-east-1"
|
||||
RECORDING_STORAGE_AWS_ACCESS_KEY_ID: str | None = None
|
||||
RECORDING_STORAGE_AWS_SECRET_ACCESS_KEY: str | None = None
|
||||
# Daily.co storage configuration
|
||||
DAILYCO_STORAGE_AWS_BUCKET_NAME: str | None = None
|
||||
DAILYCO_STORAGE_AWS_REGION: str | None = None
|
||||
DAILYCO_STORAGE_AWS_ROLE_ARN: str | None = None
|
||||
|
||||
# Translate into the target language
|
||||
TRANSLATION_BACKEND: str = "passthrough"
|
||||
@@ -125,8 +128,6 @@ class Settings(BaseSettings):
|
||||
WHEREBY_API_URL: str = "https://api.whereby.dev/v1"
|
||||
WHEREBY_API_KEY: NonEmptyString | None = None
|
||||
WHEREBY_WEBHOOK_SECRET: str | None = None
|
||||
AWS_WHEREBY_ACCESS_KEY_ID: str | None = None
|
||||
AWS_WHEREBY_ACCESS_KEY_SECRET: str | None = None
|
||||
AWS_PROCESS_RECORDING_QUEUE_URL: str | None = None
|
||||
SQS_POLLING_TIMEOUT_SECONDS: int = 60
|
||||
|
||||
@@ -134,14 +135,12 @@ class Settings(BaseSettings):
|
||||
DAILY_API_KEY: str | None = None
|
||||
DAILY_WEBHOOK_SECRET: str | None = None
|
||||
DAILY_SUBDOMAIN: str | None = None
|
||||
AWS_DAILY_S3_BUCKET: str | None = None
|
||||
AWS_DAILY_S3_REGION: str = "us-west-2"
|
||||
AWS_DAILY_ROLE_ARN: str | None = None
|
||||
DAILY_WEBHOOK_UUID: str | None = (
|
||||
None # Webhook UUID for this environment. Not used by production code
|
||||
)
|
||||
|
||||
# Platform Migration Feature Flags
|
||||
DAILY_MIGRATION_ENABLED: bool = False
|
||||
DAILY_MIGRATION_ROOM_IDS: list[str] = []
|
||||
DEFAULT_VIDEO_PLATFORM: Platform = "whereby"
|
||||
# Platform Configuration
|
||||
DEFAULT_VIDEO_PLATFORM: Platform = WHEREBY_PLATFORM
|
||||
|
||||
# Zulip integration
|
||||
ZULIP_REALM: str | None = None
|
||||
|
||||
@@ -3,6 +3,13 @@ from reflector.settings import settings
|
||||
|
||||
|
||||
def get_transcripts_storage() -> Storage:
|
||||
"""
|
||||
Get storage for processed transcript files (master credentials).
|
||||
|
||||
Also use this for ALL our file operations with bucket override:
|
||||
master = get_transcripts_storage()
|
||||
master.delete_file(key, bucket=recording.bucket_name)
|
||||
"""
|
||||
assert settings.TRANSCRIPT_STORAGE_BACKEND
|
||||
return Storage.get_instance(
|
||||
name=settings.TRANSCRIPT_STORAGE_BACKEND,
|
||||
@@ -10,8 +17,53 @@ def get_transcripts_storage() -> Storage:
|
||||
)
|
||||
|
||||
|
||||
def get_recordings_storage() -> Storage:
|
||||
def get_whereby_storage() -> Storage:
|
||||
"""
|
||||
Get storage config for Whereby (for passing to Whereby API).
|
||||
|
||||
Usage:
|
||||
whereby_storage = get_whereby_storage()
|
||||
key_id, secret = whereby_storage.key_credentials
|
||||
whereby_api.create_meeting(
|
||||
bucket=whereby_storage.bucket_name,
|
||||
access_key_id=key_id,
|
||||
secret=secret,
|
||||
)
|
||||
|
||||
Do NOT use for our file operations - use get_transcripts_storage() instead.
|
||||
"""
|
||||
if not settings.WHEREBY_STORAGE_AWS_BUCKET_NAME:
|
||||
raise ValueError(
|
||||
"WHEREBY_STORAGE_AWS_BUCKET_NAME required for Whereby with AWS storage"
|
||||
)
|
||||
|
||||
return Storage.get_instance(
|
||||
name=settings.RECORDING_STORAGE_BACKEND,
|
||||
settings_prefix="RECORDING_STORAGE_",
|
||||
name="aws",
|
||||
settings_prefix="WHEREBY_STORAGE_",
|
||||
)
|
||||
|
||||
|
||||
def get_dailyco_storage() -> Storage:
|
||||
"""
|
||||
Get storage config for Daily.co (for passing to Daily API).
|
||||
|
||||
Usage:
|
||||
daily_storage = get_dailyco_storage()
|
||||
daily_api.create_meeting(
|
||||
bucket=daily_storage.bucket_name,
|
||||
region=daily_storage.region,
|
||||
role_arn=daily_storage.role_credential,
|
||||
)
|
||||
|
||||
Do NOT use for our file operations - use get_transcripts_storage() instead.
|
||||
"""
|
||||
# Fail fast if platform-specific config missing
|
||||
if not settings.DAILYCO_STORAGE_AWS_BUCKET_NAME:
|
||||
raise ValueError(
|
||||
"DAILYCO_STORAGE_AWS_BUCKET_NAME required for Daily.co with AWS storage"
|
||||
)
|
||||
|
||||
return Storage.get_instance(
|
||||
name="aws",
|
||||
settings_prefix="DAILYCO_STORAGE_",
|
||||
)
|
||||
|
||||
@@ -1,10 +1,23 @@
|
||||
import importlib
|
||||
from typing import BinaryIO, Union
|
||||
|
||||
from pydantic import BaseModel
|
||||
|
||||
from reflector.settings import settings
|
||||
|
||||
|
||||
class StorageError(Exception):
|
||||
"""Base exception for storage operations."""
|
||||
|
||||
pass
|
||||
|
||||
|
||||
class StoragePermissionError(StorageError):
|
||||
"""Exception raised when storage operation fails due to permission issues."""
|
||||
|
||||
pass
|
||||
|
||||
|
||||
class FileResult(BaseModel):
|
||||
filename: str
|
||||
url: str
|
||||
@@ -36,26 +49,113 @@ class Storage:
|
||||
|
||||
return cls._registry[name](**config)
|
||||
|
||||
async def put_file(self, filename: str, data: bytes) -> FileResult:
|
||||
return await self._put_file(filename, data)
|
||||
|
||||
async def _put_file(self, filename: str, data: bytes) -> FileResult:
|
||||
# Credential properties for API passthrough
|
||||
@property
|
||||
def bucket_name(self) -> str:
|
||||
"""Default bucket name for this storage instance."""
|
||||
raise NotImplementedError
|
||||
|
||||
async def delete_file(self, filename: str):
|
||||
return await self._delete_file(filename)
|
||||
|
||||
async def _delete_file(self, filename: str):
|
||||
@property
|
||||
def region(self) -> str:
|
||||
"""AWS region for this storage instance."""
|
||||
raise NotImplementedError
|
||||
|
||||
async def get_file_url(self, filename: str) -> str:
|
||||
return await self._get_file_url(filename)
|
||||
@property
|
||||
def access_key_id(self) -> str | None:
|
||||
"""AWS access key ID (None for role-based auth). Prefer key_credentials property."""
|
||||
return None
|
||||
|
||||
async def _get_file_url(self, filename: str) -> str:
|
||||
@property
|
||||
def secret_access_key(self) -> str | None:
|
||||
"""AWS secret access key (None for role-based auth). Prefer key_credentials property."""
|
||||
return None
|
||||
|
||||
@property
|
||||
def role_arn(self) -> str | None:
|
||||
"""AWS IAM role ARN for role-based auth (None for key-based auth). Prefer role_credential property."""
|
||||
return None
|
||||
|
||||
@property
|
||||
def key_credentials(self) -> tuple[str, str]:
|
||||
"""
|
||||
Get (access_key_id, secret_access_key) for key-based auth.
|
||||
Raises ValueError if storage uses IAM role instead.
|
||||
"""
|
||||
raise NotImplementedError
|
||||
|
||||
async def get_file(self, filename: str):
|
||||
return await self._get_file(filename)
|
||||
|
||||
async def _get_file(self, filename: str):
|
||||
@property
|
||||
def role_credential(self) -> str:
|
||||
"""
|
||||
Get IAM role ARN for role-based auth.
|
||||
Raises ValueError if storage uses access keys instead.
|
||||
"""
|
||||
raise NotImplementedError
|
||||
|
||||
async def put_file(
|
||||
self, filename: str, data: Union[bytes, BinaryIO], *, bucket: str | None = None
|
||||
) -> FileResult:
|
||||
"""Upload data. bucket: override instance default if provided."""
|
||||
return await self._put_file(filename, data, bucket=bucket)
|
||||
|
||||
async def _put_file(
|
||||
self, filename: str, data: Union[bytes, BinaryIO], *, bucket: str | None = None
|
||||
) -> FileResult:
|
||||
raise NotImplementedError
|
||||
|
||||
async def delete_file(self, filename: str, *, bucket: str | None = None):
|
||||
"""Delete file. bucket: override instance default if provided."""
|
||||
return await self._delete_file(filename, bucket=bucket)
|
||||
|
||||
async def _delete_file(self, filename: str, *, bucket: str | None = None):
|
||||
raise NotImplementedError
|
||||
|
||||
async def get_file_url(
|
||||
self,
|
||||
filename: str,
|
||||
operation: str = "get_object",
|
||||
expires_in: int = 3600,
|
||||
*,
|
||||
bucket: str | None = None,
|
||||
) -> str:
|
||||
"""Generate presigned URL. bucket: override instance default if provided."""
|
||||
return await self._get_file_url(filename, operation, expires_in, bucket=bucket)
|
||||
|
||||
async def _get_file_url(
|
||||
self,
|
||||
filename: str,
|
||||
operation: str = "get_object",
|
||||
expires_in: int = 3600,
|
||||
*,
|
||||
bucket: str | None = None,
|
||||
) -> str:
|
||||
raise NotImplementedError
|
||||
|
||||
async def get_file(self, filename: str, *, bucket: str | None = None):
|
||||
"""Download file. bucket: override instance default if provided."""
|
||||
return await self._get_file(filename, bucket=bucket)
|
||||
|
||||
async def _get_file(self, filename: str, *, bucket: str | None = None):
|
||||
raise NotImplementedError
|
||||
|
||||
async def list_objects(
|
||||
self, prefix: str = "", *, bucket: str | None = None
|
||||
) -> list[str]:
|
||||
"""List object keys. bucket: override instance default if provided."""
|
||||
return await self._list_objects(prefix, bucket=bucket)
|
||||
|
||||
async def _list_objects(
|
||||
self, prefix: str = "", *, bucket: str | None = None
|
||||
) -> list[str]:
|
||||
raise NotImplementedError
|
||||
|
||||
async def stream_to_fileobj(
|
||||
self, filename: str, fileobj: BinaryIO, *, bucket: str | None = None
|
||||
):
|
||||
"""Stream file directly to file object without loading into memory.
|
||||
bucket: override instance default if provided."""
|
||||
return await self._stream_to_fileobj(filename, fileobj, bucket=bucket)
|
||||
|
||||
async def _stream_to_fileobj(
|
||||
self, filename: str, fileobj: BinaryIO, *, bucket: str | None = None
|
||||
):
|
||||
raise NotImplementedError
|
||||
|
||||
@@ -1,79 +1,236 @@
|
||||
from functools import wraps
|
||||
from typing import BinaryIO, Union
|
||||
|
||||
import aioboto3
|
||||
from botocore.config import Config
|
||||
from botocore.exceptions import ClientError
|
||||
|
||||
from reflector.logger import logger
|
||||
from reflector.storage.base import FileResult, Storage
|
||||
from reflector.storage.base import FileResult, Storage, StoragePermissionError
|
||||
|
||||
|
||||
def handle_s3_client_errors(operation_name: str):
|
||||
"""Decorator to handle S3 ClientError with bucket-aware messaging.
|
||||
|
||||
Args:
|
||||
operation_name: Human-readable operation name for error messages (e.g., "upload", "delete")
|
||||
"""
|
||||
|
||||
def decorator(func):
|
||||
@wraps(func)
|
||||
async def wrapper(self, *args, **kwargs):
|
||||
bucket = kwargs.get("bucket")
|
||||
try:
|
||||
return await func(self, *args, **kwargs)
|
||||
except ClientError as e:
|
||||
error_code = e.response.get("Error", {}).get("Code")
|
||||
if error_code in ("AccessDenied", "NoSuchBucket"):
|
||||
actual_bucket = bucket or self._bucket_name
|
||||
bucket_context = (
|
||||
f"overridden bucket '{actual_bucket}'"
|
||||
if bucket
|
||||
else f"default bucket '{actual_bucket}'"
|
||||
)
|
||||
raise StoragePermissionError(
|
||||
f"S3 {operation_name} failed for {bucket_context}: {error_code}. "
|
||||
f"Check TRANSCRIPT_STORAGE_AWS_* credentials have permission."
|
||||
) from e
|
||||
raise
|
||||
|
||||
return wrapper
|
||||
|
||||
return decorator
|
||||
|
||||
|
||||
class AwsStorage(Storage):
|
||||
"""AWS S3 storage with bucket override for multi-platform recording architecture.
|
||||
Master credentials access all buckets via optional bucket parameter in operations."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
aws_access_key_id: str,
|
||||
aws_secret_access_key: str,
|
||||
aws_bucket_name: str,
|
||||
aws_region: str,
|
||||
aws_access_key_id: str | None = None,
|
||||
aws_secret_access_key: str | None = None,
|
||||
aws_role_arn: str | None = None,
|
||||
):
|
||||
if not aws_access_key_id:
|
||||
raise ValueError("Storage `aws_storage` require `aws_access_key_id`")
|
||||
if not aws_secret_access_key:
|
||||
raise ValueError("Storage `aws_storage` require `aws_secret_access_key`")
|
||||
if not aws_bucket_name:
|
||||
raise ValueError("Storage `aws_storage` require `aws_bucket_name`")
|
||||
if not aws_region:
|
||||
raise ValueError("Storage `aws_storage` require `aws_region`")
|
||||
if not aws_access_key_id and not aws_role_arn:
|
||||
raise ValueError(
|
||||
"Storage `aws_storage` require either `aws_access_key_id` or `aws_role_arn`"
|
||||
)
|
||||
if aws_role_arn and (aws_access_key_id or aws_secret_access_key):
|
||||
raise ValueError(
|
||||
"Storage `aws_storage` cannot use both `aws_role_arn` and access keys"
|
||||
)
|
||||
|
||||
super().__init__()
|
||||
self.aws_bucket_name = aws_bucket_name
|
||||
self._bucket_name = aws_bucket_name
|
||||
self._region = aws_region
|
||||
self._access_key_id = aws_access_key_id
|
||||
self._secret_access_key = aws_secret_access_key
|
||||
self._role_arn = aws_role_arn
|
||||
|
||||
self.aws_folder = ""
|
||||
if "/" in aws_bucket_name:
|
||||
self.aws_bucket_name, self.aws_folder = aws_bucket_name.split("/", 1)
|
||||
self._bucket_name, self.aws_folder = aws_bucket_name.split("/", 1)
|
||||
self.boto_config = Config(retries={"max_attempts": 3, "mode": "adaptive"})
|
||||
self.session = aioboto3.Session(
|
||||
aws_access_key_id=aws_access_key_id,
|
||||
aws_secret_access_key=aws_secret_access_key,
|
||||
region_name=aws_region,
|
||||
)
|
||||
self.base_url = f"https://{aws_bucket_name}.s3.amazonaws.com/"
|
||||
self.base_url = f"https://{self._bucket_name}.s3.amazonaws.com/"
|
||||
|
||||
async def _put_file(self, filename: str, data: bytes) -> FileResult:
|
||||
bucket = self.aws_bucket_name
|
||||
folder = self.aws_folder
|
||||
logger.info(f"Uploading {filename} to S3 {bucket}/{folder}")
|
||||
s3filename = f"{folder}/{filename}" if folder else filename
|
||||
async with self.session.client("s3") as client:
|
||||
await client.put_object(
|
||||
Bucket=bucket,
|
||||
Key=s3filename,
|
||||
Body=data,
|
||||
# Implement credential properties
|
||||
@property
|
||||
def bucket_name(self) -> str:
|
||||
return self._bucket_name
|
||||
|
||||
@property
|
||||
def region(self) -> str:
|
||||
return self._region
|
||||
|
||||
@property
|
||||
def access_key_id(self) -> str | None:
|
||||
return self._access_key_id
|
||||
|
||||
@property
|
||||
def secret_access_key(self) -> str | None:
|
||||
return self._secret_access_key
|
||||
|
||||
@property
|
||||
def role_arn(self) -> str | None:
|
||||
return self._role_arn
|
||||
|
||||
@property
|
||||
def key_credentials(self) -> tuple[str, str]:
|
||||
"""Get (access_key_id, secret_access_key) for key-based auth."""
|
||||
if self._role_arn:
|
||||
raise ValueError(
|
||||
"Storage uses IAM role authentication. "
|
||||
"Use role_credential property instead of key_credentials."
|
||||
)
|
||||
if not self._access_key_id or not self._secret_access_key:
|
||||
raise ValueError("Storage access key credentials not configured")
|
||||
return (self._access_key_id, self._secret_access_key)
|
||||
|
||||
async def _get_file_url(self, filename: str) -> FileResult:
|
||||
bucket = self.aws_bucket_name
|
||||
@property
|
||||
def role_credential(self) -> str:
|
||||
"""Get IAM role ARN for role-based auth."""
|
||||
if self._access_key_id or self._secret_access_key:
|
||||
raise ValueError(
|
||||
"Storage uses access key authentication. "
|
||||
"Use key_credentials property instead of role_credential."
|
||||
)
|
||||
if not self._role_arn:
|
||||
raise ValueError("Storage IAM role ARN not configured")
|
||||
return self._role_arn
|
||||
|
||||
@handle_s3_client_errors("upload")
|
||||
async def _put_file(
|
||||
self, filename: str, data: Union[bytes, BinaryIO], *, bucket: str | None = None
|
||||
) -> FileResult:
|
||||
actual_bucket = bucket or self._bucket_name
|
||||
folder = self.aws_folder
|
||||
s3filename = f"{folder}/{filename}" if folder else filename
|
||||
async with self.session.client("s3") as client:
|
||||
logger.info(f"Uploading {filename} to S3 {actual_bucket}/{folder}")
|
||||
|
||||
async with self.session.client("s3", config=self.boto_config) as client:
|
||||
if isinstance(data, bytes):
|
||||
await client.put_object(Bucket=actual_bucket, Key=s3filename, Body=data)
|
||||
else:
|
||||
# boto3 reads file-like object in chunks
|
||||
# avoids creating extra memory copy vs bytes.getvalue() approach
|
||||
await client.upload_fileobj(data, Bucket=actual_bucket, Key=s3filename)
|
||||
|
||||
url = await self._get_file_url(filename, bucket=bucket)
|
||||
return FileResult(filename=filename, url=url)
|
||||
|
||||
@handle_s3_client_errors("presign")
|
||||
async def _get_file_url(
|
||||
self,
|
||||
filename: str,
|
||||
operation: str = "get_object",
|
||||
expires_in: int = 3600,
|
||||
*,
|
||||
bucket: str | None = None,
|
||||
) -> str:
|
||||
actual_bucket = bucket or self._bucket_name
|
||||
folder = self.aws_folder
|
||||
s3filename = f"{folder}/{filename}" if folder else filename
|
||||
async with self.session.client("s3", config=self.boto_config) as client:
|
||||
presigned_url = await client.generate_presigned_url(
|
||||
"get_object",
|
||||
Params={"Bucket": bucket, "Key": s3filename},
|
||||
ExpiresIn=3600,
|
||||
operation,
|
||||
Params={"Bucket": actual_bucket, "Key": s3filename},
|
||||
ExpiresIn=expires_in,
|
||||
)
|
||||
|
||||
return presigned_url
|
||||
|
||||
async def _delete_file(self, filename: str):
|
||||
bucket = self.aws_bucket_name
|
||||
@handle_s3_client_errors("delete")
|
||||
async def _delete_file(self, filename: str, *, bucket: str | None = None):
|
||||
actual_bucket = bucket or self._bucket_name
|
||||
folder = self.aws_folder
|
||||
logger.info(f"Deleting {filename} from S3 {bucket}/{folder}")
|
||||
logger.info(f"Deleting {filename} from S3 {actual_bucket}/{folder}")
|
||||
s3filename = f"{folder}/{filename}" if folder else filename
|
||||
async with self.session.client("s3") as client:
|
||||
await client.delete_object(Bucket=bucket, Key=s3filename)
|
||||
async with self.session.client("s3", config=self.boto_config) as client:
|
||||
await client.delete_object(Bucket=actual_bucket, Key=s3filename)
|
||||
|
||||
async def _get_file(self, filename: str):
|
||||
bucket = self.aws_bucket_name
|
||||
@handle_s3_client_errors("download")
|
||||
async def _get_file(self, filename: str, *, bucket: str | None = None):
|
||||
actual_bucket = bucket or self._bucket_name
|
||||
folder = self.aws_folder
|
||||
logger.info(f"Downloading {filename} from S3 {bucket}/{folder}")
|
||||
logger.info(f"Downloading {filename} from S3 {actual_bucket}/{folder}")
|
||||
s3filename = f"{folder}/{filename}" if folder else filename
|
||||
async with self.session.client("s3") as client:
|
||||
response = await client.get_object(Bucket=bucket, Key=s3filename)
|
||||
async with self.session.client("s3", config=self.boto_config) as client:
|
||||
response = await client.get_object(Bucket=actual_bucket, Key=s3filename)
|
||||
return await response["Body"].read()
|
||||
|
||||
@handle_s3_client_errors("list_objects")
|
||||
async def _list_objects(
|
||||
self, prefix: str = "", *, bucket: str | None = None
|
||||
) -> list[str]:
|
||||
actual_bucket = bucket or self._bucket_name
|
||||
folder = self.aws_folder
|
||||
# Combine folder and prefix
|
||||
s3prefix = f"{folder}/{prefix}" if folder else prefix
|
||||
logger.info(f"Listing objects from S3 {actual_bucket} with prefix '{s3prefix}'")
|
||||
|
||||
keys = []
|
||||
async with self.session.client("s3", config=self.boto_config) as client:
|
||||
paginator = client.get_paginator("list_objects_v2")
|
||||
async for page in paginator.paginate(Bucket=actual_bucket, Prefix=s3prefix):
|
||||
if "Contents" in page:
|
||||
for obj in page["Contents"]:
|
||||
# Strip folder prefix from keys if present
|
||||
key = obj["Key"]
|
||||
if folder:
|
||||
if key.startswith(f"{folder}/"):
|
||||
key = key[len(folder) + 1 :]
|
||||
elif key == folder:
|
||||
# Skip folder marker itself
|
||||
continue
|
||||
keys.append(key)
|
||||
|
||||
return keys
|
||||
|
||||
@handle_s3_client_errors("stream")
|
||||
async def _stream_to_fileobj(
|
||||
self, filename: str, fileobj: BinaryIO, *, bucket: str | None = None
|
||||
):
|
||||
"""Stream file from S3 directly to file object without loading into memory."""
|
||||
actual_bucket = bucket or self._bucket_name
|
||||
folder = self.aws_folder
|
||||
logger.info(f"Streaming {filename} from S3 {actual_bucket}/{folder}")
|
||||
s3filename = f"{folder}/{filename}" if folder else filename
|
||||
async with self.session.client("s3", config=self.boto_config) as client:
|
||||
await client.download_fileobj(
|
||||
Bucket=actual_bucket, Key=s3filename, Fileobj=fileobj
|
||||
)
|
||||
|
||||
|
||||
Storage.register("aws", AwsStorage)
|
||||
|
||||
26
server/reflector/utils/daily.py
Normal file
26
server/reflector/utils/daily.py
Normal file
@@ -0,0 +1,26 @@
|
||||
from reflector.utils.string import NonEmptyString
|
||||
|
||||
DailyRoomName = str
|
||||
|
||||
|
||||
def extract_base_room_name(daily_room_name: DailyRoomName) -> NonEmptyString:
|
||||
"""
|
||||
Extract base room name from Daily.co timestamped room name.
|
||||
|
||||
Daily.co creates rooms with timestamp suffix: {base_name}-YYYYMMDDHHMMSS
|
||||
This function removes the timestamp to get the original room name.
|
||||
|
||||
Examples:
|
||||
"daily-20251020193458" → "daily"
|
||||
"daily-2-20251020193458" → "daily-2"
|
||||
"my-room-name-20251020193458" → "my-room-name"
|
||||
|
||||
Args:
|
||||
daily_room_name: Full Daily.co room name with optional timestamp
|
||||
|
||||
Returns:
|
||||
Base room name without timestamp suffix
|
||||
"""
|
||||
base_name = daily_room_name.rsplit("-", 1)[0]
|
||||
assert base_name, f"Extracted base name is empty from: {daily_room_name}"
|
||||
return base_name
|
||||
9
server/reflector/utils/datetime.py
Normal file
9
server/reflector/utils/datetime.py
Normal file
@@ -0,0 +1,9 @@
|
||||
from datetime import datetime, timezone
|
||||
|
||||
|
||||
def parse_datetime_with_timezone(iso_string: str) -> datetime:
|
||||
"""Parse ISO datetime string and ensure timezone awareness (defaults to UTC if naive)."""
|
||||
dt = datetime.fromisoformat(iso_string)
|
||||
if dt.tzinfo is None:
|
||||
dt = dt.replace(tzinfo=timezone.utc)
|
||||
return dt
|
||||
@@ -1,4 +1,4 @@
|
||||
from typing import Annotated
|
||||
from typing import Annotated, TypeVar
|
||||
|
||||
from pydantic import Field, TypeAdapter, constr
|
||||
|
||||
@@ -21,3 +21,12 @@ def try_parse_non_empty_string(s: str) -> NonEmptyString | None:
|
||||
if not s:
|
||||
return None
|
||||
return parse_non_empty_string(s)
|
||||
|
||||
|
||||
T = TypeVar("T", bound=str)
|
||||
|
||||
|
||||
def assert_equal[T](s1: T, s2: T) -> T:
|
||||
if s1 != s2:
|
||||
raise ValueError(f"assert_equal: {s1} != {s2}")
|
||||
return s1
|
||||
|
||||
37
server/reflector/utils/url.py
Normal file
37
server/reflector/utils/url.py
Normal file
@@ -0,0 +1,37 @@
|
||||
"""URL manipulation utilities."""
|
||||
|
||||
from urllib.parse import parse_qs, urlencode, urlparse, urlunparse
|
||||
|
||||
|
||||
def add_query_param(url: str, key: str, value: str) -> str:
|
||||
"""
|
||||
Add or update a query parameter in a URL.
|
||||
|
||||
Properly handles URLs with or without existing query parameters,
|
||||
preserving fragments and encoding special characters.
|
||||
|
||||
Args:
|
||||
url: The URL to modify
|
||||
key: The query parameter name
|
||||
value: The query parameter value
|
||||
|
||||
Returns:
|
||||
The URL with the query parameter added or updated
|
||||
|
||||
Examples:
|
||||
>>> add_query_param("https://example.com/room", "t", "token123")
|
||||
'https://example.com/room?t=token123'
|
||||
|
||||
>>> add_query_param("https://example.com/room?existing=param", "t", "token123")
|
||||
'https://example.com/room?existing=param&t=token123'
|
||||
"""
|
||||
parsed = urlparse(url)
|
||||
|
||||
query_params = parse_qs(parsed.query, keep_blank_values=True)
|
||||
|
||||
query_params[key] = [value]
|
||||
|
||||
new_query = urlencode(query_params, doseq=True)
|
||||
|
||||
new_parsed = parsed._replace(query=new_query)
|
||||
return urlunparse(new_parsed)
|
||||
@@ -1,10 +1,3 @@
|
||||
# Video Platform Abstraction Layer
|
||||
"""
|
||||
This module provides an abstraction layer for different video conferencing platforms.
|
||||
It allows seamless switching between providers (Whereby, Daily.co, etc.) without
|
||||
changing the core application logic.
|
||||
"""
|
||||
|
||||
from .base import VideoPlatformClient
|
||||
from .models import MeetingData, VideoPlatformConfig
|
||||
from .registry import get_platform_client, register_platform
|
||||
|
||||
@@ -1,18 +1,19 @@
|
||||
from abc import ABC, abstractmethod
|
||||
from datetime import datetime
|
||||
from typing import TYPE_CHECKING, Any, Dict, Optional
|
||||
|
||||
from reflector.platform_types import Platform
|
||||
from typing import TYPE_CHECKING, Any, Dict, List, Optional
|
||||
|
||||
from ..schemas.platform import Platform
|
||||
from ..utils.string import NonEmptyString
|
||||
from .models import MeetingData, VideoPlatformConfig
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from reflector.db.rooms import Room
|
||||
|
||||
# separator doesn't guarantee there's no more "ROOM_PREFIX_SEPARATOR" strings in room name
|
||||
ROOM_PREFIX_SEPARATOR = "-"
|
||||
|
||||
|
||||
class VideoPlatformClient(ABC):
|
||||
"""Abstract base class for video platform integrations."""
|
||||
|
||||
PLATFORM_NAME: Platform
|
||||
|
||||
def __init__(self, config: VideoPlatformConfig):
|
||||
@@ -20,36 +21,29 @@ class VideoPlatformClient(ABC):
|
||||
|
||||
@abstractmethod
|
||||
async def create_meeting(
|
||||
self, room_name_prefix: str, end_date: datetime, room: "Room"
|
||||
self, room_name_prefix: NonEmptyString, end_date: datetime, room: "Room"
|
||||
) -> MeetingData:
|
||||
"""Create a new meeting room."""
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
async def get_room_sessions(self, room_name: str) -> Dict[str, Any]:
|
||||
"""Get session information for a room."""
|
||||
async def get_room_sessions(self, room_name: str) -> List[Any] | None:
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
async def delete_room(self, room_name: str) -> bool:
|
||||
"""Delete a room. Returns True if successful."""
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
async def upload_logo(self, room_name: str, logo_path: str) -> bool:
|
||||
"""Upload a logo to the room. Returns True if successful."""
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def verify_webhook_signature(
|
||||
self, body: bytes, signature: str, timestamp: Optional[str] = None
|
||||
) -> bool:
|
||||
"""Verify webhook signature for security."""
|
||||
pass
|
||||
|
||||
def format_recording_config(self, room: "Room") -> Dict[str, Any]:
|
||||
"""Format recording configuration for the platform.
|
||||
Can be overridden by specific implementations."""
|
||||
if room.recording_type == "cloud" and self.config.s3_bucket:
|
||||
return {
|
||||
"type": room.recording_type,
|
||||
|
||||
@@ -1,15 +1,20 @@
|
||||
import base64
|
||||
import hmac
|
||||
from datetime import datetime
|
||||
from hashlib import sha256
|
||||
from http import HTTPStatus
|
||||
from typing import Any, Dict, Optional
|
||||
from typing import Any, Dict, List, Optional
|
||||
|
||||
import httpx
|
||||
|
||||
from reflector.db.rooms import Room
|
||||
from reflector.platform_types import Platform
|
||||
from reflector.logger import logger
|
||||
from reflector.storage import get_dailyco_storage
|
||||
|
||||
from .base import VideoPlatformClient
|
||||
from ..schemas.platform import Platform
|
||||
from ..utils.daily import DailyRoomName
|
||||
from ..utils.string import NonEmptyString
|
||||
from .base import ROOM_PREFIX_SEPARATOR, VideoPlatformClient
|
||||
from .models import MeetingData, RecordingType, VideoPlatformConfig
|
||||
|
||||
|
||||
@@ -29,14 +34,17 @@ class DailyClient(VideoPlatformClient):
|
||||
}
|
||||
|
||||
async def create_meeting(
|
||||
self, room_name_prefix: str, end_date: datetime, room: Room
|
||||
self, room_name_prefix: NonEmptyString, end_date: datetime, room: Room
|
||||
) -> MeetingData:
|
||||
"""Create a Daily.co room."""
|
||||
"""
|
||||
Daily.co rooms vs meetings:
|
||||
- We create a NEW Daily.co room for each Reflector meeting
|
||||
- Daily.co meeting/session starts automatically when first participant joins
|
||||
- Room auto-deletes after exp time
|
||||
- Meeting.room_name stores the timestamped Daily.co room name
|
||||
"""
|
||||
timestamp = datetime.now().strftime(self.TIMESTAMP_FORMAT)
|
||||
if room_name_prefix:
|
||||
room_name = f"{room_name_prefix}-{timestamp}"
|
||||
else:
|
||||
room_name = f"room-{timestamp}"
|
||||
room_name = f"{room_name_prefix}{ROOM_PREFIX_SEPARATOR}{timestamp}"
|
||||
|
||||
data = {
|
||||
"name": room_name,
|
||||
@@ -53,18 +61,16 @@ class DailyClient(VideoPlatformClient):
|
||||
},
|
||||
}
|
||||
|
||||
# Configure S3 bucket for recordings
|
||||
# NOTE: Not checking room.recording_type - figure out later if conditional needed
|
||||
assert self.config.s3_bucket, "S3 bucket must be configured"
|
||||
# Get storage config for passing to Daily API
|
||||
daily_storage = get_dailyco_storage()
|
||||
assert daily_storage.bucket_name, "S3 bucket must be configured"
|
||||
data["properties"]["recordings_bucket"] = {
|
||||
"bucket_name": self.config.s3_bucket,
|
||||
"bucket_region": self.config.s3_region,
|
||||
"assume_role_arn": self.config.aws_role_arn,
|
||||
"bucket_name": daily_storage.bucket_name,
|
||||
"bucket_region": daily_storage.region,
|
||||
"assume_role_arn": daily_storage.role_credential,
|
||||
"allow_api_access": True,
|
||||
}
|
||||
|
||||
from reflector.logger import logger
|
||||
|
||||
async with httpx.AsyncClient() as client:
|
||||
response = await client.post(
|
||||
f"{self.BASE_URL}/rooms",
|
||||
@@ -82,7 +88,6 @@ class DailyClient(VideoPlatformClient):
|
||||
response.raise_for_status()
|
||||
result = response.json()
|
||||
|
||||
# Format response to match our standard
|
||||
room_url = result["url"]
|
||||
|
||||
return MeetingData(
|
||||
@@ -94,19 +99,11 @@ class DailyClient(VideoPlatformClient):
|
||||
extra_data=result,
|
||||
)
|
||||
|
||||
async def get_room_sessions(self, room_name: str) -> Dict[str, Any]:
|
||||
"""Get Daily.co room information."""
|
||||
async with httpx.AsyncClient() as client:
|
||||
response = await client.get(
|
||||
f"{self.BASE_URL}/rooms/{room_name}",
|
||||
headers=self.headers,
|
||||
timeout=self.TIMEOUT,
|
||||
)
|
||||
response.raise_for_status()
|
||||
return response.json()
|
||||
async def get_room_sessions(self, room_name: str) -> List[Any] | None:
|
||||
# no such api
|
||||
return None
|
||||
|
||||
async def get_room_presence(self, room_name: str) -> Dict[str, Any]:
|
||||
"""Get real-time participant data - Daily.co specific feature."""
|
||||
async with httpx.AsyncClient() as client:
|
||||
response = await client.get(
|
||||
f"{self.BASE_URL}/rooms/{room_name}/presence",
|
||||
@@ -116,19 +113,36 @@ class DailyClient(VideoPlatformClient):
|
||||
response.raise_for_status()
|
||||
return response.json()
|
||||
|
||||
async def get_meeting_participants(self, meeting_id: str) -> Dict[str, Any]:
|
||||
async with httpx.AsyncClient() as client:
|
||||
response = await client.get(
|
||||
f"{self.BASE_URL}/meetings/{meeting_id}/participants",
|
||||
headers=self.headers,
|
||||
timeout=self.TIMEOUT,
|
||||
)
|
||||
response.raise_for_status()
|
||||
return response.json()
|
||||
|
||||
async def get_recording(self, recording_id: str) -> Dict[str, Any]:
|
||||
async with httpx.AsyncClient() as client:
|
||||
response = await client.get(
|
||||
f"{self.BASE_URL}/recordings/{recording_id}",
|
||||
headers=self.headers,
|
||||
timeout=self.TIMEOUT,
|
||||
)
|
||||
response.raise_for_status()
|
||||
return response.json()
|
||||
|
||||
async def delete_room(self, room_name: str) -> bool:
|
||||
"""Delete a Daily.co room."""
|
||||
async with httpx.AsyncClient() as client:
|
||||
response = await client.delete(
|
||||
f"{self.BASE_URL}/rooms/{room_name}",
|
||||
headers=self.headers,
|
||||
timeout=self.TIMEOUT,
|
||||
)
|
||||
# Daily.co returns 200 for success, 404 if room doesn't exist
|
||||
return response.status_code in (HTTPStatus.OK, HTTPStatus.NOT_FOUND)
|
||||
|
||||
async def upload_logo(self, room_name: str, logo_path: str) -> bool:
|
||||
"""Daily.co doesn't support custom logos per room - this is a no-op."""
|
||||
return True
|
||||
|
||||
def verify_webhook_signature(
|
||||
@@ -146,8 +160,6 @@ class DailyClient(VideoPlatformClient):
|
||||
return False
|
||||
|
||||
try:
|
||||
import base64
|
||||
|
||||
secret_bytes = base64.b64decode(self.config.webhook_secret)
|
||||
|
||||
signed_content = timestamp.encode() + b"." + body
|
||||
@@ -156,17 +168,25 @@ class DailyClient(VideoPlatformClient):
|
||||
expected_b64 = base64.b64encode(expected).decode()
|
||||
|
||||
return hmac.compare_digest(expected_b64, signature)
|
||||
except Exception:
|
||||
except Exception as e:
|
||||
logger.error("Daily.co webhook signature verification failed", exc_info=e)
|
||||
return False
|
||||
|
||||
async def create_meeting_token(self, room_name: str, enable_recording: bool) -> str:
|
||||
"""Create meeting token for auto-recording."""
|
||||
async def create_meeting_token(
|
||||
self,
|
||||
room_name: DailyRoomName,
|
||||
enable_recording: bool,
|
||||
user_id: Optional[str] = None,
|
||||
) -> str:
|
||||
data = {"properties": {"room_name": room_name}}
|
||||
|
||||
if enable_recording:
|
||||
data["properties"]["start_cloud_recording"] = True
|
||||
data["properties"]["enable_recording_ui"] = False
|
||||
|
||||
if user_id:
|
||||
data["properties"]["user_id"] = user_id
|
||||
|
||||
async with httpx.AsyncClient() as client:
|
||||
response = await client.post(
|
||||
f"{self.BASE_URL}/meeting-tokens",
|
||||
|
||||
@@ -1,29 +1,30 @@
|
||||
"""Factory for creating video platform clients based on configuration."""
|
||||
|
||||
from typing import Optional
|
||||
|
||||
from reflector.settings import settings
|
||||
from reflector.storage import get_dailyco_storage, get_whereby_storage
|
||||
|
||||
from .base import Platform, VideoPlatformClient, VideoPlatformConfig
|
||||
from ..schemas.platform import WHEREBY_PLATFORM, Platform
|
||||
from .base import VideoPlatformClient, VideoPlatformConfig
|
||||
from .registry import get_platform_client
|
||||
|
||||
|
||||
def get_platform_config(platform: Platform) -> VideoPlatformConfig:
|
||||
"""Get configuration for a specific platform."""
|
||||
if platform == "whereby":
|
||||
if platform == WHEREBY_PLATFORM:
|
||||
if not settings.WHEREBY_API_KEY:
|
||||
raise ValueError(
|
||||
"WHEREBY_API_KEY is required when platform='whereby'. "
|
||||
"Set WHEREBY_API_KEY environment variable."
|
||||
)
|
||||
whereby_storage = get_whereby_storage()
|
||||
key_id, secret = whereby_storage.key_credentials
|
||||
return VideoPlatformConfig(
|
||||
api_key=settings.WHEREBY_API_KEY,
|
||||
webhook_secret=settings.WHEREBY_WEBHOOK_SECRET or "",
|
||||
api_url=settings.WHEREBY_API_URL,
|
||||
s3_bucket=settings.RECORDING_STORAGE_AWS_BUCKET_NAME,
|
||||
s3_region=settings.RECORDING_STORAGE_AWS_REGION,
|
||||
aws_access_key_id=settings.AWS_WHEREBY_ACCESS_KEY_ID,
|
||||
aws_access_key_secret=settings.AWS_WHEREBY_ACCESS_KEY_SECRET,
|
||||
s3_bucket=whereby_storage.bucket_name,
|
||||
s3_region=whereby_storage.region,
|
||||
aws_access_key_id=key_id,
|
||||
aws_access_key_secret=secret,
|
||||
)
|
||||
elif platform == "daily":
|
||||
if not settings.DAILY_API_KEY:
|
||||
@@ -36,45 +37,26 @@ def get_platform_config(platform: Platform) -> VideoPlatformConfig:
|
||||
"DAILY_SUBDOMAIN is required when platform='daily'. "
|
||||
"Set DAILY_SUBDOMAIN environment variable."
|
||||
)
|
||||
daily_storage = get_dailyco_storage()
|
||||
return VideoPlatformConfig(
|
||||
api_key=settings.DAILY_API_KEY,
|
||||
webhook_secret=settings.DAILY_WEBHOOK_SECRET or "",
|
||||
subdomain=settings.DAILY_SUBDOMAIN,
|
||||
s3_bucket=settings.AWS_DAILY_S3_BUCKET,
|
||||
s3_region=settings.AWS_DAILY_S3_REGION,
|
||||
aws_role_arn=settings.AWS_DAILY_ROLE_ARN,
|
||||
s3_bucket=daily_storage.bucket_name,
|
||||
s3_region=daily_storage.region,
|
||||
aws_role_arn=daily_storage.role_credential,
|
||||
)
|
||||
else:
|
||||
raise ValueError(f"Unknown platform: {platform}")
|
||||
|
||||
|
||||
def create_platform_client(platform: Platform) -> VideoPlatformClient:
|
||||
"""Create a video platform client instance."""
|
||||
config = get_platform_config(platform)
|
||||
return get_platform_client(platform, config)
|
||||
|
||||
|
||||
def get_platform_for_room(
|
||||
room_id: Optional[str] = None, room_platform: Optional[Platform] = None
|
||||
) -> Platform:
|
||||
"""Determine which platform to use for a room.
|
||||
|
||||
Priority order (highest to lowest):
|
||||
1. DAILY_MIGRATION_ROOM_IDS - env var override for testing/migration
|
||||
2. room_platform - database persisted platform choice
|
||||
3. DEFAULT_VIDEO_PLATFORM - env var fallback
|
||||
"""
|
||||
# If Daily migration is disabled, always use Whereby
|
||||
if not settings.DAILY_MIGRATION_ENABLED:
|
||||
return "whereby"
|
||||
|
||||
# Highest priority: If room is in migration list, use Daily (env var override)
|
||||
if room_id and room_id in settings.DAILY_MIGRATION_ROOM_IDS:
|
||||
return "daily"
|
||||
|
||||
# Second priority: Use room's persisted platform from database
|
||||
def get_platform(room_platform: Optional[Platform] = None) -> Platform:
|
||||
if room_platform:
|
||||
return room_platform
|
||||
|
||||
# Fallback: Use default platform from env var
|
||||
return settings.DEFAULT_VIDEO_PLATFORM
|
||||
|
||||
@@ -1,20 +1,13 @@
|
||||
"""Video platform data models.
|
||||
|
||||
Standard data models used across all video platform implementations.
|
||||
"""
|
||||
|
||||
from typing import Any, Dict, Literal, Optional
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from reflector.platform_types import Platform
|
||||
from reflector.schemas.platform import WHEREBY_PLATFORM, Platform
|
||||
|
||||
RecordingType = Literal["none", "local", "cloud"]
|
||||
|
||||
|
||||
class MeetingData(BaseModel):
|
||||
"""Standardized meeting data returned by all providers."""
|
||||
|
||||
platform: Platform
|
||||
meeting_id: str = Field(description="Platform-specific meeting identifier")
|
||||
room_url: str = Field(description="URL for participants to join")
|
||||
@@ -25,7 +18,7 @@ class MeetingData(BaseModel):
|
||||
class Config:
|
||||
json_schema_extra = {
|
||||
"example": {
|
||||
"platform": "whereby",
|
||||
"platform": WHEREBY_PLATFORM,
|
||||
"meeting_id": "12345678",
|
||||
"room_url": "https://subdomain.whereby.com/room-20251008120000",
|
||||
"host_room_url": "https://subdomain.whereby.com/room-20251008120000?roomKey=abc123",
|
||||
@@ -35,8 +28,6 @@ class MeetingData(BaseModel):
|
||||
|
||||
|
||||
class VideoPlatformConfig(BaseModel):
|
||||
"""Platform-agnostic configuration model."""
|
||||
|
||||
api_key: str
|
||||
webhook_secret: str
|
||||
api_url: Optional[str] = None
|
||||
|
||||
@@ -1,20 +1,18 @@
|
||||
from typing import Dict, Type
|
||||
|
||||
from .base import Platform, VideoPlatformClient, VideoPlatformConfig
|
||||
from ..schemas.platform import DAILY_PLATFORM, WHEREBY_PLATFORM, Platform
|
||||
from .base import VideoPlatformClient, VideoPlatformConfig
|
||||
|
||||
# Registry of available video platforms
|
||||
_PLATFORMS: Dict[Platform, Type[VideoPlatformClient]] = {}
|
||||
|
||||
|
||||
def register_platform(name: Platform, client_class: Type[VideoPlatformClient]):
|
||||
"""Register a video platform implementation."""
|
||||
_PLATFORMS[name] = client_class
|
||||
|
||||
|
||||
def get_platform_client(
|
||||
platform: Platform, config: VideoPlatformConfig
|
||||
) -> VideoPlatformClient:
|
||||
"""Get a video platform client instance."""
|
||||
if platform not in _PLATFORMS:
|
||||
raise ValueError(f"Unknown video platform: {platform}")
|
||||
|
||||
@@ -23,17 +21,15 @@ def get_platform_client(
|
||||
|
||||
|
||||
def get_available_platforms() -> list[Platform]:
|
||||
"""Get list of available platform names."""
|
||||
return list(_PLATFORMS.keys())
|
||||
|
||||
|
||||
# Auto-register built-in platforms
|
||||
def _register_builtin_platforms():
|
||||
from .daily import DailyClient # noqa: PLC0415
|
||||
from .whereby import WherebyClient # noqa: PLC0415
|
||||
|
||||
register_platform("whereby", WherebyClient)
|
||||
register_platform("daily", DailyClient)
|
||||
register_platform(WHEREBY_PLATFORM, WherebyClient)
|
||||
register_platform(DAILY_PLATFORM, DailyClient)
|
||||
|
||||
|
||||
_register_builtin_platforms()
|
||||
|
||||
@@ -9,14 +9,20 @@ from typing import Any, Dict, Optional
|
||||
import httpx
|
||||
|
||||
from reflector.db.rooms import Room
|
||||
from reflector.storage import get_whereby_storage
|
||||
|
||||
from .base import MeetingData, Platform, VideoPlatformClient, VideoPlatformConfig
|
||||
from ..schemas.platform import WHEREBY_PLATFORM, Platform
|
||||
from ..utils.string import NonEmptyString
|
||||
from .base import (
|
||||
MeetingData,
|
||||
VideoPlatformClient,
|
||||
VideoPlatformConfig,
|
||||
)
|
||||
from .whereby_utils import whereby_room_name_prefix
|
||||
|
||||
|
||||
class WherebyClient(VideoPlatformClient):
|
||||
"""Whereby video platform implementation."""
|
||||
|
||||
PLATFORM_NAME: Platform = "whereby"
|
||||
PLATFORM_NAME: Platform = WHEREBY_PLATFORM
|
||||
TIMEOUT = 10 # seconds
|
||||
MAX_ELAPSED_TIME = 60 * 1000 # 1 minute in milliseconds
|
||||
|
||||
@@ -28,27 +34,28 @@ class WherebyClient(VideoPlatformClient):
|
||||
}
|
||||
|
||||
async def create_meeting(
|
||||
self, room_name_prefix: str, end_date: datetime, room: Room
|
||||
self, room_name_prefix: NonEmptyString, end_date: datetime, room: Room
|
||||
) -> MeetingData:
|
||||
"""Create a Whereby meeting."""
|
||||
data = {
|
||||
"isLocked": room.is_locked,
|
||||
"roomNamePrefix": room_name_prefix,
|
||||
"roomNamePrefix": whereby_room_name_prefix(room_name_prefix),
|
||||
"roomNamePattern": "uuid",
|
||||
"roomMode": room.room_mode,
|
||||
"endDate": end_date.isoformat(),
|
||||
"fields": ["hostRoomUrl"],
|
||||
}
|
||||
|
||||
# Add recording configuration if cloud recording is enabled
|
||||
if room.recording_type == "cloud":
|
||||
# Get storage config for passing credentials to Whereby API
|
||||
whereby_storage = get_whereby_storage()
|
||||
key_id, secret = whereby_storage.key_credentials
|
||||
data["recording"] = {
|
||||
"type": room.recording_type,
|
||||
"destination": {
|
||||
"provider": "s3",
|
||||
"bucket": self.config.s3_bucket,
|
||||
"accessKeyId": self.config.aws_access_key_id,
|
||||
"accessKeySecret": self.config.aws_access_key_secret,
|
||||
"bucket": whereby_storage.bucket_name,
|
||||
"accessKeyId": key_id,
|
||||
"accessKeySecret": secret,
|
||||
"fileFormat": "mp4",
|
||||
},
|
||||
"startTrigger": room.recording_trigger,
|
||||
@@ -74,7 +81,6 @@ class WherebyClient(VideoPlatformClient):
|
||||
)
|
||||
|
||||
async def get_room_sessions(self, room_name: str) -> Dict[str, Any]:
|
||||
"""Get Whereby room session information."""
|
||||
async with httpx.AsyncClient() as client:
|
||||
response = await client.get(
|
||||
f"{self.config.api_url}/insights/room-sessions?roomName={room_name}",
|
||||
@@ -82,14 +88,12 @@ class WherebyClient(VideoPlatformClient):
|
||||
timeout=self.TIMEOUT,
|
||||
)
|
||||
response.raise_for_status()
|
||||
return response.json()
|
||||
return response.json().get("results", [])
|
||||
|
||||
async def delete_room(self, room_name: str) -> bool:
|
||||
"""Whereby doesn't support room deletion - meetings expire automatically."""
|
||||
return True
|
||||
|
||||
async def upload_logo(self, room_name: str, logo_path: str) -> bool:
|
||||
"""Upload logo to Whereby room."""
|
||||
async with httpx.AsyncClient() as client:
|
||||
with open(logo_path, "rb") as f:
|
||||
response = await client.put(
|
||||
@@ -106,7 +110,6 @@ class WherebyClient(VideoPlatformClient):
|
||||
def verify_webhook_signature(
|
||||
self, body: bytes, signature: str, timestamp: Optional[str] = None
|
||||
) -> bool:
|
||||
"""Verify Whereby webhook signature."""
|
||||
if not signature:
|
||||
return False
|
||||
|
||||
@@ -116,13 +119,11 @@ class WherebyClient(VideoPlatformClient):
|
||||
|
||||
ts, sig = matches.groups()
|
||||
|
||||
# Check timestamp to prevent replay attacks
|
||||
current_time = int(time.time() * 1000)
|
||||
diff_time = current_time - int(ts) * 1000
|
||||
if diff_time >= self.MAX_ELAPSED_TIME:
|
||||
return False
|
||||
|
||||
# Verify signature
|
||||
body_dict = json.loads(body)
|
||||
signed_payload = f"{ts}.{json.dumps(body_dict, separators=(',', ':'))}"
|
||||
hmac_obj = hmac.new(
|
||||
|
||||
38
server/reflector/video_platforms/whereby_utils.py
Normal file
38
server/reflector/video_platforms/whereby_utils.py
Normal file
@@ -0,0 +1,38 @@
|
||||
import re
|
||||
from datetime import datetime
|
||||
|
||||
from reflector.utils.datetime import parse_datetime_with_timezone
|
||||
from reflector.utils.string import NonEmptyString, parse_non_empty_string
|
||||
from reflector.video_platforms.base import ROOM_PREFIX_SEPARATOR
|
||||
|
||||
|
||||
def parse_whereby_recording_filename(
|
||||
object_key: NonEmptyString,
|
||||
) -> (NonEmptyString, datetime):
|
||||
filename = parse_non_empty_string(object_key.rsplit(".", 1)[0])
|
||||
timestamp_pattern = r"(\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}Z)"
|
||||
match = re.search(timestamp_pattern, filename)
|
||||
if not match:
|
||||
raise ValueError(f"No ISO timestamp found in filename: {object_key}")
|
||||
timestamp_str = match.group(1)
|
||||
timestamp_start = match.start(1)
|
||||
room_name_part = filename[:timestamp_start]
|
||||
if room_name_part.endswith(ROOM_PREFIX_SEPARATOR):
|
||||
room_name_part = room_name_part[: -len(ROOM_PREFIX_SEPARATOR)]
|
||||
else:
|
||||
raise ValueError(
|
||||
f"room name {room_name_part} doesnt have {ROOM_PREFIX_SEPARATOR} at the end of filename: {object_key}"
|
||||
)
|
||||
|
||||
return parse_non_empty_string(room_name_part), parse_datetime_with_timezone(
|
||||
timestamp_str
|
||||
)
|
||||
|
||||
|
||||
def whereby_room_name_prefix(room_name_prefix: NonEmptyString) -> NonEmptyString:
|
||||
return room_name_prefix + ROOM_PREFIX_SEPARATOR
|
||||
|
||||
|
||||
# room name comes with "/" from whereby api but lacks "/" e.g. in recording filenames
|
||||
def room_name_to_whereby_api_room_name(room_name: NonEmptyString) -> NonEmptyString:
|
||||
return f"/{room_name}"
|
||||
@@ -1,5 +1,3 @@
|
||||
"""Daily.co webhook handler endpoint."""
|
||||
|
||||
import json
|
||||
from typing import Any, Dict, Literal
|
||||
|
||||
@@ -7,25 +5,24 @@ from fastapi import APIRouter, HTTPException, Request
|
||||
from pydantic import BaseModel
|
||||
|
||||
from reflector.db.meetings import meetings_controller
|
||||
from reflector.logger import logger
|
||||
from reflector.logger import logger as _logger
|
||||
from reflector.settings import settings
|
||||
from reflector.utils.daily import DailyRoomName
|
||||
from reflector.video_platforms.factory import create_platform_client
|
||||
from reflector.worker.process import process_multitrack_recording
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
logger = _logger.bind(platform="daily")
|
||||
|
||||
|
||||
class DailyTrack(BaseModel):
|
||||
"""Daily.co recording track (audio or video file)."""
|
||||
|
||||
type: Literal["audio", "video"]
|
||||
s3Key: str
|
||||
size: int
|
||||
|
||||
|
||||
class DailyWebhookEvent(BaseModel):
|
||||
"""Daily webhook event structure."""
|
||||
|
||||
version: str
|
||||
type: str
|
||||
id: str
|
||||
@@ -33,7 +30,7 @@ class DailyWebhookEvent(BaseModel):
|
||||
event_ts: float
|
||||
|
||||
|
||||
def _extract_room_name(event: DailyWebhookEvent) -> str | None:
|
||||
def _extract_room_name(event: DailyWebhookEvent) -> DailyRoomName | None:
|
||||
"""Extract room name from Daily event payload.
|
||||
|
||||
Daily.co API inconsistency:
|
||||
@@ -69,13 +66,11 @@ async def webhook(request: Request):
|
||||
)
|
||||
raise HTTPException(status_code=401, detail="Invalid webhook signature")
|
||||
|
||||
# Parse the JSON body
|
||||
try:
|
||||
body_json = json.loads(body)
|
||||
except json.JSONDecodeError:
|
||||
raise HTTPException(status_code=422, detail="Invalid JSON")
|
||||
|
||||
# Handle Daily's test event during webhook creation
|
||||
if body_json.get("test") == "test":
|
||||
logger.info("Received Daily webhook test event")
|
||||
return {"status": "ok"}
|
||||
@@ -98,33 +93,39 @@ async def webhook(request: Request):
|
||||
await _handle_recording_ready(event)
|
||||
elif event.type == "recording.error":
|
||||
await _handle_recording_error(event)
|
||||
else:
|
||||
logger.warning(
|
||||
"Unhandled Daily webhook event type",
|
||||
event_type=event.type,
|
||||
payload=event.payload,
|
||||
)
|
||||
|
||||
return {"status": "ok"}
|
||||
|
||||
|
||||
async def _handle_participant_joined(event: DailyWebhookEvent):
|
||||
"""Handle participant joined event."""
|
||||
room_name = _extract_room_name(event)
|
||||
if not room_name:
|
||||
daily_room_name = _extract_room_name(event)
|
||||
if not daily_room_name:
|
||||
logger.warning("participant.joined: no room in payload", payload=event.payload)
|
||||
return
|
||||
|
||||
meeting = await meetings_controller.get_by_room_name(room_name)
|
||||
meeting = await meetings_controller.get_by_room_name(daily_room_name)
|
||||
if meeting:
|
||||
await meetings_controller.increment_num_clients(meeting.id)
|
||||
logger.info(
|
||||
"Participant joined",
|
||||
meeting_id=meeting.id,
|
||||
room_name=room_name,
|
||||
room_name=daily_room_name,
|
||||
recording_type=meeting.recording_type,
|
||||
recording_trigger=meeting.recording_trigger,
|
||||
)
|
||||
else:
|
||||
logger.warning("participant.joined: meeting not found", room_name=room_name)
|
||||
logger.warning(
|
||||
"participant.joined: meeting not found", room_name=daily_room_name
|
||||
)
|
||||
|
||||
|
||||
async def _handle_participant_left(event: DailyWebhookEvent):
|
||||
"""Handle participant left event."""
|
||||
room_name = _extract_room_name(event)
|
||||
if not room_name:
|
||||
return
|
||||
@@ -135,7 +136,6 @@ async def _handle_participant_left(event: DailyWebhookEvent):
|
||||
|
||||
|
||||
async def _handle_recording_started(event: DailyWebhookEvent):
|
||||
"""Handle recording started event."""
|
||||
room_name = _extract_room_name(event)
|
||||
if not room_name:
|
||||
logger.warning(
|
||||
@@ -182,7 +182,6 @@ async def _handle_recording_ready(event: DailyWebhookEvent):
|
||||
)
|
||||
return
|
||||
|
||||
# Validate tracks structure
|
||||
try:
|
||||
tracks = [DailyTrack(**t) for t in tracks_raw]
|
||||
except Exception as e:
|
||||
@@ -201,10 +200,10 @@ async def _handle_recording_ready(event: DailyWebhookEvent):
|
||||
platform="daily",
|
||||
)
|
||||
|
||||
bucket_name = settings.AWS_DAILY_S3_BUCKET
|
||||
bucket_name = settings.DAILYCO_STORAGE_AWS_BUCKET_NAME
|
||||
if not bucket_name:
|
||||
logger.error(
|
||||
"AWS_DAILY_S3_BUCKET not configured; cannot process Daily recording"
|
||||
"DAILYCO_STORAGE_AWS_BUCKET_NAME not configured; cannot process Daily recording"
|
||||
)
|
||||
return
|
||||
|
||||
@@ -212,14 +211,13 @@ async def _handle_recording_ready(event: DailyWebhookEvent):
|
||||
|
||||
process_multitrack_recording.delay(
|
||||
bucket_name=bucket_name,
|
||||
room_name=room_name,
|
||||
daily_room_name=room_name,
|
||||
recording_id=recording_id,
|
||||
track_keys=track_keys,
|
||||
)
|
||||
|
||||
|
||||
async def _handle_recording_error(event: DailyWebhookEvent):
|
||||
"""Handle recording error event."""
|
||||
room_name = _extract_room_name(event)
|
||||
error = event.payload.get("error", "Unknown error")
|
||||
|
||||
|
||||
@@ -15,12 +15,13 @@ from reflector.db.calendar_events import calendar_events_controller
|
||||
from reflector.db.meetings import meetings_controller
|
||||
from reflector.db.rooms import rooms_controller
|
||||
from reflector.redis_cache import RedisAsyncLock
|
||||
from reflector.schemas.platform import Platform
|
||||
from reflector.services.ics_sync import ics_sync_service
|
||||
from reflector.settings import settings
|
||||
from reflector.video_platforms.base import Platform
|
||||
from reflector.utils.url import add_query_param
|
||||
from reflector.video_platforms.factory import (
|
||||
create_platform_client,
|
||||
get_platform_for_room,
|
||||
get_platform,
|
||||
)
|
||||
from reflector.worker.webhook import test_webhook
|
||||
|
||||
@@ -45,7 +46,7 @@ class Room(BaseModel):
|
||||
ics_enabled: bool = False
|
||||
ics_last_sync: Optional[datetime] = None
|
||||
ics_last_etag: Optional[str] = None
|
||||
platform: Platform = "whereby"
|
||||
platform: Platform
|
||||
|
||||
|
||||
class RoomDetails(Room):
|
||||
@@ -73,7 +74,7 @@ class Meeting(BaseModel):
|
||||
is_active: bool = True
|
||||
calendar_event_id: str | None = None
|
||||
calendar_metadata: dict[str, Any] | None = None
|
||||
platform: Platform = "whereby"
|
||||
platform: Platform
|
||||
|
||||
|
||||
class CreateRoom(BaseModel):
|
||||
@@ -173,14 +174,6 @@ class CalendarEventResponse(BaseModel):
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
def parse_datetime_with_timezone(iso_string: str) -> datetime:
|
||||
"""Parse ISO datetime string and ensure timezone awareness (defaults to UTC if naive)."""
|
||||
dt = datetime.fromisoformat(iso_string)
|
||||
if dt.tzinfo is None:
|
||||
dt = dt.replace(tzinfo=timezone.utc)
|
||||
return dt
|
||||
|
||||
|
||||
@router.get("/rooms", response_model=Page[RoomDetails])
|
||||
async def rooms_list(
|
||||
user: Annotated[Optional[auth.UserInfo], Depends(auth.current_user_optional)],
|
||||
@@ -198,7 +191,7 @@ async def rooms_list(
|
||||
)
|
||||
|
||||
for room in paginated.items:
|
||||
room.platform = get_platform_for_room(room.id, room.platform)
|
||||
room.platform = get_platform(room.platform)
|
||||
|
||||
return paginated
|
||||
|
||||
@@ -214,7 +207,7 @@ async def rooms_get(
|
||||
raise HTTPException(status_code=404, detail="Room not found")
|
||||
if not room.is_shared and (user_id is None or room.user_id != user_id):
|
||||
raise HTTPException(status_code=403, detail="Room access denied")
|
||||
room.platform = get_platform_for_room(room.id, room.platform)
|
||||
room.platform = get_platform(room.platform)
|
||||
return room
|
||||
|
||||
|
||||
@@ -236,7 +229,7 @@ async def rooms_get_by_name(
|
||||
room_dict["webhook_url"] = None
|
||||
room_dict["webhook_secret"] = None
|
||||
|
||||
room_dict["platform"] = get_platform_for_room(room.id, room.platform)
|
||||
room_dict["platform"] = get_platform(room.platform)
|
||||
|
||||
return RoomDetails(**room_dict)
|
||||
|
||||
@@ -282,7 +275,7 @@ async def rooms_update(
|
||||
raise HTTPException(status_code=403, detail="Not authorized")
|
||||
values = info.dict(exclude_unset=True)
|
||||
await rooms_controller.update(room, values)
|
||||
room.platform = get_platform_for_room(room.id, room.platform)
|
||||
room.platform = get_platform(room.platform)
|
||||
return room
|
||||
|
||||
|
||||
@@ -330,16 +323,13 @@ async def rooms_create_meeting(
|
||||
if meeting is None:
|
||||
end_date = current_time + timedelta(hours=8)
|
||||
|
||||
# Determine which platform to use
|
||||
platform = get_platform_for_room(room.id, room.platform)
|
||||
platform = get_platform(room.platform)
|
||||
client = create_platform_client(platform)
|
||||
|
||||
# Create meeting via platform abstraction
|
||||
meeting_data = await client.create_meeting(
|
||||
room.name, end_date=end_date, room=room
|
||||
)
|
||||
|
||||
# Upload logo if supported by platform
|
||||
await client.upload_logo(meeting_data.room_name, "./images/logo.png")
|
||||
|
||||
meeting = await meetings_controller.create(
|
||||
@@ -350,7 +340,6 @@ async def rooms_create_meeting(
|
||||
start_date=current_time,
|
||||
end_date=end_date,
|
||||
room=room,
|
||||
platform=platform,
|
||||
)
|
||||
except LockError:
|
||||
logger.warning("Failed to acquire lock for room %s within timeout", room_name)
|
||||
@@ -358,17 +347,17 @@ async def rooms_create_meeting(
|
||||
status_code=503, detail="Meeting creation in progress, please try again"
|
||||
)
|
||||
|
||||
meeting.platform = get_platform_for_room(room.id, room.platform)
|
||||
|
||||
if meeting.platform == "daily" and room.recording_trigger != "none":
|
||||
client = create_platform_client(meeting.platform)
|
||||
token = await client.create_meeting_token(
|
||||
meeting.room_name, enable_recording=True
|
||||
meeting.room_name,
|
||||
enable_recording=True,
|
||||
user_id=user_id,
|
||||
)
|
||||
meeting = meeting.model_copy()
|
||||
meeting.room_url += f"?t={token}"
|
||||
meeting.room_url = add_query_param(meeting.room_url, "t", token)
|
||||
if meeting.host_room_url:
|
||||
meeting.host_room_url += f"?t={token}"
|
||||
meeting.host_room_url = add_query_param(meeting.host_room_url, "t", token)
|
||||
|
||||
if user_id != room.user_id:
|
||||
meeting.host_room_url = ""
|
||||
@@ -524,7 +513,7 @@ async def rooms_list_active_meetings(
|
||||
room=room, current_time=current_time
|
||||
)
|
||||
|
||||
effective_platform = get_platform_for_room(room.id, room.platform)
|
||||
effective_platform = get_platform(room.platform)
|
||||
for meeting in meetings:
|
||||
meeting.platform = effective_platform
|
||||
|
||||
@@ -548,17 +537,10 @@ async def rooms_get_meeting(
|
||||
if not room:
|
||||
raise HTTPException(status_code=404, detail="Room not found")
|
||||
|
||||
meeting = await meetings_controller.get_by_id(meeting_id)
|
||||
meeting = await meetings_controller.get_by_id(meeting_id, room=room)
|
||||
if not meeting:
|
||||
raise HTTPException(status_code=404, detail="Meeting not found")
|
||||
|
||||
if meeting.room_id != room.id:
|
||||
raise HTTPException(
|
||||
status_code=403, detail="Meeting does not belong to this room"
|
||||
)
|
||||
|
||||
meeting.platform = get_platform_for_room(room.id, room.platform)
|
||||
|
||||
if user_id != room.user_id and not room.is_shared:
|
||||
meeting.host_room_url = ""
|
||||
|
||||
@@ -577,16 +559,11 @@ async def rooms_join_meeting(
|
||||
if not room:
|
||||
raise HTTPException(status_code=404, detail="Room not found")
|
||||
|
||||
meeting = await meetings_controller.get_by_id(meeting_id)
|
||||
meeting = await meetings_controller.get_by_id(meeting_id, room=room)
|
||||
|
||||
if not meeting:
|
||||
raise HTTPException(status_code=404, detail="Meeting not found")
|
||||
|
||||
if meeting.room_id != room.id:
|
||||
raise HTTPException(
|
||||
status_code=403, detail="Meeting does not belong to this room"
|
||||
)
|
||||
|
||||
if not meeting.is_active:
|
||||
raise HTTPException(status_code=400, detail="Meeting is not active")
|
||||
|
||||
@@ -594,8 +571,6 @@ async def rooms_join_meeting(
|
||||
if meeting.end_date <= current_time:
|
||||
raise HTTPException(status_code=400, detail="Meeting has ended")
|
||||
|
||||
meeting.platform = get_platform_for_room(room.id, room.platform)
|
||||
|
||||
if user_id != room.user_id:
|
||||
meeting.host_room_url = ""
|
||||
|
||||
|
||||
@@ -5,7 +5,7 @@ from fastapi import APIRouter, Depends, HTTPException, Query
|
||||
from fastapi_pagination import Page
|
||||
from fastapi_pagination.ext.databases import apaginate
|
||||
from jose import jwt
|
||||
from pydantic import BaseModel, Field, constr, field_serializer
|
||||
from pydantic import AwareDatetime, BaseModel, Field, constr, field_serializer
|
||||
|
||||
import reflector.auth as auth
|
||||
from reflector.db import get_database
|
||||
@@ -133,6 +133,21 @@ SearchOffsetParam = Annotated[
|
||||
SearchOffsetBase, Query(description="Number of results to skip")
|
||||
]
|
||||
|
||||
SearchFromDatetimeParam = Annotated[
|
||||
AwareDatetime | None,
|
||||
Query(
|
||||
alias="from",
|
||||
description="Filter transcripts created on or after this datetime (ISO 8601 with timezone)",
|
||||
),
|
||||
]
|
||||
SearchToDatetimeParam = Annotated[
|
||||
AwareDatetime | None,
|
||||
Query(
|
||||
alias="to",
|
||||
description="Filter transcripts created on or before this datetime (ISO 8601 with timezone)",
|
||||
),
|
||||
]
|
||||
|
||||
|
||||
class SearchResponse(BaseModel):
|
||||
results: list[SearchResult]
|
||||
@@ -174,18 +189,23 @@ async def transcripts_search(
|
||||
offset: SearchOffsetParam = 0,
|
||||
room_id: Optional[str] = None,
|
||||
source_kind: Optional[SourceKind] = None,
|
||||
from_datetime: SearchFromDatetimeParam = None,
|
||||
to_datetime: SearchToDatetimeParam = None,
|
||||
user: Annotated[
|
||||
Optional[auth.UserInfo], Depends(auth.current_user_optional)
|
||||
] = None,
|
||||
):
|
||||
"""
|
||||
Full-text search across transcript titles and content.
|
||||
"""
|
||||
"""Full-text search across transcript titles and content."""
|
||||
if not user and not settings.PUBLIC_MODE:
|
||||
raise HTTPException(status_code=401, detail="Not authenticated")
|
||||
|
||||
user_id = user["sub"] if user else None
|
||||
|
||||
if from_datetime and to_datetime and from_datetime > to_datetime:
|
||||
raise HTTPException(
|
||||
status_code=400, detail="'from' must be less than or equal to 'to'"
|
||||
)
|
||||
|
||||
search_params = SearchParameters(
|
||||
query_text=parse_search_query_param(q),
|
||||
limit=limit,
|
||||
@@ -193,6 +213,8 @@ async def transcripts_search(
|
||||
user_id=user_id,
|
||||
room_id=room_id,
|
||||
source_kind=source_kind,
|
||||
from_datetime=from_datetime,
|
||||
to_datetime=to_datetime,
|
||||
)
|
||||
|
||||
results, total = await search_controller.search_transcripts(search_params)
|
||||
|
||||
@@ -5,8 +5,12 @@ from fastapi import APIRouter, Depends, HTTPException
|
||||
from pydantic import BaseModel
|
||||
|
||||
import reflector.auth as auth
|
||||
from reflector.db.recordings import recordings_controller
|
||||
from reflector.db.transcripts import transcripts_controller
|
||||
from reflector.pipelines.main_file_pipeline import task_pipeline_file_process
|
||||
from reflector.pipelines.main_multitrack_pipeline import (
|
||||
task_pipeline_multitrack_process,
|
||||
)
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
@@ -33,14 +37,35 @@ async def transcript_process(
|
||||
status_code=400, detail="Recording is not ready for processing"
|
||||
)
|
||||
|
||||
# avoid duplicate scheduling for either pipeline
|
||||
if task_is_scheduled_or_active(
|
||||
"reflector.pipelines.main_file_pipeline.task_pipeline_file_process",
|
||||
transcript_id=transcript_id,
|
||||
) or task_is_scheduled_or_active(
|
||||
"reflector.pipelines.main_multitrack_pipeline.task_pipeline_multitrack_process",
|
||||
transcript_id=transcript_id,
|
||||
):
|
||||
return ProcessStatus(status="already running")
|
||||
|
||||
# schedule a background task process the file
|
||||
task_pipeline_file_process.delay(transcript_id=transcript_id)
|
||||
# Determine processing mode strictly from DB to avoid S3 scans
|
||||
bucket_name = None
|
||||
track_keys: list[str] = []
|
||||
|
||||
if transcript.recording_id:
|
||||
recording = await recordings_controller.get_by_id(transcript.recording_id)
|
||||
if recording:
|
||||
bucket_name = recording.bucket_name
|
||||
track_keys = list(getattr(recording, "track_keys", []) or [])
|
||||
|
||||
if bucket_name:
|
||||
task_pipeline_multitrack_process.delay(
|
||||
transcript_id=transcript_id,
|
||||
bucket_name=bucket_name,
|
||||
track_keys=track_keys,
|
||||
)
|
||||
else:
|
||||
# Default single-file pipeline
|
||||
task_pipeline_file_process.delay(transcript_id=transcript_id)
|
||||
|
||||
return ProcessStatus(status="ok")
|
||||
|
||||
|
||||
@@ -11,7 +11,6 @@ router = APIRouter()
|
||||
class UserInfo(BaseModel):
|
||||
sub: str
|
||||
email: Optional[str]
|
||||
email_verified: Optional[bool]
|
||||
|
||||
|
||||
@router.get("/me")
|
||||
|
||||
62
server/reflector/views/user_api_keys.py
Normal file
62
server/reflector/views/user_api_keys.py
Normal file
@@ -0,0 +1,62 @@
|
||||
from datetime import datetime
|
||||
from typing import Annotated
|
||||
|
||||
import structlog
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from pydantic import BaseModel
|
||||
|
||||
import reflector.auth as auth
|
||||
from reflector.db.user_api_keys import user_api_keys_controller
|
||||
from reflector.utils.string import NonEmptyString
|
||||
|
||||
router = APIRouter()
|
||||
logger = structlog.get_logger(__name__)
|
||||
|
||||
|
||||
class CreateApiKeyRequest(BaseModel):
|
||||
name: NonEmptyString | None = None
|
||||
|
||||
|
||||
class ApiKeyResponse(BaseModel):
|
||||
id: NonEmptyString
|
||||
user_id: NonEmptyString
|
||||
name: NonEmptyString | None
|
||||
created_at: datetime
|
||||
|
||||
|
||||
class CreateApiKeyResponse(ApiKeyResponse):
|
||||
key: NonEmptyString
|
||||
|
||||
|
||||
@router.post("/user/api-keys", response_model=CreateApiKeyResponse)
|
||||
async def create_api_key(
|
||||
req: CreateApiKeyRequest,
|
||||
user: Annotated[auth.UserInfo, Depends(auth.current_user)],
|
||||
):
|
||||
api_key_model, plaintext = await user_api_keys_controller.create_key(
|
||||
user_id=user["sub"],
|
||||
name=req.name,
|
||||
)
|
||||
return CreateApiKeyResponse(
|
||||
**api_key_model.model_dump(),
|
||||
key=plaintext,
|
||||
)
|
||||
|
||||
|
||||
@router.get("/user/api-keys", response_model=list[ApiKeyResponse])
|
||||
async def list_api_keys(
|
||||
user: Annotated[auth.UserInfo, Depends(auth.current_user)],
|
||||
):
|
||||
api_keys = await user_api_keys_controller.list_by_user_id(user["sub"])
|
||||
return [ApiKeyResponse(**k.model_dump()) for k in api_keys]
|
||||
|
||||
|
||||
@router.delete("/user/api-keys/{key_id}")
|
||||
async def delete_api_key(
|
||||
key_id: NonEmptyString,
|
||||
user: Annotated[auth.UserInfo, Depends(auth.current_user)],
|
||||
):
|
||||
deleted = await user_api_keys_controller.delete_key(key_id, user["sub"])
|
||||
if not deleted:
|
||||
raise HTTPException(status_code=404)
|
||||
return {"status": "ok"}
|
||||
@@ -1,114 +0,0 @@
|
||||
import logging
|
||||
from datetime import datetime
|
||||
|
||||
import httpx
|
||||
|
||||
from reflector.db.rooms import Room
|
||||
from reflector.settings import settings
|
||||
from reflector.utils.string import parse_non_empty_string
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _get_headers():
|
||||
api_key = parse_non_empty_string(
|
||||
settings.WHEREBY_API_KEY, "WHEREBY_API_KEY value is required."
|
||||
)
|
||||
return {
|
||||
"Content-Type": "application/json; charset=utf-8",
|
||||
"Authorization": f"Bearer {api_key}",
|
||||
}
|
||||
|
||||
|
||||
TIMEOUT = 10 # seconds
|
||||
|
||||
|
||||
def _get_whereby_s3_auth():
|
||||
errors = []
|
||||
try:
|
||||
bucket_name = parse_non_empty_string(
|
||||
settings.RECORDING_STORAGE_AWS_BUCKET_NAME,
|
||||
"RECORDING_STORAGE_AWS_BUCKET_NAME value is required.",
|
||||
)
|
||||
except Exception as e:
|
||||
errors.append(e)
|
||||
try:
|
||||
key_id = parse_non_empty_string(
|
||||
settings.AWS_WHEREBY_ACCESS_KEY_ID,
|
||||
"AWS_WHEREBY_ACCESS_KEY_ID value is required.",
|
||||
)
|
||||
except Exception as e:
|
||||
errors.append(e)
|
||||
try:
|
||||
key_secret = parse_non_empty_string(
|
||||
settings.AWS_WHEREBY_ACCESS_KEY_SECRET,
|
||||
"AWS_WHEREBY_ACCESS_KEY_SECRET value is required.",
|
||||
)
|
||||
except Exception as e:
|
||||
errors.append(e)
|
||||
if len(errors) > 0:
|
||||
raise Exception(
|
||||
f"Failed to get Whereby auth settings: {', '.join(str(e) for e in errors)}"
|
||||
)
|
||||
return bucket_name, key_id, key_secret
|
||||
|
||||
|
||||
async def create_meeting(room_name_prefix: str, end_date: datetime, room: Room):
|
||||
s3_bucket_name, s3_key_id, s3_key_secret = _get_whereby_s3_auth()
|
||||
data = {
|
||||
"isLocked": room.is_locked,
|
||||
"roomNamePrefix": room_name_prefix,
|
||||
"roomNamePattern": "uuid",
|
||||
"roomMode": room.room_mode,
|
||||
"endDate": end_date.isoformat(),
|
||||
"recording": {
|
||||
"type": room.recording_type,
|
||||
"destination": {
|
||||
"provider": "s3",
|
||||
"bucket": s3_bucket_name,
|
||||
"accessKeyId": s3_key_id,
|
||||
"accessKeySecret": s3_key_secret,
|
||||
"fileFormat": "mp4",
|
||||
},
|
||||
"startTrigger": room.recording_trigger,
|
||||
},
|
||||
"fields": ["hostRoomUrl"],
|
||||
}
|
||||
async with httpx.AsyncClient() as client:
|
||||
response = await client.post(
|
||||
f"{settings.WHEREBY_API_URL}/meetings",
|
||||
headers=_get_headers(),
|
||||
json=data,
|
||||
timeout=TIMEOUT,
|
||||
)
|
||||
if response.status_code == 403:
|
||||
logger.warning(
|
||||
f"Failed to create meeting: access denied on Whereby: {response.text}"
|
||||
)
|
||||
response.raise_for_status()
|
||||
return response.json()
|
||||
|
||||
|
||||
async def get_room_sessions(room_name: str):
|
||||
async with httpx.AsyncClient() as client:
|
||||
response = await client.get(
|
||||
f"{settings.WHEREBY_API_URL}/insights/room-sessions?roomName={room_name}",
|
||||
headers=_get_headers(),
|
||||
timeout=TIMEOUT,
|
||||
)
|
||||
response.raise_for_status()
|
||||
return response.json()
|
||||
|
||||
|
||||
async def upload_logo(room_name: str, logo_path: str):
|
||||
async with httpx.AsyncClient() as client:
|
||||
with open(logo_path, "rb") as f:
|
||||
response = await client.put(
|
||||
f"{settings.WHEREBY_API_URL}/rooms{room_name}/theme/logo",
|
||||
headers={
|
||||
"Authorization": f"Bearer {settings.WHEREBY_API_KEY}",
|
||||
},
|
||||
timeout=TIMEOUT,
|
||||
files={"image": f},
|
||||
)
|
||||
response.raise_for_status()
|
||||
@@ -19,7 +19,7 @@ from reflector.db.meetings import meetings
|
||||
from reflector.db.recordings import recordings
|
||||
from reflector.db.transcripts import transcripts, transcripts_controller
|
||||
from reflector.settings import settings
|
||||
from reflector.storage import get_recordings_storage
|
||||
from reflector.storage import get_transcripts_storage
|
||||
|
||||
logger = structlog.get_logger(__name__)
|
||||
|
||||
@@ -53,8 +53,8 @@ async def delete_single_transcript(
|
||||
)
|
||||
if recording:
|
||||
try:
|
||||
await get_recordings_storage().delete_file(
|
||||
recording["object_key"]
|
||||
await get_transcripts_storage().delete_file(
|
||||
recording["object_key"], bucket=recording["bucket_name"]
|
||||
)
|
||||
except Exception as storage_error:
|
||||
logger.warning(
|
||||
|
||||
@@ -1,179 +0,0 @@
|
||||
"""Stub data for Daily.co testing - Fish conversation"""
|
||||
|
||||
import re
|
||||
from typing import Any
|
||||
|
||||
from reflector.processors.types import Word
|
||||
from reflector.utils import generate_uuid4
|
||||
|
||||
# Constants for stub data generation
|
||||
MIN_WORD_DURATION = 0.3 # Base duration per word in seconds
|
||||
WORD_LENGTH_MULTIPLIER = 0.05 # Additional duration per character
|
||||
NUM_STUB_TOPICS = 3 # Number of topics to generate
|
||||
|
||||
# The fish argument text - 2 speakers arguing about eating fish
|
||||
FISH_TEXT = """Fish for dinner are nothing wrong with you? There's nothing wrong with me. Wrong with you? Would you shut up? There's nothing wrong with me. I'm just trying to. There's nothing wrong with me. I'm trying to eat a fish. Wrong with you trying to eat a fish and it falls off the plate. Would you shut up? You're bothering me. More than a fish is bothering me. Would you shut up and leave me alone? What's your problem? I'm just trying to eat a fish is wrong with you. I'm only trying to eat a fish. Would you shut up? Wrong with you. There's nothing wrong with me. There's nothing wrong with me. Wrong with you. There's nothing wrong with me. Wrong with you. There's nothing wrong with me. Would you shut up and let me eat my fish? Wrong with you. Shut up! What is wrong with you? Would you just shut up? What's your problem? Would you shut up with you? What is wrong with you? Wrong with me? I'm just trying to get my attention. Did you shut up? You're bothering me. Would you shut up? You're beginning to bug me. What's your problem? Just trying to eat my fish. Stay on the plate. Would you shut up? Just trying to eat my fish.
|
||||
|
||||
I'm gonna hit you with my problem. You're worse than this fish. You're more of a problem than a fish. What's your problem? Would you shut up? Would you shut your mouth? I want to eat my fish. Shut up! I can't even think. What's your problem? Trying to eat my fish is wrong with you. I don't have a problem. What is wrong with you? I have a problem. What's your problem? I don't have a problem. Can't you hear me with you? Can't you hear me? I don't have a problem. I want to eat my fish. Your problem? Just want to eat. What is wrong with you? Shut up! What is wrong with you? You just shut up! What's your problem? What is wrong with you anyway? What is wrong with you? I won't stay on the plate. You shut up! What is wrong with you? Would you just shut up? Let me eat my fish. What's your problem? Shut up and leave me alone! I can't even think. Wrong with you. I don't have a problem. Problem? I don't have a problem. Wrong with you. I don't have a problem with you. That's your problem. Don't have a problem? I want to eat my fish.
|
||||
|
||||
What is wrong with you? What's your problem? Problem? I just want to eat my fish. Wrong with you. What's wrong with you? I don't have a problem. You shut up! What's wrong with you? Just shut up! What's wrong with you? Shut up! What is wrong with you? I'm trying to eat a fish. I'm trying to eat a fish and it falls off the plate. Would you shut up? What is wrong with you? Would you shut up? Is wrong with you? Would you just shut up? What is wrong with you? Would you just shut? Is wrong with you? What's your problem? You just shut. What is wrong with you? Trying to eat my fish. Would you be quiet? What's your problem? Would you just shut up? Eat my fish. I can't even eat it. Don't stay on the plate. What's your problem? Would you shut up? What is wrong with you? What is wrong with you? Would you just shut up? What's your problem? What is wrong with you? I'm gonna hit you with my fish if you don't shut up. What's your problem? Would you shut up? What's wrong with you? What is wrong? Shut up! What's your problem?"""
|
||||
|
||||
|
||||
def parse_fish_text() -> list[Word]:
|
||||
"""Parse fish text into words with timestamps and speakers.
|
||||
|
||||
Returns list of Word objects with text, start/end timestamps, and speaker ID.
|
||||
|
||||
Speaker assignment heuristic:
|
||||
- Speaker 0 (eating fish): "fish", "eat", "trying", "problem", "I"
|
||||
- Speaker 1 (annoying): "wrong with you", "shut up", "What's your problem"
|
||||
"""
|
||||
|
||||
# Split into sentences (rough)
|
||||
sentences = re.split(r"([.!?])", FISH_TEXT)
|
||||
|
||||
# Reconstruct sentences with punctuation
|
||||
full_sentences = []
|
||||
for i in range(0, len(sentences) - 1, 2):
|
||||
if sentences[i].strip():
|
||||
full_sentences.append(
|
||||
sentences[i].strip()
|
||||
+ (sentences[i + 1] if i + 1 < len(sentences) else "")
|
||||
)
|
||||
|
||||
words: list[Word] = []
|
||||
current_time = 0.0
|
||||
|
||||
for sentence in full_sentences:
|
||||
if not sentence.strip():
|
||||
continue
|
||||
|
||||
# TODO: Delete this heuristic-based speaker detection when real diarization is implemented.
|
||||
# This overly complex pattern matching is only for stub test data.
|
||||
# Real implementation should use actual speaker diarization from audio processing.
|
||||
|
||||
# Determine speaker based on content
|
||||
sentence_lower = sentence.lower()
|
||||
|
||||
# Speaker 1 patterns (annoying person)
|
||||
if any(
|
||||
p in sentence_lower
|
||||
for p in [
|
||||
"wrong with you",
|
||||
"shut up",
|
||||
"what's your problem",
|
||||
"what is wrong",
|
||||
"would you shut",
|
||||
"you shut",
|
||||
]
|
||||
):
|
||||
speaker = 1
|
||||
# Speaker 0 patterns (trying to eat)
|
||||
elif any(
|
||||
p in sentence_lower
|
||||
for p in [
|
||||
"i'm trying",
|
||||
"i'm just",
|
||||
"i want to eat",
|
||||
"eat my fish",
|
||||
"trying to eat",
|
||||
"nothing wrong with me",
|
||||
"i don't have a problem",
|
||||
"just trying",
|
||||
"leave me alone",
|
||||
"can't even",
|
||||
"i'm gonna hit",
|
||||
]
|
||||
):
|
||||
speaker = 0
|
||||
# Default: alternate or use context
|
||||
else:
|
||||
# For short phrases, guess based on keywords
|
||||
if "fish" in sentence_lower and "eat" in sentence_lower:
|
||||
speaker = 0
|
||||
elif "problem" in sentence_lower and "your" not in sentence_lower:
|
||||
speaker = 0
|
||||
else:
|
||||
speaker = 1
|
||||
|
||||
# Split sentence into words
|
||||
sentence_words = sentence.split()
|
||||
for word in sentence_words:
|
||||
word_duration = MIN_WORD_DURATION + (len(word) * WORD_LENGTH_MULTIPLIER)
|
||||
|
||||
words.append(
|
||||
Word(
|
||||
text=word + " ", # Add space
|
||||
start=current_time,
|
||||
end=current_time + word_duration,
|
||||
speaker=speaker,
|
||||
)
|
||||
)
|
||||
|
||||
current_time += word_duration
|
||||
|
||||
return words
|
||||
|
||||
|
||||
def generate_fake_topics(words: list[Word]) -> list[dict[str, Any]]:
|
||||
"""Generate fake topics from words.
|
||||
|
||||
Splits into equal topics based on word count.
|
||||
Returns list of topic dicts for database storage.
|
||||
"""
|
||||
if not words:
|
||||
return []
|
||||
|
||||
chunk_size = len(words) // NUM_STUB_TOPICS
|
||||
topics: list[dict[str, Any]] = []
|
||||
|
||||
for i in range(NUM_STUB_TOPICS):
|
||||
start_idx = i * chunk_size
|
||||
end_idx = (i + 1) * chunk_size if i < NUM_STUB_TOPICS - 1 else len(words)
|
||||
|
||||
if start_idx >= len(words):
|
||||
break
|
||||
|
||||
chunk_words = words[start_idx:end_idx]
|
||||
|
||||
topic = {
|
||||
"id": generate_uuid4(),
|
||||
"title": f"Fish Argument Part {i+1}",
|
||||
"summary": f"Argument about eating fish continues (part {i+1})",
|
||||
"timestamp": chunk_words[0].start,
|
||||
"duration": chunk_words[-1].end - chunk_words[0].start,
|
||||
"transcript": "".join(w.text for w in chunk_words),
|
||||
"words": [w.model_dump() for w in chunk_words],
|
||||
}
|
||||
|
||||
topics.append(topic)
|
||||
|
||||
return topics
|
||||
|
||||
|
||||
def generate_fake_participants() -> list[dict[str, Any]]:
|
||||
"""Generate fake participants for stub transcript."""
|
||||
return [
|
||||
{"id": generate_uuid4(), "speaker": 0, "name": "Fish Eater"},
|
||||
{"id": generate_uuid4(), "speaker": 1, "name": "Annoying Person"},
|
||||
]
|
||||
|
||||
|
||||
def get_stub_transcript_data() -> dict[str, Any]:
|
||||
"""Get complete stub transcript data for Daily.co testing.
|
||||
|
||||
Returns dict with topics, participants, title, summaries, duration.
|
||||
All data is fake/predetermined for testing webhook flow without GPU processing.
|
||||
"""
|
||||
words = parse_fish_text()
|
||||
topics = generate_fake_topics(words)
|
||||
participants = generate_fake_participants()
|
||||
|
||||
return {
|
||||
"topics": topics,
|
||||
"participants": participants,
|
||||
"title": "The Great Fish Eating Argument",
|
||||
"short_summary": "Two people argue about eating fish",
|
||||
"long_summary": "An extended argument between someone trying to eat fish and another person who won't stop asking what's wrong. The fish keeps falling off the plate.",
|
||||
"duration": words[-1].end if words else 0.0,
|
||||
}
|
||||
@@ -7,10 +7,10 @@ from celery.utils.log import get_task_logger
|
||||
from reflector.asynctask import asynctask
|
||||
from reflector.db.calendar_events import calendar_events_controller
|
||||
from reflector.db.meetings import meetings_controller
|
||||
from reflector.db.rooms import rooms_controller
|
||||
from reflector.db.rooms import Room, rooms_controller
|
||||
from reflector.redis_cache import RedisAsyncLock
|
||||
from reflector.services.ics_sync import SyncStatus, ics_sync_service
|
||||
from reflector.video_platforms.factory import create_platform_client
|
||||
from reflector.video_platforms.factory import create_platform_client, get_platform
|
||||
|
||||
logger = structlog.wrap_logger(get_task_logger(__name__))
|
||||
|
||||
@@ -86,17 +86,17 @@ def _should_sync(room) -> bool:
|
||||
MEETING_DEFAULT_DURATION = timedelta(hours=1)
|
||||
|
||||
|
||||
async def create_upcoming_meetings_for_event(event, create_window, room_id, room):
|
||||
async def create_upcoming_meetings_for_event(event, create_window, room: Room):
|
||||
if event.start_time <= create_window:
|
||||
return
|
||||
existing_meeting = await meetings_controller.get_by_calendar_event(event.id)
|
||||
existing_meeting = await meetings_controller.get_by_calendar_event(event.id, room)
|
||||
|
||||
if existing_meeting:
|
||||
return
|
||||
|
||||
logger.info(
|
||||
"Pre-creating meeting for calendar event",
|
||||
room_id=room_id,
|
||||
room_id=room.id,
|
||||
event_id=event.id,
|
||||
event_title=event.title,
|
||||
)
|
||||
@@ -104,9 +104,7 @@ async def create_upcoming_meetings_for_event(event, create_window, room_id, room
|
||||
try:
|
||||
end_date = event.end_time or (event.start_time + MEETING_DEFAULT_DURATION)
|
||||
|
||||
# Use platform abstraction to create meeting
|
||||
platform = room.platform
|
||||
client = create_platform_client(platform)
|
||||
client = create_platform_client(get_platform(room.platform))
|
||||
|
||||
meeting_data = await client.create_meeting(
|
||||
"",
|
||||
@@ -129,7 +127,6 @@ async def create_upcoming_meetings_for_event(event, create_window, room_id, room
|
||||
"description": event.description,
|
||||
"attendees": event.attendees,
|
||||
},
|
||||
platform=platform,
|
||||
)
|
||||
|
||||
logger.info(
|
||||
@@ -141,7 +138,7 @@ async def create_upcoming_meetings_for_event(event, create_window, room_id, room
|
||||
except Exception as e:
|
||||
logger.error(
|
||||
"Failed to pre-create meeting",
|
||||
room_id=room_id,
|
||||
room_id=room.id,
|
||||
event_id=event.id,
|
||||
error=str(e),
|
||||
)
|
||||
@@ -171,9 +168,7 @@ async def create_upcoming_meetings():
|
||||
)
|
||||
|
||||
for event in events:
|
||||
await create_upcoming_meetings_for_event(
|
||||
event, create_window, room.id, room
|
||||
)
|
||||
await create_upcoming_meetings_for_event(event, create_window, room)
|
||||
logger.info("Completed pre-creation check for upcoming meetings")
|
||||
|
||||
except Exception as e:
|
||||
|
||||
@@ -15,28 +15,32 @@ from redis.exceptions import LockError
|
||||
from reflector.db.meetings import meetings_controller
|
||||
from reflector.db.recordings import Recording, recordings_controller
|
||||
from reflector.db.rooms import rooms_controller
|
||||
from reflector.db.transcripts import SourceKind, transcripts_controller
|
||||
from reflector.db.transcripts import (
|
||||
SourceKind,
|
||||
TranscriptParticipant,
|
||||
transcripts_controller,
|
||||
)
|
||||
from reflector.pipelines.main_file_pipeline import task_pipeline_file_process
|
||||
from reflector.pipelines.main_live_pipeline import asynctask
|
||||
from reflector.pipelines.main_multitrack_pipeline import (
|
||||
task_pipeline_multitrack_process,
|
||||
)
|
||||
from reflector.pipelines.topic_processing import EmptyPipeline
|
||||
from reflector.processors import AudioFileWriterProcessor
|
||||
from reflector.processors.audio_waveform_processor import AudioWaveformProcessor
|
||||
from reflector.redis_cache import get_redis_client
|
||||
from reflector.settings import settings
|
||||
from reflector.whereby import get_room_sessions
|
||||
from reflector.worker.daily_stub_data import get_stub_transcript_data
|
||||
from reflector.storage import get_transcripts_storage
|
||||
from reflector.utils.daily import DailyRoomName, extract_base_room_name
|
||||
from reflector.video_platforms.factory import create_platform_client
|
||||
from reflector.video_platforms.whereby_utils import (
|
||||
parse_whereby_recording_filename,
|
||||
room_name_to_whereby_api_room_name,
|
||||
)
|
||||
|
||||
logger = structlog.wrap_logger(get_task_logger(__name__))
|
||||
|
||||
|
||||
def parse_datetime_with_timezone(iso_string: str) -> datetime:
|
||||
"""Parse ISO datetime string and ensure timezone awareness (defaults to UTC if naive)."""
|
||||
dt = datetime.fromisoformat(iso_string)
|
||||
if dt.tzinfo is None:
|
||||
dt = dt.replace(tzinfo=timezone.utc)
|
||||
return dt
|
||||
|
||||
|
||||
@shared_task
|
||||
def process_messages():
|
||||
queue_url = settings.AWS_PROCESS_RECORDING_QUEUE_URL
|
||||
@@ -78,14 +82,16 @@ def process_messages():
|
||||
logger.error("process_messages", error=str(e))
|
||||
|
||||
|
||||
# only whereby supported.
|
||||
@shared_task
|
||||
@asynctask
|
||||
async def process_recording(bucket_name: str, object_key: str):
|
||||
logger.info("Processing recording: %s/%s", bucket_name, object_key)
|
||||
|
||||
# extract a guid and a datetime from the object key
|
||||
room_name = f"/{object_key[:36]}"
|
||||
recorded_at = parse_datetime_with_timezone(object_key[37:57])
|
||||
room_name_part, recorded_at = parse_whereby_recording_filename(object_key)
|
||||
|
||||
# we store whereby api room names, NOT whereby room names
|
||||
room_name = room_name_to_whereby_api_room_name(room_name_part)
|
||||
|
||||
meeting = await meetings_controller.get_by_room_name(room_name)
|
||||
room = await rooms_controller.get_by_id(meeting.room_id)
|
||||
@@ -107,6 +113,7 @@ async def process_recording(bucket_name: str, object_key: str):
|
||||
transcript,
|
||||
{
|
||||
"topics": [],
|
||||
"participants": [],
|
||||
},
|
||||
)
|
||||
else:
|
||||
@@ -126,15 +133,15 @@ async def process_recording(bucket_name: str, object_key: str):
|
||||
upload_filename = transcript.data_path / f"upload{extension}"
|
||||
upload_filename.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
s3 = boto3.client(
|
||||
"s3",
|
||||
region_name=settings.TRANSCRIPT_STORAGE_AWS_REGION,
|
||||
aws_access_key_id=settings.TRANSCRIPT_STORAGE_AWS_ACCESS_KEY_ID,
|
||||
aws_secret_access_key=settings.TRANSCRIPT_STORAGE_AWS_SECRET_ACCESS_KEY,
|
||||
)
|
||||
storage = get_transcripts_storage()
|
||||
|
||||
with open(upload_filename, "wb") as f:
|
||||
s3.download_fileobj(bucket_name, object_key, f)
|
||||
try:
|
||||
with open(upload_filename, "wb") as f:
|
||||
await storage.stream_to_fileobj(object_key, f, bucket=bucket_name)
|
||||
except Exception:
|
||||
# Clean up partial file on stream failure
|
||||
upload_filename.unlink(missing_ok=True)
|
||||
raise
|
||||
|
||||
container = av.open(upload_filename.as_posix())
|
||||
try:
|
||||
@@ -155,14 +162,14 @@ async def process_recording(bucket_name: str, object_key: str):
|
||||
@asynctask
|
||||
async def process_multitrack_recording(
|
||||
bucket_name: str,
|
||||
room_name: str,
|
||||
daily_room_name: DailyRoomName,
|
||||
recording_id: str,
|
||||
track_keys: list[str],
|
||||
):
|
||||
logger.info(
|
||||
"Processing multitrack recording",
|
||||
bucket=bucket_name,
|
||||
room_name=room_name,
|
||||
room_name=daily_room_name,
|
||||
recording_id=recording_id,
|
||||
provided_keys=len(track_keys),
|
||||
)
|
||||
@@ -171,33 +178,38 @@ async def process_multitrack_recording(
|
||||
logger.warning("No audio track keys provided")
|
||||
return
|
||||
|
||||
recorded_at = datetime.now(timezone.utc)
|
||||
tz = timezone.utc
|
||||
recorded_at = datetime.now(tz)
|
||||
try:
|
||||
if track_keys:
|
||||
folder = os.path.basename(os.path.dirname(track_keys[0]))
|
||||
ts_match = re.search(r"(\d{14})$", folder)
|
||||
if ts_match:
|
||||
ts = ts_match.group(1)
|
||||
recorded_at = datetime.strptime(ts, "%Y%m%d%H%M%S").replace(
|
||||
tzinfo=timezone.utc
|
||||
)
|
||||
except Exception:
|
||||
logger.warning("Could not parse recorded_at from keys, using now()")
|
||||
recorded_at = datetime.strptime(ts, "%Y%m%d%H%M%S").replace(tzinfo=tz)
|
||||
except Exception as e:
|
||||
logger.warning(
|
||||
f"Could not parse recorded_at from keys, using now() {recorded_at}",
|
||||
e,
|
||||
exc_info=True,
|
||||
)
|
||||
|
||||
room_name = room_name.split("-", 1)[0]
|
||||
room = await rooms_controller.get_by_name(room_name)
|
||||
meeting = await meetings_controller.get_by_room_name(daily_room_name)
|
||||
|
||||
room_name_base = extract_base_room_name(daily_room_name)
|
||||
|
||||
room = await rooms_controller.get_by_name(room_name_base)
|
||||
if not room:
|
||||
raise Exception(f"Room not found: {room_name}")
|
||||
raise Exception(f"Room not found: {room_name_base}")
|
||||
|
||||
meeting = await meetings_controller.create(
|
||||
id=recording_id,
|
||||
room_name=room_name,
|
||||
room_url=room.name,
|
||||
host_room_url=room.name,
|
||||
start_date=recorded_at,
|
||||
end_date=recorded_at,
|
||||
room=room,
|
||||
platform=room.platform,
|
||||
if not meeting:
|
||||
raise Exception(f"Meeting not found: {room_name_base}")
|
||||
|
||||
logger.info(
|
||||
"Found existing Meeting for recording",
|
||||
meeting_id=meeting.id,
|
||||
room_name=daily_room_name,
|
||||
recording_id=recording_id,
|
||||
)
|
||||
|
||||
recording = await recordings_controller.get_by_id(recording_id)
|
||||
@@ -210,8 +222,12 @@ async def process_multitrack_recording(
|
||||
object_key=object_key_dir,
|
||||
recorded_at=recorded_at,
|
||||
meeting_id=meeting.id,
|
||||
track_keys=track_keys,
|
||||
)
|
||||
)
|
||||
else:
|
||||
# Recording already exists; assume metadata was set at creation time
|
||||
pass
|
||||
|
||||
transcript = await transcripts_controller.get_by_recording_id(recording.id)
|
||||
if transcript:
|
||||
@@ -219,6 +235,7 @@ async def process_multitrack_recording(
|
||||
transcript,
|
||||
{
|
||||
"topics": [],
|
||||
"participants": [],
|
||||
},
|
||||
)
|
||||
else:
|
||||
@@ -234,6 +251,65 @@ async def process_multitrack_recording(
|
||||
room_id=room.id,
|
||||
)
|
||||
|
||||
try:
|
||||
daily_client = create_platform_client("daily")
|
||||
|
||||
id_to_name = {}
|
||||
id_to_user_id = {}
|
||||
|
||||
mtg_session_id = None
|
||||
try:
|
||||
rec_details = await daily_client.get_recording(recording_id)
|
||||
mtg_session_id = rec_details.get("mtgSessionId")
|
||||
except Exception as e:
|
||||
logger.warning(
|
||||
"Failed to fetch Daily recording details",
|
||||
error=str(e),
|
||||
recording_id=recording_id,
|
||||
exc_info=True,
|
||||
)
|
||||
|
||||
if mtg_session_id:
|
||||
try:
|
||||
payload = await daily_client.get_meeting_participants(mtg_session_id)
|
||||
for p in payload.get("data", []):
|
||||
pid = p.get("participant_id")
|
||||
name = p.get("user_name")
|
||||
user_id = p.get("user_id")
|
||||
if pid and name:
|
||||
id_to_name[pid] = name
|
||||
if pid and user_id:
|
||||
id_to_user_id[pid] = user_id
|
||||
except Exception as e:
|
||||
logger.warning(
|
||||
"Failed to fetch Daily meeting participants",
|
||||
error=str(e),
|
||||
mtg_session_id=mtg_session_id,
|
||||
exc_info=True,
|
||||
)
|
||||
else:
|
||||
logger.warning(
|
||||
"No mtgSessionId found for recording; participant names may be generic",
|
||||
recording_id=recording_id,
|
||||
)
|
||||
|
||||
for idx, key in enumerate(track_keys):
|
||||
base = os.path.basename(key)
|
||||
m = re.search(r"\d{13,}-([0-9a-fA-F-]{36})-cam-audio-", base)
|
||||
participant_id = m.group(1) if m else None
|
||||
|
||||
default_name = f"Speaker {idx}"
|
||||
name = id_to_name.get(participant_id, default_name)
|
||||
user_id = id_to_user_id.get(participant_id)
|
||||
|
||||
participant = TranscriptParticipant(
|
||||
id=participant_id, speaker=idx, name=name, user_id=user_id
|
||||
)
|
||||
await transcripts_controller.upsert_participant(transcript, participant)
|
||||
|
||||
except Exception as e:
|
||||
logger.warning("Failed to map participant names", error=str(e), exc_info=True)
|
||||
|
||||
task_pipeline_multitrack_process.delay(
|
||||
transcript_id=transcript.id,
|
||||
bucket_name=bucket_name,
|
||||
@@ -259,7 +335,7 @@ async def process_meetings():
|
||||
Uses distributed locking to prevent race conditions when multiple workers
|
||||
process the same meeting simultaneously.
|
||||
"""
|
||||
logger.info("Processing meetings")
|
||||
logger.debug("Processing meetings")
|
||||
meetings = await meetings_controller.get_all_active()
|
||||
current_time = datetime.now(timezone.utc)
|
||||
redis_client = get_redis_client()
|
||||
@@ -284,7 +360,8 @@ async def process_meetings():
|
||||
end_date = end_date.replace(tzinfo=timezone.utc)
|
||||
|
||||
# This API call could be slow, extend lock if needed
|
||||
response = await get_room_sessions(meeting.room_name)
|
||||
client = create_platform_client(meeting.platform)
|
||||
room_sessions = await client.get_room_sessions(meeting.room_name)
|
||||
|
||||
try:
|
||||
# Extend lock after slow operation to ensure we still hold it
|
||||
@@ -293,7 +370,6 @@ async def process_meetings():
|
||||
logger_.warning("Lost lock for meeting, skipping")
|
||||
continue
|
||||
|
||||
room_sessions = response.get("results", [])
|
||||
has_active_sessions = room_sessions and any(
|
||||
rs["endedAt"] is None for rs in room_sessions
|
||||
)
|
||||
@@ -326,7 +402,7 @@ async def process_meetings():
|
||||
except LockError:
|
||||
pass # Lock already released or expired
|
||||
|
||||
logger.info(
|
||||
logger.debug(
|
||||
"Processed meetings finished",
|
||||
processed_count=processed_count,
|
||||
skipped_count=skipped_count,
|
||||
@@ -344,10 +420,6 @@ async def convert_audio_and_waveform(transcript) -> None:
|
||||
transcript_id=transcript.id,
|
||||
)
|
||||
|
||||
# Import processors we need
|
||||
from reflector.processors import AudioFileWriterProcessor
|
||||
from reflector.processors.audio_waveform_processor import AudioWaveformProcessor
|
||||
|
||||
upload_path = transcript.data_path / "upload.webm"
|
||||
mp3_path = transcript.audio_mp3_filename
|
||||
|
||||
@@ -366,21 +438,11 @@ async def convert_audio_and_waveform(transcript) -> None:
|
||||
mp3_size=mp3_path.stat().st_size,
|
||||
)
|
||||
|
||||
# Generate waveform
|
||||
waveform_processor = AudioWaveformProcessor(
|
||||
audio_path=mp3_path,
|
||||
waveform_path=transcript.audio_waveform_filename,
|
||||
)
|
||||
|
||||
# Create minimal pipeline object for processor (matching EmptyPipeline from main_file_pipeline.py)
|
||||
class MinimalPipeline:
|
||||
def __init__(self, logger_instance):
|
||||
self.logger = logger_instance
|
||||
|
||||
def get_pref(self, k, d=None):
|
||||
return d
|
||||
|
||||
waveform_processor.set_pipeline(MinimalPipeline(logger))
|
||||
waveform_processor.set_pipeline(EmptyPipeline(logger))
|
||||
await waveform_processor.flush()
|
||||
|
||||
logger.info(
|
||||
@@ -402,226 +464,58 @@ async def convert_audio_and_waveform(transcript) -> None:
|
||||
pass
|
||||
|
||||
|
||||
@shared_task
|
||||
@asynctask
|
||||
async def process_daily_recording(
|
||||
meeting_id: str, recording_id: str, tracks: list[dict]
|
||||
) -> None:
|
||||
"""Stub processor for Daily.co recordings - writes fake transcription/diarization.
|
||||
|
||||
Handles webhook retries by checking if recording already exists.
|
||||
Validates track structure before processing.
|
||||
|
||||
Args:
|
||||
meeting_id: Meeting ID
|
||||
recording_id: Recording ID from Daily.co webhook
|
||||
tracks: List of track dicts from Daily.co webhook
|
||||
[{type: 'audio'|'video', s3Key: str, size: int}, ...]
|
||||
"""
|
||||
logger.info(
|
||||
"Processing Daily.co recording (STUB)",
|
||||
meeting_id=meeting_id,
|
||||
recording_id=recording_id,
|
||||
num_tracks=len(tracks),
|
||||
)
|
||||
|
||||
# Check if recording already exists (webhook retry case)
|
||||
existing_recording = await recordings_controller.get_by_id(recording_id)
|
||||
if existing_recording:
|
||||
logger.warning(
|
||||
"Recording already exists, skipping processing (likely webhook retry)",
|
||||
recording_id=recording_id,
|
||||
)
|
||||
return
|
||||
|
||||
meeting = await meetings_controller.get_by_id(meeting_id)
|
||||
if not meeting:
|
||||
raise Exception(f"Meeting {meeting_id} not found")
|
||||
|
||||
room = await rooms_controller.get_by_id(meeting.room_id)
|
||||
|
||||
# Validate bucket configuration
|
||||
if not settings.AWS_DAILY_S3_BUCKET:
|
||||
raise ValueError("AWS_DAILY_S3_BUCKET not configured for Daily.co processing")
|
||||
|
||||
# Validate and parse tracks
|
||||
# Import at runtime to avoid circular dependency (daily.py imports from process.py)
|
||||
from reflector.views.daily import DailyTrack # noqa: PLC0415
|
||||
|
||||
try:
|
||||
validated_tracks = [DailyTrack(**t) for t in tracks]
|
||||
except Exception as e:
|
||||
logger.error(
|
||||
"Invalid track structure from Daily.co webhook",
|
||||
error=str(e),
|
||||
tracks=tracks,
|
||||
)
|
||||
raise ValueError(f"Invalid track structure: {e}")
|
||||
|
||||
# Find first audio track for Recording entity
|
||||
audio_track = next((t for t in validated_tracks if t.type == "audio"), None)
|
||||
if not audio_track:
|
||||
raise Exception(f"No audio tracks found in {len(tracks)} tracks")
|
||||
|
||||
# Create Recording entry
|
||||
recording = await recordings_controller.create(
|
||||
Recording(
|
||||
id=recording_id,
|
||||
bucket_name=settings.AWS_DAILY_S3_BUCKET,
|
||||
object_key=audio_track.s3Key,
|
||||
recorded_at=datetime.now(timezone.utc),
|
||||
meeting_id=meeting.id,
|
||||
status="completed",
|
||||
)
|
||||
)
|
||||
|
||||
logger.info(
|
||||
"Created recording",
|
||||
recording_id=recording.id,
|
||||
s3_key=audio_track.s3Key,
|
||||
)
|
||||
|
||||
# Create Transcript entry
|
||||
transcript = await transcripts_controller.add(
|
||||
"",
|
||||
source_kind=SourceKind.ROOM,
|
||||
source_language="en",
|
||||
target_language="en",
|
||||
user_id=room.user_id,
|
||||
recording_id=recording.id,
|
||||
share_mode="public",
|
||||
meeting_id=meeting.id,
|
||||
room_id=room.id,
|
||||
)
|
||||
|
||||
logger.info("Created transcript", transcript_id=transcript.id)
|
||||
|
||||
# Download audio file from Daily.co S3 for playback
|
||||
upload_filename = transcript.data_path / "upload.webm"
|
||||
upload_filename.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
s3 = boto3.client(
|
||||
"s3",
|
||||
region_name=settings.TRANSCRIPT_STORAGE_AWS_REGION,
|
||||
aws_access_key_id=settings.TRANSCRIPT_STORAGE_AWS_ACCESS_KEY_ID,
|
||||
aws_secret_access_key=settings.TRANSCRIPT_STORAGE_AWS_SECRET_ACCESS_KEY,
|
||||
)
|
||||
|
||||
try:
|
||||
logger.info(
|
||||
"Downloading audio from Daily.co S3",
|
||||
bucket=settings.AWS_DAILY_S3_BUCKET,
|
||||
key=audio_track.s3Key,
|
||||
)
|
||||
with open(upload_filename, "wb") as f:
|
||||
s3.download_fileobj(settings.AWS_DAILY_S3_BUCKET, audio_track.s3Key, f)
|
||||
|
||||
# Validate audio file
|
||||
container = av.open(upload_filename.as_posix())
|
||||
try:
|
||||
if not len(container.streams.audio):
|
||||
raise Exception("File has no audio stream")
|
||||
finally:
|
||||
container.close()
|
||||
|
||||
logger.info("Audio file downloaded and validated", file=str(upload_filename))
|
||||
except Exception as e:
|
||||
logger.error(
|
||||
"Failed to download or validate audio file",
|
||||
error=str(e),
|
||||
bucket=settings.AWS_DAILY_S3_BUCKET,
|
||||
key=audio_track.s3Key,
|
||||
)
|
||||
# Continue with stub data even if audio download fails
|
||||
pass
|
||||
|
||||
# Generate fake data
|
||||
stub_data = get_stub_transcript_data()
|
||||
|
||||
# Update transcript with fake data
|
||||
await transcripts_controller.update(
|
||||
transcript,
|
||||
{
|
||||
"topics": stub_data["topics"],
|
||||
"participants": stub_data["participants"],
|
||||
"title": stub_data["title"],
|
||||
"short_summary": stub_data["short_summary"],
|
||||
"long_summary": stub_data["long_summary"],
|
||||
"duration": stub_data["duration"],
|
||||
"status": "uploaded" if upload_filename.exists() else "ended",
|
||||
},
|
||||
)
|
||||
|
||||
logger.info(
|
||||
"Daily.co recording processed (STUB)",
|
||||
transcript_id=transcript.id,
|
||||
duration=stub_data["duration"],
|
||||
num_topics=len(stub_data["topics"]),
|
||||
has_audio=upload_filename.exists(),
|
||||
)
|
||||
|
||||
# Convert WebM to MP3 and generate waveform without full pipeline
|
||||
# (full pipeline would overwrite our stub transcription data)
|
||||
if upload_filename.exists():
|
||||
await convert_audio_and_waveform(transcript)
|
||||
|
||||
|
||||
@shared_task
|
||||
@asynctask
|
||||
async def reprocess_failed_recordings():
|
||||
"""
|
||||
Find recordings in the S3 bucket and check if they have proper transcriptions.
|
||||
Find recordings in Whereby S3 bucket and check if they have proper transcriptions.
|
||||
If not, requeue them for processing.
|
||||
"""
|
||||
logger.info("Checking for recordings that need processing or reprocessing")
|
||||
|
||||
s3 = boto3.client(
|
||||
"s3",
|
||||
region_name=settings.TRANSCRIPT_STORAGE_AWS_REGION,
|
||||
aws_access_key_id=settings.TRANSCRIPT_STORAGE_AWS_ACCESS_KEY_ID,
|
||||
aws_secret_access_key=settings.TRANSCRIPT_STORAGE_AWS_SECRET_ACCESS_KEY,
|
||||
)
|
||||
Note: Daily.co recordings are processed via webhooks, not this cron job.
|
||||
"""
|
||||
logger.info("Checking Whereby recordings that need processing or reprocessing")
|
||||
|
||||
if not settings.WHEREBY_STORAGE_AWS_BUCKET_NAME:
|
||||
raise ValueError(
|
||||
"WHEREBY_STORAGE_AWS_BUCKET_NAME required for Whereby recording reprocessing. "
|
||||
"Set WHEREBY_STORAGE_AWS_BUCKET_NAME environment variable."
|
||||
)
|
||||
|
||||
storage = get_transcripts_storage()
|
||||
bucket_name = settings.WHEREBY_STORAGE_AWS_BUCKET_NAME
|
||||
|
||||
reprocessed_count = 0
|
||||
try:
|
||||
paginator = s3.get_paginator("list_objects_v2")
|
||||
bucket_name = settings.RECORDING_STORAGE_AWS_BUCKET_NAME
|
||||
pages = paginator.paginate(Bucket=bucket_name)
|
||||
object_keys = await storage.list_objects(prefix="", bucket=bucket_name)
|
||||
|
||||
for page in pages:
|
||||
if "Contents" not in page:
|
||||
for object_key in object_keys:
|
||||
if not object_key.endswith(".mp4"):
|
||||
continue
|
||||
|
||||
for obj in page["Contents"]:
|
||||
object_key = obj["Key"]
|
||||
recording = await recordings_controller.get_by_object_key(
|
||||
bucket_name, object_key
|
||||
)
|
||||
if not recording:
|
||||
logger.info(f"Queueing recording for processing: {object_key}")
|
||||
process_recording.delay(bucket_name, object_key)
|
||||
reprocessed_count += 1
|
||||
continue
|
||||
|
||||
if not (object_key.endswith(".mp4")):
|
||||
continue
|
||||
|
||||
recording = await recordings_controller.get_by_object_key(
|
||||
bucket_name, object_key
|
||||
transcript = None
|
||||
try:
|
||||
transcript = await transcripts_controller.get_by_recording_id(
|
||||
recording.id
|
||||
)
|
||||
except ValidationError:
|
||||
await transcripts_controller.remove_by_recording_id(recording.id)
|
||||
logger.warning(
|
||||
f"Removed invalid transcript for recording: {recording.id}"
|
||||
)
|
||||
if not recording:
|
||||
logger.info(f"Queueing recording for processing: {object_key}")
|
||||
process_recording.delay(bucket_name, object_key)
|
||||
reprocessed_count += 1
|
||||
continue
|
||||
|
||||
transcript = None
|
||||
try:
|
||||
transcript = await transcripts_controller.get_by_recording_id(
|
||||
recording.id
|
||||
)
|
||||
except ValidationError:
|
||||
await transcripts_controller.remove_by_recording_id(recording.id)
|
||||
logger.warning(
|
||||
f"Removed invalid transcript for recording: {recording.id}"
|
||||
)
|
||||
|
||||
if transcript is None or transcript.status == "error":
|
||||
logger.info(f"Queueing recording for processing: {object_key}")
|
||||
process_recording.delay(bucket_name, object_key)
|
||||
reprocessed_count += 1
|
||||
if transcript is None or transcript.status == "error":
|
||||
logger.info(f"Queueing recording for processing: {object_key}")
|
||||
process_recording.delay(bucket_name, object_key)
|
||||
reprocessed_count += 1
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error checking S3 bucket: {str(e)}")
|
||||
|
||||
@@ -1,65 +0,0 @@
|
||||
#!/usr/bin/env python
|
||||
"""
|
||||
Reprocess the Daily.co multitrack recording to fix audio mixdown
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
|
||||
from reflector.pipelines.main_multitrack_pipeline import (
|
||||
task_pipeline_multitrack_process,
|
||||
)
|
||||
|
||||
|
||||
async def reprocess():
|
||||
"""Process the multitrack recording with fixed mixdown"""
|
||||
|
||||
bucket_name = "reflector-dailyco-local"
|
||||
track_keys = [
|
||||
"monadical/daily-20251020193458/1760988935484-52f7f48b-fbab-431f-9a50-87b9abfc8255-cam-audio-1760988935922",
|
||||
"monadical/daily-20251020193458/1760988935484-a37c35e3-6f8e-4274-a482-e9d0f102a732-cam-audio-1760988943823",
|
||||
]
|
||||
|
||||
# Create a new transcript with fixed mixdown
|
||||
import uuid
|
||||
|
||||
from reflector.db import get_database
|
||||
from reflector.db.transcripts import Transcript, transcripts
|
||||
|
||||
db = get_database()
|
||||
await db.connect()
|
||||
|
||||
try:
|
||||
transcript_id = str(uuid.uuid4())
|
||||
transcript = Transcript(
|
||||
id=transcript_id,
|
||||
name="Daily Multitrack - With Audio Mixdown",
|
||||
source_kind="file",
|
||||
source_language="en",
|
||||
target_language="en",
|
||||
status="idle",
|
||||
events=[],
|
||||
title="",
|
||||
)
|
||||
|
||||
query = transcripts.insert().values(**transcript.model_dump())
|
||||
await db.execute(query)
|
||||
print(f"Created transcript: {transcript_id}")
|
||||
|
||||
# Process with the fixed pipeline
|
||||
await task_pipeline_multitrack_process(
|
||||
transcript_id=transcript_id, bucket_name=bucket_name, track_keys=track_keys
|
||||
)
|
||||
|
||||
print(
|
||||
f"Processing complete! Check: http://localhost:3000/transcripts/{transcript_id}"
|
||||
)
|
||||
|
||||
return transcript_id
|
||||
finally:
|
||||
await db.disconnect()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
transcript_id = asyncio.run(reprocess())
|
||||
print(f"\n✅ Reprocessing complete!")
|
||||
print(f"📍 View at: http://localhost:3000/transcripts/{transcript_id}")
|
||||
@@ -1,5 +1,4 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Recreate Daily.co webhook (fixes circuit-breaker FAILED state)."""
|
||||
|
||||
import asyncio
|
||||
import sys
|
||||
@@ -12,8 +11,11 @@ import httpx
|
||||
from reflector.settings import settings
|
||||
|
||||
|
||||
async def recreate_webhook(webhook_url: str):
|
||||
"""Delete all webhooks and create new one."""
|
||||
async def setup_webhook(webhook_url: str):
|
||||
"""
|
||||
Create or update Daily.co webhook for this environment.
|
||||
Uses DAILY_WEBHOOK_UUID to identify existing webhook.
|
||||
"""
|
||||
if not settings.DAILY_API_KEY:
|
||||
print("Error: DAILY_API_KEY not set")
|
||||
return 1
|
||||
@@ -23,42 +25,85 @@ async def recreate_webhook(webhook_url: str):
|
||||
"Content-Type": "application/json",
|
||||
}
|
||||
|
||||
webhook_data = {
|
||||
"url": webhook_url,
|
||||
"eventTypes": [
|
||||
"participant.joined",
|
||||
"participant.left",
|
||||
"recording.started",
|
||||
"recording.ready-to-download",
|
||||
"recording.error",
|
||||
],
|
||||
"hmac": settings.DAILY_WEBHOOK_SECRET,
|
||||
}
|
||||
|
||||
async with httpx.AsyncClient() as client:
|
||||
# List existing webhooks
|
||||
resp = await client.get("https://api.daily.co/v1/webhooks", headers=headers)
|
||||
resp.raise_for_status()
|
||||
webhooks = resp.json()
|
||||
webhook_uuid = settings.DAILY_WEBHOOK_UUID
|
||||
|
||||
# Delete all existing webhooks
|
||||
for wh in webhooks:
|
||||
uuid = wh["uuid"]
|
||||
print(f"Deleting webhook {uuid} (state: {wh['state']})")
|
||||
await client.delete(
|
||||
f"https://api.daily.co/v1/webhooks/{uuid}", headers=headers
|
||||
if webhook_uuid:
|
||||
# Update existing webhook
|
||||
print(f"Updating existing webhook {webhook_uuid}...")
|
||||
try:
|
||||
resp = await client.patch(
|
||||
f"https://api.daily.co/v1/webhooks/{webhook_uuid}",
|
||||
headers=headers,
|
||||
json=webhook_data,
|
||||
)
|
||||
resp.raise_for_status()
|
||||
result = resp.json()
|
||||
print(f"✓ Updated webhook {result['uuid']} (state: {result['state']})")
|
||||
print(f" URL: {result['url']}")
|
||||
return 0
|
||||
except httpx.HTTPStatusError as e:
|
||||
if e.response.status_code == 404:
|
||||
print(f"Webhook {webhook_uuid} not found, creating new one...")
|
||||
webhook_uuid = None # Fall through to creation
|
||||
else:
|
||||
print(f"Error updating webhook: {e}")
|
||||
return 1
|
||||
|
||||
if not webhook_uuid:
|
||||
# Create new webhook
|
||||
print("Creating new webhook...")
|
||||
resp = await client.post(
|
||||
"https://api.daily.co/v1/webhooks", headers=headers, json=webhook_data
|
||||
)
|
||||
resp.raise_for_status()
|
||||
result = resp.json()
|
||||
webhook_uuid = result["uuid"]
|
||||
|
||||
# Create new webhook
|
||||
webhook_data = {
|
||||
"url": webhook_url,
|
||||
"eventTypes": [
|
||||
"participant.joined",
|
||||
"participant.left",
|
||||
"recording.started",
|
||||
"recording.ready-to-download",
|
||||
"recording.error",
|
||||
],
|
||||
"hmac": settings.DAILY_WEBHOOK_SECRET,
|
||||
}
|
||||
print(f"✓ Created webhook {webhook_uuid} (state: {result['state']})")
|
||||
print(f" URL: {result['url']}")
|
||||
print()
|
||||
print("=" * 60)
|
||||
print("IMPORTANT: Add this to your environment variables:")
|
||||
print("=" * 60)
|
||||
print(f"DAILY_WEBHOOK_UUID: {webhook_uuid}")
|
||||
print("=" * 60)
|
||||
print()
|
||||
|
||||
resp = await client.post(
|
||||
"https://api.daily.co/v1/webhooks", headers=headers, json=webhook_data
|
||||
)
|
||||
resp.raise_for_status()
|
||||
result = resp.json()
|
||||
# Try to write UUID to .env file
|
||||
env_file = Path(__file__).parent.parent / ".env"
|
||||
if env_file.exists():
|
||||
lines = env_file.read_text().splitlines()
|
||||
updated = False
|
||||
|
||||
print(f"Created webhook {result['uuid']} (state: {result['state']})")
|
||||
print(f"URL: {result['url']}")
|
||||
return 0
|
||||
# Update existing DAILY_WEBHOOK_UUID line or add it
|
||||
for i, line in enumerate(lines):
|
||||
if line.startswith("DAILY_WEBHOOK_UUID="):
|
||||
lines[i] = f"DAILY_WEBHOOK_UUID={webhook_uuid}"
|
||||
updated = True
|
||||
break
|
||||
|
||||
if not updated:
|
||||
lines.append(f"DAILY_WEBHOOK_UUID={webhook_uuid}")
|
||||
|
||||
env_file.write_text("\n".join(lines) + "\n")
|
||||
print(f"✓ Also saved to local .env file")
|
||||
else:
|
||||
print(f"⚠ Local .env file not found - please add manually")
|
||||
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
@@ -67,6 +112,12 @@ if __name__ == "__main__":
|
||||
print(
|
||||
"Example: python recreate_daily_webhook.py https://example.com/v1/daily/webhook"
|
||||
)
|
||||
print()
|
||||
print("Behavior:")
|
||||
print(" - If DAILY_WEBHOOK_UUID set: Updates existing webhook")
|
||||
print(
|
||||
" - If DAILY_WEBHOOK_UUID empty: Creates new webhook, saves UUID to .env"
|
||||
)
|
||||
sys.exit(1)
|
||||
|
||||
sys.exit(asyncio.run(recreate_webhook(sys.argv[1])))
|
||||
sys.exit(asyncio.run(setup_webhook(sys.argv[1])))
|
||||
|
||||
@@ -1,124 +0,0 @@
|
||||
#!/usr/bin/env python
|
||||
"""
|
||||
Test script to trigger multitrack recording processing with ffmpeg padding fix
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
|
||||
from reflector.pipelines.main_multitrack_pipeline import PipelineMainMultitrack
|
||||
|
||||
|
||||
async def test_processing():
|
||||
"""Manually trigger multitrack processing for the test recording"""
|
||||
|
||||
# Initialize database connection
|
||||
from reflector.db import get_database
|
||||
|
||||
db = get_database()
|
||||
await db.connect()
|
||||
|
||||
try:
|
||||
# The test recording with known speaker timeline
|
||||
bucket_name = "monadical"
|
||||
track_keys = [
|
||||
"daily-20251020193458/1760988935484-52f7f48b-fbab-431f-9a50-87b9abfc8255-cam-audio-1760988935922.webm",
|
||||
"daily-20251020193458/1760988935484-a37c35e3-6f8e-4274-a482-e9d0f102a732-cam-audio-1760988943823.webm",
|
||||
]
|
||||
|
||||
# Create a new transcript ID
|
||||
import uuid
|
||||
|
||||
transcript_id = str(uuid.uuid4())
|
||||
|
||||
# Create transcript directly with SQL
|
||||
from reflector.db.transcripts import (
|
||||
Transcript,
|
||||
transcripts,
|
||||
transcripts_controller,
|
||||
)
|
||||
|
||||
pipeline = PipelineMainMultitrack(transcript_id=transcript_id)
|
||||
|
||||
# Create transcript model
|
||||
transcript = Transcript(
|
||||
id=transcript_id,
|
||||
name="FFMPEG Test - Daily Multitrack Recording",
|
||||
source_kind="file",
|
||||
source_language="en",
|
||||
target_language="en",
|
||||
status="idle",
|
||||
events=[],
|
||||
title="",
|
||||
)
|
||||
# Insert into database
|
||||
query = transcripts.insert().values(**transcript.model_dump())
|
||||
await db.execute(query)
|
||||
print(f"Created transcript: {transcript_id}")
|
||||
|
||||
# Process the tracks using the pipeline
|
||||
print(f"Processing multitrack recording with ffmpeg padding...")
|
||||
print(f"Track 0: ...935922.webm (expected to start at ~2s)")
|
||||
print(f"Track 1: ...943823.webm (expected to start at ~51s)")
|
||||
|
||||
try:
|
||||
await pipeline.set_status(transcript_id, "processing")
|
||||
await pipeline.process(bucket_name, track_keys)
|
||||
print(f"Processing complete!")
|
||||
except Exception as e:
|
||||
await pipeline.set_status(transcript_id, "error")
|
||||
print(f"Error during processing: {e}")
|
||||
import traceback
|
||||
|
||||
traceback.print_exc()
|
||||
raise
|
||||
|
||||
# Check the results
|
||||
final_transcript = await transcripts_controller.get(transcript_id)
|
||||
print(f"\nTranscript status: {final_transcript.status}")
|
||||
print(f"Transcript title: {final_transcript.title}")
|
||||
|
||||
# Extract timeline from events
|
||||
if final_transcript.events:
|
||||
for event in final_transcript.events:
|
||||
if event.get("event") == "TRANSCRIPT":
|
||||
text = event.get("data", {}).get("text", "")
|
||||
# Show first 500 chars to check if speakers are properly separated
|
||||
print(f"\nTranscript text (first 500 chars):")
|
||||
print(text[:500])
|
||||
|
||||
# Show last 500 chars too to see if second speaker is at the end
|
||||
print(f"\nTranscript text (last 500 chars):")
|
||||
print(text[-500:])
|
||||
|
||||
# Count words per speaker
|
||||
words = text.split()
|
||||
print(f"\nTotal words in transcript: {len(words)}")
|
||||
|
||||
# Check if text has proper speaker separation
|
||||
# Expected: First ~45% from speaker 0, then ~35% from speaker 1, then ~20% from speaker 0
|
||||
first_third = " ".join(words[: len(words) // 3])
|
||||
middle_third = " ".join(
|
||||
words[len(words) // 3 : 2 * len(words) // 3]
|
||||
)
|
||||
last_third = " ".join(words[2 * len(words) // 3 :])
|
||||
|
||||
print(f"\nFirst third preview: {first_third[:100]}...")
|
||||
print(f"Middle third preview: {middle_third[:100]}...")
|
||||
print(f"Last third preview: {last_third[:100]}...")
|
||||
break
|
||||
|
||||
return transcript_id
|
||||
finally:
|
||||
await db.disconnect()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
transcript_id = asyncio.run(test_processing())
|
||||
print(f"\n✅ Test complete! Transcript ID: {transcript_id}")
|
||||
print(f"\nExpected timeline:")
|
||||
print(f" Speaker 0: ~2s to ~49s (first participant speaks)")
|
||||
print(f" Speaker 1: ~51s to ~70s (second participant speaks)")
|
||||
print(f" Speaker 0: ~73s to end (first participant speaks again)")
|
||||
print(
|
||||
f"\nIf the text shows proper chronological order (not interleaved), the fix worked!"
|
||||
)
|
||||
@@ -1,162 +0,0 @@
|
||||
#!/usr/bin/env python
|
||||
"""
|
||||
Test script to trigger multitrack recording processing with ffmpeg padding fix
|
||||
This version loads tracks from local filesystem instead of S3
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import os
|
||||
|
||||
from reflector.pipelines.main_multitrack_pipeline import PipelineMainMultitrack
|
||||
|
||||
|
||||
async def test_processing():
|
||||
"""Manually trigger multitrack processing for the test recording"""
|
||||
|
||||
# Initialize database connection
|
||||
from reflector.db import get_database
|
||||
|
||||
db = get_database()
|
||||
await db.connect()
|
||||
|
||||
try:
|
||||
# Create a new transcript ID
|
||||
import uuid
|
||||
|
||||
transcript_id = str(uuid.uuid4())
|
||||
|
||||
# Create transcript directly with SQL
|
||||
from reflector.db.transcripts import (
|
||||
Transcript,
|
||||
transcripts,
|
||||
transcripts_controller,
|
||||
)
|
||||
|
||||
pipeline = PipelineMainMultitrack(transcript_id=transcript_id)
|
||||
|
||||
# Create transcript model
|
||||
transcript = Transcript(
|
||||
id=transcript_id,
|
||||
name="FFMPEG Test - Daily Multitrack Recording",
|
||||
source_kind="file",
|
||||
source_language="en",
|
||||
target_language="en",
|
||||
status="idle",
|
||||
events=[],
|
||||
title="",
|
||||
)
|
||||
# Insert into database
|
||||
query = transcripts.insert().values(**transcript.model_dump())
|
||||
await db.execute(query)
|
||||
print(f"Created transcript: {transcript_id}")
|
||||
|
||||
# Read track files from local filesystem (in the container they'll be at /app/)
|
||||
tracks_dir = "/app"
|
||||
track_files = [
|
||||
"1760988935484-52f7f48b-fbab-431f-9a50-87b9abfc8255-cam-audio-1760988935922.webm",
|
||||
"1760988935484-a37c35e3-6f8e-4274-a482-e9d0f102a732-cam-audio-1760988943823.webm",
|
||||
]
|
||||
|
||||
# Read track data
|
||||
track_datas = []
|
||||
for track_file in track_files:
|
||||
file_path = os.path.join(tracks_dir, track_file)
|
||||
if os.path.exists(file_path):
|
||||
with open(file_path, "rb") as f:
|
||||
track_datas.append(f.read())
|
||||
print(f"Loaded track: {track_file} ({len(track_datas[-1])} bytes)")
|
||||
else:
|
||||
print(f"Track file not found: {file_path}")
|
||||
track_datas.append(b"")
|
||||
|
||||
# Process the tracks using the pipeline
|
||||
print(f"\nProcessing multitrack recording with ffmpeg padding...")
|
||||
print(f"Track 0: ...935922.webm (expected to start at ~2s)")
|
||||
print(f"Track 1: ...943823.webm (expected to start at ~51s)")
|
||||
|
||||
# Call the process method directly with track data
|
||||
# We'll need to mock S3 operations and directly work with the data
|
||||
|
||||
# Save tracks to temporary files and process them
|
||||
|
||||
try:
|
||||
await pipeline.set_status(transcript_id, "processing")
|
||||
|
||||
# Create a mock bucket and keys setup
|
||||
bucket_name = "test-bucket"
|
||||
track_keys = ["track0.webm", "track1.webm"]
|
||||
|
||||
# Mock S3 client to return our local data
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
mock_s3 = MagicMock()
|
||||
|
||||
def mock_get_object(Bucket, Key):
|
||||
idx = 0 if "track0" in Key else 1
|
||||
return {"Body": MagicMock(read=lambda: track_datas[idx])}
|
||||
|
||||
mock_s3.get_object = mock_get_object
|
||||
|
||||
# Patch boto3.client to return our mock
|
||||
with patch("boto3.client", return_value=mock_s3):
|
||||
await pipeline.process(bucket_name, track_keys)
|
||||
|
||||
print(f"Processing complete!")
|
||||
except Exception as e:
|
||||
await pipeline.set_status(transcript_id, "error")
|
||||
print(f"Error during processing: {e}")
|
||||
import traceback
|
||||
|
||||
traceback.print_exc()
|
||||
raise
|
||||
|
||||
# Check the results
|
||||
final_transcript = await transcripts_controller.get(transcript_id)
|
||||
print(f"\nTranscript status: {final_transcript.status}")
|
||||
print(f"Transcript title: {final_transcript.title}")
|
||||
|
||||
# Extract timeline from events
|
||||
if final_transcript.events:
|
||||
for event in final_transcript.events:
|
||||
if event.get("event") == "TRANSCRIPT":
|
||||
text = event.get("data", {}).get("text", "")
|
||||
# Show first 500 chars to check if speakers are properly separated
|
||||
print(f"\nTranscript text (first 500 chars):")
|
||||
print(text[:500])
|
||||
|
||||
# Show last 500 chars too to see if second speaker is at the end
|
||||
print(f"\nTranscript text (last 500 chars):")
|
||||
print(text[-500:])
|
||||
|
||||
# Count words per speaker
|
||||
words = text.split()
|
||||
print(f"\nTotal words in transcript: {len(words)}")
|
||||
|
||||
# Check if text has proper speaker separation
|
||||
# Expected: First ~45% from speaker 0, then ~35% from speaker 1, then ~20% from speaker 0
|
||||
first_third = " ".join(words[: len(words) // 3])
|
||||
middle_third = " ".join(
|
||||
words[len(words) // 3 : 2 * len(words) // 3]
|
||||
)
|
||||
last_third = " ".join(words[2 * len(words) // 3 :])
|
||||
|
||||
print(f"\nFirst third preview: {first_third[:100]}...")
|
||||
print(f"Middle third preview: {middle_third[:100]}...")
|
||||
print(f"Last third preview: {last_third[:100]}...")
|
||||
break
|
||||
|
||||
return transcript_id
|
||||
finally:
|
||||
await db.disconnect()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
transcript_id = asyncio.run(test_processing())
|
||||
print(f"\n✅ Test complete! Transcript ID: {transcript_id}")
|
||||
print(f"\nExpected timeline:")
|
||||
print(f" Speaker 0: ~2s to ~49s (first participant speaks)")
|
||||
print(f" Speaker 1: ~51s to ~70s (second participant speaks)")
|
||||
print(f" Speaker 0: ~73s to end (first participant speaks again)")
|
||||
print(
|
||||
f"\nIf the text shows proper chronological order (not interleaved), the fix worked!"
|
||||
)
|
||||
@@ -1,66 +0,0 @@
|
||||
#!/usr/bin/env python
|
||||
"""
|
||||
Test multitrack processing with correct S3 bucket configuration
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import uuid
|
||||
|
||||
from reflector.db import get_database
|
||||
from reflector.db.transcripts import Transcript, transcripts
|
||||
from reflector.pipelines.main_multitrack_pipeline import (
|
||||
task_pipeline_multitrack_process,
|
||||
)
|
||||
|
||||
|
||||
async def create_and_process():
|
||||
"""Create a new transcript and process with correct S3 bucket"""
|
||||
|
||||
# Correct S3 configuration
|
||||
bucket_name = "reflector-dailyco-local"
|
||||
track_keys = [
|
||||
"monadical/daily-20251020193458/1760988935484-52f7f48b-fbab-431f-9a50-87b9abfc8255-cam-audio-1760988935922",
|
||||
"monadical/daily-20251020193458/1760988935484-a37c35e3-6f8e-4274-a482-e9d0f102a732-cam-audio-1760988943823",
|
||||
]
|
||||
|
||||
# Create a new transcript
|
||||
db = get_database()
|
||||
await db.connect()
|
||||
|
||||
try:
|
||||
transcript_id = str(uuid.uuid4())
|
||||
transcript = Transcript(
|
||||
id=transcript_id,
|
||||
name="Daily Multitrack - Correct S3 Bucket Test",
|
||||
source_kind="file",
|
||||
source_language="en",
|
||||
target_language="en",
|
||||
status="idle",
|
||||
events=[],
|
||||
title="",
|
||||
)
|
||||
|
||||
query = transcripts.insert().values(**transcript.model_dump())
|
||||
await db.execute(query)
|
||||
print(f"Created transcript: {transcript_id}")
|
||||
|
||||
# Trigger processing with Celery
|
||||
result = task_pipeline_multitrack_process.delay(
|
||||
transcript_id=transcript_id, bucket_name=bucket_name, track_keys=track_keys
|
||||
)
|
||||
|
||||
print(f"Task ID: {result.id}")
|
||||
print(
|
||||
f"Processing started! Check: http://localhost:3000/transcripts/{transcript_id}"
|
||||
)
|
||||
print(f"API Status: http://localhost:1250/v1/transcripts/{transcript_id}")
|
||||
|
||||
return transcript_id
|
||||
finally:
|
||||
await db.disconnect()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
transcript_id = asyncio.run(create_and_process())
|
||||
print(f"\n✅ Task submitted successfully!")
|
||||
print(f"📍 Transcript ID: {transcript_id}")
|
||||
@@ -5,6 +5,8 @@ from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
|
||||
from reflector.schemas.platform import WHEREBY_PLATFORM
|
||||
|
||||
|
||||
@pytest.fixture(scope="session", autouse=True)
|
||||
def register_mock_platform():
|
||||
@@ -12,7 +14,7 @@ def register_mock_platform():
|
||||
|
||||
from reflector.video_platforms.registry import register_platform
|
||||
|
||||
register_platform("whereby", MockPlatformClient)
|
||||
register_platform(WHEREBY_PLATFORM, MockPlatformClient)
|
||||
yield
|
||||
|
||||
|
||||
|
||||
@@ -4,6 +4,7 @@ from typing import Any, Dict, Literal, Optional
|
||||
|
||||
from reflector.db.rooms import Room
|
||||
from reflector.video_platforms.base import (
|
||||
ROOM_PREFIX_SEPARATOR,
|
||||
MeetingData,
|
||||
VideoPlatformClient,
|
||||
VideoPlatformConfig,
|
||||
@@ -24,7 +25,7 @@ class MockPlatformClient(VideoPlatformClient):
|
||||
self, room_name_prefix: str, end_date: datetime, room: Room
|
||||
) -> MeetingData:
|
||||
meeting_id = str(uuid.uuid4())
|
||||
room_name = f"{room_name_prefix}-{meeting_id[:8]}"
|
||||
room_name = f"{room_name_prefix}{ROOM_PREFIX_SEPARATOR}{meeting_id[:8]}"
|
||||
room_url = f"https://mock.video/{room_name}"
|
||||
host_room_url = f"{room_url}?host=true"
|
||||
|
||||
|
||||
@@ -139,14 +139,10 @@ async def test_cleanup_deletes_associated_meeting_and_recording():
|
||||
mock_settings.PUBLIC_DATA_RETENTION_DAYS = 7
|
||||
|
||||
# Mock storage deletion
|
||||
with patch("reflector.db.transcripts.get_transcripts_storage") as mock_storage:
|
||||
with patch("reflector.worker.cleanup.get_transcripts_storage") as mock_storage:
|
||||
mock_storage.return_value.delete_file = AsyncMock()
|
||||
with patch(
|
||||
"reflector.worker.cleanup.get_recordings_storage"
|
||||
) as mock_rec_storage:
|
||||
mock_rec_storage.return_value.delete_file = AsyncMock()
|
||||
|
||||
result = await cleanup_old_public_data()
|
||||
result = await cleanup_old_public_data()
|
||||
|
||||
# Check results
|
||||
assert result["transcripts_deleted"] == 1
|
||||
|
||||
330
server/tests/test_consent_multitrack.py
Normal file
330
server/tests/test_consent_multitrack.py
Normal file
@@ -0,0 +1,330 @@
|
||||
from datetime import datetime, timezone
|
||||
from unittest.mock import AsyncMock, MagicMock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
from reflector.db.meetings import (
|
||||
MeetingConsent,
|
||||
meeting_consent_controller,
|
||||
meetings_controller,
|
||||
)
|
||||
from reflector.db.recordings import Recording, recordings_controller
|
||||
from reflector.db.rooms import rooms_controller
|
||||
from reflector.db.transcripts import SourceKind, transcripts_controller
|
||||
from reflector.pipelines.main_live_pipeline import cleanup_consent
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_consent_cleanup_deletes_multitrack_files():
|
||||
room = await rooms_controller.add(
|
||||
name="Test Room",
|
||||
user_id="test-user",
|
||||
zulip_auto_post=False,
|
||||
zulip_stream="",
|
||||
zulip_topic="",
|
||||
is_locked=False,
|
||||
room_mode="normal",
|
||||
recording_type="cloud",
|
||||
recording_trigger="automatic",
|
||||
is_shared=False,
|
||||
platform="daily",
|
||||
)
|
||||
|
||||
# Create meeting
|
||||
meeting = await meetings_controller.create(
|
||||
id="test-multitrack-meeting",
|
||||
room_name="test-room-20250101120000",
|
||||
room_url="https://test.daily.co/test-room",
|
||||
host_room_url="https://test.daily.co/test-room",
|
||||
start_date=datetime.now(timezone.utc),
|
||||
end_date=datetime.now(timezone.utc),
|
||||
room=room,
|
||||
)
|
||||
|
||||
track_keys = [
|
||||
"recordings/test-room-20250101120000/track-0.webm",
|
||||
"recordings/test-room-20250101120000/track-1.webm",
|
||||
"recordings/test-room-20250101120000/track-2.webm",
|
||||
]
|
||||
recording = await recordings_controller.create(
|
||||
Recording(
|
||||
bucket_name="test-bucket",
|
||||
object_key="recordings/test-room-20250101120000", # Folder path
|
||||
recorded_at=datetime.now(timezone.utc),
|
||||
meeting_id=meeting.id,
|
||||
track_keys=track_keys,
|
||||
)
|
||||
)
|
||||
|
||||
# Create transcript
|
||||
transcript = await transcripts_controller.add(
|
||||
name="Test Multitrack Transcript",
|
||||
source_kind=SourceKind.ROOM,
|
||||
recording_id=recording.id,
|
||||
meeting_id=meeting.id,
|
||||
)
|
||||
|
||||
# Add consent denial
|
||||
await meeting_consent_controller.upsert(
|
||||
MeetingConsent(
|
||||
meeting_id=meeting.id,
|
||||
user_id="test-user",
|
||||
consent_given=False,
|
||||
consent_timestamp=datetime.now(timezone.utc),
|
||||
)
|
||||
)
|
||||
|
||||
# Mock get_transcripts_storage (master credentials with bucket override)
|
||||
with patch(
|
||||
"reflector.pipelines.main_live_pipeline.get_transcripts_storage"
|
||||
) as mock_get_transcripts_storage:
|
||||
mock_master_storage = MagicMock()
|
||||
mock_master_storage.delete_file = AsyncMock()
|
||||
mock_get_transcripts_storage.return_value = mock_master_storage
|
||||
|
||||
await cleanup_consent(transcript_id=transcript.id)
|
||||
|
||||
# Verify master storage was used with bucket override for all track keys
|
||||
assert mock_master_storage.delete_file.call_count == 3
|
||||
deleted_keys = []
|
||||
for call_args in mock_master_storage.delete_file.call_args_list:
|
||||
key = call_args[0][0]
|
||||
bucket_kwarg = call_args[1].get("bucket")
|
||||
deleted_keys.append(key)
|
||||
assert bucket_kwarg == "test-bucket" # Verify bucket override!
|
||||
assert set(deleted_keys) == set(track_keys)
|
||||
|
||||
updated_transcript = await transcripts_controller.get_by_id(transcript.id)
|
||||
assert updated_transcript.audio_deleted is True
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_consent_cleanup_handles_missing_track_keys():
|
||||
room = await rooms_controller.add(
|
||||
name="Test Room 2",
|
||||
user_id="test-user",
|
||||
zulip_auto_post=False,
|
||||
zulip_stream="",
|
||||
zulip_topic="",
|
||||
is_locked=False,
|
||||
room_mode="normal",
|
||||
recording_type="cloud",
|
||||
recording_trigger="automatic",
|
||||
is_shared=False,
|
||||
platform="daily",
|
||||
)
|
||||
|
||||
# Create meeting
|
||||
meeting = await meetings_controller.create(
|
||||
id="test-multitrack-meeting-2",
|
||||
room_name="test-room-20250101120001",
|
||||
room_url="https://test.daily.co/test-room-2",
|
||||
host_room_url="https://test.daily.co/test-room-2",
|
||||
start_date=datetime.now(timezone.utc),
|
||||
end_date=datetime.now(timezone.utc),
|
||||
room=room,
|
||||
)
|
||||
|
||||
recording = await recordings_controller.create(
|
||||
Recording(
|
||||
bucket_name="test-bucket",
|
||||
object_key="recordings/old-style-recording.mp4",
|
||||
recorded_at=datetime.now(timezone.utc),
|
||||
meeting_id=meeting.id,
|
||||
track_keys=None,
|
||||
)
|
||||
)
|
||||
|
||||
transcript = await transcripts_controller.add(
|
||||
name="Test Old-Style Transcript",
|
||||
source_kind=SourceKind.ROOM,
|
||||
recording_id=recording.id,
|
||||
meeting_id=meeting.id,
|
||||
)
|
||||
|
||||
# Add consent denial
|
||||
await meeting_consent_controller.upsert(
|
||||
MeetingConsent(
|
||||
meeting_id=meeting.id,
|
||||
user_id="test-user-2",
|
||||
consent_given=False,
|
||||
consent_timestamp=datetime.now(timezone.utc),
|
||||
)
|
||||
)
|
||||
|
||||
# Mock get_transcripts_storage (master credentials with bucket override)
|
||||
with patch(
|
||||
"reflector.pipelines.main_live_pipeline.get_transcripts_storage"
|
||||
) as mock_get_transcripts_storage:
|
||||
mock_master_storage = MagicMock()
|
||||
mock_master_storage.delete_file = AsyncMock()
|
||||
mock_get_transcripts_storage.return_value = mock_master_storage
|
||||
|
||||
await cleanup_consent(transcript_id=transcript.id)
|
||||
|
||||
# Verify master storage was used with bucket override
|
||||
assert mock_master_storage.delete_file.call_count == 1
|
||||
call_args = mock_master_storage.delete_file.call_args
|
||||
assert call_args[0][0] == recording.object_key
|
||||
assert call_args[1].get("bucket") == "test-bucket" # Verify bucket override!
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_consent_cleanup_empty_track_keys_falls_back():
|
||||
room = await rooms_controller.add(
|
||||
name="Test Room 3",
|
||||
user_id="test-user",
|
||||
zulip_auto_post=False,
|
||||
zulip_stream="",
|
||||
zulip_topic="",
|
||||
is_locked=False,
|
||||
room_mode="normal",
|
||||
recording_type="cloud",
|
||||
recording_trigger="automatic",
|
||||
is_shared=False,
|
||||
platform="daily",
|
||||
)
|
||||
|
||||
# Create meeting
|
||||
meeting = await meetings_controller.create(
|
||||
id="test-multitrack-meeting-3",
|
||||
room_name="test-room-20250101120002",
|
||||
room_url="https://test.daily.co/test-room-3",
|
||||
host_room_url="https://test.daily.co/test-room-3",
|
||||
start_date=datetime.now(timezone.utc),
|
||||
end_date=datetime.now(timezone.utc),
|
||||
room=room,
|
||||
)
|
||||
|
||||
recording = await recordings_controller.create(
|
||||
Recording(
|
||||
bucket_name="test-bucket",
|
||||
object_key="recordings/fallback-recording.mp4",
|
||||
recorded_at=datetime.now(timezone.utc),
|
||||
meeting_id=meeting.id,
|
||||
track_keys=[],
|
||||
)
|
||||
)
|
||||
|
||||
transcript = await transcripts_controller.add(
|
||||
name="Test Empty Track Keys Transcript",
|
||||
source_kind=SourceKind.ROOM,
|
||||
recording_id=recording.id,
|
||||
meeting_id=meeting.id,
|
||||
)
|
||||
|
||||
# Add consent denial
|
||||
await meeting_consent_controller.upsert(
|
||||
MeetingConsent(
|
||||
meeting_id=meeting.id,
|
||||
user_id="test-user-3",
|
||||
consent_given=False,
|
||||
consent_timestamp=datetime.now(timezone.utc),
|
||||
)
|
||||
)
|
||||
|
||||
# Mock get_transcripts_storage (master credentials with bucket override)
|
||||
with patch(
|
||||
"reflector.pipelines.main_live_pipeline.get_transcripts_storage"
|
||||
) as mock_get_transcripts_storage:
|
||||
mock_master_storage = MagicMock()
|
||||
mock_master_storage.delete_file = AsyncMock()
|
||||
mock_get_transcripts_storage.return_value = mock_master_storage
|
||||
|
||||
# Run cleanup
|
||||
await cleanup_consent(transcript_id=transcript.id)
|
||||
|
||||
# Verify master storage was used with bucket override
|
||||
assert mock_master_storage.delete_file.call_count == 1
|
||||
call_args = mock_master_storage.delete_file.call_args
|
||||
assert call_args[0][0] == recording.object_key
|
||||
assert call_args[1].get("bucket") == "test-bucket" # Verify bucket override!
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_consent_cleanup_partial_failure_doesnt_mark_deleted():
|
||||
room = await rooms_controller.add(
|
||||
name="Test Room 4",
|
||||
user_id="test-user",
|
||||
zulip_auto_post=False,
|
||||
zulip_stream="",
|
||||
zulip_topic="",
|
||||
is_locked=False,
|
||||
room_mode="normal",
|
||||
recording_type="cloud",
|
||||
recording_trigger="automatic",
|
||||
is_shared=False,
|
||||
platform="daily",
|
||||
)
|
||||
|
||||
# Create meeting
|
||||
meeting = await meetings_controller.create(
|
||||
id="test-multitrack-meeting-4",
|
||||
room_name="test-room-20250101120003",
|
||||
room_url="https://test.daily.co/test-room-4",
|
||||
host_room_url="https://test.daily.co/test-room-4",
|
||||
start_date=datetime.now(timezone.utc),
|
||||
end_date=datetime.now(timezone.utc),
|
||||
room=room,
|
||||
)
|
||||
|
||||
track_keys = [
|
||||
"recordings/test-room-20250101120003/track-0.webm",
|
||||
"recordings/test-room-20250101120003/track-1.webm",
|
||||
"recordings/test-room-20250101120003/track-2.webm",
|
||||
]
|
||||
recording = await recordings_controller.create(
|
||||
Recording(
|
||||
bucket_name="test-bucket",
|
||||
object_key="recordings/test-room-20250101120003",
|
||||
recorded_at=datetime.now(timezone.utc),
|
||||
meeting_id=meeting.id,
|
||||
track_keys=track_keys,
|
||||
)
|
||||
)
|
||||
|
||||
# Create transcript
|
||||
transcript = await transcripts_controller.add(
|
||||
name="Test Partial Failure Transcript",
|
||||
source_kind=SourceKind.ROOM,
|
||||
recording_id=recording.id,
|
||||
meeting_id=meeting.id,
|
||||
)
|
||||
|
||||
# Add consent denial
|
||||
await meeting_consent_controller.upsert(
|
||||
MeetingConsent(
|
||||
meeting_id=meeting.id,
|
||||
user_id="test-user-4",
|
||||
consent_given=False,
|
||||
consent_timestamp=datetime.now(timezone.utc),
|
||||
)
|
||||
)
|
||||
|
||||
# Mock get_transcripts_storage (master credentials with bucket override) with partial failure
|
||||
with patch(
|
||||
"reflector.pipelines.main_live_pipeline.get_transcripts_storage"
|
||||
) as mock_get_transcripts_storage:
|
||||
mock_master_storage = MagicMock()
|
||||
|
||||
call_count = 0
|
||||
|
||||
async def delete_side_effect(key, bucket=None):
|
||||
nonlocal call_count
|
||||
call_count += 1
|
||||
if call_count == 2:
|
||||
raise Exception("S3 deletion failed")
|
||||
|
||||
mock_master_storage.delete_file = AsyncMock(side_effect=delete_side_effect)
|
||||
mock_get_transcripts_storage.return_value = mock_master_storage
|
||||
|
||||
await cleanup_consent(transcript_id=transcript.id)
|
||||
|
||||
# Verify master storage was called with bucket override
|
||||
assert mock_master_storage.delete_file.call_count == 3
|
||||
|
||||
updated_transcript = await transcripts_controller.get_by_id(transcript.id)
|
||||
assert (
|
||||
updated_transcript.audio_deleted is None
|
||||
or updated_transcript.audio_deleted is False
|
||||
)
|
||||
@@ -127,18 +127,27 @@ async def mock_storage():
|
||||
from reflector.storage.base import Storage
|
||||
|
||||
class TestStorage(Storage):
|
||||
async def _put_file(self, path, data):
|
||||
async def _put_file(self, path, data, bucket=None):
|
||||
return None
|
||||
|
||||
async def _get_file_url(self, path):
|
||||
async def _get_file_url(
|
||||
self,
|
||||
path,
|
||||
operation: str = "get_object",
|
||||
expires_in: int = 3600,
|
||||
bucket=None,
|
||||
):
|
||||
return f"http://test-storage/{path}"
|
||||
|
||||
async def _get_file(self, path):
|
||||
async def _get_file(self, path, bucket=None):
|
||||
return b"test_audio_data"
|
||||
|
||||
async def _delete_file(self, path):
|
||||
async def _delete_file(self, path, bucket=None):
|
||||
return None
|
||||
|
||||
async def _stream_to_fileobj(self, path, fileobj, bucket=None):
|
||||
fileobj.write(b"test_audio_data")
|
||||
|
||||
storage = TestStorage()
|
||||
# Add mock tracking for verification
|
||||
storage._put_file = AsyncMock(side_effect=storage._put_file)
|
||||
@@ -181,7 +190,7 @@ async def mock_waveform_processor():
|
||||
async def mock_topic_detector():
|
||||
"""Mock TranscriptTopicDetectorProcessor"""
|
||||
with patch(
|
||||
"reflector.pipelines.main_file_pipeline.TranscriptTopicDetectorProcessor"
|
||||
"reflector.pipelines.topic_processing.TranscriptTopicDetectorProcessor"
|
||||
) as mock_topic_class:
|
||||
mock_topic = AsyncMock()
|
||||
mock_topic.set_pipeline = MagicMock()
|
||||
@@ -218,7 +227,7 @@ async def mock_topic_detector():
|
||||
async def mock_title_processor():
|
||||
"""Mock TranscriptFinalTitleProcessor"""
|
||||
with patch(
|
||||
"reflector.pipelines.main_file_pipeline.TranscriptFinalTitleProcessor"
|
||||
"reflector.pipelines.topic_processing.TranscriptFinalTitleProcessor"
|
||||
) as mock_title_class:
|
||||
mock_title = AsyncMock()
|
||||
mock_title.set_pipeline = MagicMock()
|
||||
@@ -247,7 +256,7 @@ async def mock_title_processor():
|
||||
async def mock_summary_processor():
|
||||
"""Mock TranscriptFinalSummaryProcessor"""
|
||||
with patch(
|
||||
"reflector.pipelines.main_file_pipeline.TranscriptFinalSummaryProcessor"
|
||||
"reflector.pipelines.topic_processing.TranscriptFinalSummaryProcessor"
|
||||
) as mock_summary_class:
|
||||
mock_summary = AsyncMock()
|
||||
mock_summary.set_pipeline = MagicMock()
|
||||
|
||||
@@ -48,6 +48,7 @@ async def test_create_room_with_ics_fields(authenticated_client):
|
||||
"ics_url": "https://calendar.example.com/test.ics",
|
||||
"ics_fetch_interval": 600,
|
||||
"ics_enabled": True,
|
||||
"platform": "daily",
|
||||
},
|
||||
)
|
||||
assert response.status_code == 200
|
||||
@@ -75,6 +76,7 @@ async def test_update_room_ics_configuration(authenticated_client):
|
||||
"is_shared": False,
|
||||
"webhook_url": "",
|
||||
"webhook_secret": "",
|
||||
"platform": "daily",
|
||||
},
|
||||
)
|
||||
assert response.status_code == 200
|
||||
@@ -111,6 +113,7 @@ async def test_trigger_ics_sync(authenticated_client):
|
||||
is_shared=False,
|
||||
ics_url="https://calendar.example.com/api.ics",
|
||||
ics_enabled=True,
|
||||
platform="daily",
|
||||
)
|
||||
|
||||
cal = Calendar()
|
||||
@@ -154,6 +157,7 @@ async def test_trigger_ics_sync_unauthorized(client):
|
||||
is_shared=False,
|
||||
ics_url="https://calendar.example.com/api.ics",
|
||||
ics_enabled=True,
|
||||
platform="daily",
|
||||
)
|
||||
|
||||
response = await client.post(f"/rooms/{room.name}/ics/sync")
|
||||
@@ -176,6 +180,7 @@ async def test_trigger_ics_sync_not_configured(authenticated_client):
|
||||
recording_trigger="automatic-2nd-participant",
|
||||
is_shared=False,
|
||||
ics_enabled=False,
|
||||
platform="daily",
|
||||
)
|
||||
|
||||
response = await client.post(f"/rooms/{room.name}/ics/sync")
|
||||
@@ -200,6 +205,7 @@ async def test_get_ics_status(authenticated_client):
|
||||
ics_url="https://calendar.example.com/status.ics",
|
||||
ics_enabled=True,
|
||||
ics_fetch_interval=300,
|
||||
platform="daily",
|
||||
)
|
||||
|
||||
now = datetime.now(timezone.utc)
|
||||
@@ -231,6 +237,7 @@ async def test_get_ics_status_unauthorized(client):
|
||||
is_shared=False,
|
||||
ics_url="https://calendar.example.com/status.ics",
|
||||
ics_enabled=True,
|
||||
platform="daily",
|
||||
)
|
||||
|
||||
response = await client.get(f"/rooms/{room.name}/ics/status")
|
||||
@@ -252,6 +259,7 @@ async def test_list_room_meetings(authenticated_client):
|
||||
recording_type="cloud",
|
||||
recording_trigger="automatic-2nd-participant",
|
||||
is_shared=False,
|
||||
platform="daily",
|
||||
)
|
||||
|
||||
now = datetime.now(timezone.utc)
|
||||
@@ -298,6 +306,7 @@ async def test_list_room_meetings_non_owner(client):
|
||||
recording_type="cloud",
|
||||
recording_trigger="automatic-2nd-participant",
|
||||
is_shared=False,
|
||||
platform="daily",
|
||||
)
|
||||
|
||||
event = CalendarEvent(
|
||||
@@ -334,6 +343,7 @@ async def test_list_upcoming_meetings(authenticated_client):
|
||||
recording_type="cloud",
|
||||
recording_trigger="automatic-2nd-participant",
|
||||
is_shared=False,
|
||||
platform="daily",
|
||||
)
|
||||
|
||||
now = datetime.now(timezone.utc)
|
||||
|
||||
256
server/tests/test_search_date_filtering.py
Normal file
256
server/tests/test_search_date_filtering.py
Normal file
@@ -0,0 +1,256 @@
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
import pytest
|
||||
|
||||
from reflector.db import get_database
|
||||
from reflector.db.search import SearchParameters, search_controller
|
||||
from reflector.db.transcripts import SourceKind, transcripts
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
class TestDateRangeIntegration:
|
||||
async def setup_test_transcripts(self):
|
||||
# Use a test user_id that will match in our search parameters
|
||||
test_user_id = "test-user-123"
|
||||
|
||||
test_data = [
|
||||
{
|
||||
"id": "test-before-range",
|
||||
"created_at": datetime(2024, 1, 15, tzinfo=timezone.utc),
|
||||
"title": "Before Range Transcript",
|
||||
"user_id": test_user_id,
|
||||
},
|
||||
{
|
||||
"id": "test-start-boundary",
|
||||
"created_at": datetime(2024, 6, 1, tzinfo=timezone.utc),
|
||||
"title": "Start Boundary Transcript",
|
||||
"user_id": test_user_id,
|
||||
},
|
||||
{
|
||||
"id": "test-middle-range",
|
||||
"created_at": datetime(2024, 6, 15, tzinfo=timezone.utc),
|
||||
"title": "Middle Range Transcript",
|
||||
"user_id": test_user_id,
|
||||
},
|
||||
{
|
||||
"id": "test-end-boundary",
|
||||
"created_at": datetime(2024, 6, 30, 23, 59, 59, tzinfo=timezone.utc),
|
||||
"title": "End Boundary Transcript",
|
||||
"user_id": test_user_id,
|
||||
},
|
||||
{
|
||||
"id": "test-after-range",
|
||||
"created_at": datetime(2024, 12, 31, tzinfo=timezone.utc),
|
||||
"title": "After Range Transcript",
|
||||
"user_id": test_user_id,
|
||||
},
|
||||
]
|
||||
|
||||
for data in test_data:
|
||||
full_data = {
|
||||
"id": data["id"],
|
||||
"name": data["id"],
|
||||
"status": "ended",
|
||||
"locked": False,
|
||||
"duration": 60.0,
|
||||
"created_at": data["created_at"],
|
||||
"title": data["title"],
|
||||
"short_summary": "Test summary",
|
||||
"long_summary": "Test long summary",
|
||||
"share_mode": "public",
|
||||
"source_kind": SourceKind.FILE,
|
||||
"audio_deleted": False,
|
||||
"reviewed": False,
|
||||
"user_id": data["user_id"],
|
||||
}
|
||||
|
||||
await get_database().execute(transcripts.insert().values(**full_data))
|
||||
|
||||
return test_data
|
||||
|
||||
async def cleanup_test_transcripts(self, test_data):
|
||||
"""Clean up test transcripts."""
|
||||
for data in test_data:
|
||||
await get_database().execute(
|
||||
transcripts.delete().where(transcripts.c.id == data["id"])
|
||||
)
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_filter_with_from_datetime_only(self):
|
||||
"""Test filtering with only from_datetime parameter."""
|
||||
test_data = await self.setup_test_transcripts()
|
||||
test_user_id = "test-user-123"
|
||||
|
||||
try:
|
||||
params = SearchParameters(
|
||||
query_text=None,
|
||||
from_datetime=datetime(2024, 6, 1, tzinfo=timezone.utc),
|
||||
to_datetime=None,
|
||||
user_id=test_user_id,
|
||||
)
|
||||
|
||||
results, total = await search_controller.search_transcripts(params)
|
||||
|
||||
# Should include: start_boundary, middle, end_boundary, after
|
||||
result_ids = [r.id for r in results]
|
||||
assert "test-before-range" not in result_ids
|
||||
assert "test-start-boundary" in result_ids
|
||||
assert "test-middle-range" in result_ids
|
||||
assert "test-end-boundary" in result_ids
|
||||
assert "test-after-range" in result_ids
|
||||
|
||||
finally:
|
||||
await self.cleanup_test_transcripts(test_data)
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_filter_with_to_datetime_only(self):
|
||||
"""Test filtering with only to_datetime parameter."""
|
||||
test_data = await self.setup_test_transcripts()
|
||||
test_user_id = "test-user-123"
|
||||
|
||||
try:
|
||||
params = SearchParameters(
|
||||
query_text=None,
|
||||
from_datetime=None,
|
||||
to_datetime=datetime(2024, 6, 30, tzinfo=timezone.utc),
|
||||
user_id=test_user_id,
|
||||
)
|
||||
|
||||
results, total = await search_controller.search_transcripts(params)
|
||||
|
||||
result_ids = [r.id for r in results]
|
||||
assert "test-before-range" in result_ids
|
||||
assert "test-start-boundary" in result_ids
|
||||
assert "test-middle-range" in result_ids
|
||||
assert "test-end-boundary" not in result_ids
|
||||
assert "test-after-range" not in result_ids
|
||||
|
||||
finally:
|
||||
await self.cleanup_test_transcripts(test_data)
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_filter_with_both_datetimes(self):
|
||||
test_data = await self.setup_test_transcripts()
|
||||
test_user_id = "test-user-123"
|
||||
|
||||
try:
|
||||
params = SearchParameters(
|
||||
query_text=None,
|
||||
from_datetime=datetime(2024, 6, 1, tzinfo=timezone.utc),
|
||||
to_datetime=datetime(
|
||||
2024, 7, 1, tzinfo=timezone.utc
|
||||
), # Inclusive of 6/30
|
||||
user_id=test_user_id,
|
||||
)
|
||||
|
||||
results, total = await search_controller.search_transcripts(params)
|
||||
|
||||
result_ids = [r.id for r in results]
|
||||
assert "test-before-range" not in result_ids
|
||||
assert "test-start-boundary" in result_ids
|
||||
assert "test-middle-range" in result_ids
|
||||
assert "test-end-boundary" in result_ids
|
||||
assert "test-after-range" not in result_ids
|
||||
|
||||
finally:
|
||||
await self.cleanup_test_transcripts(test_data)
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_date_filter_with_room_and_source_kind(self):
|
||||
test_data = await self.setup_test_transcripts()
|
||||
test_user_id = "test-user-123"
|
||||
|
||||
try:
|
||||
params = SearchParameters(
|
||||
query_text=None,
|
||||
from_datetime=datetime(2024, 6, 1, tzinfo=timezone.utc),
|
||||
to_datetime=datetime(2024, 7, 1, tzinfo=timezone.utc),
|
||||
source_kind=SourceKind.FILE,
|
||||
room_id=None,
|
||||
user_id=test_user_id,
|
||||
)
|
||||
|
||||
results, total = await search_controller.search_transcripts(params)
|
||||
|
||||
for result in results:
|
||||
assert result.source_kind == SourceKind.FILE
|
||||
assert result.created_at >= datetime(2024, 6, 1, tzinfo=timezone.utc)
|
||||
assert result.created_at <= datetime(2024, 7, 1, tzinfo=timezone.utc)
|
||||
|
||||
finally:
|
||||
await self.cleanup_test_transcripts(test_data)
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_empty_results_for_future_dates(self):
|
||||
test_data = await self.setup_test_transcripts()
|
||||
test_user_id = "test-user-123"
|
||||
|
||||
try:
|
||||
params = SearchParameters(
|
||||
query_text=None,
|
||||
from_datetime=datetime(2099, 1, 1, tzinfo=timezone.utc),
|
||||
to_datetime=datetime(2099, 12, 31, tzinfo=timezone.utc),
|
||||
user_id=test_user_id,
|
||||
)
|
||||
|
||||
results, total = await search_controller.search_transcripts(params)
|
||||
|
||||
assert results == []
|
||||
assert total == 0
|
||||
|
||||
finally:
|
||||
await self.cleanup_test_transcripts(test_data)
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_date_only_input_handling(self):
|
||||
test_data = await self.setup_test_transcripts()
|
||||
test_user_id = "test-user-123"
|
||||
|
||||
try:
|
||||
# Pydantic will parse date-only strings to datetime at midnight
|
||||
from_dt = datetime(2024, 6, 15, 0, 0, 0, tzinfo=timezone.utc)
|
||||
to_dt = datetime(2024, 6, 16, 0, 0, 0, tzinfo=timezone.utc)
|
||||
|
||||
params = SearchParameters(
|
||||
query_text=None,
|
||||
from_datetime=from_dt,
|
||||
to_datetime=to_dt,
|
||||
user_id=test_user_id,
|
||||
)
|
||||
|
||||
results, total = await search_controller.search_transcripts(params)
|
||||
|
||||
result_ids = [r.id for r in results]
|
||||
assert "test-middle-range" in result_ids
|
||||
assert "test-before-range" not in result_ids
|
||||
assert "test-after-range" not in result_ids
|
||||
|
||||
finally:
|
||||
await self.cleanup_test_transcripts(test_data)
|
||||
|
||||
|
||||
class TestDateValidationEdgeCases:
|
||||
"""Edge case tests for datetime validation."""
|
||||
|
||||
def test_timezone_aware_comparison(self):
|
||||
"""Test that timezone-aware comparisons work correctly."""
|
||||
# PST time (UTC-8)
|
||||
pst = timezone(timedelta(hours=-8))
|
||||
pst_dt = datetime(2024, 6, 15, 8, 0, 0, tzinfo=pst)
|
||||
|
||||
# UTC time equivalent (8AM PST = 4PM UTC)
|
||||
utc_dt = datetime(2024, 6, 15, 16, 0, 0, tzinfo=timezone.utc)
|
||||
|
||||
assert pst_dt == utc_dt
|
||||
|
||||
def test_mixed_timezone_input(self):
|
||||
"""Test handling mixed timezone inputs."""
|
||||
pst = timezone(timedelta(hours=-8))
|
||||
ist = timezone(timedelta(hours=5, minutes=30))
|
||||
|
||||
from_date = datetime(2024, 6, 15, 0, 0, 0, tzinfo=pst) # PST midnight
|
||||
to_date = datetime(2024, 6, 15, 23, 59, 59, tzinfo=ist) # IST end of day
|
||||
|
||||
assert from_date.tzinfo is not None
|
||||
assert to_date.tzinfo is not None
|
||||
assert from_date < to_date
|
||||
321
server/tests/test_storage.py
Normal file
321
server/tests/test_storage.py
Normal file
@@ -0,0 +1,321 @@
|
||||
"""Tests for storage abstraction layer."""
|
||||
|
||||
import io
|
||||
from unittest.mock import AsyncMock, MagicMock, patch
|
||||
|
||||
import pytest
|
||||
from botocore.exceptions import ClientError
|
||||
|
||||
from reflector.storage.base import StoragePermissionError
|
||||
from reflector.storage.storage_aws import AwsStorage
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_aws_storage_stream_to_fileobj():
|
||||
"""Test that AWS storage can stream directly to a file object without loading into memory."""
|
||||
# Setup
|
||||
storage = AwsStorage(
|
||||
aws_bucket_name="test-bucket",
|
||||
aws_region="us-east-1",
|
||||
aws_access_key_id="test-key",
|
||||
aws_secret_access_key="test-secret",
|
||||
)
|
||||
|
||||
# Mock download_fileobj to write data
|
||||
async def mock_download(Bucket, Key, Fileobj, **kwargs):
|
||||
Fileobj.write(b"chunk1chunk2")
|
||||
|
||||
mock_client = AsyncMock()
|
||||
mock_client.download_fileobj = AsyncMock(side_effect=mock_download)
|
||||
mock_client.__aenter__ = AsyncMock(return_value=mock_client)
|
||||
mock_client.__aexit__ = AsyncMock(return_value=None)
|
||||
|
||||
# Patch the session client
|
||||
with patch.object(storage.session, "client", return_value=mock_client):
|
||||
# Create a file-like object to stream to
|
||||
output = io.BytesIO()
|
||||
|
||||
# Act - stream to file object
|
||||
await storage.stream_to_fileobj("test-file.mp4", output, bucket="test-bucket")
|
||||
|
||||
# Assert
|
||||
mock_client.download_fileobj.assert_called_once_with(
|
||||
Bucket="test-bucket", Key="test-file.mp4", Fileobj=output
|
||||
)
|
||||
|
||||
# Check that data was written to output
|
||||
output.seek(0)
|
||||
assert output.read() == b"chunk1chunk2"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_aws_storage_stream_to_fileobj_with_folder():
|
||||
"""Test streaming with folder prefix in bucket name."""
|
||||
storage = AwsStorage(
|
||||
aws_bucket_name="test-bucket/recordings",
|
||||
aws_region="us-east-1",
|
||||
aws_access_key_id="test-key",
|
||||
aws_secret_access_key="test-secret",
|
||||
)
|
||||
|
||||
async def mock_download(Bucket, Key, Fileobj, **kwargs):
|
||||
Fileobj.write(b"data")
|
||||
|
||||
mock_client = AsyncMock()
|
||||
mock_client.download_fileobj = AsyncMock(side_effect=mock_download)
|
||||
mock_client.__aenter__ = AsyncMock(return_value=mock_client)
|
||||
mock_client.__aexit__ = AsyncMock(return_value=None)
|
||||
|
||||
with patch.object(storage.session, "client", return_value=mock_client):
|
||||
output = io.BytesIO()
|
||||
await storage.stream_to_fileobj("file.mp4", output, bucket="other-bucket")
|
||||
|
||||
# Should use folder prefix from instance config
|
||||
mock_client.download_fileobj.assert_called_once_with(
|
||||
Bucket="other-bucket", Key="recordings/file.mp4", Fileobj=output
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_storage_base_class_stream_to_fileobj():
|
||||
"""Test that base Storage class has stream_to_fileobj method."""
|
||||
from reflector.storage.base import Storage
|
||||
|
||||
# Verify method exists in base class
|
||||
assert hasattr(Storage, "stream_to_fileobj")
|
||||
|
||||
# Create a mock storage instance
|
||||
storage = MagicMock(spec=Storage)
|
||||
storage.stream_to_fileobj = AsyncMock()
|
||||
|
||||
# Should be callable
|
||||
await storage.stream_to_fileobj("file.mp4", io.BytesIO())
|
||||
storage.stream_to_fileobj.assert_called_once()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_aws_storage_stream_uses_download_fileobj():
|
||||
"""Test that download_fileobj is called correctly."""
|
||||
storage = AwsStorage(
|
||||
aws_bucket_name="test-bucket",
|
||||
aws_region="us-east-1",
|
||||
aws_access_key_id="test-key",
|
||||
aws_secret_access_key="test-secret",
|
||||
)
|
||||
|
||||
async def mock_download(Bucket, Key, Fileobj, **kwargs):
|
||||
Fileobj.write(b"data")
|
||||
|
||||
mock_client = AsyncMock()
|
||||
mock_client.download_fileobj = AsyncMock(side_effect=mock_download)
|
||||
mock_client.__aenter__ = AsyncMock(return_value=mock_client)
|
||||
mock_client.__aexit__ = AsyncMock(return_value=None)
|
||||
|
||||
with patch.object(storage.session, "client", return_value=mock_client):
|
||||
output = io.BytesIO()
|
||||
await storage.stream_to_fileobj("test.mp4", output)
|
||||
|
||||
# Verify download_fileobj was called with correct parameters
|
||||
mock_client.download_fileobj.assert_called_once_with(
|
||||
Bucket="test-bucket", Key="test.mp4", Fileobj=output
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_aws_storage_handles_access_denied_error():
|
||||
"""Test that AccessDenied errors are caught and wrapped in StoragePermissionError."""
|
||||
storage = AwsStorage(
|
||||
aws_bucket_name="test-bucket",
|
||||
aws_region="us-east-1",
|
||||
aws_access_key_id="test-key",
|
||||
aws_secret_access_key="test-secret",
|
||||
)
|
||||
|
||||
# Mock ClientError with AccessDenied
|
||||
error_response = {"Error": {"Code": "AccessDenied", "Message": "Access Denied"}}
|
||||
mock_client = AsyncMock()
|
||||
mock_client.put_object = AsyncMock(
|
||||
side_effect=ClientError(error_response, "PutObject")
|
||||
)
|
||||
mock_client.__aenter__ = AsyncMock(return_value=mock_client)
|
||||
mock_client.__aexit__ = AsyncMock(return_value=None)
|
||||
|
||||
with patch.object(storage.session, "client", return_value=mock_client):
|
||||
with pytest.raises(StoragePermissionError) as exc_info:
|
||||
await storage.put_file("test.txt", b"data")
|
||||
|
||||
# Verify error message contains expected information
|
||||
error_msg = str(exc_info.value)
|
||||
assert "AccessDenied" in error_msg
|
||||
assert "default bucket 'test-bucket'" in error_msg
|
||||
assert "S3 upload failed" in error_msg
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_aws_storage_handles_no_such_bucket_error():
|
||||
"""Test that NoSuchBucket errors are caught and wrapped in StoragePermissionError."""
|
||||
storage = AwsStorage(
|
||||
aws_bucket_name="test-bucket",
|
||||
aws_region="us-east-1",
|
||||
aws_access_key_id="test-key",
|
||||
aws_secret_access_key="test-secret",
|
||||
)
|
||||
|
||||
# Mock ClientError with NoSuchBucket
|
||||
error_response = {
|
||||
"Error": {
|
||||
"Code": "NoSuchBucket",
|
||||
"Message": "The specified bucket does not exist",
|
||||
}
|
||||
}
|
||||
mock_client = AsyncMock()
|
||||
mock_client.delete_object = AsyncMock(
|
||||
side_effect=ClientError(error_response, "DeleteObject")
|
||||
)
|
||||
mock_client.__aenter__ = AsyncMock(return_value=mock_client)
|
||||
mock_client.__aexit__ = AsyncMock(return_value=None)
|
||||
|
||||
with patch.object(storage.session, "client", return_value=mock_client):
|
||||
with pytest.raises(StoragePermissionError) as exc_info:
|
||||
await storage.delete_file("test.txt")
|
||||
|
||||
# Verify error message contains expected information
|
||||
error_msg = str(exc_info.value)
|
||||
assert "NoSuchBucket" in error_msg
|
||||
assert "default bucket 'test-bucket'" in error_msg
|
||||
assert "S3 delete failed" in error_msg
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_aws_storage_error_message_with_bucket_override():
|
||||
"""Test that error messages correctly show overridden bucket."""
|
||||
storage = AwsStorage(
|
||||
aws_bucket_name="default-bucket",
|
||||
aws_region="us-east-1",
|
||||
aws_access_key_id="test-key",
|
||||
aws_secret_access_key="test-secret",
|
||||
)
|
||||
|
||||
# Mock ClientError with AccessDenied
|
||||
error_response = {"Error": {"Code": "AccessDenied", "Message": "Access Denied"}}
|
||||
mock_client = AsyncMock()
|
||||
mock_client.get_object = AsyncMock(
|
||||
side_effect=ClientError(error_response, "GetObject")
|
||||
)
|
||||
mock_client.__aenter__ = AsyncMock(return_value=mock_client)
|
||||
mock_client.__aexit__ = AsyncMock(return_value=None)
|
||||
|
||||
with patch.object(storage.session, "client", return_value=mock_client):
|
||||
with pytest.raises(StoragePermissionError) as exc_info:
|
||||
await storage.get_file("test.txt", bucket="override-bucket")
|
||||
|
||||
# Verify error message shows overridden bucket, not default
|
||||
error_msg = str(exc_info.value)
|
||||
assert "overridden bucket 'override-bucket'" in error_msg
|
||||
assert "default-bucket" not in error_msg
|
||||
assert "S3 download failed" in error_msg
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_aws_storage_reraises_non_handled_errors():
|
||||
"""Test that non-AccessDenied/NoSuchBucket errors are re-raised as-is."""
|
||||
storage = AwsStorage(
|
||||
aws_bucket_name="test-bucket",
|
||||
aws_region="us-east-1",
|
||||
aws_access_key_id="test-key",
|
||||
aws_secret_access_key="test-secret",
|
||||
)
|
||||
|
||||
# Mock ClientError with different error code
|
||||
error_response = {
|
||||
"Error": {"Code": "InternalError", "Message": "Internal Server Error"}
|
||||
}
|
||||
mock_client = AsyncMock()
|
||||
mock_client.put_object = AsyncMock(
|
||||
side_effect=ClientError(error_response, "PutObject")
|
||||
)
|
||||
mock_client.__aenter__ = AsyncMock(return_value=mock_client)
|
||||
mock_client.__aexit__ = AsyncMock(return_value=None)
|
||||
|
||||
with patch.object(storage.session, "client", return_value=mock_client):
|
||||
# Should raise ClientError, not StoragePermissionError
|
||||
with pytest.raises(ClientError) as exc_info:
|
||||
await storage.put_file("test.txt", b"data")
|
||||
|
||||
# Verify it's the original ClientError
|
||||
assert exc_info.value.response["Error"]["Code"] == "InternalError"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_aws_storage_presign_url_handles_errors():
|
||||
"""Test that presigned URL generation handles permission errors."""
|
||||
storage = AwsStorage(
|
||||
aws_bucket_name="test-bucket",
|
||||
aws_region="us-east-1",
|
||||
aws_access_key_id="test-key",
|
||||
aws_secret_access_key="test-secret",
|
||||
)
|
||||
|
||||
# Mock ClientError with AccessDenied during presign operation
|
||||
error_response = {"Error": {"Code": "AccessDenied", "Message": "Access Denied"}}
|
||||
mock_client = AsyncMock()
|
||||
mock_client.generate_presigned_url = AsyncMock(
|
||||
side_effect=ClientError(error_response, "GeneratePresignedUrl")
|
||||
)
|
||||
mock_client.__aenter__ = AsyncMock(return_value=mock_client)
|
||||
mock_client.__aexit__ = AsyncMock(return_value=None)
|
||||
|
||||
with patch.object(storage.session, "client", return_value=mock_client):
|
||||
with pytest.raises(StoragePermissionError) as exc_info:
|
||||
await storage.get_file_url("test.txt")
|
||||
|
||||
# Verify error message
|
||||
error_msg = str(exc_info.value)
|
||||
assert "S3 presign failed" in error_msg
|
||||
assert "AccessDenied" in error_msg
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_aws_storage_list_objects_handles_errors():
|
||||
"""Test that list_objects handles permission errors."""
|
||||
storage = AwsStorage(
|
||||
aws_bucket_name="test-bucket",
|
||||
aws_region="us-east-1",
|
||||
aws_access_key_id="test-key",
|
||||
aws_secret_access_key="test-secret",
|
||||
)
|
||||
|
||||
# Mock ClientError during list operation
|
||||
error_response = {"Error": {"Code": "AccessDenied", "Message": "Access Denied"}}
|
||||
mock_paginator = MagicMock()
|
||||
|
||||
async def mock_paginate(*args, **kwargs):
|
||||
raise ClientError(error_response, "ListObjectsV2")
|
||||
yield # Make it an async generator
|
||||
|
||||
mock_paginator.paginate = mock_paginate
|
||||
|
||||
mock_client = AsyncMock()
|
||||
mock_client.get_paginator = MagicMock(return_value=mock_paginator)
|
||||
mock_client.__aenter__ = AsyncMock(return_value=mock_client)
|
||||
mock_client.__aexit__ = AsyncMock(return_value=None)
|
||||
|
||||
with patch.object(storage.session, "client", return_value=mock_client):
|
||||
with pytest.raises(StoragePermissionError) as exc_info:
|
||||
await storage.list_objects(prefix="test/")
|
||||
|
||||
error_msg = str(exc_info.value)
|
||||
assert "S3 list_objects failed" in error_msg
|
||||
assert "AccessDenied" in error_msg
|
||||
|
||||
|
||||
def test_aws_storage_constructor_rejects_mixed_auth():
|
||||
"""Test that constructor rejects both role_arn and access keys."""
|
||||
with pytest.raises(ValueError, match="cannot use both.*role_arn.*access keys"):
|
||||
AwsStorage(
|
||||
aws_bucket_name="test-bucket",
|
||||
aws_region="us-east-1",
|
||||
aws_access_key_id="test-key",
|
||||
aws_secret_access_key="test-secret",
|
||||
aws_role_arn="arn:aws:iam::123456789012:role/test-role",
|
||||
)
|
||||
@@ -22,13 +22,16 @@ async def test_recording_deleted_with_transcript():
|
||||
recording_id=recording.id,
|
||||
)
|
||||
|
||||
with patch("reflector.db.transcripts.get_recordings_storage") as mock_get_storage:
|
||||
with patch("reflector.db.transcripts.get_transcripts_storage") as mock_get_storage:
|
||||
storage_instance = mock_get_storage.return_value
|
||||
storage_instance.delete_file = AsyncMock()
|
||||
|
||||
await transcripts_controller.remove_by_id(transcript.id)
|
||||
|
||||
storage_instance.delete_file.assert_awaited_once_with(recording.object_key)
|
||||
# Should be called with bucket override
|
||||
storage_instance.delete_file.assert_awaited_once_with(
|
||||
recording.object_key, bucket=recording.bucket_name
|
||||
)
|
||||
|
||||
assert await recordings_controller.get_by_id(recording.id) is None
|
||||
assert await transcripts_controller.get_by_id(transcript.id) is None
|
||||
|
||||
70
server/tests/test_user_api_keys.py
Normal file
70
server/tests/test_user_api_keys.py
Normal file
@@ -0,0 +1,70 @@
|
||||
import pytest
|
||||
|
||||
from reflector.db.user_api_keys import user_api_keys_controller
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_api_key_creation_and_verification():
|
||||
api_key_model, plaintext = await user_api_keys_controller.create_key(
|
||||
user_id="test_user",
|
||||
name="Test API Key",
|
||||
)
|
||||
|
||||
verified = await user_api_keys_controller.verify_key(plaintext)
|
||||
assert verified is not None
|
||||
assert verified.user_id == "test_user"
|
||||
assert verified.name == "Test API Key"
|
||||
|
||||
invalid = await user_api_keys_controller.verify_key("fake_key")
|
||||
assert invalid is None
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_api_key_hashing():
|
||||
_, plaintext = await user_api_keys_controller.create_key(
|
||||
user_id="test_user_2",
|
||||
)
|
||||
|
||||
api_keys = await user_api_keys_controller.list_by_user_id("test_user_2")
|
||||
assert len(api_keys) == 1
|
||||
assert api_keys[0].key_hash != plaintext
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_generate_api_key_uniqueness():
|
||||
key1 = user_api_keys_controller.generate_key()
|
||||
key2 = user_api_keys_controller.generate_key()
|
||||
assert key1 != key2
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_hash_api_key_deterministic():
|
||||
key = "test_key_123"
|
||||
hash1 = user_api_keys_controller.hash_key(key)
|
||||
hash2 = user_api_keys_controller.hash_key(key)
|
||||
assert hash1 == hash2
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_by_user_id_empty():
|
||||
api_keys = await user_api_keys_controller.list_by_user_id("nonexistent_user")
|
||||
assert api_keys == []
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_by_user_id_multiple():
|
||||
user_id = "multi_key_user"
|
||||
|
||||
_, plaintext1 = await user_api_keys_controller.create_key(
|
||||
user_id=user_id,
|
||||
name="API Key 1",
|
||||
)
|
||||
_, plaintext2 = await user_api_keys_controller.create_key(
|
||||
user_id=user_id,
|
||||
name="API Key 2",
|
||||
)
|
||||
|
||||
api_keys = await user_api_keys_controller.list_by_user_id(user_id)
|
||||
assert len(api_keys) == 2
|
||||
names = {k.name for k in api_keys}
|
||||
assert names == {"API Key 1", "API Key 2"}
|
||||
17
server/tests/test_utils_daily.py
Normal file
17
server/tests/test_utils_daily.py
Normal file
@@ -0,0 +1,17 @@
|
||||
import pytest
|
||||
|
||||
from reflector.utils.daily import extract_base_room_name
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"daily_room_name,expected",
|
||||
[
|
||||
("daily-20251020193458", "daily"),
|
||||
("daily-2-20251020193458", "daily-2"),
|
||||
("my-room-name-20251020193458", "my-room-name"),
|
||||
("room-with-numbers-123-20251020193458", "room-with-numbers-123"),
|
||||
("x-20251020193458", "x"),
|
||||
],
|
||||
)
|
||||
def test_extract_base_room_name(daily_room_name, expected):
|
||||
assert extract_base_room_name(daily_room_name) == expected
|
||||
63
server/tests/test_utils_url.py
Normal file
63
server/tests/test_utils_url.py
Normal file
@@ -0,0 +1,63 @@
|
||||
"""Tests for URL utility functions."""
|
||||
|
||||
from reflector.utils.url import add_query_param
|
||||
|
||||
|
||||
class TestAddQueryParam:
|
||||
"""Test the add_query_param function."""
|
||||
|
||||
def test_add_param_to_url_without_query(self):
|
||||
"""Should add query param with ? to URL without existing params."""
|
||||
url = "https://example.com/room"
|
||||
result = add_query_param(url, "t", "token123")
|
||||
assert result == "https://example.com/room?t=token123"
|
||||
|
||||
def test_add_param_to_url_with_existing_query(self):
|
||||
"""Should add query param with & to URL with existing params."""
|
||||
url = "https://example.com/room?existing=param"
|
||||
result = add_query_param(url, "t", "token123")
|
||||
assert result == "https://example.com/room?existing=param&t=token123"
|
||||
|
||||
def test_add_param_to_url_with_multiple_existing_params(self):
|
||||
"""Should add query param to URL with multiple existing params."""
|
||||
url = "https://example.com/room?param1=value1¶m2=value2"
|
||||
result = add_query_param(url, "t", "token123")
|
||||
assert (
|
||||
result == "https://example.com/room?param1=value1¶m2=value2&t=token123"
|
||||
)
|
||||
|
||||
def test_add_param_with_special_characters(self):
|
||||
"""Should properly encode special characters in param value."""
|
||||
url = "https://example.com/room"
|
||||
result = add_query_param(url, "name", "hello world")
|
||||
assert result == "https://example.com/room?name=hello+world"
|
||||
|
||||
def test_add_param_to_url_with_fragment(self):
|
||||
"""Should preserve URL fragment when adding query param."""
|
||||
url = "https://example.com/room#section"
|
||||
result = add_query_param(url, "t", "token123")
|
||||
assert result == "https://example.com/room?t=token123#section"
|
||||
|
||||
def test_add_param_to_url_with_query_and_fragment(self):
|
||||
"""Should preserve fragment when adding param to URL with existing query."""
|
||||
url = "https://example.com/room?existing=param#section"
|
||||
result = add_query_param(url, "t", "token123")
|
||||
assert result == "https://example.com/room?existing=param&t=token123#section"
|
||||
|
||||
def test_add_param_overwrites_existing_param(self):
|
||||
"""Should overwrite existing param with same name."""
|
||||
url = "https://example.com/room?t=oldtoken"
|
||||
result = add_query_param(url, "t", "newtoken")
|
||||
assert result == "https://example.com/room?t=newtoken"
|
||||
|
||||
def test_url_without_scheme(self):
|
||||
"""Should handle URLs without scheme (relative URLs)."""
|
||||
url = "/room/path"
|
||||
result = add_query_param(url, "t", "token123")
|
||||
assert result == "/room/path?t=token123"
|
||||
|
||||
def test_empty_url(self):
|
||||
"""Should handle empty URL."""
|
||||
url = ""
|
||||
result = add_query_param(url, "t", "token123")
|
||||
assert result == "?t=token123"
|
||||
58
server/tests/test_video_platforms_factory.py
Normal file
58
server/tests/test_video_platforms_factory.py
Normal file
@@ -0,0 +1,58 @@
|
||||
"""Tests for video_platforms.factory module."""
|
||||
|
||||
from unittest.mock import patch
|
||||
|
||||
from reflector.video_platforms.factory import get_platform
|
||||
|
||||
|
||||
class TestGetPlatformF:
|
||||
"""Test suite for get_platform function."""
|
||||
|
||||
@patch("reflector.video_platforms.factory.settings")
|
||||
def test_with_room_platform(self, mock_settings):
|
||||
"""When room_platform provided, should return room_platform."""
|
||||
mock_settings.DEFAULT_VIDEO_PLATFORM = "whereby"
|
||||
|
||||
# Should return the room's platform when provided
|
||||
assert get_platform(room_platform="daily") == "daily"
|
||||
assert get_platform(room_platform="whereby") == "whereby"
|
||||
|
||||
@patch("reflector.video_platforms.factory.settings")
|
||||
def test_without_room_platform_uses_default(self, mock_settings):
|
||||
"""When no room_platform, should return DEFAULT_VIDEO_PLATFORM."""
|
||||
mock_settings.DEFAULT_VIDEO_PLATFORM = "whereby"
|
||||
|
||||
# Should return default when room_platform is None
|
||||
assert get_platform(room_platform=None) == "whereby"
|
||||
|
||||
@patch("reflector.video_platforms.factory.settings")
|
||||
def test_with_daily_default(self, mock_settings):
|
||||
"""When DEFAULT_VIDEO_PLATFORM is 'daily', should return 'daily' when no room_platform."""
|
||||
mock_settings.DEFAULT_VIDEO_PLATFORM = "daily"
|
||||
|
||||
# Should return default 'daily' when room_platform is None
|
||||
assert get_platform(room_platform=None) == "daily"
|
||||
|
||||
@patch("reflector.video_platforms.factory.settings")
|
||||
def test_no_room_id_provided(self, mock_settings):
|
||||
"""Should work correctly even when room_id is not provided."""
|
||||
mock_settings.DEFAULT_VIDEO_PLATFORM = "whereby"
|
||||
|
||||
# Should use room_platform when provided
|
||||
assert get_platform(room_platform="daily") == "daily"
|
||||
|
||||
# Should use default when room_platform not provided
|
||||
assert get_platform(room_platform=None) == "whereby"
|
||||
|
||||
@patch("reflector.video_platforms.factory.settings")
|
||||
def test_room_platform_always_takes_precedence(self, mock_settings):
|
||||
"""room_platform should always be used when provided."""
|
||||
mock_settings.DEFAULT_VIDEO_PLATFORM = "whereby"
|
||||
|
||||
# room_platform should take precedence over default
|
||||
assert get_platform(room_platform="daily") == "daily"
|
||||
assert get_platform(room_platform="whereby") == "whereby"
|
||||
|
||||
# Different default shouldn't matter when room_platform provided
|
||||
mock_settings.DEFAULT_VIDEO_PLATFORM = "daily"
|
||||
assert get_platform(room_platform="whereby") == "whereby"
|
||||
@@ -1,23 +0,0 @@
|
||||
#!/usr/bin/env python
|
||||
"""
|
||||
Trigger reprocessing of Daily.co multitrack recording via Celery
|
||||
"""
|
||||
|
||||
from reflector.pipelines.main_multitrack_pipeline import (
|
||||
task_pipeline_multitrack_process,
|
||||
)
|
||||
|
||||
# Trigger the Celery task
|
||||
result = task_pipeline_multitrack_process.delay(
|
||||
transcript_id="32fad706-f8cf-434c-94c8-1ee69f7be081", # The ID that was created
|
||||
bucket_name="reflector-dailyco-local",
|
||||
track_keys=[
|
||||
"monadical/daily-20251020193458/1760988935484-52f7f48b-fbab-431f-9a50-87b9abfc8255-cam-audio-1760988935922",
|
||||
"monadical/daily-20251020193458/1760988935484-a37c35e3-6f8e-4274-a482-e9d0f102a732-cam-audio-1760988943823",
|
||||
],
|
||||
)
|
||||
|
||||
print(f"Task ID: {result.id}")
|
||||
print(
|
||||
f"Processing started! Check: http://localhost:3000/transcripts/32fad706-f8cf-434c-94c8-1ee69f7be081"
|
||||
)
|
||||
@@ -1,5 +1,5 @@
|
||||
"use client";
|
||||
import React, { useState, useEffect } from "react";
|
||||
import React, { useState, useEffect, useMemo } from "react";
|
||||
import {
|
||||
Flex,
|
||||
Spinner,
|
||||
@@ -235,15 +235,26 @@ export default function TranscriptBrowser() {
|
||||
|
||||
const pageSize = 20;
|
||||
|
||||
// must be json-able
|
||||
const searchFilters = useMemo(
|
||||
() => ({
|
||||
q: urlSearchQuery,
|
||||
extras: {
|
||||
room_id: urlRoomId || undefined,
|
||||
source_kind: urlSourceKind || undefined,
|
||||
},
|
||||
}),
|
||||
[urlSearchQuery, urlRoomId, urlSourceKind],
|
||||
);
|
||||
|
||||
const {
|
||||
data: searchData,
|
||||
isLoading: searchLoading,
|
||||
refetch: reloadSearch,
|
||||
} = useTranscriptsSearch(urlSearchQuery, {
|
||||
} = useTranscriptsSearch(searchFilters.q, {
|
||||
limit: pageSize,
|
||||
offset: paginationPageTo0Based(page) * pageSize,
|
||||
room_id: urlRoomId || undefined,
|
||||
source_kind: urlSourceKind || undefined,
|
||||
...searchFilters.extras,
|
||||
});
|
||||
|
||||
const results = searchData?.results || [];
|
||||
@@ -255,6 +266,12 @@ export default function TranscriptBrowser() {
|
||||
|
||||
const totalPages = getTotalPages(totalResults, pageSize);
|
||||
|
||||
// reset pagination when search results change (detected by total change; good enough approximation)
|
||||
useEffect(() => {
|
||||
// operation is idempotent
|
||||
setPage(FIRST_PAGE).then(() => {});
|
||||
}, [JSON.stringify(searchFilters)]);
|
||||
|
||||
const userName = useUserName();
|
||||
const [deletionLoading, setDeletionLoading] = useState(false);
|
||||
const cancelRef = React.useRef(null);
|
||||
|
||||
@@ -78,6 +78,14 @@ export default async function AppLayout({
|
||||
)}
|
||||
{featureEnabled("requireLogin") ? (
|
||||
<>
|
||||
·
|
||||
<Link
|
||||
href="/settings/api-keys"
|
||||
as={NextLink}
|
||||
className="font-light px-2"
|
||||
>
|
||||
Settings
|
||||
</Link>
|
||||
·
|
||||
<UserInfo />
|
||||
</>
|
||||
|
||||
341
www/app/(app)/settings/api-keys/page.tsx
Normal file
341
www/app/(app)/settings/api-keys/page.tsx
Normal file
@@ -0,0 +1,341 @@
|
||||
"use client";
|
||||
import React, { useState, useRef } from "react";
|
||||
import {
|
||||
Box,
|
||||
Button,
|
||||
Heading,
|
||||
Stack,
|
||||
Text,
|
||||
Input,
|
||||
Table,
|
||||
Flex,
|
||||
IconButton,
|
||||
Code,
|
||||
Dialog,
|
||||
} from "@chakra-ui/react";
|
||||
import { LuTrash2, LuCopy, LuPlus } from "react-icons/lu";
|
||||
import { useQueryClient } from "@tanstack/react-query";
|
||||
import { $api } from "../../../lib/apiClient";
|
||||
import { toaster } from "../../../components/ui/toaster";
|
||||
|
||||
interface CreateApiKeyResponse {
|
||||
id: string;
|
||||
user_id: string;
|
||||
name: string | null;
|
||||
created_at: string;
|
||||
key: string;
|
||||
}
|
||||
|
||||
export default function ApiKeysPage() {
|
||||
const [newKeyName, setNewKeyName] = useState("");
|
||||
const [isCreating, setIsCreating] = useState(false);
|
||||
const [createdKey, setCreatedKey] = useState<CreateApiKeyResponse | null>(
|
||||
null,
|
||||
);
|
||||
const [keyToDelete, setKeyToDelete] = useState<string | null>(null);
|
||||
const queryClient = useQueryClient();
|
||||
const cancelRef = useRef<HTMLButtonElement>(null);
|
||||
|
||||
const { data: apiKeys, isLoading } = $api.useQuery(
|
||||
"get",
|
||||
"/v1/user/api-keys",
|
||||
);
|
||||
|
||||
const createKeyMutation = $api.useMutation("post", "/v1/user/api-keys", {
|
||||
onSuccess: (data) => {
|
||||
setCreatedKey(data);
|
||||
setNewKeyName("");
|
||||
setIsCreating(false);
|
||||
queryClient.invalidateQueries({ queryKey: ["get", "/v1/user/api-keys"] });
|
||||
toaster.create({
|
||||
duration: 5000,
|
||||
render: () => (
|
||||
<Box bg="green.500" color="white" px={4} py={3} borderRadius="md">
|
||||
<Text fontWeight="bold">API key created</Text>
|
||||
<Text fontSize="sm">
|
||||
Make sure to copy it now - you won't see it again!
|
||||
</Text>
|
||||
</Box>
|
||||
),
|
||||
});
|
||||
|
||||
setTimeout(() => {
|
||||
const keyElement = document.querySelector(".api-key-code");
|
||||
if (keyElement) {
|
||||
const range = document.createRange();
|
||||
range.selectNodeContents(keyElement);
|
||||
const selection = window.getSelection();
|
||||
selection?.removeAllRanges();
|
||||
selection?.addRange(range);
|
||||
}
|
||||
}, 100);
|
||||
},
|
||||
onError: () => {
|
||||
toaster.create({
|
||||
duration: 3000,
|
||||
render: () => (
|
||||
<Box bg="red.500" color="white" px={4} py={3} borderRadius="md">
|
||||
<Text fontWeight="bold">Error</Text>
|
||||
<Text fontSize="sm">Failed to create API key</Text>
|
||||
</Box>
|
||||
),
|
||||
});
|
||||
},
|
||||
});
|
||||
|
||||
const deleteKeyMutation = $api.useMutation(
|
||||
"delete",
|
||||
"/v1/user/api-keys/{key_id}",
|
||||
{
|
||||
onSuccess: () => {
|
||||
queryClient.invalidateQueries({
|
||||
queryKey: ["get", "/v1/user/api-keys"],
|
||||
});
|
||||
toaster.create({
|
||||
duration: 3000,
|
||||
render: () => (
|
||||
<Box bg="green.500" color="white" px={4} py={3} borderRadius="md">
|
||||
<Text fontWeight="bold">API key deleted</Text>
|
||||
</Box>
|
||||
),
|
||||
});
|
||||
},
|
||||
onError: () => {
|
||||
toaster.create({
|
||||
duration: 3000,
|
||||
render: () => (
|
||||
<Box bg="red.500" color="white" px={4} py={3} borderRadius="md">
|
||||
<Text fontWeight="bold">Error</Text>
|
||||
<Text fontSize="sm">Failed to delete API key</Text>
|
||||
</Box>
|
||||
),
|
||||
});
|
||||
},
|
||||
},
|
||||
);
|
||||
|
||||
const handleCreateKey = () => {
|
||||
createKeyMutation.mutate({
|
||||
body: { name: newKeyName || null },
|
||||
});
|
||||
};
|
||||
|
||||
const handleCopyKey = (key: string) => {
|
||||
navigator.clipboard.writeText(key);
|
||||
toaster.create({
|
||||
duration: 2000,
|
||||
render: () => (
|
||||
<Box bg="green.500" color="white" px={4} py={3} borderRadius="md">
|
||||
<Text fontWeight="bold">Copied to clipboard</Text>
|
||||
</Box>
|
||||
),
|
||||
});
|
||||
};
|
||||
|
||||
const handleDeleteRequest = (keyId: string) => {
|
||||
setKeyToDelete(keyId);
|
||||
};
|
||||
|
||||
const confirmDelete = () => {
|
||||
if (keyToDelete) {
|
||||
deleteKeyMutation.mutate({
|
||||
params: { path: { key_id: keyToDelete } },
|
||||
});
|
||||
setKeyToDelete(null);
|
||||
}
|
||||
};
|
||||
|
||||
const formatDate = (dateString: string) => {
|
||||
return new Date(dateString).toLocaleDateString("en-US", {
|
||||
year: "numeric",
|
||||
month: "short",
|
||||
day: "numeric",
|
||||
hour: "2-digit",
|
||||
minute: "2-digit",
|
||||
});
|
||||
};
|
||||
|
||||
return (
|
||||
<Box maxW="800px" w="100%" mx="auto" p={8}>
|
||||
<Heading mb={2}>API Keys</Heading>
|
||||
<Text color="gray.600" mb={6}>
|
||||
Manage your API keys for programmatic access to Reflector
|
||||
</Text>
|
||||
|
||||
{/* Show newly created key */}
|
||||
{createdKey && (
|
||||
<Box
|
||||
mb={6}
|
||||
p={4}
|
||||
bg="green.50"
|
||||
borderWidth={1}
|
||||
borderColor="green.200"
|
||||
borderRadius="md"
|
||||
>
|
||||
<Flex justify="space-between" align="start" mb={2}>
|
||||
<Heading size="sm" color="green.800">
|
||||
API Key Created
|
||||
</Heading>
|
||||
<Button
|
||||
size="sm"
|
||||
variant="ghost"
|
||||
onClick={() => setCreatedKey(null)}
|
||||
>
|
||||
×
|
||||
</Button>
|
||||
</Flex>
|
||||
<Text mb={2} fontSize="sm" color="green.700">
|
||||
Make sure to copy your API key now. You won't be able to see it
|
||||
again!
|
||||
</Text>
|
||||
<Flex gap={2} align="center">
|
||||
<Code
|
||||
p={2}
|
||||
flex={1}
|
||||
fontSize="sm"
|
||||
bg="white"
|
||||
className="api-key-code"
|
||||
>
|
||||
{createdKey.key}
|
||||
</Code>
|
||||
<IconButton
|
||||
aria-label="Copy API key"
|
||||
size="sm"
|
||||
onClick={() => handleCopyKey(createdKey.key)}
|
||||
>
|
||||
<LuCopy />
|
||||
</IconButton>
|
||||
</Flex>
|
||||
</Box>
|
||||
)}
|
||||
|
||||
{/* Create new key */}
|
||||
<Box mb={8} p={6} borderWidth={1} borderRadius="md">
|
||||
<Heading size="md" mb={4}>
|
||||
Create New API Key
|
||||
</Heading>
|
||||
{!isCreating ? (
|
||||
<Button onClick={() => setIsCreating(true)} colorPalette="blue">
|
||||
<LuPlus /> Create API Key
|
||||
</Button>
|
||||
) : (
|
||||
<Stack gap={4}>
|
||||
<Box>
|
||||
<Text mb={2}>Name (optional)</Text>
|
||||
<Input
|
||||
placeholder="e.g., Production API Key"
|
||||
value={newKeyName}
|
||||
onChange={(e) => setNewKeyName(e.target.value)}
|
||||
/>
|
||||
</Box>
|
||||
<Flex gap={2}>
|
||||
<Button
|
||||
onClick={handleCreateKey}
|
||||
colorPalette="blue"
|
||||
loading={createKeyMutation.isPending}
|
||||
>
|
||||
Create
|
||||
</Button>
|
||||
<Button
|
||||
onClick={() => {
|
||||
setIsCreating(false);
|
||||
setNewKeyName("");
|
||||
}}
|
||||
variant="outline"
|
||||
>
|
||||
Cancel
|
||||
</Button>
|
||||
</Flex>
|
||||
</Stack>
|
||||
)}
|
||||
</Box>
|
||||
|
||||
{/* List of API keys */}
|
||||
<Box>
|
||||
<Heading size="md" mb={4}>
|
||||
Your API Keys
|
||||
</Heading>
|
||||
{isLoading ? (
|
||||
<Text>Loading...</Text>
|
||||
) : !apiKeys || apiKeys.length === 0 ? (
|
||||
<Text color="gray.600">
|
||||
No API keys yet. Create one to get started.
|
||||
</Text>
|
||||
) : (
|
||||
<Table.Root>
|
||||
<Table.Header>
|
||||
<Table.Row>
|
||||
<Table.ColumnHeader>Name</Table.ColumnHeader>
|
||||
<Table.ColumnHeader>Created</Table.ColumnHeader>
|
||||
<Table.ColumnHeader>Actions</Table.ColumnHeader>
|
||||
</Table.Row>
|
||||
</Table.Header>
|
||||
<Table.Body>
|
||||
{apiKeys.map((key) => (
|
||||
<Table.Row key={key.id}>
|
||||
<Table.Cell>
|
||||
{key.name || <Text color="gray.500">Unnamed</Text>}
|
||||
</Table.Cell>
|
||||
<Table.Cell>{formatDate(key.created_at)}</Table.Cell>
|
||||
<Table.Cell>
|
||||
<IconButton
|
||||
aria-label="Delete API key"
|
||||
size="sm"
|
||||
colorPalette="red"
|
||||
variant="ghost"
|
||||
onClick={() => handleDeleteRequest(key.id)}
|
||||
loading={
|
||||
deleteKeyMutation.isPending &&
|
||||
deleteKeyMutation.variables?.params?.path?.key_id ===
|
||||
key.id
|
||||
}
|
||||
>
|
||||
<LuTrash2 />
|
||||
</IconButton>
|
||||
</Table.Cell>
|
||||
</Table.Row>
|
||||
))}
|
||||
</Table.Body>
|
||||
</Table.Root>
|
||||
)}
|
||||
</Box>
|
||||
|
||||
{/* Delete confirmation dialog */}
|
||||
<Dialog.Root
|
||||
open={!!keyToDelete}
|
||||
onOpenChange={(e) => {
|
||||
if (!e.open) setKeyToDelete(null);
|
||||
}}
|
||||
initialFocusEl={() => cancelRef.current}
|
||||
>
|
||||
<Dialog.Backdrop />
|
||||
<Dialog.Positioner>
|
||||
<Dialog.Content>
|
||||
<Dialog.Header fontSize="lg" fontWeight="bold">
|
||||
Delete API Key
|
||||
</Dialog.Header>
|
||||
<Dialog.Body>
|
||||
<Text>
|
||||
Are you sure you want to delete this API key? This action cannot
|
||||
be undone.
|
||||
</Text>
|
||||
</Dialog.Body>
|
||||
<Dialog.Footer>
|
||||
<Button
|
||||
ref={cancelRef}
|
||||
onClick={() => setKeyToDelete(null)}
|
||||
variant="outline"
|
||||
colorPalette="gray"
|
||||
>
|
||||
Cancel
|
||||
</Button>
|
||||
<Button colorPalette="red" onClick={confirmDelete} ml={3}>
|
||||
Delete
|
||||
</Button>
|
||||
</Dialog.Footer>
|
||||
</Dialog.Content>
|
||||
</Dialog.Positioner>
|
||||
</Dialog.Root>
|
||||
</Box>
|
||||
);
|
||||
}
|
||||
@@ -1,4 +1,4 @@
|
||||
import { useEffect, useRef, useState } from "react";
|
||||
import { useEffect, useState } from "react";
|
||||
import React from "react";
|
||||
import Markdown from "react-markdown";
|
||||
import "../../../styles/markdown.css";
|
||||
@@ -16,17 +16,15 @@ import {
|
||||
} from "@chakra-ui/react";
|
||||
import { LuPen } from "react-icons/lu";
|
||||
import { useError } from "../../../(errors)/errorContext";
|
||||
import ShareAndPrivacy from "../shareAndPrivacy";
|
||||
|
||||
type FinalSummaryProps = {
|
||||
transcriptResponse: GetTranscript;
|
||||
topicsResponse: GetTranscriptTopic[];
|
||||
onUpdate?: (newSummary) => void;
|
||||
transcript: GetTranscript;
|
||||
topics: GetTranscriptTopic[];
|
||||
onUpdate: (newSummary: string) => void;
|
||||
finalSummaryRef: React.Dispatch<React.SetStateAction<HTMLDivElement | null>>;
|
||||
};
|
||||
|
||||
export default function FinalSummary(props: FinalSummaryProps) {
|
||||
const finalSummaryRef = useRef<HTMLParagraphElement>(null);
|
||||
|
||||
const [isEditMode, setIsEditMode] = useState(false);
|
||||
const [preEditSummary, setPreEditSummary] = useState("");
|
||||
const [editedSummary, setEditedSummary] = useState("");
|
||||
@@ -35,10 +33,10 @@ export default function FinalSummary(props: FinalSummaryProps) {
|
||||
const updateTranscriptMutation = useTranscriptUpdate();
|
||||
|
||||
useEffect(() => {
|
||||
setEditedSummary(props.transcriptResponse?.long_summary || "");
|
||||
}, [props.transcriptResponse?.long_summary]);
|
||||
setEditedSummary(props.transcript?.long_summary || "");
|
||||
}, [props.transcript?.long_summary]);
|
||||
|
||||
if (!props.topicsResponse || !props.transcriptResponse) {
|
||||
if (!props.topics || !props.transcript) {
|
||||
return null;
|
||||
}
|
||||
|
||||
@@ -54,9 +52,7 @@ export default function FinalSummary(props: FinalSummaryProps) {
|
||||
long_summary: newSummary,
|
||||
},
|
||||
});
|
||||
if (props.onUpdate) {
|
||||
props.onUpdate(newSummary);
|
||||
}
|
||||
props.onUpdate(newSummary);
|
||||
console.log("Updated long summary:", updatedTranscript);
|
||||
} catch (err) {
|
||||
console.error("Failed to update long summary:", err);
|
||||
@@ -75,7 +71,7 @@ export default function FinalSummary(props: FinalSummaryProps) {
|
||||
};
|
||||
|
||||
const onSaveClick = () => {
|
||||
updateSummary(editedSummary, props.transcriptResponse.id);
|
||||
updateSummary(editedSummary, props.transcript.id);
|
||||
setIsEditMode(false);
|
||||
};
|
||||
|
||||
@@ -133,11 +129,6 @@ export default function FinalSummary(props: FinalSummaryProps) {
|
||||
>
|
||||
<LuPen />
|
||||
</IconButton>
|
||||
<ShareAndPrivacy
|
||||
finalSummaryRef={finalSummaryRef}
|
||||
transcriptResponse={props.transcriptResponse}
|
||||
topicsResponse={props.topicsResponse}
|
||||
/>
|
||||
</>
|
||||
)}
|
||||
</Flex>
|
||||
@@ -153,7 +144,7 @@ export default function FinalSummary(props: FinalSummaryProps) {
|
||||
mt={2}
|
||||
/>
|
||||
) : (
|
||||
<div ref={finalSummaryRef} className="markdown">
|
||||
<div ref={props.finalSummaryRef} className="markdown">
|
||||
<Markdown>{editedSummary}</Markdown>
|
||||
</div>
|
||||
)}
|
||||
|
||||
@@ -41,6 +41,8 @@ export default function TranscriptDetails(details: TranscriptDetails) {
|
||||
waiting || mp3.audioDeleted === true,
|
||||
);
|
||||
const useActiveTopic = useState<Topic | null>(null);
|
||||
const [finalSummaryElement, setFinalSummaryElement] =
|
||||
useState<HTMLDivElement | null>(null);
|
||||
|
||||
useEffect(() => {
|
||||
if (waiting) {
|
||||
@@ -124,9 +126,12 @@ export default function TranscriptDetails(details: TranscriptDetails) {
|
||||
<TranscriptTitle
|
||||
title={transcript.data?.title || "Unnamed Transcript"}
|
||||
transcriptId={transcriptId}
|
||||
onUpdate={(newTitle) => {
|
||||
onUpdate={() => {
|
||||
transcript.refetch().then(() => {});
|
||||
}}
|
||||
transcript={transcript.data || null}
|
||||
topics={topics.topics}
|
||||
finalSummaryElement={finalSummaryElement}
|
||||
/>
|
||||
</Flex>
|
||||
{mp3.audioDeleted && (
|
||||
@@ -148,11 +153,12 @@ export default function TranscriptDetails(details: TranscriptDetails) {
|
||||
{transcript.data && topics.topics ? (
|
||||
<>
|
||||
<FinalSummary
|
||||
transcriptResponse={transcript.data}
|
||||
topicsResponse={topics.topics}
|
||||
transcript={transcript.data}
|
||||
topics={topics.topics}
|
||||
onUpdate={() => {
|
||||
transcript.refetch();
|
||||
transcript.refetch().then(() => {});
|
||||
}}
|
||||
finalSummaryRef={setFinalSummaryElement}
|
||||
/>
|
||||
</>
|
||||
) : (
|
||||
|
||||
@@ -26,9 +26,9 @@ import { useAuth } from "../../lib/AuthProvider";
|
||||
import { featureEnabled } from "../../lib/features";
|
||||
|
||||
type ShareAndPrivacyProps = {
|
||||
finalSummaryRef: any;
|
||||
transcriptResponse: GetTranscript;
|
||||
topicsResponse: GetTranscriptTopic[];
|
||||
finalSummaryElement: HTMLDivElement | null;
|
||||
transcript: GetTranscript;
|
||||
topics: GetTranscriptTopic[];
|
||||
};
|
||||
|
||||
type ShareOption = { value: ShareMode; label: string };
|
||||
@@ -48,7 +48,7 @@ export default function ShareAndPrivacy(props: ShareAndPrivacyProps) {
|
||||
const [isOwner, setIsOwner] = useState(false);
|
||||
const [shareMode, setShareMode] = useState<ShareOption>(
|
||||
shareOptionsData.find(
|
||||
(option) => option.value === props.transcriptResponse.share_mode,
|
||||
(option) => option.value === props.transcript.share_mode,
|
||||
) || shareOptionsData[0],
|
||||
);
|
||||
const [shareLoading, setShareLoading] = useState(false);
|
||||
@@ -70,7 +70,7 @@ export default function ShareAndPrivacy(props: ShareAndPrivacyProps) {
|
||||
try {
|
||||
const updatedTranscript = await updateTranscriptMutation.mutateAsync({
|
||||
params: {
|
||||
path: { transcript_id: props.transcriptResponse.id },
|
||||
path: { transcript_id: props.transcript.id },
|
||||
},
|
||||
body: requestBody,
|
||||
});
|
||||
@@ -90,8 +90,8 @@ export default function ShareAndPrivacy(props: ShareAndPrivacyProps) {
|
||||
const userId = auth.status === "authenticated" ? auth.user?.id : null;
|
||||
|
||||
useEffect(() => {
|
||||
setIsOwner(!!(requireLogin && userId === props.transcriptResponse.user_id));
|
||||
}, [userId, props.transcriptResponse.user_id]);
|
||||
setIsOwner(!!(requireLogin && userId === props.transcript.user_id));
|
||||
}, [userId, props.transcript.user_id]);
|
||||
|
||||
return (
|
||||
<>
|
||||
@@ -171,19 +171,19 @@ export default function ShareAndPrivacy(props: ShareAndPrivacyProps) {
|
||||
<Flex gap={2} mb={2}>
|
||||
{requireLogin && (
|
||||
<ShareZulip
|
||||
transcriptResponse={props.transcriptResponse}
|
||||
topicsResponse={props.topicsResponse}
|
||||
transcript={props.transcript}
|
||||
topics={props.topics}
|
||||
disabled={toShareMode(shareMode.value) === "private"}
|
||||
/>
|
||||
)}
|
||||
<ShareCopy
|
||||
finalSummaryRef={props.finalSummaryRef}
|
||||
transcriptResponse={props.transcriptResponse}
|
||||
topicsResponse={props.topicsResponse}
|
||||
finalSummaryElement={props.finalSummaryElement}
|
||||
transcript={props.transcript}
|
||||
topics={props.topics}
|
||||
/>
|
||||
</Flex>
|
||||
|
||||
<ShareLink transcriptId={props.transcriptResponse.id} />
|
||||
<ShareLink transcriptId={props.transcript.id} />
|
||||
</Dialog.Body>
|
||||
</Dialog.Content>
|
||||
</Dialog.Positioner>
|
||||
|
||||
@@ -5,34 +5,35 @@ type GetTranscriptTopic = components["schemas"]["GetTranscriptTopic"];
|
||||
import { Button, BoxProps, Box } from "@chakra-ui/react";
|
||||
|
||||
type ShareCopyProps = {
|
||||
finalSummaryRef: any;
|
||||
transcriptResponse: GetTranscript;
|
||||
topicsResponse: GetTranscriptTopic[];
|
||||
finalSummaryElement: HTMLDivElement | null;
|
||||
transcript: GetTranscript;
|
||||
topics: GetTranscriptTopic[];
|
||||
};
|
||||
|
||||
export default function ShareCopy({
|
||||
finalSummaryRef,
|
||||
transcriptResponse,
|
||||
topicsResponse,
|
||||
finalSummaryElement,
|
||||
transcript,
|
||||
topics,
|
||||
...boxProps
|
||||
}: ShareCopyProps & BoxProps) {
|
||||
const [isCopiedSummary, setIsCopiedSummary] = useState(false);
|
||||
const [isCopiedTranscript, setIsCopiedTranscript] = useState(false);
|
||||
|
||||
const onCopySummaryClick = () => {
|
||||
let text_to_copy = finalSummaryRef.current?.innerText;
|
||||
const text_to_copy = finalSummaryElement?.innerText;
|
||||
|
||||
text_to_copy &&
|
||||
if (text_to_copy) {
|
||||
navigator.clipboard.writeText(text_to_copy).then(() => {
|
||||
setIsCopiedSummary(true);
|
||||
// Reset the copied state after 2 seconds
|
||||
setTimeout(() => setIsCopiedSummary(false), 2000);
|
||||
});
|
||||
}
|
||||
};
|
||||
|
||||
const onCopyTranscriptClick = () => {
|
||||
let text_to_copy =
|
||||
topicsResponse
|
||||
topics
|
||||
?.map((topic) => topic.transcript)
|
||||
.join("\n\n")
|
||||
.replace(/ +/g, " ")
|
||||
|
||||
@@ -26,8 +26,8 @@ import {
|
||||
import { featureEnabled } from "../../lib/features";
|
||||
|
||||
type ShareZulipProps = {
|
||||
transcriptResponse: GetTranscript;
|
||||
topicsResponse: GetTranscriptTopic[];
|
||||
transcript: GetTranscript;
|
||||
topics: GetTranscriptTopic[];
|
||||
disabled: boolean;
|
||||
};
|
||||
|
||||
@@ -88,14 +88,14 @@ export default function ShareZulip(props: ShareZulipProps & BoxProps) {
|
||||
}, [stream, streams]);
|
||||
|
||||
const handleSendToZulip = async () => {
|
||||
if (!props.transcriptResponse) return;
|
||||
if (!props.transcript) return;
|
||||
|
||||
if (stream && topic) {
|
||||
try {
|
||||
await postToZulipMutation.mutateAsync({
|
||||
params: {
|
||||
path: {
|
||||
transcript_id: props.transcriptResponse.id,
|
||||
transcript_id: props.transcript.id,
|
||||
},
|
||||
query: {
|
||||
stream,
|
||||
|
||||
@@ -2,14 +2,22 @@ import { useState } from "react";
|
||||
import type { components } from "../../reflector-api";
|
||||
|
||||
type UpdateTranscript = components["schemas"]["UpdateTranscript"];
|
||||
type GetTranscript = components["schemas"]["GetTranscript"];
|
||||
type GetTranscriptTopic = components["schemas"]["GetTranscriptTopic"];
|
||||
import { useTranscriptUpdate } from "../../lib/apiHooks";
|
||||
import { Heading, IconButton, Input, Flex, Spacer } from "@chakra-ui/react";
|
||||
import { LuPen } from "react-icons/lu";
|
||||
import ShareAndPrivacy from "./shareAndPrivacy";
|
||||
|
||||
type TranscriptTitle = {
|
||||
title: string;
|
||||
transcriptId: string;
|
||||
onUpdate?: (newTitle: string) => void;
|
||||
onUpdate: (newTitle: string) => void;
|
||||
|
||||
// share props
|
||||
transcript: GetTranscript | null;
|
||||
topics: GetTranscriptTopic[] | null;
|
||||
finalSummaryElement: HTMLDivElement | null;
|
||||
};
|
||||
|
||||
const TranscriptTitle = (props: TranscriptTitle) => {
|
||||
@@ -29,9 +37,7 @@ const TranscriptTitle = (props: TranscriptTitle) => {
|
||||
},
|
||||
body: requestBody,
|
||||
});
|
||||
if (props.onUpdate) {
|
||||
props.onUpdate(newTitle);
|
||||
}
|
||||
props.onUpdate(newTitle);
|
||||
console.log("Updated transcript title:", newTitle);
|
||||
} catch (err) {
|
||||
console.error("Failed to update transcript:", err);
|
||||
@@ -62,11 +68,11 @@ const TranscriptTitle = (props: TranscriptTitle) => {
|
||||
}
|
||||
setIsEditing(false);
|
||||
};
|
||||
const handleChange = (e) => {
|
||||
const handleChange = (e: React.ChangeEvent<HTMLInputElement>) => {
|
||||
setDisplayedTitle(e.target.value);
|
||||
};
|
||||
|
||||
const handleKeyDown = (e) => {
|
||||
const handleKeyDown = (e: React.KeyboardEvent<HTMLInputElement>) => {
|
||||
if (e.key === "Enter") {
|
||||
updateTitle(displayedTitle, props.transcriptId);
|
||||
setIsEditing(false);
|
||||
@@ -111,6 +117,13 @@ const TranscriptTitle = (props: TranscriptTitle) => {
|
||||
>
|
||||
<LuPen />
|
||||
</IconButton>
|
||||
{props.transcript && props.topics && (
|
||||
<ShareAndPrivacy
|
||||
finalSummaryElement={props.finalSummaryElement}
|
||||
transcript={props.transcript}
|
||||
topics={props.topics}
|
||||
/>
|
||||
)}
|
||||
</Flex>
|
||||
)}
|
||||
</>
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
import Room from "../room";
|
||||
import RoomContainer from "../components/RoomContainer";
|
||||
|
||||
export default Room;
|
||||
export default RoomContainer;
|
||||
|
||||
@@ -1,166 +1,26 @@
|
||||
"use client";
|
||||
|
||||
import { useCallback, useEffect, useRef, useState } from "react";
|
||||
import { Box, Button, Text, VStack, HStack, Icon } from "@chakra-ui/react";
|
||||
import { toaster } from "../../components/ui/toaster";
|
||||
import { useCallback, useEffect, useRef } from "react";
|
||||
import { Box } from "@chakra-ui/react";
|
||||
import { useRouter } from "next/navigation";
|
||||
import { useRecordingConsent } from "../../recordingConsentContext";
|
||||
import { useMeetingAudioConsent } from "../../lib/apiHooks";
|
||||
import { FaBars } from "react-icons/fa6";
|
||||
import DailyIframe, { DailyCall } from "@daily-co/daily-js";
|
||||
import type { components } from "../../reflector-api";
|
||||
import { useAuth } from "../../lib/AuthProvider";
|
||||
import {
|
||||
ConsentDialogButton,
|
||||
recordingTypeRequiresConsent,
|
||||
} from "../../lib/consent";
|
||||
|
||||
type Meeting = components["schemas"]["Meeting"];
|
||||
|
||||
const CONSENT_BUTTON_TOP_OFFSET = "56px";
|
||||
const TOAST_CHECK_INTERVAL_MS = 100;
|
||||
|
||||
interface DailyRoomProps {
|
||||
meeting: Meeting;
|
||||
}
|
||||
|
||||
function ConsentDialogButton({ meetingId }: { meetingId: string }) {
|
||||
const { state: consentState, touch, hasConsent } = useRecordingConsent();
|
||||
const [modalOpen, setModalOpen] = useState(false);
|
||||
const audioConsentMutation = useMeetingAudioConsent();
|
||||
|
||||
const handleConsent = useCallback(
|
||||
async (meetingId: string, given: boolean) => {
|
||||
try {
|
||||
await audioConsentMutation.mutateAsync({
|
||||
params: {
|
||||
path: {
|
||||
meeting_id: meetingId,
|
||||
},
|
||||
},
|
||||
body: {
|
||||
consent_given: given,
|
||||
},
|
||||
});
|
||||
|
||||
touch(meetingId);
|
||||
} catch (error) {
|
||||
console.error("Error submitting consent:", error);
|
||||
}
|
||||
},
|
||||
[audioConsentMutation, touch],
|
||||
);
|
||||
|
||||
const showConsentModal = useCallback(() => {
|
||||
if (modalOpen) return;
|
||||
|
||||
setModalOpen(true);
|
||||
|
||||
const toastId = toaster.create({
|
||||
placement: "top",
|
||||
duration: null,
|
||||
render: ({ dismiss }) => (
|
||||
<Box
|
||||
p={6}
|
||||
bg="rgba(255, 255, 255, 0.7)"
|
||||
borderRadius="lg"
|
||||
boxShadow="lg"
|
||||
maxW="md"
|
||||
mx="auto"
|
||||
>
|
||||
<VStack gap={4} alignItems="center">
|
||||
<Text fontSize="md" textAlign="center" fontWeight="medium">
|
||||
Can we have your permission to store this meeting's audio
|
||||
recording on our servers?
|
||||
</Text>
|
||||
<HStack gap={4} justifyContent="center">
|
||||
<Button
|
||||
variant="ghost"
|
||||
size="sm"
|
||||
onClick={() => {
|
||||
handleConsent(meetingId, false).then(() => {
|
||||
/*signifies it's ok to now wait here.*/
|
||||
});
|
||||
dismiss();
|
||||
}}
|
||||
>
|
||||
No, delete after transcription
|
||||
</Button>
|
||||
<Button
|
||||
colorPalette="primary"
|
||||
size="sm"
|
||||
onClick={() => {
|
||||
handleConsent(meetingId, true).then(() => {
|
||||
/*signifies it's ok to now wait here.*/
|
||||
});
|
||||
dismiss();
|
||||
}}
|
||||
>
|
||||
Yes, store the audio
|
||||
</Button>
|
||||
</HStack>
|
||||
</VStack>
|
||||
</Box>
|
||||
),
|
||||
});
|
||||
|
||||
toastId.then((id) => {
|
||||
const checkToastStatus = setInterval(() => {
|
||||
if (!toaster.isActive(id)) {
|
||||
setModalOpen(false);
|
||||
clearInterval(checkToastStatus);
|
||||
}
|
||||
}, TOAST_CHECK_INTERVAL_MS);
|
||||
});
|
||||
|
||||
const handleKeyDown = (event: KeyboardEvent) => {
|
||||
if (event.key === "Escape") {
|
||||
toastId.then((id) => toaster.dismiss(id));
|
||||
}
|
||||
};
|
||||
|
||||
document.addEventListener("keydown", handleKeyDown);
|
||||
|
||||
const cleanup = () => {
|
||||
toastId.then((id) => toaster.dismiss(id));
|
||||
document.removeEventListener("keydown", handleKeyDown);
|
||||
};
|
||||
|
||||
return cleanup;
|
||||
}, [meetingId, handleConsent, modalOpen]);
|
||||
|
||||
if (
|
||||
!consentState.ready ||
|
||||
hasConsent(meetingId) ||
|
||||
audioConsentMutation.isPending
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
|
||||
return (
|
||||
<Button
|
||||
position="absolute"
|
||||
top={CONSENT_BUTTON_TOP_OFFSET}
|
||||
left="8px"
|
||||
zIndex={1000}
|
||||
colorPalette="blue"
|
||||
size="sm"
|
||||
onClick={showConsentModal}
|
||||
>
|
||||
Meeting is being recorded
|
||||
<Icon as={FaBars} ml={2} />
|
||||
</Button>
|
||||
);
|
||||
}
|
||||
|
||||
const recordingTypeRequiresConsent = (
|
||||
recordingType: Meeting["recording_type"],
|
||||
) => {
|
||||
return recordingType === "cloud";
|
||||
};
|
||||
|
||||
export default function DailyRoom({ meeting }: DailyRoomProps) {
|
||||
const router = useRouter();
|
||||
const auth = useAuth();
|
||||
const status = auth.status;
|
||||
const isAuthenticated = status === "authenticated";
|
||||
const [callFrame, setCallFrame] = useState<DailyCall | null>(null);
|
||||
const containerRef = useRef<HTMLDivElement>(null);
|
||||
|
||||
const roomUrl = meeting?.host_room_url || meeting?.room_url;
|
||||
@@ -201,10 +61,6 @@ export default function DailyRoom({ meeting }: DailyRoomProps) {
|
||||
|
||||
frame.on("left-meeting", handleLeave);
|
||||
await frame.join({ url: roomUrl });
|
||||
|
||||
if (!destroyed) {
|
||||
setCallFrame(frame);
|
||||
}
|
||||
} catch (error) {
|
||||
console.error("Error creating Daily frame:", error);
|
||||
}
|
||||
@@ -229,9 +85,9 @@ export default function DailyRoom({ meeting }: DailyRoomProps) {
|
||||
return (
|
||||
<Box position="relative" width="100vw" height="100vh">
|
||||
<div ref={containerRef} style={{ width: "100%", height: "100%" }} />
|
||||
{recordingTypeRequiresConsent(meeting.recording_type) && (
|
||||
<ConsentDialogButton meetingId={meeting.id} />
|
||||
)}
|
||||
{meeting.recording_type &&
|
||||
recordingTypeRequiresConsent(meeting.recording_type) &&
|
||||
meeting.id && <ConsentDialogButton meetingId={meeting.id} />}
|
||||
</Box>
|
||||
);
|
||||
}
|
||||
|
||||
@@ -1,16 +1,16 @@
|
||||
"use client";
|
||||
|
||||
import { useCallback, useEffect, useRef, useState, RefObject } from "react";
|
||||
import { Box, Button, Text, VStack, HStack, Icon } from "@chakra-ui/react";
|
||||
import { toaster } from "../../components/ui/toaster";
|
||||
import { useCallback, useEffect, useRef, RefObject } from "react";
|
||||
import { useRouter } from "next/navigation";
|
||||
import { useRecordingConsent } from "../../recordingConsentContext";
|
||||
import { useMeetingAudioConsent } from "../../lib/apiHooks";
|
||||
import type { components } from "../../reflector-api";
|
||||
import { FaBars } from "react-icons/fa6";
|
||||
import { useAuth } from "../../lib/AuthProvider";
|
||||
import { getWherebyUrl, useWhereby } from "../../lib/wherebyClient";
|
||||
import { assertExistsAndNonEmptyString, NonEmptyString } from "../../lib/utils";
|
||||
import {
|
||||
ConsentDialogButton as BaseConsentDialogButton,
|
||||
useConsentDialog,
|
||||
recordingTypeRequiresConsent,
|
||||
} from "../../lib/consent";
|
||||
|
||||
type Meeting = components["schemas"]["Meeting"];
|
||||
|
||||
@@ -18,206 +18,36 @@ interface WherebyRoomProps {
|
||||
meeting: Meeting;
|
||||
}
|
||||
|
||||
const useConsentWherebyFocusManagement = (
|
||||
acceptButtonRef: RefObject<HTMLButtonElement>,
|
||||
wherebyRef: RefObject<HTMLElement>,
|
||||
) => {
|
||||
const currentFocusRef = useRef<HTMLElement | null>(null);
|
||||
useEffect(() => {
|
||||
if (acceptButtonRef.current) {
|
||||
acceptButtonRef.current.focus();
|
||||
} else {
|
||||
console.error(
|
||||
"accept button ref not available yet for focus management - seems to be illegal state",
|
||||
);
|
||||
}
|
||||
|
||||
const handleWherebyReady = () => {
|
||||
console.log("whereby ready - refocusing consent button");
|
||||
currentFocusRef.current = document.activeElement as HTMLElement;
|
||||
if (acceptButtonRef.current) {
|
||||
acceptButtonRef.current.focus();
|
||||
}
|
||||
};
|
||||
|
||||
if (wherebyRef.current) {
|
||||
wherebyRef.current.addEventListener("ready", handleWherebyReady);
|
||||
} else {
|
||||
console.warn(
|
||||
"whereby ref not available yet for focus management - seems to be illegal state. not waiting, focus management off.",
|
||||
);
|
||||
}
|
||||
|
||||
return () => {
|
||||
wherebyRef.current?.removeEventListener("ready", handleWherebyReady);
|
||||
currentFocusRef.current?.focus();
|
||||
};
|
||||
}, []);
|
||||
};
|
||||
|
||||
const useConsentDialog = (
|
||||
meetingId: string,
|
||||
wherebyRef: RefObject<HTMLElement>,
|
||||
) => {
|
||||
const { state: consentState, touch, hasConsent } = useRecordingConsent();
|
||||
const [modalOpen, setModalOpen] = useState(false);
|
||||
const audioConsentMutation = useMeetingAudioConsent();
|
||||
|
||||
const handleConsent = useCallback(
|
||||
async (meetingId: string, given: boolean) => {
|
||||
try {
|
||||
await audioConsentMutation.mutateAsync({
|
||||
params: {
|
||||
path: {
|
||||
meeting_id: meetingId,
|
||||
},
|
||||
},
|
||||
body: {
|
||||
consent_given: given,
|
||||
},
|
||||
});
|
||||
|
||||
touch(meetingId);
|
||||
} catch (error) {
|
||||
console.error("Error submitting consent:", error);
|
||||
}
|
||||
},
|
||||
[audioConsentMutation, touch],
|
||||
);
|
||||
|
||||
const showConsentModal = useCallback(() => {
|
||||
if (modalOpen) return;
|
||||
|
||||
setModalOpen(true);
|
||||
|
||||
const toastId = toaster.create({
|
||||
placement: "top",
|
||||
duration: null,
|
||||
render: ({ dismiss }) => {
|
||||
const AcceptButton = () => {
|
||||
const buttonRef = useRef<HTMLButtonElement>(null);
|
||||
useConsentWherebyFocusManagement(buttonRef, wherebyRef);
|
||||
return (
|
||||
<Button
|
||||
ref={buttonRef}
|
||||
colorPalette="primary"
|
||||
size="sm"
|
||||
onClick={() => {
|
||||
handleConsent(meetingId, true).then(() => {
|
||||
/*signifies it's ok to now wait here.*/
|
||||
});
|
||||
dismiss();
|
||||
}}
|
||||
>
|
||||
Yes, store the audio
|
||||
</Button>
|
||||
);
|
||||
};
|
||||
|
||||
return (
|
||||
<Box
|
||||
p={6}
|
||||
bg="rgba(255, 255, 255, 0.7)"
|
||||
borderRadius="lg"
|
||||
boxShadow="lg"
|
||||
maxW="md"
|
||||
mx="auto"
|
||||
>
|
||||
<VStack gap={4} alignItems="center">
|
||||
<Text fontSize="md" textAlign="center" fontWeight="medium">
|
||||
Can we have your permission to store this meeting's audio
|
||||
recording on our servers?
|
||||
</Text>
|
||||
<HStack gap={4} justifyContent="center">
|
||||
<Button
|
||||
variant="ghost"
|
||||
size="sm"
|
||||
onClick={() => {
|
||||
handleConsent(meetingId, false).then(() => {
|
||||
/*signifies it's ok to now wait here.*/
|
||||
});
|
||||
dismiss();
|
||||
}}
|
||||
>
|
||||
No, delete after transcription
|
||||
</Button>
|
||||
<AcceptButton />
|
||||
</HStack>
|
||||
</VStack>
|
||||
</Box>
|
||||
);
|
||||
},
|
||||
});
|
||||
|
||||
toastId.then((id) => {
|
||||
const checkToastStatus = setInterval(() => {
|
||||
if (!toaster.isActive(id)) {
|
||||
setModalOpen(false);
|
||||
clearInterval(checkToastStatus);
|
||||
}
|
||||
}, 100);
|
||||
});
|
||||
|
||||
const handleKeyDown = (event: KeyboardEvent) => {
|
||||
if (event.key === "Escape") {
|
||||
toastId.then((id) => toaster.dismiss(id));
|
||||
}
|
||||
};
|
||||
|
||||
document.addEventListener("keydown", handleKeyDown);
|
||||
|
||||
const cleanup = () => {
|
||||
toastId.then((id) => toaster.dismiss(id));
|
||||
document.removeEventListener("keydown", handleKeyDown);
|
||||
};
|
||||
|
||||
return cleanup;
|
||||
}, [meetingId, handleConsent, wherebyRef, modalOpen]);
|
||||
|
||||
return {
|
||||
showConsentModal,
|
||||
consentState,
|
||||
hasConsent,
|
||||
consentLoading: audioConsentMutation.isPending,
|
||||
};
|
||||
};
|
||||
|
||||
function ConsentDialogButton({
|
||||
function WherebyConsentDialogButton({
|
||||
meetingId,
|
||||
wherebyRef,
|
||||
}: {
|
||||
meetingId: NonEmptyString;
|
||||
wherebyRef: React.RefObject<HTMLElement>;
|
||||
}) {
|
||||
const { showConsentModal, consentState, hasConsent, consentLoading } =
|
||||
useConsentDialog(meetingId, wherebyRef);
|
||||
const previousFocusRef = useRef<HTMLElement | null>(null);
|
||||
|
||||
if (!consentState.ready || hasConsent(meetingId) || consentLoading) {
|
||||
return null;
|
||||
}
|
||||
useEffect(() => {
|
||||
const element = wherebyRef.current;
|
||||
if (!element) return;
|
||||
|
||||
return (
|
||||
<Button
|
||||
position="absolute"
|
||||
top="56px"
|
||||
left="8px"
|
||||
zIndex={1000}
|
||||
colorPalette="blue"
|
||||
size="sm"
|
||||
onClick={showConsentModal}
|
||||
>
|
||||
Meeting is being recorded
|
||||
<Icon as={FaBars} ml={2} />
|
||||
</Button>
|
||||
);
|
||||
const handleWherebyReady = () => {
|
||||
previousFocusRef.current = document.activeElement as HTMLElement;
|
||||
};
|
||||
|
||||
element.addEventListener("ready", handleWherebyReady);
|
||||
|
||||
return () => {
|
||||
element.removeEventListener("ready", handleWherebyReady);
|
||||
if (previousFocusRef.current && document.activeElement === element) {
|
||||
previousFocusRef.current.focus();
|
||||
}
|
||||
};
|
||||
}, [wherebyRef]);
|
||||
|
||||
return <BaseConsentDialogButton meetingId={meetingId} />;
|
||||
}
|
||||
|
||||
const recordingTypeRequiresConsent = (
|
||||
recordingType: NonNullable<Meeting["recording_type"]>,
|
||||
) => {
|
||||
return recordingType === "cloud";
|
||||
};
|
||||
|
||||
export default function WherebyRoom({ meeting }: WherebyRoomProps) {
|
||||
const wherebyLoaded = useWhereby();
|
||||
const wherebyRef = useRef<HTMLElement>(null);
|
||||
@@ -261,7 +91,7 @@ export default function WherebyRoom({ meeting }: WherebyRoomProps) {
|
||||
{recordingType &&
|
||||
recordingTypeRequiresConsent(recordingType) &&
|
||||
meetingId && (
|
||||
<ConsentDialogButton
|
||||
<WherebyConsentDialogButton
|
||||
meetingId={assertExistsAndNonEmptyString(meetingId)}
|
||||
wherebyRef={wherebyRef}
|
||||
/>
|
||||
|
||||
36
www/app/lib/consent/ConsentDialog.tsx
Normal file
36
www/app/lib/consent/ConsentDialog.tsx
Normal file
@@ -0,0 +1,36 @@
|
||||
"use client";
|
||||
|
||||
import { Box, Button, Text, VStack, HStack } from "@chakra-ui/react";
|
||||
import { CONSENT_DIALOG_TEXT } from "./constants";
|
||||
|
||||
interface ConsentDialogProps {
|
||||
onAccept: () => void;
|
||||
onReject: () => void;
|
||||
}
|
||||
|
||||
export function ConsentDialog({ onAccept, onReject }: ConsentDialogProps) {
|
||||
return (
|
||||
<Box
|
||||
p={6}
|
||||
bg="rgba(255, 255, 255, 0.7)"
|
||||
borderRadius="lg"
|
||||
boxShadow="lg"
|
||||
maxW="md"
|
||||
mx="auto"
|
||||
>
|
||||
<VStack gap={4} alignItems="center">
|
||||
<Text fontSize="md" textAlign="center" fontWeight="medium">
|
||||
{CONSENT_DIALOG_TEXT.question}
|
||||
</Text>
|
||||
<HStack gap={4} justifyContent="center">
|
||||
<Button variant="ghost" size="sm" onClick={onReject}>
|
||||
{CONSENT_DIALOG_TEXT.rejectButton}
|
||||
</Button>
|
||||
<Button colorPalette="primary" size="sm" onClick={onAccept}>
|
||||
{CONSENT_DIALOG_TEXT.acceptButton}
|
||||
</Button>
|
||||
</HStack>
|
||||
</VStack>
|
||||
</Box>
|
||||
);
|
||||
}
|
||||
39
www/app/lib/consent/ConsentDialogButton.tsx
Normal file
39
www/app/lib/consent/ConsentDialogButton.tsx
Normal file
@@ -0,0 +1,39 @@
|
||||
"use client";
|
||||
|
||||
import { Button, Icon } from "@chakra-ui/react";
|
||||
import { FaBars } from "react-icons/fa6";
|
||||
import { useConsentDialog } from "./useConsentDialog";
|
||||
import {
|
||||
CONSENT_BUTTON_TOP_OFFSET,
|
||||
CONSENT_BUTTON_LEFT_OFFSET,
|
||||
CONSENT_BUTTON_Z_INDEX,
|
||||
CONSENT_DIALOG_TEXT,
|
||||
} from "./constants";
|
||||
|
||||
interface ConsentDialogButtonProps {
|
||||
meetingId: string;
|
||||
}
|
||||
|
||||
export function ConsentDialogButton({ meetingId }: ConsentDialogButtonProps) {
|
||||
const { showConsentModal, consentState, hasConsent, consentLoading } =
|
||||
useConsentDialog(meetingId);
|
||||
|
||||
if (!consentState.ready || hasConsent(meetingId) || consentLoading) {
|
||||
return null;
|
||||
}
|
||||
|
||||
return (
|
||||
<Button
|
||||
position="absolute"
|
||||
top={CONSENT_BUTTON_TOP_OFFSET}
|
||||
left={CONSENT_BUTTON_LEFT_OFFSET}
|
||||
zIndex={CONSENT_BUTTON_Z_INDEX}
|
||||
colorPalette="blue"
|
||||
size="sm"
|
||||
onClick={showConsentModal}
|
||||
>
|
||||
{CONSENT_DIALOG_TEXT.triggerButton}
|
||||
<Icon as={FaBars} ml={2} />
|
||||
</Button>
|
||||
);
|
||||
}
|
||||
12
www/app/lib/consent/constants.ts
Normal file
12
www/app/lib/consent/constants.ts
Normal file
@@ -0,0 +1,12 @@
|
||||
export const CONSENT_BUTTON_TOP_OFFSET = "56px";
|
||||
export const CONSENT_BUTTON_LEFT_OFFSET = "8px";
|
||||
export const CONSENT_BUTTON_Z_INDEX = 1000;
|
||||
export const TOAST_CHECK_INTERVAL_MS = 100;
|
||||
|
||||
export const CONSENT_DIALOG_TEXT = {
|
||||
question:
|
||||
"Can we have your permission to store this meeting's audio recording on our servers?",
|
||||
acceptButton: "Yes, store the audio",
|
||||
rejectButton: "No, delete after transcription",
|
||||
triggerButton: "Meeting is being recorded",
|
||||
} as const;
|
||||
8
www/app/lib/consent/index.ts
Normal file
8
www/app/lib/consent/index.ts
Normal file
@@ -0,0 +1,8 @@
|
||||
"use client";
|
||||
|
||||
export { ConsentDialogButton } from "./ConsentDialogButton";
|
||||
export { ConsentDialog } from "./ConsentDialog";
|
||||
export { useConsentDialog } from "./useConsentDialog";
|
||||
export { recordingTypeRequiresConsent } from "./utils";
|
||||
export * from "./constants";
|
||||
export * from "./types";
|
||||
9
www/app/lib/consent/types.ts
Normal file
9
www/app/lib/consent/types.ts
Normal file
@@ -0,0 +1,9 @@
|
||||
export interface ConsentDialogResult {
|
||||
showConsentModal: () => void;
|
||||
consentState: {
|
||||
ready: boolean;
|
||||
consentAnsweredForMeetings?: Set<string>;
|
||||
};
|
||||
hasConsent: (meetingId: string) => boolean;
|
||||
consentLoading: boolean;
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user