chore(main): release 0.12.1 (#658)

* feat: calendar integration

* feat: add ICS calendar API endpoints for room configuration and sync

* feat: add Celery background tasks for ICS sync

* feat: implement Phase 2 - Multiple active meetings per room with grace period

This commit adds support for multiple concurrent meetings per room, implementing
grace period logic and improved meeting lifecycle management for calendar integration.

## Database Changes
- Remove unique constraint preventing multiple active meetings per room
- Add last_participant_left_at field to track when meeting becomes empty
- Add grace_period_minutes field (default: 15) for configurable grace period

## Meeting Controller Enhancements
- Add get_all_active_for_room() to retrieve all active meetings for a room
- Add get_active_by_calendar_event() to find meetings by calendar event ID
- Maintain backward compatibility with existing get_active() method

## New API Endpoints
- GET /rooms/{room_name}/meetings/active - List all active meetings
- POST /rooms/{room_name}/meetings/{meeting_id}/join - Join specific meeting

## Meeting Lifecycle Improvements
- 15-minute grace period after last participant leaves
- Automatic reactivation when participant rejoins during grace period
- Force close calendar meetings 30 minutes after scheduled end time
- Update process_meetings task to handle multiple active meetings

## Whereby Integration
- Clear grace period when participants join via webhook events
- Track participant count for grace period management

## Testing
- Add comprehensive tests for multiple active meetings
- Test grace period behavior and participant rejoin scenarios
- Test calendar meeting force closure logic
- All 5 new tests passing

This enables proper calendar integration with overlapping meetings while
preventing accidental meeting closures through the grace period mechanism.

* feat: implement frontend for calendar integration (Phase 3 & 4)

- Created MeetingSelection component for choosing between multiple active meetings
- Shows both active meetings and upcoming calendar events (30 min ahead)
- Displays meeting metadata with privacy controls (owner-only details)
- Supports creation of unscheduled meetings alongside calendar meetings

- Added waiting page for users joining before scheduled start time
- Shows countdown timer until meeting begins
- Auto-transitions to meeting when calendar event becomes active
- Handles early joining with proper routing

- Created collapsible info panel showing meeting details
- Displays calendar metadata (title, description, attendees)
- Shows participant count and duration
- Privacy-aware: sensitive info only visible to room owners

- Integrated ICS settings into room configuration dialog
- Test connection functionality with immediate feedback
- Manual sync trigger with detailed results
- Shows last sync time and ETag for monitoring
- Configurable sync intervals (1 min to 1 hour)

- New /room/{roomName} route for meeting selection
- Waiting room at /room/{roomName}/wait?eventId={id}
- Classic room page at /{roomName} with meeting info
- Uses sessionStorage to pass selected meeting between pages

- Added new endpoints for active/upcoming meetings
- Regenerated TypeScript client with latest OpenAPI spec
- Proper error handling and loading states
- Auto-refresh every 30 seconds for live updates

- Color-coded badges for meeting status
- Attendee status indicators (accepted/declined/tentative)
- Responsive design with Chakra UI components
- Clear visual hierarchy between active and upcoming meetings
- Smart truncation for long attendee lists

This completes the frontend implementation for calendar integration,
enabling users to seamlessly join scheduled meetings from their
calendar applications.

* WIP: Migrate calendar integration frontend to React Query

- Migrate all calendar components from useApi to React Query hooks
- Fix Chakra UI v3 compatibility issues (Card, Progress, spacing props, leftIcon)
- Update backend Meeting model to include calendar fields
- Replace imperative API calls with declarative React Query patterns
- Remove old OpenAPI generated files that conflict with new structure

* fix: alembic migrations

* feat: add calendar migration

* feat: update ics, first version working

* feat: implement tabbed interface for room edit dialog

- Add General, Calendar, and Share tabs to organize room settings
- Move ICS settings to dedicated Calendar tab
- Move Zulip configuration to Share tab
- Keep basic room settings and webhooks in General tab
- Remove redundant migration file
- Fix Chakra UI v3 compatibility issues in calendar components

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: infinite loop

* feat: improve ICS calendar sync UX and fix room URL matching

- Replace "Test Connection" button with "Force Sync" button (Edit Room only)
- Show detailed sync results: total events downloaded vs room matches
- Remove emoticons and auto-hide timeout for cleaner UX
- Fix room URL matching to use UI_BASE_URL instead of BASE_URL
- Replace FaSync icon with LuRefreshCw for consistency
- Clear sync results when dialog closes or Force Sync pressed
- Update tests to reflect UI_BASE_URL change and exact URL matching

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* feat: reorganize room edit dialog and fix Force Sync button

- Move WebHook configuration from General to dedicated WebHook tab
- Add WebHook tab after Share tab in room edit dialog
- Fix Force Sync button not appearing by adding missing isEditing prop
- Fix indentation issues in MeetingSelection component

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* feat: complete calendar integration with UI improvements and code cleanup

Calendar Integration Tasks:
- Update upcoming meetings window from 30 to 120 minutes
- Include currently happening events in upcoming meetings API
- Create shared time utility functions (formatDateTime, formatCountdown, formatStartedAgo)
- Improve ongoing meetings UI logic with proper time detection
- Fix backend code organization and remove excessive documentation

UI/UX Improvements:
- Restructure room page layout using MinimalHeader pattern
- Remove borders from header and footer elements
- Change button text from "Leave Meeting" to "Leave Room"
- Remove "Back to Reflector" footer for cleaner design
- Extract WaitPageClient component for better separation

Backend Changes:
- calendar_events.py: Fix import organization and extend timing window
- rooms.py: Update API default from 30 to 120 minutes
- Enhanced test coverage for ongoing meeting scenarios

Frontend Changes:
- MinimalHeader: Add onLeave prop for custom navigation
- MeetingSelection: Complete layout restructure with shared utilities
- timeUtils: New shared utility file for consistent time formatting

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* feat: remove wait page and simplify Join button with 5-minute disable logic

- Remove entire wait page directory and associated files
- Update handleJoinUpcoming to create unscheduled meeting directly
- Simplify Join button to single state:
  - Always shows "Join" text
  - Blue when meeting can be joined (ongoing or within 5 minutes)
  - Gray/disabled when more than 5 minutes away
- Remove confusing "Join Now", "Join Early" text variations

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* feat: improve calendar integration and meeting UI

- Refactor ICS sync tasks to use @asynctask decorator for cleaner async handling
- Extract meeting creation logic into reusable function
- Improve meeting selection UI with distinct current/upcoming sections
- Add early join functionality for upcoming meetings within 5-minute window
- Simplify non-ICS room workflow with direct Whereby embed
- Fix import paths and component organization

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* feat: restore original recording consent functionality

- Remove custom ConsentDialogButton and WherebyEmbed components
- Merge RoomClient logic back into main room page
- Restore original consent UI: blue button with toast modal
- Maintain calendar integration features for ICS-enabled rooms
- Add consent-handler.md documentation of original functionality
- Preserve focus management and accessibility features

* fix: redirect Join Now button to local meeting page

- Change handleJoinDirect to use onMeetingSelect instead of opening external URL
- Join Now button now navigates to /{roomName}/{meetingId} instead of whereby.com
- Maintains proper routing within the application

* feat: remove restrictive message for non-owners in private rooms

- Remove confusing message about room owner permissions
- Cleaner UI for all users regardless of ownership status
- Users will only see available meetings and join options

* feat: improve meeting selection UI for better readability

- Limit page content to max 800px width for better 4K display readability
- Remove LIVE tag badge for cleaner interface
- Remove shadow from main live meeting box
- Remove blue border and hover effects for minimal design
- Change background to neutral gray for less visual noise

* feat: add room by name endpoint for non-authenticated access

- Add GET /rooms/name/{room_name} backend endpoint
- Endpoint supports non-authenticated access for public rooms
- Returns RoomDetails with webhook fields hidden for non-owners
- Update useRoomGetByName hook to use new direct endpoint
- Remove authentication requirement from frontend hook
- Regenerate API client types

Fixes: Non-authenticated users can now access room lobbies

* feat: add friendly message when no meetings are ongoing

- Show centered message with calendar icon when no meetings are active
- Message text: 'No meetings right now' with helpful description
- Contextual text for owners/shared rooms mentioning quick meeting option
- Consistent gray styling matching the rest of the interface
- Only displays when both currentMeetings and upcomingMeetings are empty

* style: center no meetings message and remove background

- Change from Box to Flex with flex=1 for vertical centering
- Remove gray background, border radius, and padding
- Message now appears cleanly centered in available space
- Maintains horizontal and vertical centering

* feat: move Create Meeting button to header

- Remove 'Start a Quick Meeting' box from main content area
- Add showCreateButton and onCreateMeeting props to MinimalHeader
- Create Meeting button now appears in header left of Leave Room
- Only shows for room owners or shared room users
- Update no meetings message to remove reference to quick meeting below
- Cleaner, more accessible UI with actions in the header

* style: change room title and no meetings text to pure black

- Update room title in MinimalHeader from gray.700 to black
- Update 'No meetings right now' text from gray.700 to black
- Improves visual hierarchy and readability
- Consistent with other pages' styling

* style: linting

* fix: remove plan files

* fix: alembic migration with named foreign keys

* feat: add SyncStatus enum and refactor ICS sync to use rooms controller

- Add SyncStatus enum to replace string literals in ICS sync status
- Replace direct SQL queries in worker with rooms_controller.get_ics_enabled()
- Improve type safety and maintainability of ICS sync code
- Enum values: SUCCESS, UNCHANGED, ERROR, SKIPPED maintain backward compatibility

* refactor: remove unnecessary docstring from get_ics_enabled method

The function name is self-explanatory

* fix: import top level

* feat: use Literal type for ICSStatus.status field

- Changed ICSStatus.status from str to Literal['enabled', 'disabled']
- Improves type safety and API documentation

* feat: update TypeScript definitions for ICSStatus Literal type

- OpenAPI generation now properly reflects Literal['enabled', 'disabled'] type
- Improves type safety for frontend consumers of the API
- Applied automatic formatting via pre-commit hooks

* refactor: replace loguru with structlog in ics_sync service

- Replace loguru import with structlog in services/ics_sync.py
- Update logging calls to use structlog's structured format with keyword args
- Maintains consistency with other services using structlog
- Changes: logger.info(f'...') -> logger.info('...', key=value) format

* chore: remove loguru dependency and improve type annotations

- Remove loguru from dependencies in pyproject.toml (replaced with structlog)
- Update meeting controller methods to properly return Optional types
- Update dependency lock file after loguru removal

* fix: resolve pyflakes warnings in ics_sync and meetings modules

Remove unused imports and variables to clean up code quality

* Remove grace period logic and improve meeting deactivation

- Removed grace_period_minutes and last_participant_left_at fields
- Simplified deactivation logic based on actual usage patterns:
  * Active sessions: Keep meeting active regardless of scheduled time
  * Calendar meetings: Wait until scheduled end if unused, deactivate immediately once used and empty
  * On-the-fly meetings: Deactivate immediately when empty
- Created migration to drop unused database columns
- Updated tests to remove grace period test cases

* Update test to match new deactivation logic for calendar meetings

* fix: remove unwanted file

* fix: incompleted changes from EVENT_WINDOW*

* fix: update room ICS API tests to include required webhook fields and correct URL

- Add webhook_url and webhook_secret fields to room creation tests
- Fix room URL matching in ICS sync test to use UI_BASE_URL instead of BASE_URL
- Aligns test with actual API requirements and ICS sync service implementation

* fix: add Redis distributed locking to prevent race conditions in process_meetings

- Implement per-meeting locks using Redis to prevent concurrent processing
- Add lock extension after slow API calls (Whereby) to handle long-running operations
- Use redis-py's built-in lock.extend() with replace_ttl=True for simple TTL refresh
- Track and log skipped meetings when locked by other workers
- Document SSRF analysis showing it's low-risk due to async worker isolation

This prevents multiple workers from processing the same meeting simultaneously,
which could cause state corruption or duplicate deactivations.

* refactor: rename MinimalHeader to MeetingMinimalHeader for clarity

* fix: minor code quality improvements - add emoji constants, fix type safety, cleanup comments

* fix: database migration

* self-pr review

* self-pr review

* self-pr review treeshake

* fix: local fixes

* fix: creation of meeting

* fix: meeting selection create button

* compile fix

* fix: meeting selection responsive

* fix: rework process logic for meeting

* fix: meeting useEffect frontend-only dedupe (#647)

* meeting useEffect frontend-only dedupe

* format

* also get room by name backend fix

---------

Co-authored-by: Igor Loskutov <igor.loskutoff@gmail.com>

* invalidate meeting list on new meeting

* test fix

* room url copy button for ics

* calendar refresh quick action icon

* remove work.md

* meeting page frontend fixes

* hide number of meeting participants

* Revert "hide number of meeting participants"

This reverts commit 38906c5d1a.

* ui bits

* ui bits

* remove log

* room name typing stricten

* feat: protect atomic operation involving external service with redlock

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Igor Monadical <igor@monadical.com>
Co-authored-by: Igor Loskutov <igor.loskutoff@gmail.com>

.github/workflows/db_migrations.yml

@@ -2,6 +2,8 @@ name: Test Database Migrations
 
 on:
   push:
+    branches:
+      - main
     paths:
       - "server/migrations/**"
       - "server/reflector/db/**"
@@ -17,6 +19,9 @@ on:
 jobs:
   test-migrations:
     runs-on: ubuntu-latest
+    concurrency:
+      group: db-ubuntu-latest-${{ github.ref }}
+      cancel-in-progress: true
     services:
       postgres:
         image: postgres:17

.github/workflows/test_next_server.yml

@@ -0,0 +1,45 @@
+name: Test Next Server
+
+on:
+  pull_request:
+    paths:
+      - "www/**"
+  push:
+    branches:
+      - main
+    paths:
+      - "www/**"
+
+jobs:
+  test-next-server:
+    runs-on: ubuntu-latest
+
+    defaults:
+      run:
+        working-directory: ./www
+
+    steps:
+    - uses: actions/checkout@v4
+
+    - name: Setup Node.js
+      uses: actions/setup-node@v4
+      with:
+        node-version: '20'
+
+    - name: Install pnpm
+      uses: pnpm/action-setup@v4
+      with:
+        version: 8
+
+    - name: Setup Node.js cache
+      uses: actions/setup-node@v4
+      with:
+        node-version: '20'
+        cache: 'pnpm'
+        cache-dependency-path: './www/pnpm-lock.yaml'
+
+    - name: Install dependencies
+      run: pnpm install
+
+    - name: Run tests
+      run: pnpm test

.github/workflows/test_server.yml

@@ -5,12 +5,17 @@ on:
     paths:
       - "server/**"
   push:
+    branches:
+      - main
     paths:
       - "server/**"
 
 jobs:
   pytest:
     runs-on: ubuntu-latest
+    concurrency:
+      group: pytest-${{ github.ref }}
+      cancel-in-progress: true
     services:
       redis:
         image: redis:6
@@ -30,6 +35,9 @@ jobs:
 
   docker-amd64:
     runs-on: linux-amd64
+    concurrency:
+      group: docker-amd64-${{ github.ref }}
+      cancel-in-progress: true
     steps:
       - uses: actions/checkout@v4
       - name: Set up Docker Buildx
@@ -45,6 +53,9 @@ jobs:
 
   docker-arm64:
     runs-on: linux-arm64
+    concurrency:
+      group: docker-arm64-${{ github.ref }}
+      cancel-in-progress: true
     steps:
       - uses: actions/checkout@v4
       - name: Set up Docker Buildx

.gitignore

@@ -14,4 +14,7 @@ data/
 www/REFACTOR.md
 www/reload-frontend
 server/test.sqlite
-CLAUDE.local.md
+CLAUDE.local.md
+www/.env.development
+www/.env.production
+.playwright-mcp

.gitleaksignore

@@ -0,0 +1 @@
+b9d891d3424f371642cb032ecfd0e2564470a72c:server/tests/test_transcripts_recording_deletion.py:generic-api-key:15

.pre-commit-config.yaml

@@ -27,3 +27,8 @@ repos:
         files: ^server/
       - id: ruff-format
         files: ^server/
+
+  - repo: https://github.com/gitleaks/gitleaks
+    rev: v8.28.0
+    hooks:
+      - id: gitleaks

CHANGELOG.md

@@ -1,5 +1,107 @@
 # Changelog
 
+## [0.12.1](https://github.com/Monadical-SAS/reflector/compare/v0.12.0...v0.12.1) (2025-09-17)
+
+
+### Bug Fixes
+
+* production blocked because having existing meeting with room_id null ([#657](https://github.com/Monadical-SAS/reflector/issues/657)) ([870e860](https://github.com/Monadical-SAS/reflector/commit/870e8605171a27155a9cbee215eeccb9a8d6c0a2))
+
+## [0.12.0](https://github.com/Monadical-SAS/reflector/compare/v0.11.0...v0.12.0) (2025-09-17)
+
+
+### Features
+
+* calendar integration ([#608](https://github.com/Monadical-SAS/reflector/issues/608)) ([6f680b5](https://github.com/Monadical-SAS/reflector/commit/6f680b57954c688882c4ed49f40f161c52a00a24))
+* self-hosted gpu api ([#636](https://github.com/Monadical-SAS/reflector/issues/636)) ([ab859d6](https://github.com/Monadical-SAS/reflector/commit/ab859d65a6bded904133a163a081a651b3938d42))
+
+
+### Bug Fixes
+
+* ignore player hotkeys for text inputs ([#646](https://github.com/Monadical-SAS/reflector/issues/646)) ([fa049e8](https://github.com/Monadical-SAS/reflector/commit/fa049e8d068190ce7ea015fd9fcccb8543f54a3f))
+
+## [0.11.0](https://github.com/Monadical-SAS/reflector/compare/v0.10.0...v0.11.0) (2025-09-16)
+
+
+### Features
+
+* remove profanity filter that was there for conference ([#652](https://github.com/Monadical-SAS/reflector/issues/652)) ([b42f7cf](https://github.com/Monadical-SAS/reflector/commit/b42f7cfc606783afcee792590efcc78b507468ab))
+
+
+### Bug Fixes
+
+* zulip and consent handler on the file pipeline ([#645](https://github.com/Monadical-SAS/reflector/issues/645)) ([5f143fe](https://github.com/Monadical-SAS/reflector/commit/5f143fe3640875dcb56c26694254a93189281d17))
+* zulip stream and topic selection in share dialog ([#644](https://github.com/Monadical-SAS/reflector/issues/644)) ([c546e69](https://github.com/Monadical-SAS/reflector/commit/c546e69739e68bb74fbc877eb62609928e5b8de6))
+
+## [0.10.0](https://github.com/Monadical-SAS/reflector/compare/v0.9.0...v0.10.0) (2025-09-11)
+
+
+### Features
+
+* replace nextjs-config with environment variables ([#632](https://github.com/Monadical-SAS/reflector/issues/632)) ([369ecdf](https://github.com/Monadical-SAS/reflector/commit/369ecdff13f3862d926a9c0b87df52c9d94c4dde))
+
+
+### Bug Fixes
+
+* anonymous users transcript permissions ([#621](https://github.com/Monadical-SAS/reflector/issues/621)) ([f81fe99](https://github.com/Monadical-SAS/reflector/commit/f81fe9948a9237b3e0001b2d8ca84f54d76878f9))
+* auth post ([#624](https://github.com/Monadical-SAS/reflector/issues/624)) ([cde99ca](https://github.com/Monadical-SAS/reflector/commit/cde99ca2716f84ba26798f289047732f0448742e))
+* auth post ([#626](https://github.com/Monadical-SAS/reflector/issues/626)) ([3b85ff3](https://github.com/Monadical-SAS/reflector/commit/3b85ff3bdf4fb053b103070646811bc990c0e70a))
+* auth post ([#627](https://github.com/Monadical-SAS/reflector/issues/627)) ([962038e](https://github.com/Monadical-SAS/reflector/commit/962038ee3f2a555dc3c03856be0e4409456e0996))
+* missing follow_redirects=True on modal endpoint ([#630](https://github.com/Monadical-SAS/reflector/issues/630)) ([fc363bd](https://github.com/Monadical-SAS/reflector/commit/fc363bd49b17b075e64f9186e5e0185abc325ea7))
+* sync backend and frontend token refresh logic ([#614](https://github.com/Monadical-SAS/reflector/issues/614)) ([5a5b323](https://github.com/Monadical-SAS/reflector/commit/5a5b3233820df9536da75e87ce6184a983d4713a))
+
+## [0.9.0](https://github.com/Monadical-SAS/reflector/compare/v0.8.2...v0.9.0) (2025-09-06)
+
+
+### Features
+
+* frontend openapi react query ([#606](https://github.com/Monadical-SAS/reflector/issues/606)) ([c4d2825](https://github.com/Monadical-SAS/reflector/commit/c4d2825c81f81ad8835629fbf6ea8c7383f8c31b))
+
+
+### Bug Fixes
+
+* align whisper transcriber api with parakeet ([#602](https://github.com/Monadical-SAS/reflector/issues/602)) ([0663700](https://github.com/Monadical-SAS/reflector/commit/0663700a615a4af69a03c96c410f049e23ec9443))
+* kv use tls explicit ([#610](https://github.com/Monadical-SAS/reflector/issues/610)) ([08d88ec](https://github.com/Monadical-SAS/reflector/commit/08d88ec349f38b0d13e0fa4cb73486c8dfd31836))
+* source kind for file processing ([#601](https://github.com/Monadical-SAS/reflector/issues/601)) ([dc82f8b](https://github.com/Monadical-SAS/reflector/commit/dc82f8bb3bdf3ab3d4088e592a30fd63907319e1))
+* token refresh locking ([#613](https://github.com/Monadical-SAS/reflector/issues/613)) ([7f5a4c9](https://github.com/Monadical-SAS/reflector/commit/7f5a4c9ddc7fd098860c8bdda2ca3b57f63ded2f))
+
+## [0.8.2](https://github.com/Monadical-SAS/reflector/compare/v0.8.1...v0.8.2) (2025-08-29)
+
+
+### Bug Fixes
+
+* search-logspam ([#593](https://github.com/Monadical-SAS/reflector/issues/593)) ([695d1a9](https://github.com/Monadical-SAS/reflector/commit/695d1a957d4cd862753049f9beed88836cabd5ab))
+
+## [0.8.1](https://github.com/Monadical-SAS/reflector/compare/v0.8.0...v0.8.1) (2025-08-29)
+
+
+### Bug Fixes
+
+* make webhook secret/url allowing null ([#590](https://github.com/Monadical-SAS/reflector/issues/590)) ([84a3812](https://github.com/Monadical-SAS/reflector/commit/84a381220bc606231d08d6f71d4babc818fa3c75))
+
+## [0.8.0](https://github.com/Monadical-SAS/reflector/compare/v0.7.3...v0.8.0) (2025-08-29)
+
+
+### Features
+
+* **cleanup:** add automatic data retention for public instances ([#574](https://github.com/Monadical-SAS/reflector/issues/574)) ([6f0c7c1](https://github.com/Monadical-SAS/reflector/commit/6f0c7c1a5e751713366886c8e764c2009e12ba72))
+* **rooms:** add webhook for transcript completion ([#578](https://github.com/Monadical-SAS/reflector/issues/578)) ([88ed7cf](https://github.com/Monadical-SAS/reflector/commit/88ed7cfa7804794b9b54cad4c3facc8a98cf85fd))
+
+
+### Bug Fixes
+
+* file pipeline status reporting and websocket updates ([#589](https://github.com/Monadical-SAS/reflector/issues/589)) ([9dfd769](https://github.com/Monadical-SAS/reflector/commit/9dfd76996f851cc52be54feea078adbc0816dc57))
+* Igor/evaluation ([#575](https://github.com/Monadical-SAS/reflector/issues/575)) ([124ce03](https://github.com/Monadical-SAS/reflector/commit/124ce03bf86044c18313d27228a25da4bc20c9c5))
+* optimize parakeet transcription batching algorithm ([#577](https://github.com/Monadical-SAS/reflector/issues/577)) ([7030e0f](https://github.com/Monadical-SAS/reflector/commit/7030e0f23649a8cf6c1eb6d5889684a41ce849ec))
+
+## [0.7.3](https://github.com/Monadical-SAS/reflector/compare/v0.7.2...v0.7.3) (2025-08-22)
+
+
+### Bug Fixes
+
+* cleaned repo, and get git-leaks clean ([359280d](https://github.com/Monadical-SAS/reflector/commit/359280dd340433ba4402ed69034094884c825e67))
+* restore previous behavior on live pipeline + audio downscaler ([#561](https://github.com/Monadical-SAS/reflector/issues/561)) ([9265d20](https://github.com/Monadical-SAS/reflector/commit/9265d201b590d23c628c5f19251b70f473859043))
+
 ## [0.7.2](https://github.com/Monadical-SAS/reflector/compare/v0.7.1...v0.7.2) (2025-08-21)
 
 

CLAUDE.md

@@ -66,7 +66,6 @@ pnpm install
 
 # Copy configuration templates
 cp .env_template .env
-cp config-template.ts config.ts
 ```
 
 **Development:**

README.md

@@ -1,43 +1,60 @@
 <div align="center">
+<img width="100" alt="image" src="https://github.com/user-attachments/assets/66fb367b-2c89-4516-9912-f47ac59c6a7f"/>
 
 # Reflector
 
-Reflector Audio Management and Analysis is a cutting-edge web application under development by Monadical. It utilizes AI to record meetings, providing a permanent record with transcripts, translations, and automated summaries.
+Reflector is an AI-powered audio transcription and meeting analysis platform that provides real-time transcription, speaker diarization, translation and summarization for audio content and live meetings. It works 100% with local models (whisper/parakeet, pyannote, seamless-m4t, and your local llm like phi-4).
 
-[![Tests](https://github.com/monadical-sas/reflector/actions/workflows/pytests.yml/badge.svg?branch=main&event=push)](https://github.com/monadical-sas/reflector/actions/workflows/pytests.yml)
+[![Tests](https://github.com/monadical-sas/reflector/actions/workflows/test_server.yml/badge.svg?branch=main&event=push)](https://github.com/monadical-sas/reflector/actions/workflows/test_server.yml)
 [![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](https://opensource.org/licenses/MIT)
 </div>
-
-## Screenshots
+</div>
 <table>
   <tr>
     <td>
-      <a href="https://github.com/user-attachments/assets/3a976930-56c1-47ef-8c76-55d3864309e3">
-        <img width="700" alt="image" src="https://github.com/user-attachments/assets/3a976930-56c1-47ef-8c76-55d3864309e3" />
+      <a href="https://github.com/user-attachments/assets/21f5597c-2930-4899-a154-f7bd61a59e97">
+        <img width="700" alt="image" src="https://github.com/user-attachments/assets/21f5597c-2930-4899-a154-f7bd61a59e97" />
       </a>
     </td>
     <td>
-      <a href="https://github.com/user-attachments/assets/bfe3bde3-08af-4426-a9a1-11ad5cd63b33">
-        <img width="700" alt="image" src="https://github.com/user-attachments/assets/bfe3bde3-08af-4426-a9a1-11ad5cd63b33" />
+      <a href="https://github.com/user-attachments/assets/f6b9399a-5e51-4bae-b807-59128d0a940c">
+        <img width="700" alt="image" src="https://github.com/user-attachments/assets/f6b9399a-5e51-4bae-b807-59128d0a940c" />
       </a>
     </td>
     <td>
-      <a href="https://github.com/user-attachments/assets/7b60c9d0-efe4-474f-a27b-ea13bd0fabdc">
-        <img width="700" alt="image" src="https://github.com/user-attachments/assets/7b60c9d0-efe4-474f-a27b-ea13bd0fabdc" />
+      <a href="https://github.com/user-attachments/assets/a42ce460-c1fd-4489-a995-270516193897">
+        <img width="700" alt="image" src="https://github.com/user-attachments/assets/a42ce460-c1fd-4489-a995-270516193897" />
+      </a>
+    </td>
+    <td>
+      <a href="https://github.com/user-attachments/assets/21929f6d-c309-42fe-9c11-f1299e50fbd4">
+        <img width="700" alt="image" src="https://github.com/user-attachments/assets/21929f6d-c309-42fe-9c11-f1299e50fbd4" />
       </a>
     </td>
   </tr>
 </table>
 
+## What is Reflector?
+
+Reflector is a web application that utilizes local models to process audio content, providing:
+
+- **Real-time Transcription**: Convert speech to text using [Whisper](https://github.com/openai/whisper) (multi-language) or [Parakeet](https://huggingface.co/nvidia/parakeet-tdt-0.6b-v2) (English) models
+- **Speaker Diarization**: Identify and label different speakers using [Pyannote](https://github.com/pyannote/pyannote-audio) 3.1
+- **Live Translation**: Translate audio content in real-time to many languages with [Facebook Seamless-M4T](https://github.com/facebookresearch/seamless_communication)
+- **Topic Detection & Summarization**: Extract key topics and generate concise summaries using LLMs
+- **Meeting Recording**: Create permanent records of meetings with searchable transcripts
+
+Currently we provide [modal.com](https://modal.com/) gpu template to deploy.
+
 ## Background
 
 The project architecture consists of three primary components:
 
-- **Front-End**: NextJS React project hosted on Vercel, located in `www/`.
 - **Back-End**: Python server that offers an API and data persistence, found in `server/`.
-- **GPU implementation**: Providing services such as speech-to-text transcription, topic generation, automated summaries, and translations. Most reliable option is Modal deployment
+- **Front-End**: NextJS React project hosted on Vercel, located in `www/`.
+- **GPU implementation**: Providing services such as speech-to-text transcription, topic generation, automated summaries, and translations.
 
-It also uses authentik for authentication if activated, and Vercel for deployment and configuration of the front-end.
+It also uses authentik for authentication if activated.
 
 ## Contribution Guidelines
 
@@ -72,6 +89,8 @@ Note: We currently do not have instructions for Windows users.
 
 ## Installation
 
+*Note: we're working toward better installation, theses instructions are not accurate for now*
+
 ### Frontend
 
 Start with `cd www`.
@@ -80,11 +99,10 @@ Start with `cd www`.
 
 ```bash
 pnpm install
-cp .env_template .env
-cp config-template.ts config.ts
+cp .env.example .env
 ```
 
-Then, fill in the environment variables in `.env` and the configuration in `config.ts` as needed. If you are unsure on how to proceed, ask in Zulip.
+Then, fill in the environment variables in `.env` as needed. If you are unsure on how to proceed, ask in Zulip.
 
 **Run in development mode**
 
@@ -149,3 +167,34 @@ You can manually process an audio file by calling the process tool:
 ```bash
 uv run python -m reflector.tools.process path/to/audio.wav
 ```
+
+
+## Feature Flags
+
+Reflector uses environment variable-based feature flags to control application functionality. These flags allow you to enable or disable features without code changes.
+
+### Available Feature Flags
+
+| Feature Flag | Environment Variable |
+|-------------|---------------------|
+| `requireLogin` | `NEXT_PUBLIC_FEATURE_REQUIRE_LOGIN` |
+| `privacy` | `NEXT_PUBLIC_FEATURE_PRIVACY` |
+| `browse` | `NEXT_PUBLIC_FEATURE_BROWSE` |
+| `sendToZulip` | `NEXT_PUBLIC_FEATURE_SEND_TO_ZULIP` |
+| `rooms` | `NEXT_PUBLIC_FEATURE_ROOMS` |
+
+### Setting Feature Flags
+
+Feature flags are controlled via environment variables using the pattern `NEXT_PUBLIC_FEATURE_{FEATURE_NAME}` where `{FEATURE_NAME}` is the SCREAMING_SNAKE_CASE version of the feature name.
+
+**Examples:**
+```bash
+# Enable user authentication requirement
+NEXT_PUBLIC_FEATURE_REQUIRE_LOGIN=true
+
+# Disable browse functionality
+NEXT_PUBLIC_FEATURE_BROWSE=false
+
+# Enable Zulip integration
+NEXT_PUBLIC_FEATURE_SEND_TO_ZULIP=true
+```

compose.yml

@@ -6,6 +6,7 @@ services:
       - 1250:1250
     volumes:
       - ./server/:/app/
+      - /app/.venv
     env_file:
       - ./server/.env
     environment:
@@ -16,6 +17,7 @@ services:
       context: server
     volumes:
       - ./server/:/app/
+      - /app/.venv
     env_file:
       - ./server/.env
     environment:
@@ -26,6 +28,7 @@ services:
       context: server
     volumes:
       - ./server/:/app/
+      - /app/.venv
     env_file:
       - ./server/.env
     environment:

gpu/modal_deployments/.gitignore

@@ -0,0 +1,33 @@
+# OS / Editor
+.DS_Store
+.vscode/
+.idea/
+
+# Python
+__pycache__/
+*.py[cod]
+*$py.class
+
+# Logs
+*.log
+
+# Env and secrets
+.env
+.env.*
+*.env
+*.secret
+
+# Build / dist
+build/
+dist/
+.eggs/
+*.egg-info/
+
+# Coverage / test
+.pytest_cache/
+.coverage*
+htmlcov/
+
+# Modal local state (if any)
+modal_mounts/
+.modal_cache/

gpu/modal_deployments/reflector_transcriber.py

@@ -0,0 +1,608 @@
+import os
+import sys
+import threading
+import uuid
+from typing import Generator, Mapping, NamedTuple, NewType, TypedDict
+from urllib.parse import urlparse
+
+import modal
+
+MODEL_NAME = "large-v2"
+MODEL_COMPUTE_TYPE: str = "float16"
+MODEL_NUM_WORKERS: int = 1
+MINUTES = 60  # seconds
+SAMPLERATE = 16000
+UPLOADS_PATH = "/uploads"
+CACHE_PATH = "/models"
+SUPPORTED_FILE_EXTENSIONS = ["mp3", "mp4", "mpeg", "mpga", "m4a", "wav", "webm"]
+VAD_CONFIG = {
+    "batch_max_duration": 30.0,
+    "silence_padding": 0.5,
+    "window_size": 512,
+}
+
+
+WhisperUniqFilename = NewType("WhisperUniqFilename", str)
+AudioFileExtension = NewType("AudioFileExtension", str)
+
+app = modal.App("reflector-transcriber")
+
+model_cache = modal.Volume.from_name("models", create_if_missing=True)
+upload_volume = modal.Volume.from_name("whisper-uploads", create_if_missing=True)
+
+
+class TimeSegment(NamedTuple):
+    """Represents a time segment with start and end times."""
+
+    start: float
+    end: float
+
+
+class AudioSegment(NamedTuple):
+    """Represents an audio segment with timing and audio data."""
+
+    start: float
+    end: float
+    audio: any
+
+
+class TranscriptResult(NamedTuple):
+    """Represents a transcription result with text and word timings."""
+
+    text: str
+    words: list["WordTiming"]
+
+
+class WordTiming(TypedDict):
+    """Represents a word with its timing information."""
+
+    word: str
+    start: float
+    end: float
+
+
+def download_model():
+    from faster_whisper import download_model
+
+    model_cache.reload()
+
+    download_model(MODEL_NAME, cache_dir=CACHE_PATH)
+
+    model_cache.commit()
+
+
+image = (
+    modal.Image.debian_slim(python_version="3.12")
+    .env(
+        {
+            "HF_HUB_ENABLE_HF_TRANSFER": "1",
+            "LD_LIBRARY_PATH": (
+                "/usr/local/lib/python3.12/site-packages/nvidia/cudnn/lib/:"
+                "/opt/conda/lib/python3.12/site-packages/nvidia/cublas/lib/"
+            ),
+        }
+    )
+    .apt_install("ffmpeg")
+    .pip_install(
+        "huggingface_hub==0.27.1",
+        "hf-transfer==0.1.9",
+        "torch==2.5.1",
+        "faster-whisper==1.1.1",
+        "fastapi==0.115.12",
+        "requests",
+        "librosa==0.10.1",
+        "numpy<2",
+        "silero-vad==5.1.0",
+    )
+    .run_function(download_model, volumes={CACHE_PATH: model_cache})
+)
+
+
+def detect_audio_format(url: str, headers: Mapping[str, str]) -> AudioFileExtension:
+    parsed_url = urlparse(url)
+    url_path = parsed_url.path
+
+    for ext in SUPPORTED_FILE_EXTENSIONS:
+        if url_path.lower().endswith(f".{ext}"):
+            return AudioFileExtension(ext)
+
+    content_type = headers.get("content-type", "").lower()
+    if "audio/mpeg" in content_type or "audio/mp3" in content_type:
+        return AudioFileExtension("mp3")
+    if "audio/wav" in content_type:
+        return AudioFileExtension("wav")
+    if "audio/mp4" in content_type:
+        return AudioFileExtension("mp4")
+
+    raise ValueError(
+        f"Unsupported audio format for URL: {url}. "
+        f"Supported extensions: {', '.join(SUPPORTED_FILE_EXTENSIONS)}"
+    )
+
+
+def download_audio_to_volume(
+    audio_file_url: str,
+) -> tuple[WhisperUniqFilename, AudioFileExtension]:
+    import requests
+    from fastapi import HTTPException
+
+    response = requests.head(audio_file_url, allow_redirects=True)
+    if response.status_code == 404:
+        raise HTTPException(status_code=404, detail="Audio file not found")
+
+    response = requests.get(audio_file_url, allow_redirects=True)
+    response.raise_for_status()
+
+    audio_suffix = detect_audio_format(audio_file_url, response.headers)
+    unique_filename = WhisperUniqFilename(f"{uuid.uuid4()}.{audio_suffix}")
+    file_path = f"{UPLOADS_PATH}/{unique_filename}"
+
+    with open(file_path, "wb") as f:
+        f.write(response.content)
+
+    upload_volume.commit()
+    return unique_filename, audio_suffix
+
+
+def pad_audio(audio_array, sample_rate: int = SAMPLERATE):
+    """Add 0.5s of silence if audio is shorter than the silence_padding window.
+
+    Whisper does not require this strictly, but aligning behavior with Parakeet
+    avoids edge-case crashes on extremely short inputs and makes comparisons easier.
+    """
+    import numpy as np
+
+    audio_duration = len(audio_array) / sample_rate
+    if audio_duration < VAD_CONFIG["silence_padding"]:
+        silence_samples = int(sample_rate * VAD_CONFIG["silence_padding"])
+        silence = np.zeros(silence_samples, dtype=np.float32)
+        return np.concatenate([audio_array, silence])
+    return audio_array
+
+
+@app.cls(
+    gpu="A10G",
+    timeout=5 * MINUTES,
+    scaledown_window=5 * MINUTES,
+    image=image,
+    volumes={CACHE_PATH: model_cache, UPLOADS_PATH: upload_volume},
+)
+@modal.concurrent(max_inputs=10)
+class TranscriberWhisperLive:
+    """Live transcriber class for small audio segments (A10G).
+
+    Mirrors the Parakeet live class API but uses Faster-Whisper under the hood.
+    """
+
+    @modal.enter()
+    def enter(self):
+        import faster_whisper
+        import torch
+
+        self.lock = threading.Lock()
+        self.use_gpu = torch.cuda.is_available()
+        self.device = "cuda" if self.use_gpu else "cpu"
+        self.model = faster_whisper.WhisperModel(
+            MODEL_NAME,
+            device=self.device,
+            compute_type=MODEL_COMPUTE_TYPE,
+            num_workers=MODEL_NUM_WORKERS,
+            download_root=CACHE_PATH,
+            local_files_only=True,
+        )
+        print(f"Model is on device: {self.device}")
+
+    @modal.method()
+    def transcribe_segment(
+        self,
+        filename: str,
+        language: str = "en",
+    ):
+        """Transcribe a single uploaded audio file by filename."""
+        upload_volume.reload()
+
+        file_path = f"{UPLOADS_PATH}/{filename}"
+        if not os.path.exists(file_path):
+            raise FileNotFoundError(f"File not found: {file_path}")
+
+        with self.lock:
+            with NoStdStreams():
+                segments, _ = self.model.transcribe(
+                    file_path,
+                    language=language,
+                    beam_size=5,
+                    word_timestamps=True,
+                    vad_filter=True,
+                    vad_parameters={"min_silence_duration_ms": 500},
+                )
+
+        segments = list(segments)
+        text = "".join(segment.text for segment in segments).strip()
+        words = [
+            {
+                "word": word.word,
+                "start": round(float(word.start), 2),
+                "end": round(float(word.end), 2),
+            }
+            for segment in segments
+            for word in segment.words
+        ]
+
+        return {"text": text, "words": words}
+
+    @modal.method()
+    def transcribe_batch(
+        self,
+        filenames: list[str],
+        language: str = "en",
+    ):
+        """Transcribe multiple uploaded audio files and return per-file results."""
+        upload_volume.reload()
+
+        results = []
+        for filename in filenames:
+            file_path = f"{UPLOADS_PATH}/{filename}"
+            if not os.path.exists(file_path):
+                raise FileNotFoundError(f"Batch file not found: {file_path}")
+
+            with self.lock:
+                with NoStdStreams():
+                    segments, _ = self.model.transcribe(
+                        file_path,
+                        language=language,
+                        beam_size=5,
+                        word_timestamps=True,
+                        vad_filter=True,
+                        vad_parameters={"min_silence_duration_ms": 500},
+                    )
+
+            segments = list(segments)
+            text = "".join(seg.text for seg in segments).strip()
+            words = [
+                {
+                    "word": w.word,
+                    "start": round(float(w.start), 2),
+                    "end": round(float(w.end), 2),
+                }
+                for seg in segments
+                for w in seg.words
+            ]
+
+            results.append(
+                {
+                    "filename": filename,
+                    "text": text,
+                    "words": words,
+                }
+            )
+
+        return results
+
+
+@app.cls(
+    gpu="L40S",
+    timeout=15 * MINUTES,
+    image=image,
+    volumes={CACHE_PATH: model_cache, UPLOADS_PATH: upload_volume},
+)
+class TranscriberWhisperFile:
+    """File transcriber for larger/longer audio, using VAD-driven batching (L40S)."""
+
+    @modal.enter()
+    def enter(self):
+        import faster_whisper
+        import torch
+        from silero_vad import load_silero_vad
+
+        self.lock = threading.Lock()
+        self.use_gpu = torch.cuda.is_available()
+        self.device = "cuda" if self.use_gpu else "cpu"
+        self.model = faster_whisper.WhisperModel(
+            MODEL_NAME,
+            device=self.device,
+            compute_type=MODEL_COMPUTE_TYPE,
+            num_workers=MODEL_NUM_WORKERS,
+            download_root=CACHE_PATH,
+            local_files_only=True,
+        )
+        self.vad_model = load_silero_vad(onnx=False)
+
+    @modal.method()
+    def transcribe_segment(
+        self, filename: str, timestamp_offset: float = 0.0, language: str = "en"
+    ):
+        import librosa
+        import numpy as np
+        from silero_vad import VADIterator
+
+        def vad_segments(
+            audio_array,
+            sample_rate: int = SAMPLERATE,
+            window_size: int = VAD_CONFIG["window_size"],
+        ) -> Generator[TimeSegment, None, None]:
+            """Generate speech segments as TimeSegment using Silero VAD."""
+            iterator = VADIterator(self.vad_model, sampling_rate=sample_rate)
+            start = None
+            for i in range(0, len(audio_array), window_size):
+                chunk = audio_array[i : i + window_size]
+                if len(chunk) < window_size:
+                    chunk = np.pad(
+                        chunk, (0, window_size - len(chunk)), mode="constant"
+                    )
+                speech = iterator(chunk)
+                if not speech:
+                    continue
+                if "start" in speech:
+                    start = speech["start"]
+                    continue
+                if "end" in speech and start is not None:
+                    end = speech["end"]
+                    yield TimeSegment(
+                        start / float(SAMPLERATE), end / float(SAMPLERATE)
+                    )
+                    start = None
+            iterator.reset_states()
+
+        upload_volume.reload()
+        file_path = f"{UPLOADS_PATH}/{filename}"
+        if not os.path.exists(file_path):
+            raise FileNotFoundError(f"File not found: {file_path}")
+
+        audio_array, _sr = librosa.load(file_path, sr=SAMPLERATE, mono=True)
+
+        # Batch segments up to ~30s windows by merging contiguous VAD segments
+        merged_batches: list[TimeSegment] = []
+        batch_start = None
+        batch_end = None
+        max_duration = VAD_CONFIG["batch_max_duration"]
+        for segment in vad_segments(audio_array):
+            seg_start, seg_end = segment.start, segment.end
+            if batch_start is None:
+                batch_start, batch_end = seg_start, seg_end
+                continue
+            if seg_end - batch_start <= max_duration:
+                batch_end = seg_end
+            else:
+                merged_batches.append(TimeSegment(batch_start, batch_end))
+                batch_start, batch_end = seg_start, seg_end
+        if batch_start is not None and batch_end is not None:
+            merged_batches.append(TimeSegment(batch_start, batch_end))
+
+        all_text = []
+        all_words = []
+
+        for segment in merged_batches:
+            start_time, end_time = segment.start, segment.end
+            s_idx = int(start_time * SAMPLERATE)
+            e_idx = int(end_time * SAMPLERATE)
+            segment = audio_array[s_idx:e_idx]
+            segment = pad_audio(segment, SAMPLERATE)
+
+            with self.lock:
+                segments, _ = self.model.transcribe(
+                    segment,
+                    language=language,
+                    beam_size=5,
+                    word_timestamps=True,
+                    vad_filter=True,
+                    vad_parameters={"min_silence_duration_ms": 500},
+                )
+
+            segments = list(segments)
+            text = "".join(seg.text for seg in segments).strip()
+            words = [
+                {
+                    "word": w.word,
+                    "start": round(float(w.start) + start_time + timestamp_offset, 2),
+                    "end": round(float(w.end) + start_time + timestamp_offset, 2),
+                }
+                for seg in segments
+                for w in seg.words
+            ]
+            if text:
+                all_text.append(text)
+            all_words.extend(words)
+
+        return {"text": " ".join(all_text), "words": all_words}
+
+
+def detect_audio_format(url: str, headers: dict) -> str:
+    from urllib.parse import urlparse
+
+    from fastapi import HTTPException
+
+    url_path = urlparse(url).path
+    for ext in SUPPORTED_FILE_EXTENSIONS:
+        if url_path.lower().endswith(f".{ext}"):
+            return ext
+
+    content_type = headers.get("content-type", "").lower()
+    if "audio/mpeg" in content_type or "audio/mp3" in content_type:
+        return "mp3"
+    if "audio/wav" in content_type:
+        return "wav"
+    if "audio/mp4" in content_type:
+        return "mp4"
+
+    raise HTTPException(
+        status_code=400,
+        detail=(
+            f"Unsupported audio format for URL. Supported extensions: {', '.join(SUPPORTED_FILE_EXTENSIONS)}"
+        ),
+    )
+
+
+def download_audio_to_volume(audio_file_url: str) -> tuple[str, str]:
+    import requests
+    from fastapi import HTTPException
+
+    response = requests.head(audio_file_url, allow_redirects=True)
+    if response.status_code == 404:
+        raise HTTPException(status_code=404, detail="Audio file not found")
+
+    response = requests.get(audio_file_url, allow_redirects=True)
+    response.raise_for_status()
+
+    audio_suffix = detect_audio_format(audio_file_url, response.headers)
+    unique_filename = f"{uuid.uuid4()}.{audio_suffix}"
+    file_path = f"{UPLOADS_PATH}/{unique_filename}"
+
+    with open(file_path, "wb") as f:
+        f.write(response.content)
+
+    upload_volume.commit()
+    return unique_filename, audio_suffix
+
+
+@app.function(
+    scaledown_window=60,
+    timeout=600,
+    secrets=[
+        modal.Secret.from_name("reflector-gpu"),
+    ],
+    volumes={CACHE_PATH: model_cache, UPLOADS_PATH: upload_volume},
+    image=image,
+)
+@modal.concurrent(max_inputs=40)
+@modal.asgi_app()
+def web():
+    from fastapi import (
+        Body,
+        Depends,
+        FastAPI,
+        Form,
+        HTTPException,
+        UploadFile,
+        status,
+    )
+    from fastapi.security import OAuth2PasswordBearer
+
+    transcriber_live = TranscriberWhisperLive()
+    transcriber_file = TranscriberWhisperFile()
+
+    app = FastAPI()
+
+    oauth2_scheme = OAuth2PasswordBearer(tokenUrl="token")
+
+    def apikey_auth(apikey: str = Depends(oauth2_scheme)):
+        if apikey == os.environ["REFLECTOR_GPU_APIKEY"]:
+            return
+        raise HTTPException(
+            status_code=status.HTTP_401_UNAUTHORIZED,
+            detail="Invalid API key",
+            headers={"WWW-Authenticate": "Bearer"},
+        )
+
+    class TranscriptResponse(dict):
+        pass
+
+    @app.post("/v1/audio/transcriptions", dependencies=[Depends(apikey_auth)])
+    def transcribe(
+        file: UploadFile = None,
+        files: list[UploadFile] | None = None,
+        model: str = Form(MODEL_NAME),
+        language: str = Form("en"),
+        batch: bool = Form(False),
+    ):
+        if not file and not files:
+            raise HTTPException(
+                status_code=400, detail="Either 'file' or 'files' parameter is required"
+            )
+        if batch and not files:
+            raise HTTPException(
+                status_code=400, detail="Batch transcription requires 'files'"
+            )
+
+        upload_files = [file] if file else files
+
+        uploaded_filenames: list[str] = []
+        for upload_file in upload_files:
+            audio_suffix = upload_file.filename.split(".")[-1]
+            if audio_suffix not in SUPPORTED_FILE_EXTENSIONS:
+                raise HTTPException(
+                    status_code=400,
+                    detail=(
+                        f"Unsupported audio format. Supported extensions: {', '.join(SUPPORTED_FILE_EXTENSIONS)}"
+                    ),
+                )
+
+            unique_filename = f"{uuid.uuid4()}.{audio_suffix}"
+            file_path = f"{UPLOADS_PATH}/{unique_filename}"
+            with open(file_path, "wb") as f:
+                content = upload_file.file.read()
+                f.write(content)
+            uploaded_filenames.append(unique_filename)
+
+        upload_volume.commit()
+
+        try:
+            if batch and len(upload_files) > 1:
+                func = transcriber_live.transcribe_batch.spawn(
+                    filenames=uploaded_filenames,
+                    language=language,
+                )
+                results = func.get()
+                return {"results": results}
+
+            results = []
+            for filename in uploaded_filenames:
+                func = transcriber_live.transcribe_segment.spawn(
+                    filename=filename,
+                    language=language,
+                )
+                result = func.get()
+                result["filename"] = filename
+                results.append(result)
+
+            return {"results": results} if len(results) > 1 else results[0]
+        finally:
+            for filename in uploaded_filenames:
+                try:
+                    file_path = f"{UPLOADS_PATH}/{filename}"
+                    os.remove(file_path)
+                except Exception:
+                    pass
+            upload_volume.commit()
+
+    @app.post("/v1/audio/transcriptions-from-url", dependencies=[Depends(apikey_auth)])
+    def transcribe_from_url(
+        audio_file_url: str = Body(
+            ..., description="URL of the audio file to transcribe"
+        ),
+        model: str = Body(MODEL_NAME),
+        language: str = Body("en"),
+        timestamp_offset: float = Body(0.0),
+    ):
+        unique_filename, _audio_suffix = download_audio_to_volume(audio_file_url)
+        try:
+            func = transcriber_file.transcribe_segment.spawn(
+                filename=unique_filename,
+                timestamp_offset=timestamp_offset,
+                language=language,
+            )
+            result = func.get()
+            return result
+        finally:
+            try:
+                file_path = f"{UPLOADS_PATH}/{unique_filename}"
+                os.remove(file_path)
+                upload_volume.commit()
+            except Exception:
+                pass
+
+    return app
+
+
+class NoStdStreams:
+    def __init__(self):
+        self.devnull = open(os.devnull, "w")
+
+    def __enter__(self):
+        self._stdout, self._stderr = sys.stdout, sys.stderr
+        self._stdout.flush()
+        self._stderr.flush()
+        sys.stdout, sys.stderr = self.devnull, self.devnull
+
+    def __exit__(self, exc_type, exc_value, traceback):
+        sys.stdout, sys.stderr = self._stdout, self._stderr
+        self.devnull.close()

gpu/modal_deployments/reflector_transcriber_parakeet.py

@@ -3,7 +3,7 @@ import os
 import sys
 import threading
 import uuid
-from typing import Mapping, NewType
+from typing import Generator, Mapping, NamedTuple, NewType, TypedDict
 from urllib.parse import urlparse
 
 import modal
@@ -14,10 +14,7 @@ SAMPLERATE = 16000
 UPLOADS_PATH = "/uploads"
 CACHE_PATH = "/cache"
 VAD_CONFIG = {
-    "max_segment_duration": 30.0,
-    "batch_max_files": 10,
-    "batch_max_duration": 5.0,
-    "min_segment_duration": 0.02,
+    "batch_max_duration": 30.0,
     "silence_padding": 0.5,
     "window_size": 512,
 }
@@ -25,6 +22,37 @@ VAD_CONFIG = {
 ParakeetUniqFilename = NewType("ParakeetUniqFilename", str)
 AudioFileExtension = NewType("AudioFileExtension", str)
 
+
+class TimeSegment(NamedTuple):
+    """Represents a time segment with start and end times."""
+
+    start: float
+    end: float
+
+
+class AudioSegment(NamedTuple):
+    """Represents an audio segment with timing and audio data."""
+
+    start: float
+    end: float
+    audio: any
+
+
+class TranscriptResult(NamedTuple):
+    """Represents a transcription result with text and word timings."""
+
+    text: str
+    words: list["WordTiming"]
+
+
+class WordTiming(TypedDict):
+    """Represents a word with its timing information."""
+
+    word: str
+    start: float
+    end: float
+
+
 app = modal.App("reflector-transcriber-parakeet")
 
 # Volume for caching model weights
@@ -170,12 +198,14 @@ class TranscriberParakeetLive:
                 (output,) = self.model.transcribe([padded_audio], timestamps=True)
 
         text = output.text.strip()
-        words = [
-            {
-                "word": word_info["word"] + " ",
-                "start": round(word_info["start"], 2),
-                "end": round(word_info["end"], 2),
-            }
+        words: list[WordTiming] = [
+            WordTiming(
+                # XXX the space added here is to match the output of whisper
+                # whisper add space to each words, while parakeet don't
+                word=word_info["word"] + " ",
+                start=round(word_info["start"], 2),
+                end=round(word_info["end"], 2),
+            )
             for word_info in output.timestamp["word"]
         ]
 
@@ -211,12 +241,12 @@ class TranscriberParakeetLive:
         for i, (filename, output) in enumerate(zip(filenames, outputs)):
             text = output.text.strip()
 
-            words = [
-                {
-                    "word": word_info["word"] + " ",
-                    "start": round(word_info["start"], 2),
-                    "end": round(word_info["end"], 2),
-                }
+            words: list[WordTiming] = [
+                WordTiming(
+                    word=word_info["word"] + " ",
+                    start=round(word_info["start"], 2),
+                    end=round(word_info["end"], 2),
+                )
                 for word_info in output.timestamp["word"]
             ]
 
@@ -271,7 +301,9 @@ class TranscriberParakeetFile:
             audio_array, sample_rate = librosa.load(file_path, sr=SAMPLERATE, mono=True)
             return audio_array
 
-        def vad_segment_generator(audio_array):
+        def vad_segment_generator(
+            audio_array,
+        ) -> Generator[TimeSegment, None, None]:
             """Generate speech segments using VAD with start/end sample indices"""
             vad_iterator = VADIterator(self.vad_model, sampling_rate=SAMPLERATE)
             window_size = VAD_CONFIG["window_size"]
@@ -297,107 +329,121 @@ class TranscriberParakeetFile:
                     start_time = start / float(SAMPLERATE)
                     end_time = end / float(SAMPLERATE)
 
-                    # Extract the actual audio segment
-                    audio_segment = audio_array[start:end]
-
-                    yield (start_time, end_time, audio_segment)
+                    yield TimeSegment(start_time, end_time)
                     start = None
 
             vad_iterator.reset_states()
 
-        def vad_segment_filter(segments):
-            """Filter VAD segments by duration and chunk large segments"""
-            min_dur = VAD_CONFIG["min_segment_duration"]
-            max_dur = VAD_CONFIG["max_segment_duration"]
+        def batch_speech_segments(
+            segments: Generator[TimeSegment, None, None], max_duration: int
+        ) -> Generator[TimeSegment, None, None]:
+            """
+            Input segments:
+              [0-2] [3-5] [6-8] [10-11] [12-15] [17-19] [20-22]
 
-            for start_time, end_time, audio_segment in segments:
-                segment_duration = end_time - start_time
+                                  ↓ (max_duration=10)
 
-                # Skip very small segments
-                if segment_duration < min_dur:
+              Output batches:
+              [0-8]           [10-19]          [20-22]
+
+            Note: silences are kept for better transcription, previous implementation was
+            passing segments separatly, but the output was less accurate.
+            """
+            batch_start_time = None
+            batch_end_time = None
+
+            for segment in segments:
+                start_time, end_time = segment.start, segment.end
+                if batch_start_time is None or batch_end_time is None:
+                    batch_start_time = start_time
+                    batch_end_time = end_time
                     continue
 
-                # If segment is within max duration, yield as-is
-                if segment_duration <= max_dur:
-                    yield (start_time, end_time, audio_segment)
+                total_duration = end_time - batch_start_time
+
+                if total_duration <= max_duration:
+                    batch_end_time = end_time
                     continue
 
-                # Chunk large segments into smaller pieces
-                chunk_samples = int(max_dur * SAMPLERATE)
-                current_start = start_time
+                yield TimeSegment(batch_start_time, batch_end_time)
+                batch_start_time = start_time
+                batch_end_time = end_time
 
-                for chunk_offset in range(0, len(audio_segment), chunk_samples):
-                    chunk_audio = audio_segment[
-                        chunk_offset : chunk_offset + chunk_samples
-                    ]
-                    if len(chunk_audio) == 0:
-                        break
+            if batch_start_time is None or batch_end_time is None:
+                return
 
-                    chunk_duration = len(chunk_audio) / float(SAMPLERATE)
-                    chunk_end = current_start + chunk_duration
+            yield TimeSegment(batch_start_time, batch_end_time)
 
-                    # Only yield chunks that meet minimum duration
-                    if chunk_duration >= min_dur:
-                        yield (current_start, chunk_end, chunk_audio)
+        def batch_segment_to_audio_segment(
+            segments: Generator[TimeSegment, None, None],
+            audio_array,
+        ) -> Generator[AudioSegment, None, None]:
+            """Extract audio segments and apply padding for Parakeet compatibility.
 
-                    current_start = chunk_end
+            Uses pad_audio to ensure segments are at least 0.5s long, preventing
+            Parakeet crashes. This padding may cause slight timing overlaps between
+            segments, which are corrected by enforce_word_timing_constraints.
+            """
+            for segment in segments:
+                start_time, end_time = segment.start, segment.end
+                start_sample = int(start_time * SAMPLERATE)
+                end_sample = int(end_time * SAMPLERATE)
+                audio_segment = audio_array[start_sample:end_sample]
 
-        def batch_segments(segments, max_files=10, max_duration=5.0):
-            batch = []
-            batch_duration = 0.0
+                padded_segment = pad_audio(audio_segment, SAMPLERATE)
 
-            for start_time, end_time, audio_segment in segments:
-                segment_duration = end_time - start_time
+                yield AudioSegment(start_time, end_time, padded_segment)
 
-                if segment_duration < VAD_CONFIG["silence_padding"]:
-                    silence_samples = int(
-                        (VAD_CONFIG["silence_padding"] - segment_duration) * SAMPLERATE
-                    )
-                    padding = np.zeros(silence_samples, dtype=np.float32)
-                    audio_segment = np.concatenate([audio_segment, padding])
-                    segment_duration = VAD_CONFIG["silence_padding"]
-
-                batch.append((start_time, end_time, audio_segment))
-                batch_duration += segment_duration
-
-                if len(batch) >= max_files or batch_duration >= max_duration:
-                    yield batch
-                    batch = []
-                    batch_duration = 0.0
-
-            if batch:
-                yield batch
-
-        def transcribe_batch(model, audio_segments):
+        def transcribe_batch(model, audio_segments: list) -> list:
             with NoStdStreams():
                 outputs = model.transcribe(audio_segments, timestamps=True)
             return outputs
 
+        def enforce_word_timing_constraints(
+            words: list[WordTiming],
+        ) -> list[WordTiming]:
+            """Enforce that word end times don't exceed the start time of the next word.
+
+            Due to silence padding added in batch_segment_to_audio_segment for better
+            transcription accuracy, word timings from different segments may overlap.
+            This function ensures there are no overlaps by adjusting end times.
+            """
+            if len(words) <= 1:
+                return words
+
+            enforced_words = []
+            for i, word in enumerate(words):
+                enforced_word = word.copy()
+
+                if i < len(words) - 1:
+                    next_start = words[i + 1]["start"]
+                    if enforced_word["end"] > next_start:
+                        enforced_word["end"] = next_start
+
+                enforced_words.append(enforced_word)
+
+            return enforced_words
+
         def emit_results(
-            results,
-            segments_info,
-            batch_index,
-            total_batches,
-        ):
+            results: list,
+            segments_info: list[AudioSegment],
+        ) -> Generator[TranscriptResult, None, None]:
             """Yield transcribed text and word timings from model output, adjusting timestamps to absolute positions."""
-            for i, (output, (start_time, end_time, _)) in enumerate(
-                zip(results, segments_info)
-            ):
+            for i, (output, segment) in enumerate(zip(results, segments_info)):
+                start_time, end_time = segment.start, segment.end
                 text = output.text.strip()
-                words = [
-                    {
-                        "word": word_info["word"] + " ",
-                        "start": round(
+                words: list[WordTiming] = [
+                    WordTiming(
+                        word=word_info["word"] + " ",
+                        start=round(
                             word_info["start"] + start_time + timestamp_offset, 2
                         ),
-                        "end": round(
-                            word_info["end"] + start_time + timestamp_offset, 2
-                        ),
-                    }
+                        end=round(word_info["end"] + start_time + timestamp_offset, 2),
+                    )
                     for word_info in output.timestamp["word"]
                 ]
 
-                yield text, words
+                yield TranscriptResult(text, words)
 
         upload_volume.reload()
 
@@ -407,41 +453,31 @@ class TranscriberParakeetFile:
 
         audio_array = load_and_convert_audio(file_path)
         total_duration = len(audio_array) / float(SAMPLERATE)
-        processed_duration = 0.0
 
-        all_text_parts = []
-        all_words = []
+        all_text_parts: list[str] = []
+        all_words: list[WordTiming] = []
 
         raw_segments = vad_segment_generator(audio_array)
-        filtered_segments = vad_segment_filter(raw_segments)
-        batches = batch_segments(
-            filtered_segments,
-            VAD_CONFIG["batch_max_files"],
+        speech_segments = batch_speech_segments(
+            raw_segments,
             VAD_CONFIG["batch_max_duration"],
         )
+        audio_segments = batch_segment_to_audio_segment(speech_segments, audio_array)
 
-        batch_index = 0
-        total_batches = max(
-            1, int(total_duration / VAD_CONFIG["batch_max_duration"]) + 1
-        )
+        for batch in audio_segments:
+            audio_segment = batch.audio
+            results = transcribe_batch(self.model, [audio_segment])
 
-        for batch in batches:
-            batch_index += 1
-            audio_segments = [seg[2] for seg in batch]
-            results = transcribe_batch(self.model, audio_segments)
-
-            for text, words in emit_results(
+            for result in emit_results(
                 results,
-                batch,
-                batch_index,
-                total_batches,
+                [batch],
             ):
-                if not text:
+                if not result.text:
                     continue
-                all_text_parts.append(text)
-                all_words.extend(words)
+                all_text_parts.append(result.text)
+                all_words.extend(result.words)
 
-            processed_duration += sum(len(seg[2]) / float(SAMPLERATE) for seg in batch)
+        all_words = enforce_word_timing_constraints(all_words)
 
         combined_text = " ".join(all_text_parts)
         return {"text": combined_text, "words": all_words}

gpu/self_hosted/.env.example

@@ -0,0 +1,2 @@
+REFLECTOR_GPU_APIKEY=
+HF_TOKEN=

gpu/self_hosted/.gitignore

@@ -0,0 +1,38 @@
+cache/
+
+# OS / Editor
+.DS_Store
+.vscode/
+.idea/
+
+# Python
+__pycache__/
+*.py[cod]
+*$py.class
+
+# Env and secrets
+.env
+*.env
+*.secret
+HF_TOKEN
+REFLECTOR_GPU_APIKEY
+
+# Virtual env / uv
+.venv/
+venv/
+ENV/
+uv/
+
+# Build / dist
+build/
+dist/
+.eggs/
+*.egg-info/
+
+# Coverage / test
+.pytest_cache/
+.coverage*
+htmlcov/
+
+# Logs
+*.log

gpu/self_hosted/Dockerfile

@@ -0,0 +1,46 @@
+FROM python:3.12-slim
+
+ENV PYTHONUNBUFFERED=1 \
+    UV_LINK_MODE=copy \
+    UV_NO_CACHE=1
+
+WORKDIR /tmp
+RUN apt-get update \
+ && apt-get install -y \
+    ffmpeg \
+    curl \
+    ca-certificates \
+    gnupg \
+    wget \
+ && apt-get clean
+# Add NVIDIA CUDA repo for Debian 12 (bookworm) and install cuDNN 9 for CUDA 12
+ADD https://developer.download.nvidia.com/compute/cuda/repos/debian12/x86_64/cuda-keyring_1.1-1_all.deb /cuda-keyring.deb
+RUN dpkg -i /cuda-keyring.deb \
+ && rm /cuda-keyring.deb \
+ && apt-get update \
+ && apt-get install -y --no-install-recommends \
+    cuda-cudart-12-6 \
+    libcublas-12-6 \
+    libcudnn9-cuda-12 \
+    libcudnn9-dev-cuda-12 \
+ && apt-get clean \
+ && rm -rf /var/lib/apt/lists/*
+ADD https://astral.sh/uv/install.sh /uv-installer.sh
+RUN sh /uv-installer.sh && rm /uv-installer.sh
+ENV PATH="/root/.local/bin/:$PATH"
+ENV LD_LIBRARY_PATH="/usr/local/cuda/lib64:/usr/lib/x86_64-linux-gnu:$LD_LIBRARY_PATH"
+
+RUN mkdir -p /app
+WORKDIR /app
+COPY pyproject.toml uv.lock /app/
+
+
+COPY ./app /app/app
+COPY ./main.py /app/
+COPY ./runserver.sh /app/
+
+EXPOSE 8000
+
+CMD ["sh", "/app/runserver.sh"]
+
+

gpu/self_hosted/README.md

@@ -0,0 +1,73 @@
+# Self-hosted Model API
+
+Run transcription, translation, and diarization services compatible with Reflector's GPU Model API. Works on CPU or GPU.
+
+Environment variables
+
+- REFLECTOR_GPU_APIKEY: Optional Bearer token. If unset, auth is disabled.
+- HF_TOKEN: Optional. Required for diarization to download pyannote pipelines
+
+Requirements
+
+- FFmpeg must be installed and on PATH (used for URL-based and segmented transcription)
+- Python 3.12+
+- NVIDIA GPU optional. If available, it will be used automatically
+
+Local run
+Set env vars in self_hosted/.env file
+uv sync
+
+uv run uvicorn main:app --host 0.0.0.0 --port 8000
+
+Authentication
+
+- If REFLECTOR_GPU_APIKEY is set, include header: Authorization: Bearer <key>
+
+Endpoints
+
+- POST /v1/audio/transcriptions
+
+  - multipart/form-data
+  - fields: file (single file) OR files[] (multiple files), language, batch (true/false)
+  - response: single { text, words, filename } or { results: [ ... ] }
+
+- POST /v1/audio/transcriptions-from-url
+
+  - application/json
+  - body: { audio_file_url, language, timestamp_offset }
+  - response: { text, words }
+
+- POST /translate
+
+  - text: query parameter
+  - body (application/json): { source_language, target_language }
+  - response: { text: { <src>: original, <tgt>: translated } }
+
+- POST /diarize
+  - query parameters: audio_file_url, timestamp (optional)
+  - requires HF_TOKEN to be set (for pyannote)
+  - response: { diarization: [ { start, end, speaker } ] }
+
+OpenAPI docs
+
+- Visit /docs when the server is running
+
+Docker
+
+- Not yet provided in this directory. A Dockerfile will be added later. For now, use Local run above
+
+Conformance tests
+
+# From this directory
+
+TRANSCRIPT_URL=http://localhost:8000 \
+TRANSCRIPT_API_KEY=dev-key \
+uv run -m pytest -m model_api --no-cov ../../server/tests/test_model_api_transcript.py
+
+TRANSLATION_URL=http://localhost:8000 \
+TRANSLATION_API_KEY=dev-key \
+uv run -m pytest -m model_api --no-cov ../../server/tests/test_model_api_translation.py
+
+DIARIZATION_URL=http://localhost:8000 \
+DIARIZATION_API_KEY=dev-key \
+uv run -m pytest -m model_api --no-cov ../../server/tests/test_model_api_diarization.py

gpu/self_hosted/app/auth.py

@@ -0,0 +1,19 @@
+import os
+
+from fastapi import Depends, HTTPException, status
+from fastapi.security import OAuth2PasswordBearer
+
+oauth2_scheme = OAuth2PasswordBearer(tokenUrl="token")
+
+
+def apikey_auth(apikey: str = Depends(oauth2_scheme)):
+    required_key = os.environ.get("REFLECTOR_GPU_APIKEY")
+    if not required_key:
+        return
+    if apikey == required_key:
+        return
+    raise HTTPException(
+        status_code=status.HTTP_401_UNAUTHORIZED,
+        detail="Invalid API key",
+        headers={"WWW-Authenticate": "Bearer"},
+    )

gpu/self_hosted/app/config.py

@@ -0,0 +1,12 @@
+from pathlib import Path
+
+SUPPORTED_FILE_EXTENSIONS = ["mp3", "mp4", "mpeg", "mpga", "m4a", "wav", "webm"]
+SAMPLE_RATE = 16000
+VAD_CONFIG = {
+    "batch_max_duration": 30.0,
+    "silence_padding": 0.5,
+    "window_size": 512,
+}
+
+# App-level paths
+UPLOADS_PATH = Path("/tmp/whisper-uploads")

gpu/self_hosted/app/factory.py

@@ -0,0 +1,30 @@
+from contextlib import asynccontextmanager
+
+from fastapi import FastAPI
+
+from .routers.diarization import router as diarization_router
+from .routers.transcription import router as transcription_router
+from .routers.translation import router as translation_router
+from .services.transcriber import WhisperService
+from .services.diarizer import PyannoteDiarizationService
+from .utils import ensure_dirs
+
+
+@asynccontextmanager
+async def lifespan(app: FastAPI):
+    ensure_dirs()
+    whisper_service = WhisperService()
+    whisper_service.load()
+    app.state.whisper = whisper_service
+    diarization_service = PyannoteDiarizationService()
+    diarization_service.load()
+    app.state.diarizer = diarization_service
+    yield
+
+
+def create_app() -> FastAPI:
+    app = FastAPI(lifespan=lifespan)
+    app.include_router(transcription_router)
+    app.include_router(translation_router)
+    app.include_router(diarization_router)
+    return app

gpu/self_hosted/app/routers/diarization.py

@@ -0,0 +1,30 @@
+from typing import List
+
+from fastapi import APIRouter, Depends, Request
+from pydantic import BaseModel
+
+from ..auth import apikey_auth
+from ..services.diarizer import PyannoteDiarizationService
+from ..utils import download_audio_file
+
+router = APIRouter(tags=["diarization"])
+
+
+class DiarizationSegment(BaseModel):
+    start: float
+    end: float
+    speaker: int
+
+
+class DiarizationResponse(BaseModel):
+    diarization: List[DiarizationSegment]
+
+
+@router.post(
+    "/diarize", dependencies=[Depends(apikey_auth)], response_model=DiarizationResponse
+)
+def diarize(request: Request, audio_file_url: str, timestamp: float = 0.0):
+    with download_audio_file(audio_file_url) as (file_path, _ext):
+        file_path = str(file_path)
+        diarizer: PyannoteDiarizationService = request.app.state.diarizer
+        return diarizer.diarize_file(file_path, timestamp=timestamp)

gpu/self_hosted/app/routers/transcription.py

@@ -0,0 +1,109 @@
+import uuid
+from typing import Optional, Union
+
+from fastapi import APIRouter, Body, Depends, Form, HTTPException, Request, UploadFile
+from pydantic import BaseModel
+from pathlib import Path
+from ..auth import apikey_auth
+from ..config import SUPPORTED_FILE_EXTENSIONS, UPLOADS_PATH
+from ..services.transcriber import MODEL_NAME
+from ..utils import cleanup_uploaded_files, download_audio_file
+
+router = APIRouter(prefix="/v1/audio", tags=["transcription"])
+
+
+class WordTiming(BaseModel):
+    word: str
+    start: float
+    end: float
+
+
+class TranscriptResult(BaseModel):
+    text: str
+    words: list[WordTiming]
+    filename: Optional[str] = None
+
+
+class TranscriptBatchResponse(BaseModel):
+    results: list[TranscriptResult]
+
+
+@router.post(
+    "/transcriptions",
+    dependencies=[Depends(apikey_auth)],
+    response_model=Union[TranscriptResult, TranscriptBatchResponse],
+)
+def transcribe(
+    request: Request,
+    file: UploadFile = None,
+    files: list[UploadFile] | None = None,
+    model: str = Form(MODEL_NAME),
+    language: str = Form("en"),
+    batch: bool = Form(False),
+):
+    service = request.app.state.whisper
+    if not file and not files:
+        raise HTTPException(
+            status_code=400, detail="Either 'file' or 'files' parameter is required"
+        )
+    if batch and not files:
+        raise HTTPException(
+            status_code=400, detail="Batch transcription requires 'files'"
+        )
+
+    upload_files = [file] if file else files
+
+    uploaded_paths: list[Path] = []
+    with cleanup_uploaded_files(uploaded_paths):
+        for upload_file in upload_files:
+            audio_suffix = upload_file.filename.split(".")[-1].lower()
+            if audio_suffix not in SUPPORTED_FILE_EXTENSIONS:
+                raise HTTPException(
+                    status_code=400,
+                    detail=(
+                        f"Unsupported audio format. Supported extensions: {', '.join(SUPPORTED_FILE_EXTENSIONS)}"
+                    ),
+                )
+            unique_filename = f"{uuid.uuid4()}.{audio_suffix}"
+            file_path = UPLOADS_PATH / unique_filename
+            with open(file_path, "wb") as f:
+                content = upload_file.file.read()
+                f.write(content)
+            uploaded_paths.append(file_path)
+
+        if batch and len(upload_files) > 1:
+            results = []
+            for path in uploaded_paths:
+                result = service.transcribe_file(str(path), language=language)
+                result["filename"] = path.name
+                results.append(result)
+            return {"results": results}
+
+        results = []
+        for path in uploaded_paths:
+            result = service.transcribe_file(str(path), language=language)
+            result["filename"] = path.name
+            results.append(result)
+
+        return {"results": results} if len(results) > 1 else results[0]
+
+
+@router.post(
+    "/transcriptions-from-url",
+    dependencies=[Depends(apikey_auth)],
+    response_model=TranscriptResult,
+)
+def transcribe_from_url(
+    request: Request,
+    audio_file_url: str = Body(..., description="URL of the audio file to transcribe"),
+    model: str = Body(MODEL_NAME),
+    language: str = Body("en"),
+    timestamp_offset: float = Body(0.0),
+):
+    service = request.app.state.whisper
+    with download_audio_file(audio_file_url) as (file_path, _ext):
+        file_path = str(file_path)
+        result = service.transcribe_vad_url_segment(
+            file_path=file_path, timestamp_offset=timestamp_offset, language=language
+        )
+        return result

gpu/self_hosted/app/routers/translation.py

@@ -0,0 +1,28 @@
+from typing import Dict
+
+from fastapi import APIRouter, Body, Depends
+from pydantic import BaseModel
+
+from ..auth import apikey_auth
+from ..services.translator import TextTranslatorService
+
+router = APIRouter(tags=["translation"])
+
+translator = TextTranslatorService()
+
+
+class TranslationResponse(BaseModel):
+    text: Dict[str, str]
+
+
+@router.post(
+    "/translate",
+    dependencies=[Depends(apikey_auth)],
+    response_model=TranslationResponse,
+)
+def translate(
+    text: str,
+    source_language: str = Body("en"),
+    target_language: str = Body("fr"),
+):
+    return translator.translate(text, source_language, target_language)

gpu/self_hosted/app/services/diarizer.py

@@ -0,0 +1,42 @@
+import os
+import threading
+
+import torch
+import torchaudio
+from pyannote.audio import Pipeline
+
+
+class PyannoteDiarizationService:
+    def __init__(self):
+        self._pipeline = None
+        self._device = "cpu"
+        self._lock = threading.Lock()
+
+    def load(self):
+        self._device = "cuda" if torch.cuda.is_available() else "cpu"
+        self._pipeline = Pipeline.from_pretrained(
+            "pyannote/speaker-diarization-3.1",
+            use_auth_token=os.environ.get("HF_TOKEN"),
+        )
+        self._pipeline.to(torch.device(self._device))
+
+    def diarize_file(self, file_path: str, timestamp: float = 0.0) -> dict:
+        if self._pipeline is None:
+            self.load()
+        waveform, sample_rate = torchaudio.load(file_path)
+        with self._lock:
+            diarization = self._pipeline(
+                {"waveform": waveform, "sample_rate": sample_rate}
+            )
+        words = []
+        for diarization_segment, _, speaker in diarization.itertracks(yield_label=True):
+            words.append(
+                {
+                    "start": round(timestamp + diarization_segment.start, 3),
+                    "end": round(timestamp + diarization_segment.end, 3),
+                    "speaker": int(speaker[-2:])
+                    if speaker and speaker[-2:].isdigit()
+                    else 0,
+                }
+            )
+        return {"diarization": words}

gpu/self_hosted/app/services/transcriber.py

@@ -0,0 +1,208 @@
+import os
+import shutil
+import subprocess
+import threading
+from typing import Generator
+
+import faster_whisper
+import librosa
+import numpy as np
+import torch
+from fastapi import HTTPException
+from silero_vad import VADIterator, load_silero_vad
+
+from ..config import SAMPLE_RATE, VAD_CONFIG
+
+# Whisper configuration (service-local defaults)
+MODEL_NAME = "large-v2"
+# None delegates compute type to runtime: float16 on CUDA, int8 on CPU
+MODEL_COMPUTE_TYPE = None
+MODEL_NUM_WORKERS = 1
+CACHE_PATH = os.path.join(os.path.expanduser("~"), ".cache", "reflector-whisper")
+from ..utils import NoStdStreams
+
+
+class WhisperService:
+    def __init__(self):
+        self.model = None
+        self.device = "cpu"
+        self.lock = threading.Lock()
+
+    def load(self):
+        self.device = "cuda" if torch.cuda.is_available() else "cpu"
+        compute_type = MODEL_COMPUTE_TYPE or (
+            "float16" if self.device == "cuda" else "int8"
+        )
+        self.model = faster_whisper.WhisperModel(
+            MODEL_NAME,
+            device=self.device,
+            compute_type=compute_type,
+            num_workers=MODEL_NUM_WORKERS,
+            download_root=CACHE_PATH,
+        )
+
+    def pad_audio(self, audio_array, sample_rate: int = SAMPLE_RATE):
+        audio_duration = len(audio_array) / sample_rate
+        if audio_duration < VAD_CONFIG["silence_padding"]:
+            silence_samples = int(sample_rate * VAD_CONFIG["silence_padding"])
+            silence = np.zeros(silence_samples, dtype=np.float32)
+            return np.concatenate([audio_array, silence])
+        return audio_array
+
+    def enforce_word_timing_constraints(self, words: list[dict]) -> list[dict]:
+        if len(words) <= 1:
+            return words
+        enforced: list[dict] = []
+        for i, word in enumerate(words):
+            current = dict(word)
+            if i < len(words) - 1:
+                next_start = words[i + 1]["start"]
+                if current["end"] > next_start:
+                    current["end"] = next_start
+            enforced.append(current)
+        return enforced
+
+    def transcribe_file(self, file_path: str, language: str = "en") -> dict:
+        input_for_model: str | "object" = file_path
+        try:
+            audio_array, _sample_rate = librosa.load(
+                file_path, sr=SAMPLE_RATE, mono=True
+            )
+            if len(audio_array) / float(SAMPLE_RATE) < VAD_CONFIG["silence_padding"]:
+                input_for_model = self.pad_audio(audio_array, SAMPLE_RATE)
+        except Exception:
+            pass
+
+        with self.lock:
+            with NoStdStreams():
+                segments, _ = self.model.transcribe(
+                    input_for_model,
+                    language=language,
+                    beam_size=5,
+                    word_timestamps=True,
+                    vad_filter=True,
+                    vad_parameters={"min_silence_duration_ms": 500},
+                )
+
+        segments = list(segments)
+        text = "".join(segment.text for segment in segments).strip()
+        words = [
+            {
+                "word": word.word,
+                "start": round(float(word.start), 2),
+                "end": round(float(word.end), 2),
+            }
+            for segment in segments
+            for word in segment.words
+        ]
+        words = self.enforce_word_timing_constraints(words)
+        return {"text": text, "words": words}
+
+    def transcribe_vad_url_segment(
+        self, file_path: str, timestamp_offset: float = 0.0, language: str = "en"
+    ) -> dict:
+        def load_audio_via_ffmpeg(input_path: str, sample_rate: int) -> np.ndarray:
+            ffmpeg_bin = shutil.which("ffmpeg") or "ffmpeg"
+            cmd = [
+                ffmpeg_bin,
+                "-nostdin",
+                "-threads",
+                "1",
+                "-i",
+                input_path,
+                "-f",
+                "f32le",
+                "-acodec",
+                "pcm_f32le",
+                "-ac",
+                "1",
+                "-ar",
+                str(sample_rate),
+                "pipe:1",
+            ]
+            try:
+                proc = subprocess.run(
+                    cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, check=True
+                )
+            except Exception as e:
+                raise HTTPException(status_code=400, detail=f"ffmpeg failed: {e}")
+            audio = np.frombuffer(proc.stdout, dtype=np.float32)
+            return audio
+
+        def vad_segments(
+            audio_array,
+            sample_rate: int = SAMPLE_RATE,
+            window_size: int = VAD_CONFIG["window_size"],
+        ) -> Generator[tuple[float, float], None, None]:
+            vad_model = load_silero_vad(onnx=False)
+            iterator = VADIterator(vad_model, sampling_rate=sample_rate)
+            start = None
+            for i in range(0, len(audio_array), window_size):
+                chunk = audio_array[i : i + window_size]
+                if len(chunk) < window_size:
+                    chunk = np.pad(
+                        chunk, (0, window_size - len(chunk)), mode="constant"
+                    )
+                speech = iterator(chunk)
+                if not speech:
+                    continue
+                if "start" in speech:
+                    start = speech["start"]
+                    continue
+                if "end" in speech and start is not None:
+                    end = speech["end"]
+                    yield (start / float(SAMPLE_RATE), end / float(SAMPLE_RATE))
+                    start = None
+            iterator.reset_states()
+
+        audio_array = load_audio_via_ffmpeg(file_path, SAMPLE_RATE)
+
+        merged_batches: list[tuple[float, float]] = []
+        batch_start = None
+        batch_end = None
+        max_duration = VAD_CONFIG["batch_max_duration"]
+        for seg_start, seg_end in vad_segments(audio_array):
+            if batch_start is None:
+                batch_start, batch_end = seg_start, seg_end
+                continue
+            if seg_end - batch_start <= max_duration:
+                batch_end = seg_end
+            else:
+                merged_batches.append((batch_start, batch_end))
+                batch_start, batch_end = seg_start, seg_end
+        if batch_start is not None and batch_end is not None:
+            merged_batches.append((batch_start, batch_end))
+
+        all_text = []
+        all_words = []
+        for start_time, end_time in merged_batches:
+            s_idx = int(start_time * SAMPLE_RATE)
+            e_idx = int(end_time * SAMPLE_RATE)
+            segment = audio_array[s_idx:e_idx]
+            segment = self.pad_audio(segment, SAMPLE_RATE)
+            with self.lock:
+                segments, _ = self.model.transcribe(
+                    segment,
+                    language=language,
+                    beam_size=5,
+                    word_timestamps=True,
+                    vad_filter=True,
+                    vad_parameters={"min_silence_duration_ms": 500},
+                )
+            segments = list(segments)
+            text = "".join(seg.text for seg in segments).strip()
+            words = [
+                {
+                    "word": w.word,
+                    "start": round(float(w.start) + start_time + timestamp_offset, 2),
+                    "end": round(float(w.end) + start_time + timestamp_offset, 2),
+                }
+                for seg in segments
+                for w in seg.words
+            ]
+            if text:
+                all_text.append(text)
+            all_words.extend(words)
+
+        all_words = self.enforce_word_timing_constraints(all_words)
+        return {"text": " ".join(all_text), "words": all_words}

gpu/self_hosted/app/services/translator.py

@@ -0,0 +1,44 @@
+import threading
+
+from transformers import MarianMTModel, MarianTokenizer, pipeline
+
+
+class TextTranslatorService:
+    """Simple text-to-text translator using HuggingFace MarianMT models.
+
+    This mirrors the modal translator API shape but uses text translation only.
+    """
+
+    def __init__(self):
+        self._pipeline = None
+        self._lock = threading.Lock()
+
+    def load(self, source_language: str = "en", target_language: str = "fr"):
+        # Pick a default MarianMT model pair if available; fall back to Helsinki-NLP en->fr
+        model_name = self._resolve_model_name(source_language, target_language)
+        tokenizer = MarianTokenizer.from_pretrained(model_name)
+        model = MarianMTModel.from_pretrained(model_name)
+        self._pipeline = pipeline("translation", model=model, tokenizer=tokenizer)
+
+    def _resolve_model_name(self, src: str, tgt: str) -> str:
+        # Minimal mapping; extend as needed
+        pair = (src.lower(), tgt.lower())
+        mapping = {
+            ("en", "fr"): "Helsinki-NLP/opus-mt-en-fr",
+            ("fr", "en"): "Helsinki-NLP/opus-mt-fr-en",
+            ("en", "es"): "Helsinki-NLP/opus-mt-en-es",
+            ("es", "en"): "Helsinki-NLP/opus-mt-es-en",
+            ("en", "de"): "Helsinki-NLP/opus-mt-en-de",
+            ("de", "en"): "Helsinki-NLP/opus-mt-de-en",
+        }
+        return mapping.get(pair, "Helsinki-NLP/opus-mt-en-fr")
+
+    def translate(self, text: str, source_language: str, target_language: str) -> dict:
+        if self._pipeline is None:
+            self.load(source_language, target_language)
+        with self._lock:
+            results = self._pipeline(
+                text, src_lang=source_language, tgt_lang=target_language
+            )
+        translated = results[0]["translation_text"] if results else ""
+        return {"text": {source_language: text, target_language: translated}}

gpu/self_hosted/app/utils.py

@@ -0,0 +1,107 @@
+import logging
+import os
+import sys
+import uuid
+from contextlib import contextmanager
+from typing import Mapping
+from urllib.parse import urlparse
+from pathlib import Path
+
+import requests
+from fastapi import HTTPException
+
+from .config import SUPPORTED_FILE_EXTENSIONS, UPLOADS_PATH
+
+logger = logging.getLogger(__name__)
+
+
+class NoStdStreams:
+    def __init__(self):
+        self.devnull = open(os.devnull, "w")
+
+    def __enter__(self):
+        self._stdout, self._stderr = sys.stdout, sys.stderr
+        self._stdout.flush()
+        self._stderr.flush()
+        sys.stdout, sys.stderr = self.devnull, self.devnull
+
+    def __exit__(self, exc_type, exc_value, traceback):
+        sys.stdout, sys.stderr = self._stdout, self._stderr
+        self.devnull.close()
+
+
+def ensure_dirs():
+    UPLOADS_PATH.mkdir(parents=True, exist_ok=True)
+
+
+def detect_audio_format(url: str, headers: Mapping[str, str]) -> str:
+    url_path = urlparse(url).path
+    for ext in SUPPORTED_FILE_EXTENSIONS:
+        if url_path.lower().endswith(f".{ext}"):
+            return ext
+
+    content_type = headers.get("content-type", "").lower()
+    if "audio/mpeg" in content_type or "audio/mp3" in content_type:
+        return "mp3"
+    if "audio/wav" in content_type:
+        return "wav"
+    if "audio/mp4" in content_type:
+        return "mp4"
+
+    raise HTTPException(
+        status_code=400,
+        detail=(
+            f"Unsupported audio format for URL. Supported extensions: {', '.join(SUPPORTED_FILE_EXTENSIONS)}"
+        ),
+    )
+
+
+def download_audio_to_uploads(audio_file_url: str) -> tuple[Path, str]:
+    response = requests.head(audio_file_url, allow_redirects=True)
+    if response.status_code == 404:
+        raise HTTPException(status_code=404, detail="Audio file not found")
+
+    response = requests.get(audio_file_url, allow_redirects=True)
+    response.raise_for_status()
+
+    audio_suffix = detect_audio_format(audio_file_url, response.headers)
+    unique_filename = f"{uuid.uuid4()}.{audio_suffix}"
+    file_path: Path = UPLOADS_PATH / unique_filename
+
+    with open(file_path, "wb") as f:
+        f.write(response.content)
+
+    return file_path, audio_suffix
+
+
+@contextmanager
+def download_audio_file(audio_file_url: str):
+    """Download an audio file to UPLOADS_PATH and remove it after use.
+
+    Yields (file_path: Path, audio_suffix: str).
+    """
+    file_path, audio_suffix = download_audio_to_uploads(audio_file_url)
+    try:
+        yield file_path, audio_suffix
+    finally:
+        try:
+            file_path.unlink(missing_ok=True)
+        except Exception as e:
+            logger.error("Error deleting temporary file %s: %s", file_path, e)
+
+
+@contextmanager
+def cleanup_uploaded_files(file_paths: list[Path]):
+    """Ensure provided file paths are removed after use.
+
+    The provided list can be populated inside the context; all present entries
+    at exit will be deleted.
+    """
+    try:
+        yield file_paths
+    finally:
+        for path in list(file_paths):
+            try:
+                path.unlink(missing_ok=True)
+            except Exception as e:
+                logger.error("Error deleting temporary file %s: %s", path, e)

gpu/self_hosted/compose.yml

@@ -0,0 +1,10 @@
+services:
+  reflector_gpu:
+    build:
+      context: .
+    ports:
+      - "8000:8000"
+    env_file:
+      - .env
+    volumes:
+      - ./cache:/root/.cache

gpu/self_hosted/main.py

@@ -0,0 +1,3 @@
+from app.factory import create_app
+
+app = create_app()

gpu/self_hosted/pyproject.toml

@@ -0,0 +1,19 @@
+[project]
+name = "reflector-gpu"
+version = "0.1.0"
+description = "Self-hosted GPU service for speech transcription, diarization, and translation via FastAPI."
+readme = "README.md"
+requires-python = ">=3.12"
+dependencies = [
+    "fastapi[standard]>=0.116.1",
+    "uvicorn[standard]>=0.30.0",
+    "torch>=2.3.0",
+    "faster-whisper>=1.1.0",
+    "librosa==0.10.1",
+    "numpy<2",
+    "silero-vad==5.1.0",
+    "transformers>=4.35.0",
+    "sentencepiece",
+    "pyannote.audio==3.1.0",
+    "torchaudio>=2.3.0",
+]

gpu/self_hosted/runserver.sh

@@ -0,0 +1,17 @@
+#!/bin/sh
+set -e
+
+export PATH="/root/.local/bin:$PATH"
+cd /app
+
+# Install Python dependencies at runtime (first run or when FORCE_SYNC=1)
+if [ ! -d "/app/.venv" ] || [ "$FORCE_SYNC" = "1" ]; then
+  echo "[startup] Installing Python dependencies with uv..."
+  uv sync --compile-bytecode --locked
+else
+  echo "[startup] Using existing virtual environment at /app/.venv"
+fi
+
+exec uv run uvicorn main:app --host 0.0.0.0 --port 8000
+
+

server/docs/data_retention.md

@@ -0,0 +1,95 @@
+# Data Retention and Cleanup
+
+## Overview
+
+For public instances of Reflector, a data retention policy is automatically enforced to delete anonymous user data after a configurable period (default: 7 days). This ensures compliance with privacy expectations and prevents unbounded storage growth.
+
+## Configuration
+
+### Environment Variables
+
+- `PUBLIC_MODE` (bool): Must be set to `true` to enable automatic cleanup
+- `PUBLIC_DATA_RETENTION_DAYS` (int): Number of days to retain anonymous data (default: 7)
+
+### What Gets Deleted
+
+When data reaches the retention period, the following items are automatically removed:
+
+1. **Transcripts** from anonymous users (where `user_id` is NULL):
+   - Database records
+   - Local files (audio.wav, audio.mp3, audio.json waveform)
+   - Storage files (cloud storage if configured)
+
+## Automatic Cleanup
+
+### Celery Beat Schedule
+
+When `PUBLIC_MODE=true`, a Celery beat task runs daily at 3 AM to clean up old data:
+
+```python
+# Automatically scheduled when PUBLIC_MODE=true
+"cleanup_old_public_data": {
+    "task": "reflector.worker.cleanup.cleanup_old_public_data",
+    "schedule": crontab(hour=3, minute=0),  # Daily at 3 AM
+}
+```
+
+### Running the Worker
+
+Ensure both Celery worker and beat scheduler are running:
+
+```bash
+# Start Celery worker
+uv run celery -A reflector.worker.app worker --loglevel=info
+
+# Start Celery beat scheduler (in another terminal)
+uv run celery -A reflector.worker.app beat
+```
+
+## Manual Cleanup
+
+For testing or manual intervention, use the cleanup tool:
+
+```bash
+# Delete data older than 7 days (default)
+uv run python -m reflector.tools.cleanup_old_data
+
+# Delete data older than 30 days
+uv run python -m reflector.tools.cleanup_old_data --days 30
+```
+
+Note: The manual tool uses the same implementation as the Celery worker task to ensure consistency.
+
+## Important Notes
+
+1. **User Data Deletion**: Only anonymous data (where `user_id` is NULL) is deleted. Authenticated user data is preserved.
+
+2. **Storage Cleanup**: The system properly cleans up both local files and cloud storage when configured.
+
+3. **Error Handling**: If individual deletions fail, the cleanup continues and logs errors. Failed deletions are reported in the task output.
+
+4. **Public Instance Only**: The automatic cleanup task only runs when `PUBLIC_MODE=true` to prevent accidental data loss in private deployments.
+
+## Testing
+
+Run the cleanup tests:
+
+```bash
+uv run pytest tests/test_cleanup.py -v
+```
+
+## Monitoring
+
+Check Celery logs for cleanup task execution:
+
+```bash
+# Look for cleanup task logs
+grep "cleanup_old_public_data" celery.log
+grep "Starting cleanup of old public data" celery.log
+```
+
+Task statistics are logged after each run:
+- Number of transcripts deleted
+- Number of meetings deleted
+- Number of orphaned recordings deleted
+- Any errors encountered

server/docs/gpu/api-transcription.md

@@ -0,0 +1,194 @@
+## Reflector GPU Transcription API (Specification)
+
+This document defines the Reflector GPU transcription API that all implementations must adhere to. Current implementations include NVIDIA Parakeet (NeMo) and Whisper (faster-whisper), both deployed on Modal.com. The API surface and response shapes are OpenAI/Whisper-compatible, so clients can switch implementations by changing only the base URL.
+
+### Base URL and Authentication
+
+- Example base URLs (Modal web endpoints):
+
+  - Parakeet: `https://<account>--reflector-transcriber-parakeet-web.modal.run`
+  - Whisper: `https://<account>--reflector-transcriber-web.modal.run`
+
+- All endpoints are served under `/v1` and require a Bearer token:
+
+```
+Authorization: Bearer <REFLECTOR_GPU_APIKEY>
+```
+
+Note: To switch implementations, deploy the desired variant and point `TRANSCRIPT_URL` to its base URL. The API is identical.
+
+### Supported file types
+
+`mp3, mp4, mpeg, mpga, m4a, wav, webm`
+
+### Models and languages
+
+- Parakeet (NVIDIA NeMo): default `nvidia/parakeet-tdt-0.6b-v2`
+  - Language support: only `en`. Other languages return HTTP 400.
+- Whisper (faster-whisper): default `large-v2` (or deployment-specific)
+  - Language support: multilingual (per Whisper model capabilities).
+
+Note: The `model` parameter is accepted by all implementations for interface parity. Some backends may treat it as informational.
+
+### Endpoints
+
+#### POST /v1/audio/transcriptions
+
+Transcribe one or more uploaded audio files.
+
+Request: multipart/form-data
+
+- `file` (File) — optional. Single file to transcribe.
+- `files` (File[]) — optional. One or more files to transcribe.
+- `model` (string) — optional. Defaults to the implementation-specific model (see above).
+- `language` (string) — optional, defaults to `en`.
+  - Parakeet: only `en` is accepted; other values return HTTP 400
+  - Whisper: model-dependent; typically multilingual
+- `batch` (boolean) — optional, defaults to `false`.
+
+Notes:
+
+- Provide either `file` or `files`, not both. If neither is provided, HTTP 400.
+- `batch` requires `files`; using `batch=true` without `files` returns HTTP 400.
+- Response shape for multiple files is the same regardless of `batch`.
+- Files sent to this endpoint are processed in a single pass (no VAD/chunking). This is intended for short clips (roughly ≤ 30s; depends on GPU memory/model). For longer audio, prefer `/v1/audio/transcriptions-from-url` which supports VAD-based chunking.
+
+Responses
+
+Single file response:
+
+```json
+{
+  "text": "transcribed text",
+  "words": [
+    { "word": "hello", "start": 0.0, "end": 0.5 },
+    { "word": "world", "start": 0.5, "end": 1.0 }
+  ],
+  "filename": "audio.mp3"
+}
+```
+
+Multiple files response:
+
+```json
+{
+  "results": [
+    {"filename": "a1.mp3", "text": "...", "words": [...]},
+    {"filename": "a2.mp3", "text": "...", "words": [...]}]
+}
+```
+
+Notes:
+
+- Word objects always include keys: `word`, `start`, `end`.
+- Some implementations may include a trailing space in `word` to match Whisper tokenization behavior; clients should trim if needed.
+
+Example curl (single file):
+
+```bash
+curl -X POST \
+  -H "Authorization: Bearer $REFLECTOR_GPU_APIKEY" \
+  -F "file=@/path/to/audio.mp3" \
+  -F "language=en" \
+  "$BASE_URL/v1/audio/transcriptions"
+```
+
+Example curl (multiple files, batch):
+
+```bash
+curl -X POST \
+  -H "Authorization: Bearer $REFLECTOR_GPU_APIKEY" \
+  -F "files=@/path/a1.mp3" -F "files=@/path/a2.mp3" \
+  -F "batch=true" -F "language=en" \
+  "$BASE_URL/v1/audio/transcriptions"
+```
+
+#### POST /v1/audio/transcriptions-from-url
+
+Transcribe a single remote audio file by URL.
+
+Request: application/json
+
+Body parameters:
+
+- `audio_file_url` (string) — required. URL of the audio file to transcribe.
+- `model` (string) — optional. Defaults to the implementation-specific model (see above).
+- `language` (string) — optional, defaults to `en`. Parakeet only accepts `en`.
+- `timestamp_offset` (number) — optional, defaults to `0.0`. Added to each word's `start`/`end` in the response.
+
+```json
+{
+  "audio_file_url": "https://example.com/audio.mp3",
+  "model": "nvidia/parakeet-tdt-0.6b-v2",
+  "language": "en",
+  "timestamp_offset": 0.0
+}
+```
+
+Response:
+
+```json
+{
+  "text": "transcribed text",
+  "words": [
+    { "word": "hello", "start": 10.0, "end": 10.5 },
+    { "word": "world", "start": 10.5, "end": 11.0 }
+  ]
+}
+```
+
+Notes:
+
+- `timestamp_offset` is added to each word’s `start`/`end` in the response.
+- Implementations may perform VAD-based chunking and batching for long-form audio; word timings are adjusted accordingly.
+
+Example curl:
+
+```bash
+curl -X POST \
+  -H "Authorization: Bearer $REFLECTOR_GPU_APIKEY" \
+  -H "Content-Type: application/json" \
+  -d '{
+        "audio_file_url": "https://example.com/audio.mp3",
+        "language": "en",
+        "timestamp_offset": 0
+      }' \
+  "$BASE_URL/v1/audio/transcriptions-from-url"
+```
+
+### Error handling
+
+- 400 Bad Request
+  - Parakeet: `language` other than `en`
+  - Missing required parameters (`file`/`files` for upload; `audio_file_url` for URL endpoint)
+  - Unsupported file extension
+- 401 Unauthorized
+  - Missing or invalid Bearer token
+- 404 Not Found
+  - `audio_file_url` does not exist
+
+### Implementation details
+
+- GPUs: A10G for small-file/live, L40S for large-file URL transcription (subject to deployment)
+- VAD chunking and segment batching; word timings adjusted and overlapping ends constrained
+- Pads very short segments (< 0.5s) to avoid model crashes on some backends
+
+### Server configuration (Reflector API)
+
+Set the Reflector server to use the Modal backend and point `TRANSCRIPT_URL` to your chosen deployment:
+
+```
+TRANSCRIPT_BACKEND=modal
+TRANSCRIPT_URL=https://<account>--reflector-transcriber-parakeet-web.modal.run
+TRANSCRIPT_MODAL_API_KEY=<REFLECTOR_GPU_APIKEY>
+```
+
+### Conformance tests
+
+Use the pytest-based conformance tests to validate any new implementation (including self-hosted) against this spec:
+
+```
+TRANSCRIPT_URL=https://<your-deployment-base> \
+TRANSCRIPT_MODAL_API_KEY=your-api-key \
+uv run -m pytest -m model_api --no-cov server/tests/test_model_api_transcript.py
+```

server/docs/webhook.md

@@ -0,0 +1,212 @@
+# Reflector Webhook Documentation
+
+## Overview
+
+Reflector supports webhook notifications to notify external systems when transcript processing is completed. Webhooks can be configured per room and are triggered automatically after a transcript is successfully processed.
+
+## Configuration
+
+Webhooks are configured at the room level with two fields:
+- `webhook_url`: The HTTPS endpoint to receive webhook notifications
+- `webhook_secret`: Optional secret key for HMAC signature verification (auto-generated if not provided)
+
+## Events
+
+### `transcript.completed`
+
+Triggered when a transcript has been fully processed, including transcription, diarization, summarization, and topic detection.
+
+### `test`
+
+A test event that can be triggered manually to verify webhook configuration.
+
+## Webhook Request Format
+
+### Headers
+
+All webhook requests include the following headers:
+
+| Header | Description | Example |
+|--------|-------------|---------|
+| `Content-Type` | Always `application/json` | `application/json` |
+| `User-Agent` | Identifies Reflector as the source | `Reflector-Webhook/1.0` |
+| `X-Webhook-Event` | The event type | `transcript.completed` or `test` |
+| `X-Webhook-Retry` | Current retry attempt number | `0`, `1`, `2`... |
+| `X-Webhook-Signature` | HMAC signature (if secret configured) | `t=1735306800,v1=abc123...` |
+
+### Signature Verification
+
+If a webhook secret is configured, Reflector includes an HMAC-SHA256 signature in the `X-Webhook-Signature` header to verify the webhook authenticity.
+
+The signature format is: `t={timestamp},v1={signature}`
+
+To verify the signature:
+1. Extract the timestamp and signature from the header
+2. Create the signed payload: `{timestamp}.{request_body}`
+3. Compute HMAC-SHA256 of the signed payload using your webhook secret
+4. Compare the computed signature with the received signature
+
+Example verification (Python):
+```python
+import hmac
+import hashlib
+
+def verify_webhook_signature(payload: bytes, signature_header: str, secret: str) -> bool:
+    # Parse header: "t=1735306800,v1=abc123..."
+    parts = dict(part.split("=") for part in signature_header.split(","))
+    timestamp = parts["t"]
+    received_signature = parts["v1"]
+
+    # Create signed payload
+    signed_payload = f"{timestamp}.{payload.decode('utf-8')}"
+
+    # Compute expected signature
+    expected_signature = hmac.new(
+        secret.encode("utf-8"),
+        signed_payload.encode("utf-8"),
+        hashlib.sha256
+    ).hexdigest()
+
+    # Compare signatures
+    return hmac.compare_digest(expected_signature, received_signature)
+```
+
+## Event Payloads
+
+### `transcript.completed` Event
+
+This event includes a convenient URL for accessing the transcript:
+- `frontend_url`: Direct link to view the transcript in the web interface
+
+```json
+{
+  "event": "transcript.completed",
+  "event_id": "transcript.completed-abc-123-def-456",
+  "timestamp": "2025-08-27T12:34:56.789012Z",
+  "transcript": {
+    "id": "abc-123-def-456",
+    "room_id": "room-789",
+    "created_at": "2025-08-27T12:00:00Z",
+    "duration": 1800.5,
+    "title": "Q3 Product Planning Meeting",
+    "short_summary": "Team discussed Q3 product roadmap, prioritizing mobile app features and API improvements.",
+    "long_summary": "The product team met to finalize the Q3 roadmap. Key decisions included...",
+    "webvtt": "WEBVTT\n\n00:00:00.000 --> 00:00:05.000\n<v Speaker 1>Welcome everyone to today's meeting...",
+    "topics": [
+      {
+        "title": "Introduction and Agenda",
+        "summary": "Meeting kickoff with agenda review",
+        "timestamp": 0.0,
+        "duration": 120.0,
+        "webvtt": "WEBVTT\n\n00:00:00.000 --> 00:00:05.000\n<v Speaker 1>Welcome everyone..."
+      },
+      {
+        "title": "Mobile App Features Discussion",
+        "summary": "Team reviewed proposed mobile app features for Q3",
+        "timestamp": 120.0,
+        "duration": 600.0,
+        "webvtt": "WEBVTT\n\n00:02:00.000 --> 00:02:10.000\n<v Speaker 2>Let's talk about the mobile app..."
+      }
+    ],
+    "participants": [
+      {
+        "id": "participant-1",
+        "name": "John Doe",
+        "speaker": "Speaker 1"
+      },
+      {
+        "id": "participant-2",
+        "name": "Jane Smith",
+        "speaker": "Speaker 2"
+      }
+    ],
+    "source_language": "en",
+    "target_language": "en",
+    "status": "completed",
+    "frontend_url": "https://app.reflector.com/transcripts/abc-123-def-456"
+  },
+  "room": {
+    "id": "room-789",
+    "name": "Product Team Room"
+  }
+}
+```
+
+### `test` Event
+
+```json
+{
+  "event": "test",
+  "event_id": "test.2025-08-27T12:34:56.789012Z",
+  "timestamp": "2025-08-27T12:34:56.789012Z",
+  "message": "This is a test webhook from Reflector",
+  "room": {
+    "id": "room-789",
+    "name": "Product Team Room"
+  }
+}
+```
+
+## Retry Policy
+
+Webhooks are delivered with automatic retry logic to handle transient failures. When a webhook delivery fails due to server errors or network issues, Reflector will automatically retry the delivery multiple times over an extended period.
+
+### Retry Mechanism
+
+Reflector implements an exponential backoff strategy for webhook retries:
+
+- **Initial retry delay**: 60 seconds after the first failure
+- **Exponential backoff**: Each subsequent retry waits approximately twice as long as the previous one
+- **Maximum retry interval**: 1 hour (backoff is capped at this duration)
+- **Maximum retry attempts**: 30 attempts total
+- **Total retry duration**: Retries continue for approximately 24 hours
+
+### How Retries Work
+
+When a webhook fails, Reflector will:
+1. Wait 60 seconds, then retry (attempt #1)
+2. If it fails again, wait ~2 minutes, then retry (attempt #2)
+3. Continue doubling the wait time up to a maximum of 1 hour between attempts
+4. Keep retrying at 1-hour intervals until successful or 30 attempts are exhausted
+
+The `X-Webhook-Retry` header indicates the current retry attempt number (0 for the initial attempt, 1 for first retry, etc.), allowing your endpoint to track retry attempts.
+
+### Retry Behavior by HTTP Status Code
+
+| Status Code | Behavior |
+|-------------|----------|
+| 2xx (Success) | No retry, webhook marked as delivered |
+| 4xx (Client Error) | No retry, request is considered permanently failed |
+| 5xx (Server Error) | Automatic retry with exponential backoff |
+| Network/Timeout Error | Automatic retry with exponential backoff |
+
+**Important Notes:**
+- Webhooks timeout after 30 seconds. If your endpoint takes longer to respond, it will be considered a timeout error and retried.
+- During the retry period (~24 hours), you may receive the same webhook multiple times if your endpoint experiences intermittent failures.
+- There is no mechanism to manually retry failed webhooks after the retry period expires.
+
+## Testing Webhooks
+
+You can test your webhook configuration before processing transcripts:
+
+```http
+POST /v1/rooms/{room_id}/webhook/test
+```
+
+Response:
+```json
+{
+  "success": true,
+  "status_code": 200,
+  "message": "Webhook test successful",
+  "response_preview": "OK"
+}
+```
+
+Or in case of failure:
+```json
+{
+  "success": false,
+  "error": "Webhook request timed out (10 seconds)"
+}
+```

server/gpu/modal_deployments/reflector_transcriber.py

@@ -1,161 +0,0 @@
-import os
-import tempfile
-import threading
-
-import modal
-from pydantic import BaseModel
-
-MODELS_DIR = "/models"
-
-MODEL_NAME = "large-v2"
-MODEL_COMPUTE_TYPE: str = "float16"
-MODEL_NUM_WORKERS: int = 1
-
-MINUTES = 60  # seconds
-
-volume = modal.Volume.from_name("models", create_if_missing=True)
-
-app = modal.App("reflector-transcriber")
-
-
-def download_model():
-    from faster_whisper import download_model
-
-    volume.reload()
-
-    download_model(MODEL_NAME, cache_dir=MODELS_DIR)
-
-    volume.commit()
-
-
-image = (
-    modal.Image.debian_slim(python_version="3.12")
-    .pip_install(
-        "huggingface_hub==0.27.1",
-        "hf-transfer==0.1.9",
-        "torch==2.5.1",
-        "faster-whisper==1.1.1",
-    )
-    .env(
-        {
-            "HF_HUB_ENABLE_HF_TRANSFER": "1",
-            "LD_LIBRARY_PATH": (
-                "/usr/local/lib/python3.12/site-packages/nvidia/cudnn/lib/:"
-                "/opt/conda/lib/python3.12/site-packages/nvidia/cublas/lib/"
-            ),
-        }
-    )
-    .run_function(download_model, volumes={MODELS_DIR: volume})
-)
-
-
-@app.cls(
-    gpu="A10G",
-    timeout=5 * MINUTES,
-    scaledown_window=5 * MINUTES,
-    allow_concurrent_inputs=6,
-    image=image,
-    volumes={MODELS_DIR: volume},
-)
-class Transcriber:
-    @modal.enter()
-    def enter(self):
-        import faster_whisper
-        import torch
-
-        self.lock = threading.Lock()
-        self.use_gpu = torch.cuda.is_available()
-        self.device = "cuda" if self.use_gpu else "cpu"
-        self.model = faster_whisper.WhisperModel(
-            MODEL_NAME,
-            device=self.device,
-            compute_type=MODEL_COMPUTE_TYPE,
-            num_workers=MODEL_NUM_WORKERS,
-            download_root=MODELS_DIR,
-            local_files_only=True,
-        )
-
-    @modal.method()
-    def transcribe_segment(
-        self,
-        audio_data: str,
-        audio_suffix: str,
-        language: str,
-    ):
-        with tempfile.NamedTemporaryFile("wb+", suffix=f".{audio_suffix}") as fp:
-            fp.write(audio_data)
-
-            with self.lock:
-                segments, _ = self.model.transcribe(
-                    fp.name,
-                    language=language,
-                    beam_size=5,
-                    word_timestamps=True,
-                    vad_filter=True,
-                    vad_parameters={"min_silence_duration_ms": 500},
-                )
-
-            segments = list(segments)
-            text = "".join(segment.text for segment in segments)
-            words = [
-                {"word": word.word, "start": word.start, "end": word.end}
-                for segment in segments
-                for word in segment.words
-            ]
-
-            return {"text": text, "words": words}
-
-
-@app.function(
-    scaledown_window=60,
-    timeout=60,
-    allow_concurrent_inputs=40,
-    secrets=[
-        modal.Secret.from_name("reflector-gpu"),
-    ],
-    volumes={MODELS_DIR: volume},
-)
-@modal.asgi_app()
-def web():
-    from fastapi import Body, Depends, FastAPI, HTTPException, UploadFile, status
-    from fastapi.security import OAuth2PasswordBearer
-    from typing_extensions import Annotated
-
-    transcriber = Transcriber()
-
-    app = FastAPI()
-
-    oauth2_scheme = OAuth2PasswordBearer(tokenUrl="token")
-
-    supported_file_types = ["mp3", "mp4", "mpeg", "mpga", "m4a", "wav", "webm"]
-
-    def apikey_auth(apikey: str = Depends(oauth2_scheme)):
-        if apikey != os.environ["REFLECTOR_GPU_APIKEY"]:
-            raise HTTPException(
-                status_code=status.HTTP_401_UNAUTHORIZED,
-                detail="Invalid API key",
-                headers={"WWW-Authenticate": "Bearer"},
-            )
-
-    class TranscriptResponse(BaseModel):
-        result: dict
-
-    @app.post("/v1/audio/transcriptions", dependencies=[Depends(apikey_auth)])
-    def transcribe(
-        file: UploadFile,
-        model: str = "whisper-1",
-        language: Annotated[str, Body(...)] = "en",
-    ) -> TranscriptResponse:
-        audio_data = file.file.read()
-        audio_suffix = file.filename.split(".")[-1]
-        assert audio_suffix in supported_file_types
-
-        func = transcriber.transcribe_segment.spawn(
-            audio_data=audio_data,
-            audio_suffix=audio_suffix,
-            language=language,
-        )
-        result = func.get()
-        return result
-
-    return app

server/migrations/versions/0194f65cd6d3_add_webhook_fields_to_rooms.py

@@ -0,0 +1,36 @@
+"""Add webhook fields to rooms
+
+Revision ID: 0194f65cd6d3
+Revises: 5a8907fd1d78
+Create Date: 2025-08-27 09:03:19.610995
+
+"""
+
+from typing import Sequence, Union
+
+import sqlalchemy as sa
+from alembic import op
+
+# revision identifiers, used by Alembic.
+revision: str = "0194f65cd6d3"
+down_revision: Union[str, None] = "5a8907fd1d78"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    # ### commands auto generated by Alembic - please adjust! ###
+    with op.batch_alter_table("room", schema=None) as batch_op:
+        batch_op.add_column(sa.Column("webhook_url", sa.String(), nullable=True))
+        batch_op.add_column(sa.Column("webhook_secret", sa.String(), nullable=True))
+
+    # ### end Alembic commands ###
+
+
+def downgrade() -> None:
+    # ### commands auto generated by Alembic - please adjust! ###
+    with op.batch_alter_table("room", schema=None) as batch_op:
+        batch_op.drop_column("webhook_secret")
+        batch_op.drop_column("webhook_url")
+
+    # ### end Alembic commands ###

server/migrations/versions/0ce521cda2ee_remove_user_id_from_meeting_table.py

@@ -0,0 +1,36 @@
+"""remove user_id from meeting table
+
+Revision ID: 0ce521cda2ee
+Revises: 6dec9fb5b46c
+Create Date: 2025-09-10 12:40:55.688899
+
+"""
+
+from typing import Sequence, Union
+
+import sqlalchemy as sa
+from alembic import op
+
+# revision identifiers, used by Alembic.
+revision: str = "0ce521cda2ee"
+down_revision: Union[str, None] = "6dec9fb5b46c"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    # ### commands auto generated by Alembic - please adjust! ###
+    with op.batch_alter_table("meeting", schema=None) as batch_op:
+        batch_op.drop_column("user_id")
+
+    # ### end Alembic commands ###
+
+
+def downgrade() -> None:
+    # ### commands auto generated by Alembic - please adjust! ###
+    with op.batch_alter_table("meeting", schema=None) as batch_op:
+        batch_op.add_column(
+            sa.Column("user_id", sa.VARCHAR(), autoincrement=False, nullable=True)
+        )
+
+    # ### end Alembic commands ###

server/migrations/versions/2ae3db106d4e_clean_up_orphaned_room_id_references_in_.py

@@ -0,0 +1,32 @@
+"""clean up orphaned room_id references in meeting table
+
+Revision ID: 2ae3db106d4e
+Revises: def1b5867d4c
+Create Date: 2025-09-11 10:35:15.759967
+
+"""
+
+from typing import Sequence, Union
+
+from alembic import op
+
+# revision identifiers, used by Alembic.
+revision: str = "2ae3db106d4e"
+down_revision: Union[str, None] = "def1b5867d4c"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    # Set room_id to NULL for meetings that reference non-existent rooms
+    op.execute("""
+        UPDATE meeting
+        SET room_id = NULL
+        WHERE room_id IS NOT NULL
+          AND room_id NOT IN (SELECT id FROM room WHERE id IS NOT NULL)
+    """)
+
+
+def downgrade() -> None:
+    # Cannot restore orphaned references - no operation needed
+    pass

server/migrations/versions/5a8907fd1d78_add_cascade_delete_to_meeting_consent_.py

@@ -0,0 +1,50 @@
+"""add cascade delete to meeting consent foreign key
+
+Revision ID: 5a8907fd1d78
+Revises: 0ab2d7ffaa16
+Create Date: 2025-08-26 17:26:50.945491
+
+"""
+
+from typing import Sequence, Union
+
+from alembic import op
+
+# revision identifiers, used by Alembic.
+revision: str = "5a8907fd1d78"
+down_revision: Union[str, None] = "0ab2d7ffaa16"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    # ### commands auto generated by Alembic - please adjust! ###
+    with op.batch_alter_table("meeting_consent", schema=None) as batch_op:
+        batch_op.drop_constraint(
+            batch_op.f("meeting_consent_meeting_id_fkey"), type_="foreignkey"
+        )
+        batch_op.create_foreign_key(
+            batch_op.f("meeting_consent_meeting_id_fkey"),
+            "meeting",
+            ["meeting_id"],
+            ["id"],
+            ondelete="CASCADE",
+        )
+
+    # ### end Alembic commands ###
+
+
+def downgrade() -> None:
+    # ### commands auto generated by Alembic - please adjust! ###
+    with op.batch_alter_table("meeting_consent", schema=None) as batch_op:
+        batch_op.drop_constraint(
+            batch_op.f("meeting_consent_meeting_id_fkey"), type_="foreignkey"
+        )
+        batch_op.create_foreign_key(
+            batch_op.f("meeting_consent_meeting_id_fkey"),
+            "meeting",
+            ["meeting_id"],
+            ["id"],
+        )
+
+    # ### end Alembic commands ###

server/migrations/versions/6025e9b2bef2_remove_one_active_meeting_per_room_.py

@@ -0,0 +1,53 @@
+"""remove_one_active_meeting_per_room_constraint
+
+Revision ID: 6025e9b2bef2
+Revises: 2ae3db106d4e
+Create Date: 2025-08-18 18:45:44.418392
+
+"""
+
+from typing import Sequence, Union
+
+import sqlalchemy as sa
+from alembic import op
+
+# revision identifiers, used by Alembic.
+revision: str = "6025e9b2bef2"
+down_revision: Union[str, None] = "2ae3db106d4e"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    # Remove the unique constraint that prevents multiple active meetings per room
+    # This is needed to support calendar integration with overlapping meetings
+    # Check if index exists before trying to drop it
+    from alembic import context
+
+    if context.get_context().dialect.name == "postgresql":
+        conn = op.get_bind()
+        result = conn.execute(
+            sa.text(
+                "SELECT 1 FROM pg_indexes WHERE indexname = 'idx_one_active_meeting_per_room'"
+            )
+        )
+        if result.fetchone():
+            op.drop_index("idx_one_active_meeting_per_room", table_name="meeting")
+    else:
+        # For SQLite, just try to drop it
+        try:
+            op.drop_index("idx_one_active_meeting_per_room", table_name="meeting")
+        except:
+            pass
+
+
+def downgrade() -> None:
+    # Restore the unique constraint
+    op.create_index(
+        "idx_one_active_meeting_per_room",
+        "meeting",
+        ["room_id"],
+        unique=True,
+        postgresql_where=sa.text("is_active = true"),
+        sqlite_where=sa.text("is_active = 1"),
+    )

server/migrations/versions/61882a919591_webhook_url_and_secret_null_by_default.py

@@ -0,0 +1,28 @@
+"""webhook url and secret null by default
+
+
+Revision ID: 61882a919591
+Revises: 0194f65cd6d3
+Create Date: 2025-08-29 11:46:36.738091
+
+"""
+
+from typing import Sequence, Union
+
+# revision identifiers, used by Alembic.
+revision: str = "61882a919591"
+down_revision: Union[str, None] = "0194f65cd6d3"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    # ### commands auto generated by Alembic - please adjust! ###
+    pass
+    # ### end Alembic commands ###
+
+
+def downgrade() -> None:
+    # ### commands auto generated by Alembic - please adjust! ###
+    pass
+    # ### end Alembic commands ###

server/migrations/versions/6dec9fb5b46c_make_meeting_room_id_required_and_add_.py

@@ -0,0 +1,35 @@
+"""make meeting room_id required and add foreign key
+
+Revision ID: 6dec9fb5b46c
+Revises: 61882a919591
+Create Date: 2025-09-10 10:47:06.006819
+
+"""
+
+from typing import Sequence, Union
+
+from alembic import op
+
+# revision identifiers, used by Alembic.
+revision: str = "6dec9fb5b46c"
+down_revision: Union[str, None] = "61882a919591"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    # ### commands auto generated by Alembic - please adjust! ###
+    with op.batch_alter_table("meeting", schema=None) as batch_op:
+        batch_op.create_foreign_key(
+            None, "room", ["room_id"], ["id"], ondelete="CASCADE"
+        )
+
+    # ### end Alembic commands ###
+
+
+def downgrade() -> None:
+    # ### commands auto generated by Alembic - please adjust! ###
+    with op.batch_alter_table("meeting", schema=None) as batch_op:
+        batch_op.drop_constraint("meeting_room_id_fkey", type_="foreignkey")
+
+    # ### end Alembic commands ###

server/migrations/versions/d4a1c446458c_add_grace_period_fields_to_meeting.py

@@ -0,0 +1,34 @@
+"""add_grace_period_fields_to_meeting
+
+Revision ID: d4a1c446458c
+Revises: 6025e9b2bef2
+Create Date: 2025-08-18 18:50:37.768052
+
+"""
+
+from typing import Sequence, Union
+
+import sqlalchemy as sa
+from alembic import op
+
+# revision identifiers, used by Alembic.
+revision: str = "d4a1c446458c"
+down_revision: Union[str, None] = "6025e9b2bef2"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    # Add fields to track when participants left for grace period logic
+    op.add_column(
+        "meeting", sa.Column("last_participant_left_at", sa.DateTime(timezone=True))
+    )
+    op.add_column(
+        "meeting",
+        sa.Column("grace_period_minutes", sa.Integer, server_default=sa.text("15")),
+    )
+
+
+def downgrade() -> None:
+    op.drop_column("meeting", "grace_period_minutes")
+    op.drop_column("meeting", "last_participant_left_at")

server/migrations/versions/d8e204bbf615_add_calendar.py

@@ -0,0 +1,129 @@
+"""add calendar
+
+Revision ID: d8e204bbf615
+Revises: d4a1c446458c
+Create Date: 2025-09-10 19:56:22.295756
+
+"""
+
+from typing import Sequence, Union
+
+import sqlalchemy as sa
+from alembic import op
+from sqlalchemy.dialects import postgresql
+
+# revision identifiers, used by Alembic.
+revision: str = "d8e204bbf615"
+down_revision: Union[str, None] = "d4a1c446458c"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    # ### commands auto generated by Alembic - please adjust! ###
+    op.create_table(
+        "calendar_event",
+        sa.Column("id", sa.String(), nullable=False),
+        sa.Column("room_id", sa.String(), nullable=False),
+        sa.Column("ics_uid", sa.Text(), nullable=False),
+        sa.Column("title", sa.Text(), nullable=True),
+        sa.Column("description", sa.Text(), nullable=True),
+        sa.Column("start_time", sa.DateTime(timezone=True), nullable=False),
+        sa.Column("end_time", sa.DateTime(timezone=True), nullable=False),
+        sa.Column("attendees", postgresql.JSONB(astext_type=sa.Text()), nullable=True),
+        sa.Column("location", sa.Text(), nullable=True),
+        sa.Column("ics_raw_data", sa.Text(), nullable=True),
+        sa.Column("last_synced", sa.DateTime(timezone=True), nullable=False),
+        sa.Column(
+            "is_deleted", sa.Boolean(), server_default=sa.text("false"), nullable=False
+        ),
+        sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
+        sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
+        sa.ForeignKeyConstraint(
+            ["room_id"],
+            ["room.id"],
+            name="fk_calendar_event_room_id",
+            ondelete="CASCADE",
+        ),
+        sa.PrimaryKeyConstraint("id"),
+        sa.UniqueConstraint("room_id", "ics_uid", name="uq_room_calendar_event"),
+    )
+    with op.batch_alter_table("calendar_event", schema=None) as batch_op:
+        batch_op.create_index(
+            "idx_calendar_event_deleted",
+            ["is_deleted"],
+            unique=False,
+            postgresql_where=sa.text("NOT is_deleted"),
+        )
+        batch_op.create_index(
+            "idx_calendar_event_room_start", ["room_id", "start_time"], unique=False
+        )
+
+    with op.batch_alter_table("meeting", schema=None) as batch_op:
+        batch_op.add_column(sa.Column("calendar_event_id", sa.String(), nullable=True))
+        batch_op.add_column(
+            sa.Column(
+                "calendar_metadata",
+                postgresql.JSONB(astext_type=sa.Text()),
+                nullable=True,
+            )
+        )
+        batch_op.create_index(
+            "idx_meeting_calendar_event", ["calendar_event_id"], unique=False
+        )
+        batch_op.create_foreign_key(
+            "fk_meeting_calendar_event_id",
+            "calendar_event",
+            ["calendar_event_id"],
+            ["id"],
+            ondelete="SET NULL",
+        )
+
+    with op.batch_alter_table("room", schema=None) as batch_op:
+        batch_op.add_column(sa.Column("ics_url", sa.Text(), nullable=True))
+        batch_op.add_column(
+            sa.Column(
+                "ics_fetch_interval", sa.Integer(), server_default="300", nullable=True
+            )
+        )
+        batch_op.add_column(
+            sa.Column(
+                "ics_enabled",
+                sa.Boolean(),
+                server_default=sa.text("false"),
+                nullable=False,
+            )
+        )
+        batch_op.add_column(
+            sa.Column("ics_last_sync", sa.DateTime(timezone=True), nullable=True)
+        )
+        batch_op.add_column(sa.Column("ics_last_etag", sa.Text(), nullable=True))
+        batch_op.create_index("idx_room_ics_enabled", ["ics_enabled"], unique=False)
+
+    # ### end Alembic commands ###
+
+
+def downgrade() -> None:
+    # ### commands auto generated by Alembic - please adjust! ###
+    with op.batch_alter_table("room", schema=None) as batch_op:
+        batch_op.drop_index("idx_room_ics_enabled")
+        batch_op.drop_column("ics_last_etag")
+        batch_op.drop_column("ics_last_sync")
+        batch_op.drop_column("ics_enabled")
+        batch_op.drop_column("ics_fetch_interval")
+        batch_op.drop_column("ics_url")
+
+    with op.batch_alter_table("meeting", schema=None) as batch_op:
+        batch_op.drop_constraint("fk_meeting_calendar_event_id", type_="foreignkey")
+        batch_op.drop_index("idx_meeting_calendar_event")
+        batch_op.drop_column("calendar_metadata")
+        batch_op.drop_column("calendar_event_id")
+
+    with op.batch_alter_table("calendar_event", schema=None) as batch_op:
+        batch_op.drop_index("idx_calendar_event_room_start")
+        batch_op.drop_index(
+            "idx_calendar_event_deleted", postgresql_where=sa.text("NOT is_deleted")
+        )
+
+    op.drop_table("calendar_event")
+    # ### end Alembic commands ###

server/migrations/versions/dc035ff72fd5_remove_grace_period_fields.py

@@ -0,0 +1,43 @@
+"""remove_grace_period_fields
+
+Revision ID: dc035ff72fd5
+Revises: d8e204bbf615
+Create Date: 2025-09-11 10:36:45.197588
+
+"""
+
+from typing import Sequence, Union
+
+import sqlalchemy as sa
+from alembic import op
+
+# revision identifiers, used by Alembic.
+revision: str = "dc035ff72fd5"
+down_revision: Union[str, None] = "d8e204bbf615"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    # Remove grace period columns from meeting table
+    op.drop_column("meeting", "last_participant_left_at")
+    op.drop_column("meeting", "grace_period_minutes")
+
+
+def downgrade() -> None:
+    # Add back grace period columns to meeting table
+    op.add_column(
+        "meeting",
+        sa.Column(
+            "last_participant_left_at", sa.DateTime(timezone=True), nullable=True
+        ),
+    )
+    op.add_column(
+        "meeting",
+        sa.Column(
+            "grace_period_minutes",
+            sa.Integer(),
+            server_default=sa.text("15"),
+            nullable=True,
+        ),
+    )

server/migrations/versions/def1b5867d4c_make_meeting_room_id_nullable_but_keep_.py

@@ -0,0 +1,34 @@
+"""make meeting room_id nullable but keep foreign key
+
+Revision ID: def1b5867d4c
+Revises: 0ce521cda2ee
+Create Date: 2025-09-11 09:42:18.697264
+
+"""
+
+from typing import Sequence, Union
+
+import sqlalchemy as sa
+from alembic import op
+
+# revision identifiers, used by Alembic.
+revision: str = "def1b5867d4c"
+down_revision: Union[str, None] = "0ce521cda2ee"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    # ### commands auto generated by Alembic - please adjust! ###
+    with op.batch_alter_table("meeting", schema=None) as batch_op:
+        batch_op.alter_column("room_id", existing_type=sa.VARCHAR(), nullable=True)
+
+    # ### end Alembic commands ###
+
+
+def downgrade() -> None:
+    # ### commands auto generated by Alembic - please adjust! ###
+    with op.batch_alter_table("meeting", schema=None) as batch_op:
+        batch_op.alter_column("room_id", existing_type=sa.VARCHAR(), nullable=False)
+
+    # ### end Alembic commands ###

server/pyproject.toml

@@ -12,7 +12,6 @@ dependencies = [
     "requests>=2.31.0",
     "aiortc>=1.5.0",
     "sortedcontainers>=2.4.0",
-    "loguru>=0.7.0",
     "pydantic-settings>=2.0.2",
     "structlog>=23.1.0",
     "uvicorn[standard]>=0.23.1",
@@ -27,7 +26,6 @@ dependencies = [
     "prometheus-fastapi-instrumentator>=6.1.0",
     "sentencepiece>=0.1.99",
     "protobuf>=4.24.3",
-    "profanityfilter>=2.0.6",
     "celery>=5.3.4",
     "redis>=5.0.1",
     "python-jose[cryptography]>=3.3.0",
@@ -40,6 +38,7 @@ dependencies = [
     "llama-index-llms-openai-like>=0.4.0",
     "pytest-env>=1.1.5",
     "webvtt-py>=0.5.0",
+    "icalendar>=6.0.0",
 ]
 
 [dependency-groups]
@@ -119,7 +118,7 @@ addopts = "-ra -q --disable-pytest-warnings --cov --cov-report html -v"
 testpaths = ["tests"]
 asyncio_mode = "auto"
 markers = [
-    "gpu_modal: mark test to run only with GPU Modal endpoints (deselect with '-m \"not gpu_modal\"')",
+    "model_api: tests for the unified model-serving HTTP API (backend- and hardware-agnostic)",
 ]
 
 [tool.ruff.lint]
@@ -131,7 +130,7 @@ select = [
 
 [tool.ruff.lint.per-file-ignores]
 "reflector/processors/summary/summary_builder.py" = ["E501"]
-"gpu/**.py" = ["PLC0415"]
+"gpu/modal_deployments/**.py" = ["PLC0415"]
 "reflector/tools/**.py" = ["PLC0415"]
 "migrations/versions/**.py" = ["PLC0415"]
 "tests/**.py" = ["PLC0415"]

server/reflector/asynctask.py

@@ -0,0 +1,27 @@
+import asyncio
+import functools
+
+from reflector.db import get_database
+
+
+def asynctask(f):
+    @functools.wraps(f)
+    def wrapper(*args, **kwargs):
+        async def run_with_db():
+            database = get_database()
+            await database.connect()
+            try:
+                return await f(*args, **kwargs)
+            finally:
+                await database.disconnect()
+
+        coro = run_with_db()
+        try:
+            loop = asyncio.get_running_loop()
+        except RuntimeError:
+            loop = None
+        if loop and loop.is_running():
+            return loop.run_until_complete(coro)
+        return asyncio.run(coro)
+
+    return wrapper

server/reflector/auth/auth_jwt.py

@@ -67,7 +67,8 @@ def current_user(
     try:
         payload = jwtauth.verify_token(token)
         sub = payload["sub"]
-        return UserInfo(sub=sub)
+        email = payload["email"]
+        return UserInfo(sub=sub, email=email)
     except JWTError as e:
         logger.error(f"JWT error: {e}")
         raise HTTPException(status_code=401, detail="Invalid authentication")

server/reflector/db/__init__.py

@@ -24,6 +24,7 @@ def get_database() -> databases.Database:
 
 
 # import models
+import reflector.db.calendar_events  # noqa
 import reflector.db.meetings  # noqa
 import reflector.db.recordings  # noqa
 import reflector.db.rooms  # noqa

server/reflector/db/calendar_events.py

@@ -0,0 +1,182 @@
+from datetime import datetime, timedelta, timezone
+from typing import Any
+
+import sqlalchemy as sa
+from pydantic import BaseModel, Field
+from sqlalchemy.dialects.postgresql import JSONB
+
+from reflector.db import get_database, metadata
+from reflector.utils import generate_uuid4
+
+calendar_events = sa.Table(
+    "calendar_event",
+    metadata,
+    sa.Column("id", sa.String, primary_key=True),
+    sa.Column(
+        "room_id",
+        sa.String,
+        sa.ForeignKey("room.id", ondelete="CASCADE", name="fk_calendar_event_room_id"),
+        nullable=False,
+    ),
+    sa.Column("ics_uid", sa.Text, nullable=False),
+    sa.Column("title", sa.Text),
+    sa.Column("description", sa.Text),
+    sa.Column("start_time", sa.DateTime(timezone=True), nullable=False),
+    sa.Column("end_time", sa.DateTime(timezone=True), nullable=False),
+    sa.Column("attendees", JSONB),
+    sa.Column("location", sa.Text),
+    sa.Column("ics_raw_data", sa.Text),
+    sa.Column("last_synced", sa.DateTime(timezone=True), nullable=False),
+    sa.Column("is_deleted", sa.Boolean, nullable=False, server_default=sa.false()),
+    sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
+    sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
+    sa.UniqueConstraint("room_id", "ics_uid", name="uq_room_calendar_event"),
+    sa.Index("idx_calendar_event_room_start", "room_id", "start_time"),
+    sa.Index(
+        "idx_calendar_event_deleted",
+        "is_deleted",
+        postgresql_where=sa.text("NOT is_deleted"),
+    ),
+)
+
+
+class CalendarEvent(BaseModel):
+    id: str = Field(default_factory=generate_uuid4)
+    room_id: str
+    ics_uid: str
+    title: str | None = None
+    description: str | None = None
+    start_time: datetime
+    end_time: datetime
+    attendees: list[dict[str, Any]] | None = None
+    location: str | None = None
+    ics_raw_data: str | None = None
+    last_synced: datetime = Field(default_factory=lambda: datetime.now(timezone.utc))
+    is_deleted: bool = False
+    created_at: datetime = Field(default_factory=lambda: datetime.now(timezone.utc))
+    updated_at: datetime = Field(default_factory=lambda: datetime.now(timezone.utc))
+
+
+class CalendarEventController:
+    async def get_by_room(
+        self,
+        room_id: str,
+        include_deleted: bool = False,
+        start_after: datetime | None = None,
+        end_before: datetime | None = None,
+    ) -> list[CalendarEvent]:
+        query = calendar_events.select().where(calendar_events.c.room_id == room_id)
+
+        if not include_deleted:
+            query = query.where(calendar_events.c.is_deleted == False)
+
+        if start_after:
+            query = query.where(calendar_events.c.start_time >= start_after)
+
+        if end_before:
+            query = query.where(calendar_events.c.end_time <= end_before)
+
+        query = query.order_by(calendar_events.c.start_time.asc())
+
+        results = await get_database().fetch_all(query)
+        return [CalendarEvent(**result) for result in results]
+
+    async def get_upcoming(
+        self, room_id: str, minutes_ahead: int = 120
+    ) -> list[CalendarEvent]:
+        """Get upcoming events for a room within the specified minutes, including currently happening events."""
+        now = datetime.now(timezone.utc)
+        future_time = now + timedelta(minutes=minutes_ahead)
+
+        query = (
+            calendar_events.select()
+            .where(
+                sa.and_(
+                    calendar_events.c.room_id == room_id,
+                    calendar_events.c.is_deleted == False,
+                    calendar_events.c.start_time <= future_time,
+                    calendar_events.c.end_time >= now,
+                )
+            )
+            .order_by(calendar_events.c.start_time.asc())
+        )
+
+        results = await get_database().fetch_all(query)
+        return [CalendarEvent(**result) for result in results]
+
+    async def get_by_ics_uid(self, room_id: str, ics_uid: str) -> CalendarEvent | None:
+        query = calendar_events.select().where(
+            sa.and_(
+                calendar_events.c.room_id == room_id,
+                calendar_events.c.ics_uid == ics_uid,
+            )
+        )
+        result = await get_database().fetch_one(query)
+        return CalendarEvent(**result) if result else None
+
+    async def upsert(self, event: CalendarEvent) -> CalendarEvent:
+        existing = await self.get_by_ics_uid(event.room_id, event.ics_uid)
+
+        if existing:
+            event.id = existing.id
+            event.created_at = existing.created_at
+            event.updated_at = datetime.now(timezone.utc)
+
+            query = (
+                calendar_events.update()
+                .where(calendar_events.c.id == existing.id)
+                .values(**event.model_dump())
+            )
+        else:
+            query = calendar_events.insert().values(**event.model_dump())
+
+        await get_database().execute(query)
+        return event
+
+    async def soft_delete_missing(
+        self, room_id: str, current_ics_uids: list[str]
+    ) -> int:
+        """Soft delete future events that are no longer in the calendar."""
+        now = datetime.now(timezone.utc)
+
+        select_query = calendar_events.select().where(
+            sa.and_(
+                calendar_events.c.room_id == room_id,
+                calendar_events.c.start_time > now,
+                calendar_events.c.is_deleted == False,
+                calendar_events.c.ics_uid.notin_(current_ics_uids)
+                if current_ics_uids
+                else True,
+            )
+        )
+
+        to_delete = await get_database().fetch_all(select_query)
+        delete_count = len(to_delete)
+
+        if delete_count > 0:
+            update_query = (
+                calendar_events.update()
+                .where(
+                    sa.and_(
+                        calendar_events.c.room_id == room_id,
+                        calendar_events.c.start_time > now,
+                        calendar_events.c.is_deleted == False,
+                        calendar_events.c.ics_uid.notin_(current_ics_uids)
+                        if current_ics_uids
+                        else True,
+                    )
+                )
+                .values(is_deleted=True, updated_at=now)
+            )
+
+            await get_database().execute(update_query)
+
+        return delete_count
+
+    async def delete_by_room(self, room_id: str) -> int:
+        query = calendar_events.delete().where(calendar_events.c.room_id == room_id)
+        result = await get_database().execute(query)
+        return result.rowcount
+
+
+calendar_events_controller = CalendarEventController()

server/reflector/db/meetings.py

@@ -1,9 +1,9 @@
 from datetime import datetime
-from typing import Literal
+from typing import Any, Literal
 
 import sqlalchemy as sa
-from fastapi import HTTPException
 from pydantic import BaseModel, Field
+from sqlalchemy.dialects.postgresql import JSONB
 
 from reflector.db import get_database, metadata
 from reflector.db.rooms import Room
@@ -18,8 +18,12 @@ meetings = sa.Table(
     sa.Column("host_room_url", sa.String),
     sa.Column("start_date", sa.DateTime(timezone=True)),
     sa.Column("end_date", sa.DateTime(timezone=True)),
-    sa.Column("user_id", sa.String),
-    sa.Column("room_id", sa.String),
+    sa.Column(
+        "room_id",
+        sa.String,
+        sa.ForeignKey("room.id", ondelete="CASCADE"),
+        nullable=True,
+    ),
     sa.Column("is_locked", sa.Boolean, nullable=False, server_default=sa.false()),
     sa.Column("room_mode", sa.String, nullable=False, server_default="normal"),
     sa.Column("recording_type", sa.String, nullable=False, server_default="cloud"),
@@ -41,20 +45,30 @@ meetings = sa.Table(
         nullable=False,
         server_default=sa.true(),
     ),
-    sa.Index("idx_meeting_room_id", "room_id"),
-    sa.Index(
-        "idx_one_active_meeting_per_room",
-        "room_id",
-        unique=True,
-        postgresql_where=sa.text("is_active = true"),
+    sa.Column(
+        "calendar_event_id",
+        sa.String,
+        sa.ForeignKey(
+            "calendar_event.id",
+            ondelete="SET NULL",
+            name="fk_meeting_calendar_event_id",
+        ),
     ),
+    sa.Column("calendar_metadata", JSONB),
+    sa.Index("idx_meeting_room_id", "room_id"),
+    sa.Index("idx_meeting_calendar_event", "calendar_event_id"),
 )
 
 meeting_consent = sa.Table(
     "meeting_consent",
     metadata,
     sa.Column("id", sa.String, primary_key=True),
-    sa.Column("meeting_id", sa.String, sa.ForeignKey("meeting.id"), nullable=False),
+    sa.Column(
+        "meeting_id",
+        sa.String,
+        sa.ForeignKey("meeting.id", ondelete="CASCADE"),
+        nullable=False,
+    ),
     sa.Column("user_id", sa.String),
     sa.Column("consent_given", sa.Boolean, nullable=False),
     sa.Column("consent_timestamp", sa.DateTime(timezone=True), nullable=False),
@@ -76,8 +90,7 @@ class Meeting(BaseModel):
     host_room_url: str
     start_date: datetime
     end_date: datetime
-    user_id: str | None = None
-    room_id: str | None = None
+    room_id: str | None
     is_locked: bool = False
     room_mode: Literal["normal", "group"] = "normal"
     recording_type: Literal["none", "local", "cloud"] = "cloud"
@@ -85,6 +98,9 @@ class Meeting(BaseModel):
         "none", "prompt", "automatic", "automatic-2nd-participant"
     ] = "automatic-2nd-participant"
     num_clients: int = 0
+    is_active: bool = True
+    calendar_event_id: str | None = None
+    calendar_metadata: dict[str, Any] | None = None
 
 
 class MeetingController:
@@ -96,12 +112,10 @@ class MeetingController:
         host_room_url: str,
         start_date: datetime,
         end_date: datetime,
-        user_id: str,
         room: Room,
+        calendar_event_id: str | None = None,
+        calendar_metadata: dict[str, Any] | None = None,
     ):
-        """
-        Create a new meeting
-        """
         meeting = Meeting(
             id=id,
             room_name=room_name,
@@ -109,41 +123,46 @@ class MeetingController:
             host_room_url=host_room_url,
             start_date=start_date,
             end_date=end_date,
-            user_id=user_id,
             room_id=room.id,
             is_locked=room.is_locked,
             room_mode=room.room_mode,
             recording_type=room.recording_type,
             recording_trigger=room.recording_trigger,
+            calendar_event_id=calendar_event_id,
+            calendar_metadata=calendar_metadata,
         )
         query = meetings.insert().values(**meeting.model_dump())
         await get_database().execute(query)
         return meeting
 
     async def get_all_active(self) -> list[Meeting]:
-        """
-        Get active meetings.
-        """
         query = meetings.select().where(meetings.c.is_active)
         return await get_database().fetch_all(query)
 
     async def get_by_room_name(
         self,
         room_name: str,
-    ) -> Meeting:
+    ) -> Meeting | None:
         """
         Get a meeting by room name.
+        For backward compatibility, returns the most recent meeting.
         """
-        query = meetings.select().where(meetings.c.room_name == room_name)
+        end_date = getattr(meetings.c, "end_date")
+        query = (
+            meetings.select()
+            .where(meetings.c.room_name == room_name)
+            .order_by(end_date.desc())
+        )
         result = await get_database().fetch_one(query)
         if not result:
             return None
 
         return Meeting(**result)
 
-    async def get_active(self, room: Room, current_time: datetime) -> Meeting:
+    async def get_active(self, room: Room, current_time: datetime) -> Meeting | None:
         """
         Get latest active meeting for a room.
+        For backward compatibility, returns the most recent active meeting.
         """
         end_date = getattr(meetings.c, "end_date")
         query = (
@@ -163,32 +182,58 @@ class MeetingController:
 
         return Meeting(**result)
 
+    async def get_all_active_for_room(
+        self, room: Room, current_time: datetime
+    ) -> list[Meeting]:
+        end_date = getattr(meetings.c, "end_date")
+        query = (
+            meetings.select()
+            .where(
+                sa.and_(
+                    meetings.c.room_id == room.id,
+                    meetings.c.end_date > current_time,
+                    meetings.c.is_active,
+                )
+            )
+            .order_by(end_date.desc())
+        )
+        results = await get_database().fetch_all(query)
+        return [Meeting(**result) for result in results]
+
+    async def get_active_by_calendar_event(
+        self, room: Room, calendar_event_id: str, current_time: datetime
+    ) -> Meeting | None:
+        """
+        Get active meeting for a specific calendar event.
+        """
+        query = meetings.select().where(
+            sa.and_(
+                meetings.c.room_id == room.id,
+                meetings.c.calendar_event_id == calendar_event_id,
+                meetings.c.end_date > current_time,
+                meetings.c.is_active,
+            )
+        )
+        result = await get_database().fetch_one(query)
+        if not result:
+            return None
+        return Meeting(**result)
+
     async def get_by_id(self, meeting_id: str, **kwargs) -> Meeting | None:
-        """
-        Get a meeting by id
-        """
         query = meetings.select().where(meetings.c.id == meeting_id)
         result = await get_database().fetch_one(query)
         if not result:
             return None
         return Meeting(**result)
 
-    async def get_by_id_for_http(self, meeting_id: str, user_id: str | None) -> Meeting:
-        """
-        Get a meeting by ID for HTTP request.
-
-        If not found, it will raise a 404 error.
-        """
-        query = meetings.select().where(meetings.c.id == meeting_id)
+    async def get_by_calendar_event(self, calendar_event_id: str) -> Meeting | None:
+        query = meetings.select().where(
+            meetings.c.calendar_event_id == calendar_event_id
+        )
         result = await get_database().fetch_one(query)
         if not result:
-            raise HTTPException(status_code=404, detail="Meeting not found")
-
-        meeting = Meeting(**result)
-        if result["user_id"] != user_id:
-            meeting.host_room_url = ""
-
-        return meeting
+            return None
+        return Meeting(**result)
 
     async def update_meeting(self, meeting_id: str, **kwargs):
         query = meetings.update().where(meetings.c.id == meeting_id).values(**kwargs)
@@ -214,10 +259,9 @@ class MeetingConsentController:
         result = await get_database().fetch_one(query)
         if result is None:
             return None
-        return MeetingConsent(**result) if result else None
+        return MeetingConsent(**result)
 
     async def upsert(self, consent: MeetingConsent) -> MeetingConsent:
-        """Create new consent or update existing one for authenticated users"""
         if consent.user_id:
             # For authenticated users, check if consent already exists
             # not transactional but we're ok with that; the consents ain't deleted anyways

server/reflector/db/rooms.py

@@ -1,3 +1,4 @@
+import secrets
 from datetime import datetime, timezone
 from sqlite3 import IntegrityError
 from typing import Literal
@@ -40,7 +41,17 @@ rooms = sqlalchemy.Table(
     sqlalchemy.Column(
         "is_shared", sqlalchemy.Boolean, nullable=False, server_default=false()
     ),
+    sqlalchemy.Column("webhook_url", sqlalchemy.String, nullable=True),
+    sqlalchemy.Column("webhook_secret", sqlalchemy.String, nullable=True),
+    sqlalchemy.Column("ics_url", sqlalchemy.Text),
+    sqlalchemy.Column("ics_fetch_interval", sqlalchemy.Integer, server_default="300"),
+    sqlalchemy.Column(
+        "ics_enabled", sqlalchemy.Boolean, nullable=False, server_default=false()
+    ),
+    sqlalchemy.Column("ics_last_sync", sqlalchemy.DateTime(timezone=True)),
+    sqlalchemy.Column("ics_last_etag", sqlalchemy.Text),
     sqlalchemy.Index("idx_room_is_shared", "is_shared"),
+    sqlalchemy.Index("idx_room_ics_enabled", "ics_enabled"),
 )
 
 
@@ -59,6 +70,13 @@ class Room(BaseModel):
         "none", "prompt", "automatic", "automatic-2nd-participant"
     ] = "automatic-2nd-participant"
     is_shared: bool = False
+    webhook_url: str | None = None
+    webhook_secret: str | None = None
+    ics_url: str | None = None
+    ics_fetch_interval: int = 300
+    ics_enabled: bool = False
+    ics_last_sync: datetime | None = None
+    ics_last_etag: str | None = None
 
 
 class RoomController:
@@ -107,10 +125,18 @@ class RoomController:
         recording_type: str,
         recording_trigger: str,
         is_shared: bool,
+        webhook_url: str = "",
+        webhook_secret: str = "",
+        ics_url: str | None = None,
+        ics_fetch_interval: int = 300,
+        ics_enabled: bool = False,
     ):
         """
         Add a new room
         """
+        if webhook_url and not webhook_secret:
+            webhook_secret = secrets.token_urlsafe(32)
+
         room = Room(
             name=name,
             user_id=user_id,
@@ -122,6 +148,11 @@ class RoomController:
             recording_type=recording_type,
             recording_trigger=recording_trigger,
             is_shared=is_shared,
+            webhook_url=webhook_url,
+            webhook_secret=webhook_secret,
+            ics_url=ics_url,
+            ics_fetch_interval=ics_fetch_interval,
+            ics_enabled=ics_enabled,
         )
         query = rooms.insert().values(**room.model_dump())
         try:
@@ -134,6 +165,9 @@ class RoomController:
         """
         Update a room fields with key/values in values
         """
+        if values.get("webhook_url") and not values.get("webhook_secret"):
+            values["webhook_secret"] = secrets.token_urlsafe(32)
+
         query = rooms.update().where(rooms.c.id == room.id).values(**values)
         try:
             await get_database().execute(query)
@@ -183,6 +217,13 @@ class RoomController:
 
         return room
 
+    async def get_ics_enabled(self) -> list[Room]:
+        query = rooms.select().where(
+            rooms.c.ics_enabled == True, rooms.c.ics_url != None
+        )
+        results = await get_database().fetch_all(query)
+        return [Room(**result) for result in results]
+
     async def remove_by_id(
         self,
         room_id: str,

server/reflector/db/search.py

@@ -8,12 +8,14 @@ from typing import Annotated, Any, Dict, Iterator
 
 import sqlalchemy
 import webvtt
+from databases.interfaces import Record as DbRecord
 from fastapi import HTTPException
 from pydantic import (
     BaseModel,
     Field,
     NonNegativeFloat,
     NonNegativeInt,
+    TypeAdapter,
     ValidationError,
     constr,
     field_serializer,
@@ -21,9 +23,10 @@ from pydantic import (
 
 from reflector.db import get_database
 from reflector.db.rooms import rooms
-from reflector.db.transcripts import SourceKind, transcripts
+from reflector.db.transcripts import SourceKind, TranscriptStatus, transcripts
 from reflector.db.utils import is_postgresql
 from reflector.logger import logger
+from reflector.utils.string import NonEmptyString, try_parse_non_empty_string
 
 DEFAULT_SEARCH_LIMIT = 20
 SNIPPET_CONTEXT_LENGTH = 50  # Characters before/after match to include
@@ -31,12 +34,13 @@ DEFAULT_SNIPPET_MAX_LENGTH = NonNegativeInt(150)
 DEFAULT_MAX_SNIPPETS = NonNegativeInt(3)
 LONG_SUMMARY_MAX_SNIPPETS = 2
 
-SearchQueryBase = constr(min_length=0, strip_whitespace=True)
+SearchQueryBase = constr(min_length=1, strip_whitespace=True)
 SearchLimitBase = Annotated[int, Field(ge=1, le=100)]
 SearchOffsetBase = Annotated[int, Field(ge=0)]
 SearchTotalBase = Annotated[int, Field(ge=0)]
 
 SearchQuery = Annotated[SearchQueryBase, Field(description="Search query text")]
+search_query_adapter = TypeAdapter(SearchQuery)
 SearchLimit = Annotated[SearchLimitBase, Field(description="Results per page")]
 SearchOffset = Annotated[
     SearchOffsetBase, Field(description="Number of results to skip")
@@ -88,7 +92,7 @@ class WebVTTProcessor:
     @staticmethod
     def generate_snippets(
         webvtt_content: WebVTTContent,
-        query: str,
+        query: SearchQuery,
         max_snippets: NonNegativeInt = DEFAULT_MAX_SNIPPETS,
     ) -> list[str]:
         """Generate snippets from WebVTT content."""
@@ -125,7 +129,7 @@ class SnippetCandidate:
 class SearchParameters(BaseModel):
     """Validated search parameters for full-text search."""
 
-    query_text: SearchQuery
+    query_text: SearchQuery | None = None
     limit: SearchLimit = DEFAULT_SEARCH_LIMIT
     offset: SearchOffset = 0
     user_id: str | None = None
@@ -157,7 +161,7 @@ class SearchResult(BaseModel):
     room_name: str | None = None
     source_kind: SourceKind
     created_at: datetime
-    status: str = Field(..., min_length=1)
+    status: TranscriptStatus = Field(..., min_length=1)
     rank: float = Field(..., ge=0, le=1)
     duration: NonNegativeFloat | None = Field(..., description="Duration in seconds")
     search_snippets: list[str] = Field(
@@ -199,15 +203,13 @@ class SnippetGenerator:
             prev_start = start
 
     @staticmethod
-    def count_matches(text: str, query: str) -> NonNegativeInt:
+    def count_matches(text: str, query: SearchQuery) -> NonNegativeInt:
         """Count total number of matches for a query in text."""
         ZERO = NonNegativeInt(0)
         if not text:
             logger.warning("Empty text for search query in count_matches")
             return ZERO
-        if not query:
-            logger.warning("Empty query for search text in count_matches")
-            return ZERO
+        assert query is not None
         return NonNegativeInt(
             sum(1 for _ in SnippetGenerator.find_all_matches(text, query))
         )
@@ -243,13 +245,14 @@ class SnippetGenerator:
     @staticmethod
     def generate(
         text: str,
-        query: str,
+        query: SearchQuery,
         max_length: NonNegativeInt = DEFAULT_SNIPPET_MAX_LENGTH,
         max_snippets: NonNegativeInt = DEFAULT_MAX_SNIPPETS,
     ) -> list[str]:
         """Generate snippets from text."""
-        if not text or not query:
-            logger.warning("Empty text or query for generate_snippets")
+        assert query is not None
+        if not text:
+            logger.warning("Empty text for generate_snippets")
             return []
 
         candidates = (
@@ -270,7 +273,7 @@ class SnippetGenerator:
     @staticmethod
     def from_summary(
         summary: str,
-        query: str,
+        query: SearchQuery,
         max_snippets: NonNegativeInt = LONG_SUMMARY_MAX_SNIPPETS,
     ) -> list[str]:
         """Generate snippets from summary text."""
@@ -278,9 +281,9 @@ class SnippetGenerator:
 
     @staticmethod
     def combine_sources(
-        summary: str | None,
+        summary: NonEmptyString | None,
         webvtt: WebVTTContent | None,
-        query: str,
+        query: SearchQuery,
         max_total: NonNegativeInt = DEFAULT_MAX_SNIPPETS,
     ) -> tuple[list[str], NonNegativeInt]:
         """Combine snippets from multiple sources and return total match count.
@@ -289,6 +292,11 @@ class SnippetGenerator:
 
         snippets can be empty for real in case of e.g. title match
         """
+
+        assert (
+            summary is not None or webvtt is not None
+        ), "At least one source must be present"
+
         webvtt_matches = 0
         summary_matches = 0
 
@@ -355,8 +363,8 @@ class SearchController:
                 else_=rooms.c.name,
             ).label("room_name"),
         ]
-
-        if params.query_text:
+        search_query = None
+        if params.query_text is not None:
             search_query = sqlalchemy.func.websearch_to_tsquery(
                 "english", params.query_text
             )
@@ -373,7 +381,9 @@ class SearchController:
             transcripts.join(rooms, transcripts.c.room_id == rooms.c.id, isouter=True)
         )
 
-        if params.query_text:
+        if params.query_text is not None:
+            # because already initialized based on params.query_text presence above
+            assert search_query is not None
             base_query = base_query.where(
                 transcripts.c.search_vector_en.op("@@")(search_query)
             )
@@ -393,7 +403,7 @@ class SearchController:
                 transcripts.c.source_kind == params.source_kind
             )
 
-        if params.query_text:
+        if params.query_text is not None:
             order_by = sqlalchemy.desc(sqlalchemy.text("rank"))
         else:
             order_by = sqlalchemy.desc(transcripts.c.created_at)
@@ -407,19 +417,29 @@ class SearchController:
         )
         total = await get_database().fetch_val(count_query)
 
-        def _process_result(r) -> SearchResult:
+        def _process_result(r: DbRecord) -> SearchResult:
             r_dict: Dict[str, Any] = dict(r)
+
             webvtt_raw: str | None = r_dict.pop("webvtt", None)
+            webvtt: WebVTTContent | None
             if webvtt_raw:
                 webvtt = WebVTTProcessor.parse(webvtt_raw)
             else:
                 webvtt = None
-            long_summary: str | None = r_dict.pop("long_summary", None)
+
+            long_summary_r: str | None = r_dict.pop("long_summary", None)
+            long_summary: NonEmptyString = try_parse_non_empty_string(long_summary_r)
             room_name: str | None = r_dict.pop("room_name", None)
             db_result = SearchResultDB.model_validate(r_dict)
 
-            snippets, total_match_count = SnippetGenerator.combine_sources(
-                long_summary, webvtt, params.query_text, DEFAULT_MAX_SNIPPETS
+            at_least_one_source = webvtt is not None or long_summary is not None
+            has_query = params.query_text is not None
+            snippets, total_match_count = (
+                SnippetGenerator.combine_sources(
+                    long_summary, webvtt, params.query_text, DEFAULT_MAX_SNIPPETS
+                )
+                if has_query and at_least_one_source
+                else ([], 0)
             )
 
             return SearchResult(

server/reflector/db/transcripts.py

@@ -122,6 +122,15 @@ def generate_transcript_name() -> str:
     return f"Transcript {now.strftime('%Y-%m-%d %H:%M:%S')}"
 
 
+TranscriptStatus = Literal[
+    "idle", "uploaded", "recording", "processing", "error", "ended"
+]
+
+
+class StrValue(BaseModel):
+    value: str
+
+
 class AudioWaveform(BaseModel):
     data: list[float]
 
@@ -185,7 +194,7 @@ class Transcript(BaseModel):
     id: str = Field(default_factory=generate_uuid4)
     user_id: str | None = None
     name: str = Field(default_factory=generate_transcript_name)
-    status: str = "idle"
+    status: TranscriptStatus = "idle"
     duration: float = 0
     created_at: datetime = Field(default_factory=lambda: datetime.now(timezone.utc))
     title: str | None = None
@@ -732,5 +741,27 @@ class TranscriptController:
         transcript.delete_participant(participant_id)
         await self.update(transcript, {"participants": transcript.participants_dump()})
 
+    async def set_status(
+        self, transcript_id: str, status: TranscriptStatus
+    ) -> TranscriptEvent | None:
+        """
+        Update the status of a transcript
+
+        Will add an event STATUS + update the status field of transcript
+        """
+        async with self.transaction():
+            transcript = await self.get_by_id(transcript_id)
+            if not transcript:
+                raise Exception(f"Transcript {transcript_id} not found")
+            if transcript.status == status:
+                return
+            resp = await self.append_event(
+                transcript=transcript,
+                event="STATUS",
+                data=StrValue(value=status),
+            )
+            await self.update(transcript, {"status": status})
+        return resp
+
 
 transcripts_controller = TranscriptController()

server/reflector/pipelines/main_file_pipeline.py

@@ -7,18 +7,28 @@ Uses parallel processing for transcription, diarization, and waveform generation
 """
 
 import asyncio
+import uuid
 from pathlib import Path
 
 import av
 import structlog
-from celery import shared_task
+from celery import chain, shared_task
 
+from reflector.asynctask import asynctask
+from reflector.db.rooms import rooms_controller
 from reflector.db.transcripts import (
+    SourceKind,
     Transcript,
+    TranscriptStatus,
     transcripts_controller,
 )
 from reflector.logger import logger
-from reflector.pipelines.main_live_pipeline import PipelineMainBase, asynctask
+from reflector.pipelines.main_live_pipeline import (
+    PipelineMainBase,
+    broadcast_to_sockets,
+    task_cleanup_consent,
+    task_pipeline_post_to_zulip,
+)
 from reflector.processors import (
     AudioFileWriterProcessor,
     TranscriptFinalSummaryProcessor,
@@ -43,6 +53,7 @@ from reflector.processors.types import (
 )
 from reflector.settings import settings
 from reflector.storage import get_transcripts_storage
+from reflector.worker.webhook import send_transcript_webhook
 
 
 class EmptyPipeline:
@@ -83,12 +94,27 @@ class PipelineMainFile(PipelineMainBase):
                 exc_info=result,
             )
 
+    @broadcast_to_sockets
+    async def set_status(self, transcript_id: str, status: TranscriptStatus):
+        async with self.lock_transaction():
+            return await transcripts_controller.set_status(transcript_id, status)
+
     async def process(self, file_path: Path):
         """Main entry point for file processing"""
         self.logger.info(f"Starting file pipeline for {file_path}")
 
         transcript = await self.get_transcript()
 
+        # Clear transcript as we're going to regenerate everything
+        async with self.transaction():
+            await transcripts_controller.update(
+                transcript,
+                {
+                    "events": [],
+                    "topics": [],
+                },
+            )
+
         # Extract audio and write to transcript location
         audio_path = await self.extract_and_write_audio(file_path, transcript)
 
@@ -105,6 +131,8 @@ class PipelineMainFile(PipelineMainBase):
 
         self.logger.info("File pipeline complete")
 
+        await transcripts_controller.set_status(transcript.id, "ended")
+
     async def extract_and_write_audio(
         self, file_path: Path, transcript: Transcript
     ) -> Path:
@@ -353,6 +381,28 @@ class PipelineMainFile(PipelineMainBase):
         await processor.flush()
 
 
+@shared_task
+@asynctask
+async def task_send_webhook_if_needed(*, transcript_id: str):
+    """Send webhook if this is a room recording with webhook configured"""
+    transcript = await transcripts_controller.get_by_id(transcript_id)
+    if not transcript:
+        return
+
+    if transcript.source_kind == SourceKind.ROOM and transcript.room_id:
+        room = await rooms_controller.get_by_id(transcript.room_id)
+        if room and room.webhook_url:
+            logger.info(
+                "Dispatching webhook",
+                transcript_id=transcript_id,
+                room_id=room.id,
+                webhook_url=room.webhook_url,
+            )
+            send_transcript_webhook.delay(
+                transcript_id, room.id, event_id=uuid.uuid4().hex
+            )
+
+
 @shared_task
 @asynctask
 async def task_pipeline_file_process(*, transcript_id: str):
@@ -362,14 +412,28 @@ async def task_pipeline_file_process(*, transcript_id: str):
     if not transcript:
         raise Exception(f"Transcript {transcript_id} not found")
 
-    # Find the file to process
-    audio_file = next(transcript.data_path.glob("upload.*"), None)
-    if not audio_file:
-        audio_file = next(transcript.data_path.glob("audio.*"), None)
-
-    if not audio_file:
-        raise Exception("No audio file found to process")
-
-    # Run file pipeline
     pipeline = PipelineMainFile(transcript_id=transcript_id)
-    await pipeline.process(audio_file)
+    try:
+        await pipeline.set_status(transcript_id, "processing")
+
+        # Find the file to process
+        audio_file = next(transcript.data_path.glob("upload.*"), None)
+        if not audio_file:
+            audio_file = next(transcript.data_path.glob("audio.*"), None)
+
+        if not audio_file:
+            raise Exception("No audio file found to process")
+
+        await pipeline.process(audio_file)
+
+    except Exception:
+        await pipeline.set_status(transcript_id, "error")
+        raise
+
+    # Run post-processing chain: consent cleanup -> zulip -> webhook
+    post_chain = chain(
+        task_cleanup_consent.si(transcript_id=transcript_id),
+        task_pipeline_post_to_zulip.si(transcript_id=transcript_id),
+        task_send_webhook_if_needed.si(transcript_id=transcript_id),
+    )
+    post_chain.delay()

server/reflector/pipelines/main_live_pipeline.py

@@ -22,7 +22,7 @@ from celery import chord, current_task, group, shared_task
 from pydantic import BaseModel
 from structlog import BoundLogger as Logger
 
-from reflector.db import get_database
+from reflector.asynctask import asynctask
 from reflector.db.meetings import meeting_consent_controller, meetings_controller
 from reflector.db.recordings import recordings_controller
 from reflector.db.rooms import rooms_controller
@@ -32,6 +32,7 @@ from reflector.db.transcripts import (
     TranscriptFinalLongSummary,
     TranscriptFinalShortSummary,
     TranscriptFinalTitle,
+    TranscriptStatus,
     TranscriptText,
     TranscriptTopic,
     TranscriptWaveform,
@@ -69,29 +70,6 @@ from reflector.zulip import (
 )
 
 
-def asynctask(f):
-    @functools.wraps(f)
-    def wrapper(*args, **kwargs):
-        async def run_with_db():
-            database = get_database()
-            await database.connect()
-            try:
-                return await f(*args, **kwargs)
-            finally:
-                await database.disconnect()
-
-        coro = run_with_db()
-        try:
-            loop = asyncio.get_running_loop()
-        except RuntimeError:
-            loop = None
-        if loop and loop.is_running():
-            return loop.run_until_complete(coro)
-        return asyncio.run(coro)
-
-    return wrapper
-
-
 def broadcast_to_sockets(func):
     """
     Decorator to broadcast transcript event to websockets
@@ -188,8 +166,15 @@ class PipelineMainBase(PipelineRunner[PipelineMessage], Generic[PipelineMessage]
         ]
 
     @asynccontextmanager
-    async def transaction(self):
+    async def lock_transaction(self):
+        # This lock is to prevent multiple processor starting adding
+        # into event array at the same time
         async with self._lock:
+            yield
+
+    @asynccontextmanager
+    async def transaction(self):
+        async with self.lock_transaction():
             async with transcripts_controller.transaction():
                 yield
 
@@ -198,14 +183,14 @@ class PipelineMainBase(PipelineRunner[PipelineMessage], Generic[PipelineMessage]
         # if it's the first part, update the status of the transcript
         # but do not set the ended status yet.
         if isinstance(self, PipelineMainLive):
-            status_mapping = {
+            status_mapping: dict[str, TranscriptStatus] = {
                 "started": "recording",
                 "push": "recording",
                 "flush": "processing",
                 "error": "error",
             }
         elif isinstance(self, PipelineMainFinalSummaries):
-            status_mapping = {
+            status_mapping: dict[str, TranscriptStatus] = {
                 "push": "processing",
                 "flush": "processing",
                 "error": "error",
@@ -221,22 +206,8 @@ class PipelineMainBase(PipelineRunner[PipelineMessage], Generic[PipelineMessage]
             return
 
         # when the status of the pipeline changes, update the transcript
-        async with self.transaction():
-            transcript = await self.get_transcript()
-            if status == transcript.status:
-                return
-            resp = await transcripts_controller.append_event(
-                transcript=transcript,
-                event="STATUS",
-                data=StrValue(value=status),
-            )
-            await transcripts_controller.update(
-                transcript,
-                {
-                    "status": status,
-                },
-            )
-            return resp
+        async with self._lock:
+            return await transcripts_controller.set_status(self.transcript_id, status)
 
     @broadcast_to_sockets
     async def on_transcript(self, data):
@@ -794,7 +765,7 @@ def pipeline_post(*, transcript_id: str):
         chain_final_summaries,
     ) | task_pipeline_post_to_zulip.si(transcript_id=transcript_id)
 
-    chain.delay()
+    return chain.delay()
 
 
 @get_transcript

server/reflector/processors/file_diarization_modal.py

@@ -47,6 +47,7 @@ class FileDiarizationModalProcessor(FileDiarizationProcessor):
                     "audio_file_url": data.audio_url,
                     "timestamp": 0,
                 },
+                follow_redirects=True,
             )
             response.raise_for_status()
             diarization_data = response.json()["diarization"]

server/reflector/processors/file_transcript_modal.py

@@ -54,6 +54,7 @@ class FileTranscriptModalProcessor(FileTranscriptProcessor):
                     "language": data.language,
                     "batch": True,
                 },
+                follow_redirects=True,
             )
             response.raise_for_status()
             result = response.json()
@@ -67,6 +68,9 @@ class FileTranscriptModalProcessor(FileTranscriptProcessor):
             for word_info in result.get("words", [])
         ]
 
+        # words come not in order
+        words.sort(key=lambda w: w.start)
+
         return Transcript(words=words)
 
 

server/reflector/processors/types.py

@@ -4,11 +4,8 @@ import tempfile
 from pathlib import Path
 from typing import Annotated, TypedDict
 
-from profanityfilter import ProfanityFilter
 from pydantic import BaseModel, Field, PrivateAttr
 
-from reflector.redis_cache import redis_cache
-
 
 class DiarizationSegment(TypedDict):
     """Type definition for diarization segment containing speaker information"""
@@ -20,9 +17,6 @@ class DiarizationSegment(TypedDict):
 
 PUNC_RE = re.compile(r"[.;:?!…]")
 
-profanity_filter = ProfanityFilter()
-profanity_filter.set_censor("*")
-
 
 class AudioFile(BaseModel):
     name: str
@@ -124,21 +118,11 @@ def words_to_segments(words: list[Word]) -> list[TranscriptSegment]:
 
 class Transcript(BaseModel):
     translation: str | None = None
-    words: list[Word] = None
-
-    @property
-    def raw_text(self):
-        # Uncensored text
-        return "".join([word.text for word in self.words])
-
-    @redis_cache(prefix="profanity", duration=3600 * 24 * 7)
-    def _get_censored_text(self, text: str):
-        return profanity_filter.censor(text).strip()
+    words: list[Word] = []
 
     @property
     def text(self):
-        # Censored text
-        return self._get_censored_text(self.raw_text)
+        return "".join([word.text for word in self.words])
 
     @property
     def human_timestamp(self):
@@ -170,12 +154,6 @@ class Transcript(BaseModel):
             word.start += offset
             word.end += offset
 
-    def clone(self):
-        words = [
-            Word(text=word.text, start=word.start, end=word.end) for word in self.words
-        ]
-        return Transcript(text=self.text, translation=self.translation, words=words)
-
     def as_segments(self) -> list[TranscriptSegment]:
         return words_to_segments(self.words)
 

server/reflector/redis_cache.py

@@ -1,10 +1,17 @@
+import asyncio
 import functools
 import json
+from typing import Optional
 
 import redis
+import redis.asyncio as redis_async
+import structlog
+from redis.exceptions import LockError
 
 from reflector.settings import settings
 
+logger = structlog.get_logger(__name__)
+
 redis_clients = {}
 
 
@@ -21,6 +28,12 @@ def get_redis_client(db=0):
     return redis_clients[db]
 
 
+async def get_async_redis_client(db: int = 0):
+    return await redis_async.from_url(
+        f"redis://{settings.REDIS_HOST}:{settings.REDIS_PORT}/{db}"
+    )
+
+
 def redis_cache(prefix="cache", duration=3600, db=settings.REDIS_CACHE_DB, argidx=1):
     """
     Cache the result of a function in Redis.
@@ -49,3 +62,87 @@ def redis_cache(prefix="cache", duration=3600, db=settings.REDIS_CACHE_DB, argid
         return wrapper
 
     return decorator
+
+
+class RedisAsyncLock:
+    def __init__(
+        self,
+        key: str,
+        timeout: int = 120,
+        extend_interval: int = 30,
+        skip_if_locked: bool = False,
+        blocking: bool = True,
+        blocking_timeout: Optional[float] = None,
+    ):
+        self.key = f"async_lock:{key}"
+        self.timeout = timeout
+        self.extend_interval = extend_interval
+        self.skip_if_locked = skip_if_locked
+        self.blocking = blocking
+        self.blocking_timeout = blocking_timeout
+        self._lock = None
+        self._redis = None
+        self._extend_task = None
+        self._acquired = False
+
+    async def _extend_lock_periodically(self):
+        while True:
+            try:
+                await asyncio.sleep(self.extend_interval)
+                if self._lock:
+                    await self._lock.extend(self.timeout, replace_ttl=True)
+                    logger.debug("Extended lock", key=self.key)
+            except LockError:
+                logger.warning("Failed to extend lock", key=self.key)
+                break
+            except asyncio.CancelledError:
+                break
+            except Exception as e:
+                logger.error("Error extending lock", key=self.key, error=str(e))
+                break
+
+    async def __aenter__(self):
+        self._redis = await get_async_redis_client()
+        self._lock = self._redis.lock(
+            self.key,
+            timeout=self.timeout,
+            blocking=self.blocking,
+            blocking_timeout=self.blocking_timeout,
+        )
+
+        self._acquired = await self._lock.acquire()
+
+        if not self._acquired:
+            if self.skip_if_locked:
+                logger.warning(
+                    "Lock already acquired by another process, skipping", key=self.key
+                )
+                return self
+            else:
+                raise LockError(f"Failed to acquire lock: {self.key}")
+
+        self._extend_task = asyncio.create_task(self._extend_lock_periodically())
+        logger.info("Acquired lock", key=self.key)
+        return self
+
+    async def __aexit__(self, exc_type, exc_val, exc_tb):
+        if self._extend_task:
+            self._extend_task.cancel()
+            try:
+                await self._extend_task
+            except asyncio.CancelledError:
+                pass
+
+        if self._acquired and self._lock:
+            try:
+                await self._lock.release()
+                logger.info("Released lock", key=self.key)
+            except LockError:
+                logger.debug("Lock already released or expired", key=self.key)
+
+        if self._redis:
+            await self._redis.aclose()
+
+    @property
+    def acquired(self) -> bool:
+        return self._acquired

server/reflector/services/ics_sync.py

@@ -0,0 +1,408 @@
+"""
+ICS Calendar Synchronization Service
+
+This module provides services for fetching, parsing, and synchronizing ICS (iCalendar)
+calendar feeds with room booking data in the database.
+
+Key Components:
+- ICSFetchService: Handles HTTP fetching and parsing of ICS calendar data
+- ICSSyncService: Manages the synchronization process between ICS feeds and database
+
+Example Usage:
+    # Sync a room's calendar
+    room = Room(id="room1", name="conference-room", ics_url="https://cal.example.com/room.ics")
+    result = await ics_sync_service.sync_room_calendar(room)
+
+    # Result structure:
+    {
+        "status": "success",  # success|unchanged|error|skipped
+        "hash": "abc123...",  # MD5 hash of ICS content
+        "events_found": 5,    # Events matching this room
+        "total_events": 12,   # Total events in calendar within time window
+        "events_created": 2,  # New events added to database
+        "events_updated": 3,  # Existing events modified
+        "events_deleted": 1   # Events soft-deleted (no longer in calendar)
+    }
+
+Event Matching:
+    Events are matched to rooms by checking if the room's full URL appears in the
+    event's LOCATION or DESCRIPTION fields. Only events within a 25-hour window
+    (1 hour ago to 24 hours from now) are processed.
+
+Input: ICS calendar URL (e.g., "https://calendar.google.com/calendar/ical/...")
+Output: EventData objects with structured calendar information:
+    {
+        "ics_uid": "event123@google.com",
+        "title": "Team Meeting",
+        "description": "Weekly sync meeting",
+        "location": "https://meet.company.com/conference-room",
+        "start_time": datetime(2024, 1, 15, 14, 0, tzinfo=UTC),
+        "end_time": datetime(2024, 1, 15, 15, 0, tzinfo=UTC),
+        "attendees": [
+            {"email": "user@company.com", "name": "John Doe", "role": "ORGANIZER"},
+            {"email": "attendee@company.com", "name": "Jane Smith", "status": "ACCEPTED"}
+        ],
+        "ics_raw_data": "BEGIN:VEVENT\nUID:event123@google.com\n..."
+    }
+"""
+
+import hashlib
+from datetime import date, datetime, timedelta, timezone
+from enum import Enum
+from typing import TypedDict
+
+import httpx
+import pytz
+import structlog
+from icalendar import Calendar, Event
+
+from reflector.db.calendar_events import CalendarEvent, calendar_events_controller
+from reflector.db.rooms import Room, rooms_controller
+from reflector.redis_cache import RedisAsyncLock
+from reflector.settings import settings
+
+logger = structlog.get_logger()
+
+EVENT_WINDOW_DELTA_START = timedelta(hours=-1)
+EVENT_WINDOW_DELTA_END = timedelta(hours=24)
+
+
+class SyncStatus(str, Enum):
+    SUCCESS = "success"
+    UNCHANGED = "unchanged"
+    ERROR = "error"
+    SKIPPED = "skipped"
+
+
+class AttendeeData(TypedDict, total=False):
+    email: str | None
+    name: str | None
+    status: str | None
+    role: str | None
+
+
+class EventData(TypedDict):
+    ics_uid: str
+    title: str | None
+    description: str | None
+    location: str | None
+    start_time: datetime
+    end_time: datetime
+    attendees: list[AttendeeData]
+    ics_raw_data: str
+
+
+class SyncStats(TypedDict):
+    events_created: int
+    events_updated: int
+    events_deleted: int
+
+
+class SyncResultBase(TypedDict):
+    status: SyncStatus
+
+
+class SyncResult(SyncResultBase, total=False):
+    hash: str | None
+    events_found: int
+    total_events: int
+    events_created: int
+    events_updated: int
+    events_deleted: int
+    error: str | None
+    reason: str | None
+
+
+class ICSFetchService:
+    def __init__(self):
+        self.client = httpx.AsyncClient(
+            timeout=30.0, headers={"User-Agent": "Reflector/1.0"}
+        )
+
+    async def fetch_ics(self, url: str) -> str:
+        response = await self.client.get(url)
+        response.raise_for_status()
+
+        return response.text
+
+    def parse_ics(self, ics_content: str) -> Calendar:
+        return Calendar.from_ical(ics_content)
+
+    def extract_room_events(
+        self, calendar: Calendar, room_name: str, room_url: str
+    ) -> tuple[list[EventData], int]:
+        events = []
+        total_events = 0
+        now = datetime.now(timezone.utc)
+        window_start = now + EVENT_WINDOW_DELTA_START
+        window_end = now + EVENT_WINDOW_DELTA_END
+
+        for component in calendar.walk():
+            if component.name != "VEVENT":
+                continue
+
+            status = component.get("STATUS", "").upper()
+            if status == "CANCELLED":
+                continue
+
+            # Count total non-cancelled events in the time window
+            event_data = self._parse_event(component)
+            if event_data and window_start <= event_data["start_time"] <= window_end:
+                total_events += 1
+
+                # Check if event matches this room
+                if self._event_matches_room(component, room_name, room_url):
+                    events.append(event_data)
+
+        return events, total_events
+
+    def _event_matches_room(self, event: Event, room_name: str, room_url: str) -> bool:
+        location = str(event.get("LOCATION", ""))
+        description = str(event.get("DESCRIPTION", ""))
+
+        # Only match full room URL
+        # XXX leaved here as a patterns, to later be extended with tinyurl or such too
+        patterns = [
+            room_url,
+        ]
+
+        # Check location and description for patterns
+        text_to_check = f"{location} {description}".lower()
+        for pattern in patterns:
+            if pattern.lower() in text_to_check:
+                return True
+
+        return False
+
+    def _parse_event(self, event: Event) -> EventData | None:
+        uid = str(event.get("UID", ""))
+        summary = str(event.get("SUMMARY", ""))
+        description = str(event.get("DESCRIPTION", ""))
+        location = str(event.get("LOCATION", ""))
+        dtstart = event.get("DTSTART")
+        dtend = event.get("DTEND")
+
+        if not dtstart:
+            return None
+
+        # Convert fields
+        start_time = self._normalize_datetime(
+            dtstart.dt if hasattr(dtstart, "dt") else dtstart
+        )
+        end_time = (
+            self._normalize_datetime(dtend.dt if hasattr(dtend, "dt") else dtend)
+            if dtend
+            else start_time + timedelta(hours=1)
+        )
+        attendees = self._parse_attendees(event)
+
+        # Get raw event data for storage
+        raw_data = event.to_ical().decode("utf-8")
+
+        return {
+            "ics_uid": uid,
+            "title": summary,
+            "description": description,
+            "location": location,
+            "start_time": start_time,
+            "end_time": end_time,
+            "attendees": attendees,
+            "ics_raw_data": raw_data,
+        }
+
+    def _normalize_datetime(self, dt) -> datetime:
+        # Ensure datetime is with timezone, if not, assume UTC
+        if isinstance(dt, date) and not isinstance(dt, datetime):
+            dt = datetime.combine(dt, datetime.min.time())
+            dt = pytz.UTC.localize(dt)
+        elif isinstance(dt, datetime):
+            if dt.tzinfo is None:
+                dt = pytz.UTC.localize(dt)
+            else:
+                dt = dt.astimezone(pytz.UTC)
+
+        return dt
+
+    def _parse_attendees(self, event: Event) -> list[AttendeeData]:
+        # Extracts attendee information from both ATTENDEE and ORGANIZER properties.
+        # Handles malformed comma-separated email addresses in single ATTENDEE fields
+        # by splitting them into separate attendee entries. Returns a list of attendee
+        # data including email, name, status, and role information.
+        final_attendees = []
+
+        attendees = event.get("ATTENDEE", [])
+        if not isinstance(attendees, list):
+            attendees = [attendees]
+        for att in attendees:
+            email_str = str(att).replace("mailto:", "") if att else None
+
+            # Handle malformed comma-separated email addresses in a single ATTENDEE field
+            if email_str and "," in email_str:
+                # Split comma-separated emails and create separate attendee entries
+                email_parts = [email.strip() for email in email_str.split(",")]
+                for email in email_parts:
+                    if email and "@" in email:
+                        clean_email = email.replace("MAILTO:", "").replace(
+                            "mailto:", ""
+                        )
+                        att_data: AttendeeData = {
+                            "email": clean_email,
+                            "name": att.params.get("CN")
+                            if hasattr(att, "params") and email == email_parts[0]
+                            else None,
+                            "status": att.params.get("PARTSTAT")
+                            if hasattr(att, "params") and email == email_parts[0]
+                            else None,
+                            "role": att.params.get("ROLE")
+                            if hasattr(att, "params") and email == email_parts[0]
+                            else None,
+                        }
+                        final_attendees.append(att_data)
+            else:
+                # Normal single attendee
+                att_data: AttendeeData = {
+                    "email": email_str,
+                    "name": att.params.get("CN") if hasattr(att, "params") else None,
+                    "status": att.params.get("PARTSTAT")
+                    if hasattr(att, "params")
+                    else None,
+                    "role": att.params.get("ROLE") if hasattr(att, "params") else None,
+                }
+                final_attendees.append(att_data)
+
+        # Add organizer
+        organizer = event.get("ORGANIZER")
+        if organizer:
+            org_email = (
+                str(organizer).replace("mailto:", "").replace("MAILTO:", "")
+                if organizer
+                else None
+            )
+            org_data: AttendeeData = {
+                "email": org_email,
+                "name": organizer.params.get("CN")
+                if hasattr(organizer, "params")
+                else None,
+                "role": "ORGANIZER",
+            }
+            final_attendees.append(org_data)
+
+        return final_attendees
+
+
+class ICSSyncService:
+    def __init__(self):
+        self.fetch_service = ICSFetchService()
+
+    async def sync_room_calendar(self, room: Room) -> SyncResult:
+        async with RedisAsyncLock(
+            f"ics_sync_room:{room.id}", skip_if_locked=True
+        ) as lock:
+            if not lock.acquired:
+                logger.warning("ICS sync already in progress for room", room_id=room.id)
+                return {
+                    "status": SyncStatus.SKIPPED,
+                    "reason": "Sync already in progress",
+                }
+
+            return await self._sync_room_calendar(room)
+
+    async def _sync_room_calendar(self, room: Room) -> SyncResult:
+        if not room.ics_enabled or not room.ics_url:
+            return {"status": SyncStatus.SKIPPED, "reason": "ICS not configured"}
+
+        try:
+            if not self._should_sync(room):
+                return {"status": SyncStatus.SKIPPED, "reason": "Not time to sync yet"}
+
+            ics_content = await self.fetch_service.fetch_ics(room.ics_url)
+            calendar = self.fetch_service.parse_ics(ics_content)
+
+            content_hash = hashlib.md5(ics_content.encode()).hexdigest()
+            if room.ics_last_etag == content_hash:
+                logger.info("No changes in ICS for room", room_id=room.id)
+                room_url = f"{settings.UI_BASE_URL}/{room.name}"
+                events, total_events = self.fetch_service.extract_room_events(
+                    calendar, room.name, room_url
+                )
+                return {
+                    "status": SyncStatus.UNCHANGED,
+                    "hash": content_hash,
+                    "events_found": len(events),
+                    "total_events": total_events,
+                    "events_created": 0,
+                    "events_updated": 0,
+                    "events_deleted": 0,
+                }
+
+            # Extract matching events
+            room_url = f"{settings.UI_BASE_URL}/{room.name}"
+            events, total_events = self.fetch_service.extract_room_events(
+                calendar, room.name, room_url
+            )
+            sync_result = await self._sync_events_to_database(room.id, events)
+
+            # Update room sync metadata
+            await rooms_controller.update(
+                room,
+                {
+                    "ics_last_sync": datetime.now(timezone.utc),
+                    "ics_last_etag": content_hash,
+                },
+                mutate=False,
+            )
+
+            return {
+                "status": SyncStatus.SUCCESS,
+                "hash": content_hash,
+                "events_found": len(events),
+                "total_events": total_events,
+                **sync_result,
+            }
+
+        except Exception as e:
+            logger.error("Failed to sync ICS for room", room_id=room.id, error=str(e))
+            return {"status": SyncStatus.ERROR, "error": str(e)}
+
+    def _should_sync(self, room: Room) -> bool:
+        if not room.ics_last_sync:
+            return True
+
+        time_since_sync = datetime.now(timezone.utc) - room.ics_last_sync
+        return time_since_sync.total_seconds() >= room.ics_fetch_interval
+
+    async def _sync_events_to_database(
+        self, room_id: str, events: list[EventData]
+    ) -> SyncStats:
+        created = 0
+        updated = 0
+
+        current_ics_uids = []
+
+        for event_data in events:
+            calendar_event = CalendarEvent(room_id=room_id, **event_data)
+            existing = await calendar_events_controller.get_by_ics_uid(
+                room_id, event_data["ics_uid"]
+            )
+
+            if existing:
+                updated += 1
+            else:
+                created += 1
+
+            await calendar_events_controller.upsert(calendar_event)
+            current_ics_uids.append(event_data["ics_uid"])
+
+        # Soft delete events that are no longer in calendar
+        deleted = await calendar_events_controller.soft_delete_missing(
+            room_id, current_ics_uids
+        )
+
+        return {
+            "events_created": created,
+            "events_updated": updated,
+            "events_deleted": deleted,
+        }
+
+
+ics_sync_service = ICSSyncService()

server/reflector/settings.py

@@ -1,5 +1,8 @@
+from pydantic.types import PositiveInt
 from pydantic_settings import BaseSettings, SettingsConfigDict
 
+from reflector.utils.string import NonEmptyString
+
 
 class Settings(BaseSettings):
     model_config = SettingsConfigDict(
@@ -90,9 +93,8 @@ class Settings(BaseSettings):
     AUTH_JWT_PUBLIC_KEY: str | None = "authentik.monadical.com_public.pem"
     AUTH_JWT_AUDIENCE: str | None = None
 
-    # API public mode
-    # if set, all anonymous record will be public
     PUBLIC_MODE: bool = False
+    PUBLIC_DATA_RETENTION_DAYS: PositiveInt = 7
 
     # Min transcript length to generate topic + summary
     MIN_TRANSCRIPT_LENGTH: int = 750
@@ -120,7 +122,7 @@ class Settings(BaseSettings):
 
     # Whereby integration
     WHEREBY_API_URL: str = "https://api.whereby.dev/v1"
-    WHEREBY_API_KEY: str | None = None
+    WHEREBY_API_KEY: NonEmptyString | None = None
     WHEREBY_WEBHOOK_SECRET: str | None = None
     AWS_WHEREBY_ACCESS_KEY_ID: str | None = None
     AWS_WHEREBY_ACCESS_KEY_SECRET: str | None = None

server/reflector/tools/cleanup_old_data.py

@@ -0,0 +1,72 @@
+#!/usr/bin/env python
+"""
+Manual cleanup tool for old public data.
+Uses the same implementation as the Celery worker task.
+"""
+
+import argparse
+import asyncio
+import sys
+
+import structlog
+
+from reflector.settings import settings
+from reflector.worker.cleanup import _cleanup_old_public_data
+
+logger = structlog.get_logger(__name__)
+
+
+async def cleanup_old_data(days: int = 7):
+    logger.info(
+        "Starting manual cleanup",
+        retention_days=days,
+        public_mode=settings.PUBLIC_MODE,
+    )
+
+    if not settings.PUBLIC_MODE:
+        logger.critical(
+            "WARNING: PUBLIC_MODE is False. "
+            "This tool is intended for public instances only."
+        )
+        raise Exception("Tool intended for public instances only")
+
+    result = await _cleanup_old_public_data(days=days)
+
+    if result:
+        logger.info(
+            "Cleanup completed",
+            transcripts_deleted=result.get("transcripts_deleted", 0),
+            meetings_deleted=result.get("meetings_deleted", 0),
+            recordings_deleted=result.get("recordings_deleted", 0),
+            errors_count=len(result.get("errors", [])),
+        )
+        if result.get("errors"):
+            logger.warning(
+                "Errors encountered during cleanup:", errors=result["errors"][:10]
+            )
+    else:
+        logger.info("Cleanup skipped or completed without results")
+
+
+def main():
+    parser = argparse.ArgumentParser(
+        description="Clean up old transcripts and meetings"
+    )
+    parser.add_argument(
+        "--days",
+        type=int,
+        default=7,
+        help="Number of days to keep data (default: 7)",
+    )
+
+    args = parser.parse_args()
+
+    if args.days < 1:
+        logger.error("Days must be at least 1")
+        sys.exit(1)
+
+    asyncio.run(cleanup_old_data(days=args.days))
+
+
+if __name__ == "__main__":
+    main()

server/reflector/tools/process.py

@@ -1,294 +1,204 @@
 """
 Process audio file with diarization support
-===========================================
-
-Extended version of process.py that includes speaker diarization.
-This tool processes audio files locally without requiring the full server infrastructure.
 """
 
+import argparse
 import asyncio
-import tempfile
-import uuid
+import json
+import shutil
+import sys
+import time
 from pathlib import Path
-from typing import List
-
-import av
+from typing import Any, Dict, List, Literal
 
+from reflector.db.transcripts import SourceKind, TranscriptTopic, transcripts_controller
 from reflector.logger import logger
-from reflector.processors import (
-    AudioChunkerAutoProcessor,
-    AudioDownscaleProcessor,
-    AudioFileWriterProcessor,
-    AudioMergeProcessor,
-    AudioTranscriptAutoProcessor,
-    Pipeline,
-    PipelineEvent,
-    TranscriptFinalSummaryProcessor,
-    TranscriptFinalTitleProcessor,
-    TranscriptLinerProcessor,
-    TranscriptTopicDetectorProcessor,
-    TranscriptTranslatorAutoProcessor,
+from reflector.pipelines.main_file_pipeline import (
+    task_pipeline_file_process as task_pipeline_file_process,
 )
-from reflector.processors.base import BroadcastProcessor, Processor
-from reflector.processors.types import (
-    AudioDiarizationInput,
-    TitleSummary,
-    TitleSummaryWithId,
+from reflector.pipelines.main_live_pipeline import pipeline_post as live_pipeline_post
+from reflector.pipelines.main_live_pipeline import (
+    pipeline_process as live_pipeline_process,
 )
 
 
-class TopicCollectorProcessor(Processor):
-    """Collect topics for diarization"""
+def serialize_topics(topics: List[TranscriptTopic]) -> List[Dict[str, Any]]:
+    """Convert TranscriptTopic objects to JSON-serializable dicts"""
+    serialized = []
+    for topic in topics:
+        topic_dict = topic.model_dump()
+        serialized.append(topic_dict)
+    return serialized
 
-    INPUT_TYPE = TitleSummary
-    OUTPUT_TYPE = TitleSummary
 
-    def __init__(self, **kwargs):
-        super().__init__(**kwargs)
-        self.topics: List[TitleSummaryWithId] = []
-        self._topic_id = 0
+def debug_print_speakers(serialized_topics: List[Dict[str, Any]]) -> None:
+    """Print debug info about speakers found in topics"""
+    all_speakers = set()
+    for topic_dict in serialized_topics:
+        for word in topic_dict.get("words", []):
+            all_speakers.add(word.get("speaker", 0))
 
-    async def _push(self, data: TitleSummary):
-        # Convert to TitleSummaryWithId and collect
-        self._topic_id += 1
-        topic_with_id = TitleSummaryWithId(
-            id=str(self._topic_id),
-            title=data.title,
-            summary=data.summary,
-            timestamp=data.timestamp,
-            duration=data.duration,
-            transcript=data.transcript,
+    print(
+        f"Found {len(serialized_topics)} topics with speakers: {all_speakers}",
+        file=sys.stderr,
+    )
+
+
+TranscriptId = str
+
+
+# common interface for every flow: it needs an Entry in db with specific ceremony (file path + status + actual file in file system)
+# ideally we want to get rid of it at some point
+async def prepare_entry(
+    source_path: str,
+    source_language: str,
+    target_language: str,
+) -> TranscriptId:
+    file_path = Path(source_path)
+
+    transcript = await transcripts_controller.add(
+        file_path.name,
+        # note that the real file upload has SourceKind: LIVE for the reason of it's an error
+        source_kind=SourceKind.FILE,
+        source_language=source_language,
+        target_language=target_language,
+        user_id=None,
+    )
+
+    logger.info(
+        f"Created empty transcript {transcript.id} for file {file_path.name} because technically we need an empty transcript before we start transcript"
+    )
+
+    # pipelines expect files as upload.*
+
+    extension = file_path.suffix
+    upload_path = transcript.data_path / f"upload{extension}"
+    upload_path.parent.mkdir(parents=True, exist_ok=True)
+    shutil.copy2(source_path, upload_path)
+    logger.info(f"Copied {source_path} to {upload_path}")
+
+    # pipelines expect entity status "uploaded"
+    await transcripts_controller.update(transcript, {"status": "uploaded"})
+
+    return transcript.id
+
+
+# same reason as prepare_entry
+async def extract_result_from_entry(
+    transcript_id: TranscriptId, output_path: str
+) -> None:
+    post_final_transcript = await transcripts_controller.get_by_id(transcript_id)
+
+    # assert post_final_transcript.status == "ended"
+    # File pipeline doesn't set status to "ended", only live pipeline does https://github.com/Monadical-SAS/reflector/issues/582
+    topics = post_final_transcript.topics
+    if not topics:
+        raise RuntimeError(
+            f"No topics found for transcript {transcript_id} after processing"
         )
-        self.topics.append(topic_with_id)
 
-        # Pass through the original topic
-        await self.emit(data)
+    serialized_topics = serialize_topics(topics)
 
-    def get_topics(self) -> List[TitleSummaryWithId]:
-        return self.topics
+    if output_path:
+        # Write to JSON file
+        with open(output_path, "w") as f:
+            for topic_dict in serialized_topics:
+                json.dump(topic_dict, f)
+                f.write("\n")
+        print(f"Results written to {output_path}", file=sys.stderr)
+    else:
+        # Write to stdout as JSONL
+        for topic_dict in serialized_topics:
+            print(json.dumps(topic_dict))
+
+    debug_print_speakers(serialized_topics)
 
 
-async def process_audio_file(
-    filename,
-    event_callback,
-    only_transcript=False,
-    source_language="en",
-    target_language="en",
-    enable_diarization=True,
-    diarization_backend="pyannote",
+async def process_live_pipeline(
+    transcript_id: TranscriptId,
 ):
-    # Create temp file for audio if diarization is enabled
-    audio_temp_path = None
-    if enable_diarization:
-        audio_temp_file = tempfile.NamedTemporaryFile(suffix=".wav", delete=False)
-        audio_temp_path = audio_temp_file.name
-        audio_temp_file.close()
+    """Process transcript_id with transcription and diarization"""
 
-    # Create processor for collecting topics
-    topic_collector = TopicCollectorProcessor()
+    print(f"Processing transcript_id {transcript_id}...", file=sys.stderr)
+    await live_pipeline_process(transcript_id=transcript_id)
+    print(f"Processing complete for transcript {transcript_id}", file=sys.stderr)
 
-    # Build pipeline for audio processing
-    processors = []
+    pre_final_transcript = await transcripts_controller.get_by_id(transcript_id)
 
-    # Add audio file writer at the beginning if diarization is enabled
-    if enable_diarization:
-        processors.append(AudioFileWriterProcessor(audio_temp_path))
+    # assert documented behaviour: after process, the pipeline isn't ended. this is the reason of calling pipeline_post
+    assert pre_final_transcript.status != "ended"
 
-    # Add the rest of the processors
-    processors += [
-        AudioDownscaleProcessor(),
-        AudioChunkerAutoProcessor(),
-        AudioMergeProcessor(),
-        AudioTranscriptAutoProcessor.as_threaded(),
-        TranscriptLinerProcessor(),
-        TranscriptTranslatorAutoProcessor.as_threaded(),
-    ]
+    # at this point, diarization is running but we have no access to it. run diarization in parallel - one will hopefully win after polling
+    result = live_pipeline_post(transcript_id=transcript_id)
 
-    if not only_transcript:
-        processors += [
-            TranscriptTopicDetectorProcessor.as_threaded(),
-            # Collect topics for diarization
-            topic_collector,
-            BroadcastProcessor(
-                processors=[
-                    TranscriptFinalTitleProcessor.as_threaded(),
-                    TranscriptFinalSummaryProcessor.as_threaded(),
-                ],
-            ),
-        ]
-
-    # Create main pipeline
-    pipeline = Pipeline(*processors)
-    pipeline.set_pref("audio:source_language", source_language)
-    pipeline.set_pref("audio:target_language", target_language)
-    pipeline.describe()
-    pipeline.on(event_callback)
-
-    # Start processing audio
-    logger.info(f"Opening {filename}")
-    container = av.open(filename)
-    try:
-        logger.info("Start pushing audio into the pipeline")
-        for frame in container.decode(audio=0):
-            await pipeline.push(frame)
-    finally:
-        logger.info("Flushing the pipeline")
-        await pipeline.flush()
-
-    # Run diarization if enabled and we have topics
-    if enable_diarization and not only_transcript and audio_temp_path:
-        topics = topic_collector.get_topics()
-
-        if topics:
-            logger.info(f"Starting diarization with {len(topics)} topics")
-
-            try:
-                from reflector.processors import AudioDiarizationAutoProcessor
-
-                diarization_processor = AudioDiarizationAutoProcessor(
-                    name=diarization_backend
-                )
-
-                diarization_processor.set_pipeline(pipeline)
-
-                # For Modal backend, we need to upload the file to S3 first
-                if diarization_backend == "modal":
-                    from datetime import datetime
-
-                    from reflector.storage import get_transcripts_storage
-                    from reflector.utils.s3_temp_file import S3TemporaryFile
-
-                    storage = get_transcripts_storage()
-
-                    # Generate a unique filename in evaluation folder
-                    timestamp = datetime.utcnow().strftime("%Y%m%d_%H%M%S")
-                    audio_filename = f"evaluation/diarization_temp/{timestamp}_{uuid.uuid4().hex}.wav"
-
-                    # Use context manager for automatic cleanup
-                    async with S3TemporaryFile(storage, audio_filename) as s3_file:
-                        # Read and upload the audio file
-                        with open(audio_temp_path, "rb") as f:
-                            audio_data = f.read()
-
-                        audio_url = await s3_file.upload(audio_data)
-                        logger.info(f"Uploaded audio to S3: {audio_filename}")
-
-                        # Create diarization input with S3 URL
-                        diarization_input = AudioDiarizationInput(
-                            audio_url=audio_url, topics=topics
-                        )
-
-                        # Run diarization
-                        await diarization_processor.push(diarization_input)
-                        await diarization_processor.flush()
-
-                        logger.info("Diarization complete")
-                        # File will be automatically cleaned up when exiting the context
-                else:
-                    # For local backend, use local file path
-                    audio_url = audio_temp_path
-
-                    # Create diarization input
-                    diarization_input = AudioDiarizationInput(
-                        audio_url=audio_url, topics=topics
-                    )
-
-                    # Run diarization
-                    await diarization_processor.push(diarization_input)
-                    await diarization_processor.flush()
-
-                    logger.info("Diarization complete")
-
-            except ImportError as e:
-                logger.error(f"Failed to import diarization dependencies: {e}")
-                logger.error(
-                    "Install with: uv pip install pyannote.audio torch torchaudio"
-                )
-                logger.error(
-                    "And set HF_TOKEN environment variable for pyannote models"
-                )
-                raise SystemExit(1)
-            except Exception as e:
-                logger.error(f"Diarization failed: {e}")
-                raise SystemExit(1)
-        else:
-            logger.warning("Skipping diarization: no topics available")
-
-    # Clean up temp file
-    if audio_temp_path:
-        try:
-            Path(audio_temp_path).unlink()
-        except Exception as e:
-            logger.warning(f"Failed to clean up temp file {audio_temp_path}: {e}")
-
-    logger.info("All done!")
+    # result.ready() blocks even without await; it mutates result also
+    while not result.ready():
+        print(f"Status: {result.state}")
+        time.sleep(2)
 
 
 async def process_file_pipeline(
-    filename: str,
-    event_callback,
-    source_language="en",
-    target_language="en",
-    enable_diarization=True,
-    diarization_backend="modal",
+    transcript_id: TranscriptId,
 ):
     """Process audio/video file using the optimized file pipeline"""
+
+    # task_pipeline_file_process is a Celery task, need to use .delay() for async execution
+    result = task_pipeline_file_process.delay(transcript_id=transcript_id)
+
+    # Wait for the Celery task to complete
+    while not result.ready():
+        print(f"File pipeline status: {result.state}", file=sys.stderr)
+        time.sleep(2)
+
+    logger.info("File pipeline processing complete")
+
+
+async def process(
+    source_path: str,
+    source_language: str,
+    target_language: str,
+    pipeline: Literal["live", "file"],
+    output_path: str = None,
+):
+    from reflector.db import get_database
+
+    database = get_database()
+    # db connect is a part of ceremony
+    await database.connect()
+
     try:
-        from reflector.db import database
-        from reflector.db.transcripts import SourceKind, transcripts_controller
-        from reflector.pipelines.main_file_pipeline import PipelineMainFile
-
-        await database.connect()
-        try:
-            # Create a temporary transcript for processing
-            transcript = await transcripts_controller.add(
-                "",
-                source_kind=SourceKind.FILE,
-                source_language=source_language,
-                target_language=target_language,
-            )
-
-            # Process the file
-            pipeline = PipelineMainFile(transcript_id=transcript.id)
-            await pipeline.process(Path(filename))
-
-            logger.info("File pipeline processing complete")
-
-        finally:
-            await database.disconnect()
-    except ImportError as e:
-        logger.error(f"File pipeline not available: {e}")
-        logger.info("Falling back to stream pipeline")
-        # Fall back to stream pipeline
-        await process_audio_file(
-            filename,
-            event_callback,
-            only_transcript=False,
-            source_language=source_language,
-            target_language=target_language,
-            enable_diarization=enable_diarization,
-            diarization_backend=diarization_backend,
+        transcript_id = await prepare_entry(
+            source_path,
+            source_language,
+            target_language,
         )
 
+        pipeline_handlers = {
+            "live": process_live_pipeline,
+            "file": process_file_pipeline,
+        }
+
+        handler = pipeline_handlers.get(pipeline)
+        if not handler:
+            raise ValueError(f"Unknown pipeline type: {pipeline}")
+
+        await handler(transcript_id)
+
+        await extract_result_from_entry(transcript_id, output_path)
+    finally:
+        await database.disconnect()
+
 
 if __name__ == "__main__":
-    import argparse
-    import os
-
     parser = argparse.ArgumentParser(
-        description="Process audio files with optional speaker diarization"
+        description="Process audio files with speaker diarization"
     )
     parser.add_argument("source", help="Source file (mp3, wav, mp4...)")
     parser.add_argument(
-        "--stream",
-        action="store_true",
-        help="Use streaming pipeline (original frame-based processing)",
-    )
-    parser.add_argument(
-        "--only-transcript",
-        "-t",
-        action="store_true",
-        help="Only generate transcript without topics/summaries",
+        "--pipeline",
+        required=True,
+        choices=["live", "file"],
+        help="Pipeline type to use for processing (live: streaming/incremental, file: batch/parallel)",
     )
     parser.add_argument(
         "--source-language", default="en", help="Source language code (default: en)"
@@ -297,82 +207,14 @@ if __name__ == "__main__":
         "--target-language", default="en", help="Target language code (default: en)"
     )
     parser.add_argument("--output", "-o", help="Output file (output.jsonl)")
-    parser.add_argument(
-        "--enable-diarization",
-        "-d",
-        action="store_true",
-        help="Enable speaker diarization",
-    )
-    parser.add_argument(
-        "--diarization-backend",
-        default="pyannote",
-        choices=["pyannote", "modal"],
-        help="Diarization backend to use (default: pyannote)",
-    )
     args = parser.parse_args()
 
-    if "REDIS_HOST" not in os.environ:
-        os.environ["REDIS_HOST"] = "localhost"
-
-    output_fd = None
-    if args.output:
-        output_fd = open(args.output, "w")
-
-    async def event_callback(event: PipelineEvent):
-        processor = event.processor
-        data = event.data
-
-        # Ignore internal processors
-        if processor in (
-            "AudioDownscaleProcessor",
-            "AudioChunkerAutoProcessor",
-            "AudioMergeProcessor",
-            "AudioFileWriterProcessor",
-            "TopicCollectorProcessor",
-            "BroadcastProcessor",
-        ):
-            return
-
-        # If diarization is enabled, skip the original topic events from the pipeline
-        # The diarization processor will emit the same topics but with speaker info
-        if processor == "TranscriptTopicDetectorProcessor" and args.enable_diarization:
-            return
-
-        # Log all events
-        logger.info(f"Event: {processor} - {type(data).__name__}")
-
-        # Write to output
-        if output_fd:
-            output_fd.write(event.model_dump_json())
-            output_fd.write("\n")
-            output_fd.flush()
-
-    if args.stream:
-        # Use original streaming pipeline
-        asyncio.run(
-            process_audio_file(
-                args.source,
-                event_callback,
-                only_transcript=args.only_transcript,
-                source_language=args.source_language,
-                target_language=args.target_language,
-                enable_diarization=args.enable_diarization,
-                diarization_backend=args.diarization_backend,
-            )
+    asyncio.run(
+        process(
+            args.source,
+            args.source_language,
+            args.target_language,
+            args.pipeline,
+            args.output,
         )
-    else:
-        # Use optimized file pipeline (default)
-        asyncio.run(
-            process_file_pipeline(
-                args.source,
-                event_callback,
-                source_language=args.source_language,
-                target_language=args.target_language,
-                enable_diarization=args.enable_diarization,
-                diarization_backend=args.diarization_backend,
-            )
-        )
-
-    if output_fd:
-        output_fd.close()
-        logger.info(f"Output written to {args.output}")
+    )

server/reflector/tools/process_with_diarization.py

@@ -1,318 +0,0 @@
-"""
-@vibe-generated
-Process audio file with diarization support
-===========================================
-
-Extended version of process.py that includes speaker diarization.
-This tool processes audio files locally without requiring the full server infrastructure.
-"""
-
-import asyncio
-import tempfile
-import uuid
-from pathlib import Path
-from typing import List
-
-import av
-
-from reflector.logger import logger
-from reflector.processors import (
-    AudioChunkerAutoProcessor,
-    AudioDownscaleProcessor,
-    AudioFileWriterProcessor,
-    AudioMergeProcessor,
-    AudioTranscriptAutoProcessor,
-    Pipeline,
-    PipelineEvent,
-    TranscriptFinalSummaryProcessor,
-    TranscriptFinalTitleProcessor,
-    TranscriptLinerProcessor,
-    TranscriptTopicDetectorProcessor,
-    TranscriptTranslatorAutoProcessor,
-)
-from reflector.processors.base import BroadcastProcessor, Processor
-from reflector.processors.types import (
-    AudioDiarizationInput,
-    TitleSummary,
-    TitleSummaryWithId,
-)
-
-
-class TopicCollectorProcessor(Processor):
-    """Collect topics for diarization"""
-
-    INPUT_TYPE = TitleSummary
-    OUTPUT_TYPE = TitleSummary
-
-    def __init__(self, **kwargs):
-        super().__init__(**kwargs)
-        self.topics: List[TitleSummaryWithId] = []
-        self._topic_id = 0
-
-    async def _push(self, data: TitleSummary):
-        # Convert to TitleSummaryWithId and collect
-        self._topic_id += 1
-        topic_with_id = TitleSummaryWithId(
-            id=str(self._topic_id),
-            title=data.title,
-            summary=data.summary,
-            timestamp=data.timestamp,
-            duration=data.duration,
-            transcript=data.transcript,
-        )
-        self.topics.append(topic_with_id)
-
-        # Pass through the original topic
-        await self.emit(data)
-
-    def get_topics(self) -> List[TitleSummaryWithId]:
-        return self.topics
-
-
-async def process_audio_file_with_diarization(
-    filename,
-    event_callback,
-    only_transcript=False,
-    source_language="en",
-    target_language="en",
-    enable_diarization=True,
-    diarization_backend="modal",
-):
-    # Create temp file for audio if diarization is enabled
-    audio_temp_path = None
-    if enable_diarization:
-        audio_temp_file = tempfile.NamedTemporaryFile(suffix=".wav", delete=False)
-        audio_temp_path = audio_temp_file.name
-        audio_temp_file.close()
-
-    # Create processor for collecting topics
-    topic_collector = TopicCollectorProcessor()
-
-    # Build pipeline for audio processing
-    processors = []
-
-    # Add audio file writer at the beginning if diarization is enabled
-    if enable_diarization:
-        processors.append(AudioFileWriterProcessor(audio_temp_path))
-
-    # Add the rest of the processors
-    processors += [
-        AudioDownscaleProcessor(),
-        AudioChunkerAutoProcessor(),
-        AudioMergeProcessor(),
-        AudioTranscriptAutoProcessor.as_threaded(),
-    ]
-
-    processors += [
-        TranscriptLinerProcessor(),
-        TranscriptTranslatorAutoProcessor.as_threaded(),
-    ]
-
-    if not only_transcript:
-        processors += [
-            TranscriptTopicDetectorProcessor.as_threaded(),
-            # Collect topics for diarization
-            topic_collector,
-            BroadcastProcessor(
-                processors=[
-                    TranscriptFinalTitleProcessor.as_threaded(),
-                    TranscriptFinalSummaryProcessor.as_threaded(),
-                ],
-            ),
-        ]
-
-    # Create main pipeline
-    pipeline = Pipeline(*processors)
-    pipeline.set_pref("audio:source_language", source_language)
-    pipeline.set_pref("audio:target_language", target_language)
-    pipeline.describe()
-    pipeline.on(event_callback)
-
-    # Start processing audio
-    logger.info(f"Opening {filename}")
-    container = av.open(filename)
-    try:
-        logger.info("Start pushing audio into the pipeline")
-        for frame in container.decode(audio=0):
-            await pipeline.push(frame)
-    finally:
-        logger.info("Flushing the pipeline")
-        await pipeline.flush()
-
-    # Run diarization if enabled and we have topics
-    if enable_diarization and not only_transcript and audio_temp_path:
-        topics = topic_collector.get_topics()
-
-        if topics:
-            logger.info(f"Starting diarization with {len(topics)} topics")
-
-            try:
-                from reflector.processors import AudioDiarizationAutoProcessor
-
-                diarization_processor = AudioDiarizationAutoProcessor(
-                    name=diarization_backend
-                )
-
-                diarization_processor.set_pipeline(pipeline)
-
-                # For Modal backend, we need to upload the file to S3 first
-                if diarization_backend == "modal":
-                    from datetime import datetime, timezone
-
-                    from reflector.storage import get_transcripts_storage
-                    from reflector.utils.s3_temp_file import S3TemporaryFile
-
-                    storage = get_transcripts_storage()
-
-                    # Generate a unique filename in evaluation folder
-                    timestamp = datetime.now(timezone.utc).strftime("%Y%m%d_%H%M%S")
-                    audio_filename = f"evaluation/diarization_temp/{timestamp}_{uuid.uuid4().hex}.wav"
-
-                    # Use context manager for automatic cleanup
-                    async with S3TemporaryFile(storage, audio_filename) as s3_file:
-                        # Read and upload the audio file
-                        with open(audio_temp_path, "rb") as f:
-                            audio_data = f.read()
-
-                        audio_url = await s3_file.upload(audio_data)
-                        logger.info(f"Uploaded audio to S3: {audio_filename}")
-
-                        # Create diarization input with S3 URL
-                        diarization_input = AudioDiarizationInput(
-                            audio_url=audio_url, topics=topics
-                        )
-
-                        # Run diarization
-                        await diarization_processor.push(diarization_input)
-                        await diarization_processor.flush()
-
-                        logger.info("Diarization complete")
-                        # File will be automatically cleaned up when exiting the context
-                else:
-                    # For local backend, use local file path
-                    audio_url = audio_temp_path
-
-                    # Create diarization input
-                    diarization_input = AudioDiarizationInput(
-                        audio_url=audio_url, topics=topics
-                    )
-
-                    # Run diarization
-                    await diarization_processor.push(diarization_input)
-                    await diarization_processor.flush()
-
-                    logger.info("Diarization complete")
-
-            except ImportError as e:
-                logger.error(f"Failed to import diarization dependencies: {e}")
-                logger.error(
-                    "Install with: uv pip install pyannote.audio torch torchaudio"
-                )
-                logger.error(
-                    "And set HF_TOKEN environment variable for pyannote models"
-                )
-                raise SystemExit(1)
-            except Exception as e:
-                logger.error(f"Diarization failed: {e}")
-                raise SystemExit(1)
-        else:
-            logger.warning("Skipping diarization: no topics available")
-
-    # Clean up temp file
-    if audio_temp_path:
-        try:
-            Path(audio_temp_path).unlink()
-        except Exception as e:
-            logger.warning(f"Failed to clean up temp file {audio_temp_path}: {e}")
-
-    logger.info("All done!")
-
-
-if __name__ == "__main__":
-    import argparse
-    import os
-
-    parser = argparse.ArgumentParser(
-        description="Process audio files with optional speaker diarization"
-    )
-    parser.add_argument("source", help="Source file (mp3, wav, mp4...)")
-    parser.add_argument(
-        "--only-transcript",
-        "-t",
-        action="store_true",
-        help="Only generate transcript without topics/summaries",
-    )
-    parser.add_argument(
-        "--source-language", default="en", help="Source language code (default: en)"
-    )
-    parser.add_argument(
-        "--target-language", default="en", help="Target language code (default: en)"
-    )
-    parser.add_argument("--output", "-o", help="Output file (output.jsonl)")
-    parser.add_argument(
-        "--enable-diarization",
-        "-d",
-        action="store_true",
-        help="Enable speaker diarization",
-    )
-    parser.add_argument(
-        "--diarization-backend",
-        default="modal",
-        choices=["modal"],
-        help="Diarization backend to use (default: modal)",
-    )
-    args = parser.parse_args()
-
-    # Set REDIS_HOST to localhost if not provided
-    if "REDIS_HOST" not in os.environ:
-        os.environ["REDIS_HOST"] = "localhost"
-        logger.info("REDIS_HOST not set, defaulting to localhost")
-
-    output_fd = None
-    if args.output:
-        output_fd = open(args.output, "w")
-
-    async def event_callback(event: PipelineEvent):
-        processor = event.processor
-        data = event.data
-
-        # Ignore internal processors
-        if processor in (
-            "AudioDownscaleProcessor",
-            "AudioChunkerAutoProcessor",
-            "AudioMergeProcessor",
-            "AudioFileWriterProcessor",
-            "TopicCollectorProcessor",
-            "BroadcastProcessor",
-        ):
-            return
-
-        # If diarization is enabled, skip the original topic events from the pipeline
-        # The diarization processor will emit the same topics but with speaker info
-        if processor == "TranscriptTopicDetectorProcessor" and args.enable_diarization:
-            return
-
-        # Log all events
-        logger.info(f"Event: {processor} - {type(data).__name__}")
-
-        # Write to output
-        if output_fd:
-            output_fd.write(event.model_dump_json())
-            output_fd.write("\n")
-            output_fd.flush()
-
-    asyncio.run(
-        process_audio_file_with_diarization(
-            args.source,
-            event_callback,
-            only_transcript=args.only_transcript,
-            source_language=args.source_language,
-            target_language=args.target_language,
-            enable_diarization=args.enable_diarization,
-            diarization_backend=args.diarization_backend,
-        )
-    )
-
-    if output_fd:
-        output_fd.close()
-        logger.info(f"Output written to {args.output}")

server/reflector/tools/test_diarization.py

@@ -1,96 +0,0 @@
-#!/usr/bin/env python3
-"""
-@vibe-generated
-Test script for the diarization CLI tool
-=========================================
-
-This script helps test the diarization functionality with sample audio files.
-"""
-
-import asyncio
-import sys
-from pathlib import Path
-
-from reflector.logger import logger
-
-
-async def test_diarization(audio_file: str):
-    """Test the diarization functionality"""
-
-    # Import the processing function
-    from process_with_diarization import process_audio_file_with_diarization
-
-    # Collect events
-    events = []
-
-    async def event_callback(event):
-        events.append({"processor": event.processor, "data": event.data})
-        logger.info(f"Event from {event.processor}")
-
-    # Process the audio file
-    logger.info(f"Processing audio file: {audio_file}")
-
-    try:
-        await process_audio_file_with_diarization(
-            audio_file,
-            event_callback,
-            only_transcript=False,
-            source_language="en",
-            target_language="en",
-            enable_diarization=True,
-            diarization_backend="modal",
-        )
-
-        # Analyze results
-        logger.info(f"Processing complete. Received {len(events)} events")
-
-        # Look for diarization results
-        diarized_topics = []
-        for event in events:
-            if "TitleSummary" in event["processor"]:
-                # Check if words have speaker information
-                if hasattr(event["data"], "transcript") and event["data"].transcript:
-                    words = event["data"].transcript.words
-                    if words and hasattr(words[0], "speaker"):
-                        speakers = set(
-                            w.speaker for w in words if hasattr(w, "speaker")
-                        )
-                        logger.info(
-                            f"Found {len(speakers)} speakers in topic: {event['data'].title}"
-                        )
-                        diarized_topics.append(event["data"])
-
-        if diarized_topics:
-            logger.info(f"Successfully diarized {len(diarized_topics)} topics")
-
-            # Print sample output
-            sample_topic = diarized_topics[0]
-            logger.info("Sample diarized output:")
-            for i, word in enumerate(sample_topic.transcript.words[:10]):
-                logger.info(f"  Word {i}: '{word.text}' - Speaker {word.speaker}")
-        else:
-            logger.warning("No diarization results found in output")
-
-        return events
-
-    except Exception as e:
-        logger.error(f"Error during processing: {e}")
-        raise
-
-
-def main():
-    if len(sys.argv) < 2:
-        print("Usage: python test_diarization.py <audio_file>")
-        sys.exit(1)
-
-    audio_file = sys.argv[1]
-    if not Path(audio_file).exists():
-        print(f"Error: Audio file '{audio_file}' not found")
-        sys.exit(1)
-
-    # Run the test
-    asyncio.run(test_diarization(audio_file))
-
-
-if __name__ == "__main__":
-    main()

server/reflector/utils/string.py

@@ -0,0 +1,23 @@
+from typing import Annotated
+
+from pydantic import Field, TypeAdapter, constr
+
+NonEmptyStringBase = constr(min_length=1, strip_whitespace=False)
+NonEmptyString = Annotated[
+    NonEmptyStringBase,
+    Field(description="A non-empty string", min_length=1),
+]
+non_empty_string_adapter = TypeAdapter(NonEmptyString)
+
+
+def parse_non_empty_string(s: str, error: str | None = None) -> NonEmptyString:
+    try:
+        return non_empty_string_adapter.validate_python(s)
+    except Exception as e:
+        raise ValueError(f"{e}: {error}" if error else e) from e
+
+
+def try_parse_non_empty_string(s: str) -> NonEmptyString | None:
+    if not s:
+        return None
+    return parse_non_empty_string(s)

server/reflector/views/meetings.py

@@ -10,6 +10,7 @@ from reflector.db.meetings import (
     meeting_consent_controller,
     meetings_controller,
 )
+from reflector.db.rooms import rooms_controller
 
 router = APIRouter()
 
@@ -41,3 +42,34 @@ async def meeting_audio_consent(
     updated_consent = await meeting_consent_controller.upsert(consent)
 
     return {"status": "success", "consent_id": updated_consent.id}
+
+
+@router.patch("/meetings/{meeting_id}/deactivate")
+async def meeting_deactivate(
+    meeting_id: str,
+    user: Annotated[Optional[auth.UserInfo], Depends(auth.current_user)],
+):
+    user_id = user["sub"] if user else None
+    if not user_id:
+        raise HTTPException(status_code=401, detail="Authentication required")
+
+    meeting = await meetings_controller.get_by_id(meeting_id)
+    if not meeting:
+        raise HTTPException(status_code=404, detail="Meeting not found")
+
+    if not meeting.is_active:
+        return {"status": "success", "meeting_id": meeting_id}
+
+    # Only room owner or meeting creator can deactivate
+    room = await rooms_controller.get_by_id(meeting.room_id)
+    if not room:
+        raise HTTPException(status_code=404, detail="Room not found")
+
+    if user_id != room.user_id and user_id != meeting.user_id:
+        raise HTTPException(
+            status_code=403, detail="Only the room owner can deactivate meetings"
+        )
+
+    await meetings_controller.update_meeting(meeting_id, is_active=False)
+
+    return {"status": "success", "meeting_id": meeting_id}

server/reflector/views/rooms.py

@@ -1,33 +1,27 @@
 import logging
-import sqlite3
 from datetime import datetime, timedelta, timezone
-from typing import Annotated, Literal, Optional
+from enum import Enum
+from typing import Annotated, Any, Literal, Optional
 
-import asyncpg.exceptions
 from fastapi import APIRouter, Depends, HTTPException
 from fastapi_pagination import Page
 from fastapi_pagination.ext.databases import apaginate
 from pydantic import BaseModel
+from redis.exceptions import LockError
 
 import reflector.auth as auth
 from reflector.db import get_database
+from reflector.db.calendar_events import calendar_events_controller
 from reflector.db.meetings import meetings_controller
 from reflector.db.rooms import rooms_controller
+from reflector.redis_cache import RedisAsyncLock
+from reflector.services.ics_sync import ics_sync_service
 from reflector.settings import settings
 from reflector.whereby import create_meeting, upload_logo
+from reflector.worker.webhook import test_webhook
 
 logger = logging.getLogger(__name__)
 
-router = APIRouter()
-
-
-def parse_datetime_with_timezone(iso_string: str) -> datetime:
-    """Parse ISO datetime string and ensure timezone awareness (defaults to UTC if naive)."""
-    dt = datetime.fromisoformat(iso_string)
-    if dt.tzinfo is None:
-        dt = dt.replace(tzinfo=timezone.utc)
-    return dt
-
 
 class Room(BaseModel):
     id: str
@@ -42,16 +36,38 @@ class Room(BaseModel):
     recording_type: str
     recording_trigger: str
     is_shared: bool
+    ics_url: Optional[str] = None
+    ics_fetch_interval: int = 300
+    ics_enabled: bool = False
+    ics_last_sync: Optional[datetime] = None
+    ics_last_etag: Optional[str] = None
+
+
+class RoomDetails(Room):
+    webhook_url: str | None
+    webhook_secret: str | None
 
 
 class Meeting(BaseModel):
     id: str
     room_name: str
     room_url: str
+    # TODO it's not always present, | None
     host_room_url: str
     start_date: datetime
     end_date: datetime
+    user_id: str | None = None
+    room_id: str | None = None
+    is_locked: bool = False
+    room_mode: Literal["normal", "group"] = "normal"
     recording_type: Literal["none", "local", "cloud"] = "cloud"
+    recording_trigger: Literal[
+        "none", "prompt", "automatic", "automatic-2nd-participant"
+    ] = "automatic-2nd-participant"
+    num_clients: int = 0
+    is_active: bool = True
+    calendar_event_id: str | None = None
+    calendar_metadata: dict[str, Any] | None = None
 
 
 class CreateRoom(BaseModel):
@@ -64,28 +80,103 @@ class CreateRoom(BaseModel):
     recording_type: str
     recording_trigger: str
     is_shared: bool
+    webhook_url: str
+    webhook_secret: str
+    ics_url: Optional[str] = None
+    ics_fetch_interval: int = 300
+    ics_enabled: bool = False
 
 
 class UpdateRoom(BaseModel):
-    name: str
-    zulip_auto_post: bool
-    zulip_stream: str
-    zulip_topic: str
-    is_locked: bool
-    room_mode: str
-    recording_type: str
-    recording_trigger: str
-    is_shared: bool
+    name: Optional[str] = None
+    zulip_auto_post: Optional[bool] = None
+    zulip_stream: Optional[str] = None
+    zulip_topic: Optional[str] = None
+    is_locked: Optional[bool] = None
+    room_mode: Optional[str] = None
+    recording_type: Optional[str] = None
+    recording_trigger: Optional[str] = None
+    is_shared: Optional[bool] = None
+    webhook_url: Optional[str] = None
+    webhook_secret: Optional[str] = None
+    ics_url: Optional[str] = None
+    ics_fetch_interval: Optional[int] = None
+    ics_enabled: Optional[bool] = None
+
+
+class CreateRoomMeeting(BaseModel):
+    allow_duplicated: Optional[bool] = False
 
 
 class DeletionStatus(BaseModel):
     status: str
 
 
-@router.get("/rooms", response_model=Page[Room])
+class WebhookTestResult(BaseModel):
+    success: bool
+    message: str = ""
+    error: str = ""
+    status_code: int | None = None
+    response_preview: str | None = None
+
+
+class ICSStatus(BaseModel):
+    status: Literal["enabled", "disabled"]
+    last_sync: Optional[datetime] = None
+    next_sync: Optional[datetime] = None
+    last_etag: Optional[str] = None
+    events_count: int = 0
+
+
+class SyncStatus(str, Enum):
+    success = "success"
+    unchanged = "unchanged"
+    error = "error"
+    skipped = "skipped"
+
+
+class ICSSyncResult(BaseModel):
+    status: SyncStatus
+    hash: Optional[str] = None
+    events_found: int = 0
+    total_events: int = 0
+    events_created: int = 0
+    events_updated: int = 0
+    events_deleted: int = 0
+    error: Optional[str] = None
+    reason: Optional[str] = None
+
+
+class CalendarEventResponse(BaseModel):
+    id: str
+    room_id: str
+    ics_uid: str
+    title: Optional[str] = None
+    description: Optional[str] = None
+    start_time: datetime
+    end_time: datetime
+    attendees: Optional[list[dict]] = None
+    location: Optional[str] = None
+    last_synced: datetime
+    created_at: datetime
+    updated_at: datetime
+
+
+router = APIRouter()
+
+
+def parse_datetime_with_timezone(iso_string: str) -> datetime:
+    """Parse ISO datetime string and ensure timezone awareness (defaults to UTC if naive)."""
+    dt = datetime.fromisoformat(iso_string)
+    if dt.tzinfo is None:
+        dt = dt.replace(tzinfo=timezone.utc)
+    return dt
+
+
+@router.get("/rooms", response_model=Page[RoomDetails])
 async def rooms_list(
     user: Annotated[Optional[auth.UserInfo], Depends(auth.current_user_optional)],
-) -> list[Room]:
+) -> list[RoomDetails]:
     if not user and not settings.PUBLIC_MODE:
         raise HTTPException(status_code=401, detail="Not authenticated")
 
@@ -99,6 +190,42 @@ async def rooms_list(
     )
 
 
+@router.get("/rooms/{room_id}", response_model=RoomDetails)
+async def rooms_get(
+    room_id: str,
+    user: Annotated[Optional[auth.UserInfo], Depends(auth.current_user_optional)],
+):
+    user_id = user["sub"] if user else None
+    room = await rooms_controller.get_by_id_for_http(room_id, user_id=user_id)
+    if not room:
+        raise HTTPException(status_code=404, detail="Room not found")
+    return room
+
+
+@router.get("/rooms/name/{room_name}", response_model=RoomDetails)
+async def rooms_get_by_name(
+    room_name: str,
+    user: Annotated[Optional[auth.UserInfo], Depends(auth.current_user_optional)],
+):
+    user_id = user["sub"] if user else None
+    room = await rooms_controller.get_by_name(room_name)
+    if not room:
+        raise HTTPException(status_code=404, detail="Room not found")
+
+    # Convert to RoomDetails format (add webhook fields if user is owner)
+    room_dict = room.__dict__.copy()
+    if user_id == room.user_id:
+        # User is owner, include webhook details if available
+        room_dict["webhook_url"] = getattr(room, "webhook_url", None)
+        room_dict["webhook_secret"] = getattr(room, "webhook_secret", None)
+    else:
+        # Non-owner, hide webhook details
+        room_dict["webhook_url"] = None
+        room_dict["webhook_secret"] = None
+
+    return RoomDetails(**room_dict)
+
+
 @router.post("/rooms", response_model=Room)
 async def rooms_create(
     room: CreateRoom,
@@ -117,10 +244,15 @@ async def rooms_create(
         recording_type=room.recording_type,
         recording_trigger=room.recording_trigger,
         is_shared=room.is_shared,
+        webhook_url=room.webhook_url,
+        webhook_secret=room.webhook_secret,
+        ics_url=room.ics_url,
+        ics_fetch_interval=room.ics_fetch_interval,
+        ics_enabled=room.ics_enabled,
     )
 
 
-@router.patch("/rooms/{room_id}", response_model=Room)
+@router.patch("/rooms/{room_id}", response_model=RoomDetails)
 async def rooms_update(
     room_id: str,
     info: UpdateRoom,
@@ -151,6 +283,7 @@ async def rooms_delete(
 @router.post("/rooms/{room_name}/meeting", response_model=Meeting)
 async def rooms_create_meeting(
     room_name: str,
+    info: CreateRoomMeeting,
     user: Annotated[Optional[auth.UserInfo], Depends(auth.current_user_optional)],
 ):
     user_id = user["sub"] if user else None
@@ -158,54 +291,266 @@ async def rooms_create_meeting(
     if not room:
         raise HTTPException(status_code=404, detail="Room not found")
 
-    current_time = datetime.now(timezone.utc)
-    meeting = await meetings_controller.get_active(room=room, current_time=current_time)
+    try:
+        async with RedisAsyncLock(
+            f"create_meeting:{room_name}",
+            timeout=30,
+            extend_interval=10,
+            blocking_timeout=5.0,
+        ) as lock:
+            current_time = datetime.now(timezone.utc)
 
-    if meeting is None:
-        end_date = current_time + timedelta(hours=8)
+            meeting = None
+            if not info.allow_duplicated:
+                meeting = await meetings_controller.get_active(
+                    room=room, current_time=current_time
+                )
 
-        whereby_meeting = await create_meeting("", end_date=end_date, room=room)
-        await upload_logo(whereby_meeting["roomName"], "./images/logo.png")
-
-        # Now try to save to database
-        try:
-            meeting = await meetings_controller.create(
-                id=whereby_meeting["meetingId"],
-                room_name=whereby_meeting["roomName"],
-                room_url=whereby_meeting["roomUrl"],
-                host_room_url=whereby_meeting["hostRoomUrl"],
-                start_date=parse_datetime_with_timezone(whereby_meeting["startDate"]),
-                end_date=parse_datetime_with_timezone(whereby_meeting["endDate"]),
-                user_id=user_id,
-                room=room,
-            )
-        except (asyncpg.exceptions.UniqueViolationError, sqlite3.IntegrityError):
-            # Another request already created a meeting for this room
-            # Log this race condition occurrence
-            logger.info(
-                "Race condition detected for room %s - fetching existing meeting",
-                room.name,
-            )
-            logger.warning(
-                "Whereby meeting %s was created but not used (resource leak) for room %s",
-                whereby_meeting["meetingId"],
-                room.name,
-            )
-
-            # Fetch the meeting that was created by the other request
-            meeting = await meetings_controller.get_active(
-                room=room, current_time=current_time
-            )
             if meeting is None:
-                # Edge case: meeting was created but expired/deleted between checks
-                logger.error(
-                    "Meeting disappeared after race condition for room %s", room.name
-                )
-                raise HTTPException(
-                    status_code=503, detail="Unable to join meeting - please try again"
+                end_date = current_time + timedelta(hours=8)
+
+                whereby_meeting = await create_meeting("", end_date=end_date, room=room)
+
+                await upload_logo(whereby_meeting["roomName"], "./images/logo.png")
+
+                meeting = await meetings_controller.create(
+                    id=whereby_meeting["meetingId"],
+                    room_name=whereby_meeting["roomName"],
+                    room_url=whereby_meeting["roomUrl"],
+                    host_room_url=whereby_meeting["hostRoomUrl"],
+                    start_date=parse_datetime_with_timezone(
+                        whereby_meeting["startDate"]
+                    ),
+                    end_date=parse_datetime_with_timezone(whereby_meeting["endDate"]),
+                    room=room,
                 )
+    except LockError:
+        logger.warning("Failed to acquire lock for room %s within timeout", room_name)
+        raise HTTPException(
+            status_code=503, detail="Meeting creation in progress, please try again"
+        )
 
     if user_id != room.user_id:
         meeting.host_room_url = ""
 
     return meeting
+
+
+@router.post("/rooms/{room_id}/webhook/test", response_model=WebhookTestResult)
+async def rooms_test_webhook(
+    room_id: str,
+    user: Annotated[Optional[auth.UserInfo], Depends(auth.current_user_optional)],
+):
+    """Test webhook configuration by sending a sample payload."""
+    user_id = user["sub"] if user else None
+
+    room = await rooms_controller.get_by_id(room_id)
+    if not room:
+        raise HTTPException(status_code=404, detail="Room not found")
+
+    if user_id and room.user_id != user_id:
+        raise HTTPException(
+            status_code=403, detail="Not authorized to test this room's webhook"
+        )
+
+    result = await test_webhook(room_id)
+    return WebhookTestResult(**result)
+
+
+@router.post("/rooms/{room_name}/ics/sync", response_model=ICSSyncResult)
+async def rooms_sync_ics(
+    room_name: str,
+    user: Annotated[Optional[auth.UserInfo], Depends(auth.current_user_optional)],
+):
+    user_id = user["sub"] if user else None
+    room = await rooms_controller.get_by_name(room_name)
+
+    if not room:
+        raise HTTPException(status_code=404, detail="Room not found")
+
+    if user_id != room.user_id:
+        raise HTTPException(
+            status_code=403, detail="Only room owner can trigger ICS sync"
+        )
+
+    if not room.ics_enabled or not room.ics_url:
+        raise HTTPException(status_code=400, detail="ICS not configured for this room")
+
+    result = await ics_sync_service.sync_room_calendar(room)
+
+    if result["status"] == "error":
+        raise HTTPException(
+            status_code=500, detail=result.get("error", "Unknown error")
+        )
+
+    return ICSSyncResult(**result)
+
+
+@router.get("/rooms/{room_name}/ics/status", response_model=ICSStatus)
+async def rooms_ics_status(
+    room_name: str,
+    user: Annotated[Optional[auth.UserInfo], Depends(auth.current_user_optional)],
+):
+    user_id = user["sub"] if user else None
+    room = await rooms_controller.get_by_name(room_name)
+
+    if not room:
+        raise HTTPException(status_code=404, detail="Room not found")
+
+    if user_id != room.user_id:
+        raise HTTPException(
+            status_code=403, detail="Only room owner can view ICS status"
+        )
+
+    next_sync = None
+    if room.ics_enabled and room.ics_last_sync:
+        next_sync = room.ics_last_sync + timedelta(seconds=room.ics_fetch_interval)
+
+    events = await calendar_events_controller.get_by_room(
+        room.id, include_deleted=False
+    )
+
+    return ICSStatus(
+        status="enabled" if room.ics_enabled else "disabled",
+        last_sync=room.ics_last_sync,
+        next_sync=next_sync,
+        last_etag=room.ics_last_etag,
+        events_count=len(events),
+    )
+
+
+@router.get("/rooms/{room_name}/meetings", response_model=list[CalendarEventResponse])
+async def rooms_list_meetings(
+    room_name: str,
+    user: Annotated[Optional[auth.UserInfo], Depends(auth.current_user_optional)],
+):
+    user_id = user["sub"] if user else None
+    room = await rooms_controller.get_by_name(room_name)
+
+    if not room:
+        raise HTTPException(status_code=404, detail="Room not found")
+
+    events = await calendar_events_controller.get_by_room(
+        room.id, include_deleted=False
+    )
+
+    if user_id != room.user_id:
+        for event in events:
+            event.description = None
+            event.attendees = None
+
+    return events
+
+
+@router.get(
+    "/rooms/{room_name}/meetings/upcoming", response_model=list[CalendarEventResponse]
+)
+async def rooms_list_upcoming_meetings(
+    room_name: str,
+    user: Annotated[Optional[auth.UserInfo], Depends(auth.current_user_optional)],
+    minutes_ahead: int = 120,
+):
+    user_id = user["sub"] if user else None
+    room = await rooms_controller.get_by_name(room_name)
+
+    if not room:
+        raise HTTPException(status_code=404, detail="Room not found")
+
+    events = await calendar_events_controller.get_upcoming(
+        room.id, minutes_ahead=minutes_ahead
+    )
+
+    if user_id != room.user_id:
+        for event in events:
+            event.description = None
+            event.attendees = None
+
+    return events
+
+
+@router.get("/rooms/{room_name}/meetings/active", response_model=list[Meeting])
+async def rooms_list_active_meetings(
+    room_name: str,
+    user: Annotated[Optional[auth.UserInfo], Depends(auth.current_user_optional)],
+):
+    user_id = user["sub"] if user else None
+    room = await rooms_controller.get_by_name(room_name)
+
+    if not room:
+        raise HTTPException(status_code=404, detail="Room not found")
+
+    current_time = datetime.now(timezone.utc)
+    meetings = await meetings_controller.get_all_active_for_room(
+        room=room, current_time=current_time
+    )
+
+    # Hide host URLs from non-owners
+    if user_id != room.user_id:
+        for meeting in meetings:
+            meeting.host_room_url = ""
+
+    return meetings
+
+
+@router.get("/rooms/{room_name}/meetings/{meeting_id}", response_model=Meeting)
+async def rooms_get_meeting(
+    room_name: str,
+    meeting_id: str,
+    user: Annotated[Optional[auth.UserInfo], Depends(auth.current_user_optional)],
+):
+    """Get a single meeting by ID within a specific room."""
+    user_id = user["sub"] if user else None
+
+    room = await rooms_controller.get_by_name(room_name)
+    if not room:
+        raise HTTPException(status_code=404, detail="Room not found")
+
+    meeting = await meetings_controller.get_by_id(meeting_id)
+    if not meeting:
+        raise HTTPException(status_code=404, detail="Meeting not found")
+
+    if meeting.room_id != room.id:
+        raise HTTPException(
+            status_code=403, detail="Meeting does not belong to this room"
+        )
+
+    if user_id != room.user_id and not room.is_shared:
+        meeting.host_room_url = ""
+
+    return meeting
+
+
+@router.post("/rooms/{room_name}/meetings/{meeting_id}/join", response_model=Meeting)
+async def rooms_join_meeting(
+    room_name: str,
+    meeting_id: str,
+    user: Annotated[Optional[auth.UserInfo], Depends(auth.current_user_optional)],
+):
+    user_id = user["sub"] if user else None
+    room = await rooms_controller.get_by_name(room_name)
+
+    if not room:
+        raise HTTPException(status_code=404, detail="Room not found")
+
+    meeting = await meetings_controller.get_by_id(meeting_id)
+
+    if not meeting:
+        raise HTTPException(status_code=404, detail="Meeting not found")
+
+    if meeting.room_id != room.id:
+        raise HTTPException(
+            status_code=403, detail="Meeting does not belong to this room"
+        )
+
+    if not meeting.is_active:
+        raise HTTPException(status_code=400, detail="Meeting is not active")
+
+    current_time = datetime.now(timezone.utc)
+    if meeting.end_date <= current_time:
+        raise HTTPException(status_code=400, detail="Meeting has ended")
+
+    # Hide host URL from non-owners
+    if user_id != room.user_id:
+        meeting.host_room_url = ""
+
+    return meeting

server/reflector/views/transcripts.py

@@ -5,7 +5,7 @@ from fastapi import APIRouter, Depends, HTTPException, Query
 from fastapi_pagination import Page
 from fastapi_pagination.ext.databases import apaginate
 from jose import jwt
-from pydantic import BaseModel, Field, field_serializer
+from pydantic import BaseModel, Field, constr, field_serializer
 
 import reflector.auth as auth
 from reflector.db import get_database
@@ -19,14 +19,15 @@ from reflector.db.search import (
     SearchOffsetBase,
     SearchParameters,
     SearchQuery,
-    SearchQueryBase,
     SearchResult,
     SearchTotal,
     search_controller,
+    search_query_adapter,
 )
 from reflector.db.transcripts import (
     SourceKind,
     TranscriptParticipant,
+    TranscriptStatus,
     TranscriptTopic,
     transcripts_controller,
 )
@@ -63,7 +64,7 @@ class GetTranscriptMinimal(BaseModel):
     id: str
     user_id: str | None
     name: str
-    status: str
+    status: TranscriptStatus
     locked: bool
     duration: float
     title: str | None
@@ -96,6 +97,7 @@ class CreateTranscript(BaseModel):
     name: str
     source_language: str = Field("en")
     target_language: str = Field("en")
+    source_kind: SourceKind | None = None
 
 
 class UpdateTranscript(BaseModel):
@@ -114,7 +116,19 @@ class DeletionStatus(BaseModel):
     status: str
 
 
-SearchQueryParam = Annotated[SearchQueryBase, Query(description="Search query text")]
+SearchQueryParamBase = constr(min_length=0, strip_whitespace=True)
+SearchQueryParam = Annotated[
+    SearchQueryParamBase, Query(description="Search query text")
+]
+
+
+# http and api standards accept "q="; we would like to handle it as the absence of query, not as "empty string query"
+def parse_search_query_param(q: SearchQueryParam) -> SearchQuery | None:
+    if q == "":
+        return None
+    return search_query_adapter.validate_python(q)
+
+
 SearchLimitParam = Annotated[SearchLimitBase, Query(description="Results per page")]
 SearchOffsetParam = Annotated[
     SearchOffsetBase, Query(description="Number of results to skip")
@@ -124,7 +138,7 @@ SearchOffsetParam = Annotated[
 class SearchResponse(BaseModel):
     results: list[SearchResult]
     total: SearchTotal
-    query: SearchQuery
+    query: SearchQuery | None = None
     limit: SearchLimit
     offset: SearchOffset
 
@@ -174,7 +188,7 @@ async def transcripts_search(
     user_id = user["sub"] if user else None
 
     search_params = SearchParameters(
-        query_text=q,
+        query_text=parse_search_query_param(q),
         limit=limit,
         offset=offset,
         user_id=user_id,
@@ -201,7 +215,7 @@ async def transcripts_create(
     user_id = user["sub"] if user else None
     return await transcripts_controller.add(
         info.name,
-        source_kind=SourceKind.LIVE,
+        source_kind=info.source_kind or SourceKind.LIVE,
         source_language=info.source_language,
         target_language=info.target_language,
         user_id=user_id,
@@ -336,8 +350,6 @@ async def transcript_update(
     transcript = await transcripts_controller.get_by_id_for_http(
         transcript_id, user_id=user_id
     )
-    if not transcript:
-        raise HTTPException(status_code=404, detail="Transcript not found")
     values = info.dict(exclude_unset=True)
     updated_transcript = await transcripts_controller.update(transcript, values)
     return updated_transcript

server/reflector/views/transcripts_process.py

@@ -6,7 +6,7 @@ from pydantic import BaseModel
 
 import reflector.auth as auth
 from reflector.db.transcripts import transcripts_controller
-from reflector.pipelines.main_live_pipeline import task_pipeline_process
+from reflector.pipelines.main_file_pipeline import task_pipeline_file_process
 
 router = APIRouter()
 
@@ -34,13 +34,13 @@ async def transcript_process(
         )
 
     if task_is_scheduled_or_active(
-        "reflector.pipelines.main_live_pipeline.task_pipeline_process",
+        "reflector.pipelines.main_file_pipeline.task_pipeline_file_process",
         transcript_id=transcript_id,
     ):
         return ProcessStatus(status="already running")
 
     # schedule a background task process the file
-    task_pipeline_process.delay(transcript_id=transcript_id)
+    task_pipeline_file_process.delay(transcript_id=transcript_id)
 
     return ProcessStatus(status="ok")
 

server/reflector/views/transcripts_upload.py

@@ -6,7 +6,7 @@ from pydantic import BaseModel
 
 import reflector.auth as auth
 from reflector.db.transcripts import transcripts_controller
-from reflector.pipelines.main_live_pipeline import task_pipeline_process
+from reflector.pipelines.main_file_pipeline import task_pipeline_file_process
 
 router = APIRouter()
 
@@ -92,6 +92,6 @@ async def transcript_record_upload(
     await transcripts_controller.update(transcript, {"status": "uploaded"})
 
     # launch a background task to process the file
-    task_pipeline_process.delay(transcript_id=transcript_id)
+    task_pipeline_file_process.delay(transcript_id=transcript_id)
 
     return UploadStatus(status="ok")

server/reflector/views/whereby.py

@@ -68,8 +68,7 @@ async def whereby_webhook(event: WherebyWebhookEvent, request: Request):
         raise HTTPException(status_code=404, detail="Meeting not found")
 
     if event.type in ["room.client.joined", "room.client.left"]:
-        await meetings_controller.update_meeting(
-            meeting.id, num_clients=event.data["numClients"]
-        )
+        update_data = {"num_clients": event.data["numClients"]}
+        await meetings_controller.update_meeting(meeting.id, **update_data)
 
     return {"status": "ok"}

server/reflector/whereby.py

@@ -1,18 +1,60 @@
+import logging
 from datetime import datetime
 
 import httpx
 
 from reflector.db.rooms import Room
 from reflector.settings import settings
+from reflector.utils.string import parse_non_empty_string
+
+logger = logging.getLogger(__name__)
+
+
+def _get_headers():
+    api_key = parse_non_empty_string(
+        settings.WHEREBY_API_KEY, "WHEREBY_API_KEY value is required."
+    )
+    return {
+        "Content-Type": "application/json; charset=utf-8",
+        "Authorization": f"Bearer {api_key}",
+    }
+
 
-HEADERS = {
-    "Content-Type": "application/json; charset=utf-8",
-    "Authorization": f"Bearer {settings.WHEREBY_API_KEY}",
-}
 TIMEOUT = 10  # seconds
 
 
+def _get_whereby_s3_auth():
+    errors = []
+    try:
+        bucket_name = parse_non_empty_string(
+            settings.RECORDING_STORAGE_AWS_BUCKET_NAME,
+            "RECORDING_STORAGE_AWS_BUCKET_NAME value is required.",
+        )
+    except Exception as e:
+        errors.append(e)
+    try:
+        key_id = parse_non_empty_string(
+            settings.AWS_WHEREBY_ACCESS_KEY_ID,
+            "AWS_WHEREBY_ACCESS_KEY_ID value is required.",
+        )
+    except Exception as e:
+        errors.append(e)
+    try:
+        key_secret = parse_non_empty_string(
+            settings.AWS_WHEREBY_ACCESS_KEY_SECRET,
+            "AWS_WHEREBY_ACCESS_KEY_SECRET value is required.",
+        )
+    except Exception as e:
+        errors.append(e)
+    if len(errors) > 0:
+        raise Exception(
+            f"Failed to get Whereby auth settings: {', '.join(str(e) for e in errors)}"
+        )
+    return bucket_name, key_id, key_secret
+
+
 async def create_meeting(room_name_prefix: str, end_date: datetime, room: Room):
+    s3_bucket_name, s3_key_id, s3_key_secret = _get_whereby_s3_auth()
     data = {
         "isLocked": room.is_locked,
         "roomNamePrefix": room_name_prefix,
@@ -23,23 +65,26 @@ async def create_meeting(room_name_prefix: str, end_date: datetime, room: Room):
             "type": room.recording_type,
             "destination": {
                 "provider": "s3",
-                "bucket": settings.RECORDING_STORAGE_AWS_BUCKET_NAME,
-                "accessKeyId": settings.AWS_WHEREBY_ACCESS_KEY_ID,
-                "accessKeySecret": settings.AWS_WHEREBY_ACCESS_KEY_SECRET,
+                "bucket": s3_bucket_name,
+                "accessKeyId": s3_key_id,
+                "accessKeySecret": s3_key_secret,
                 "fileFormat": "mp4",
             },
             "startTrigger": room.recording_trigger,
         },
         "fields": ["hostRoomUrl"],
     }
-
     async with httpx.AsyncClient() as client:
         response = await client.post(
             f"{settings.WHEREBY_API_URL}/meetings",
-            headers=HEADERS,
+            headers=_get_headers(),
             json=data,
             timeout=TIMEOUT,
         )
+        if response.status_code == 403:
+            logger.warning(
+                f"Failed to create meeting: access denied on Whereby: {response.text}"
+            )
         response.raise_for_status()
         return response.json()
 
@@ -48,7 +93,7 @@ async def get_room_sessions(room_name: str):
     async with httpx.AsyncClient() as client:
         response = await client.get(
             f"{settings.WHEREBY_API_URL}/insights/room-sessions?roomName={room_name}",
-            headers=HEADERS,
+            headers=_get_headers(),
             timeout=TIMEOUT,
         )
         response.raise_for_status()

server/reflector/worker/app.py

@@ -19,6 +19,8 @@ else:
             "reflector.pipelines.main_live_pipeline",
             "reflector.worker.healthcheck",
             "reflector.worker.process",
+            "reflector.worker.cleanup",
+            "reflector.worker.ics_sync",
         ]
     )
 
@@ -36,8 +38,26 @@ else:
             "task": "reflector.worker.process.reprocess_failed_recordings",
             "schedule": crontab(hour=5, minute=0),  # Midnight EST
         },
+        "sync_all_ics_calendars": {
+            "task": "reflector.worker.ics_sync.sync_all_ics_calendars",
+            "schedule": 60.0,  # Run every minute to check which rooms need sync
+        },
+        "create_upcoming_meetings": {
+            "task": "reflector.worker.ics_sync.create_upcoming_meetings",
+            "schedule": 30.0,  # Run every 30 seconds to create upcoming meetings
+        },
     }
 
+    if settings.PUBLIC_MODE:
+        app.conf.beat_schedule["cleanup_old_public_data"] = {
+            "task": "reflector.worker.cleanup.cleanup_old_public_data_task",
+            "schedule": crontab(hour=3, minute=0),
+        }
+        logger.info(
+            "Public mode cleanup enabled",
+            retention_days=settings.PUBLIC_DATA_RETENTION_DAYS,
+        )
+
     if settings.HEALTHCHECK_URL:
         app.conf.beat_schedule["healthcheck_ping"] = {
             "task": "reflector.worker.healthcheck.healthcheck_ping",

server/reflector/worker/cleanup.py

@@ -0,0 +1,156 @@
+"""
+Main task for cleanup old public data.
+
+Deletes old anonymous transcripts and their associated meetings/recordings.
+Transcripts are the main entry point - any associated data is also removed.
+"""
+
+import asyncio
+from datetime import datetime, timedelta, timezone
+from typing import TypedDict
+
+import structlog
+from celery import shared_task
+from databases import Database
+from pydantic.types import PositiveInt
+
+from reflector.asynctask import asynctask
+from reflector.db import get_database
+from reflector.db.meetings import meetings
+from reflector.db.recordings import recordings
+from reflector.db.transcripts import transcripts, transcripts_controller
+from reflector.settings import settings
+from reflector.storage import get_recordings_storage
+
+logger = structlog.get_logger(__name__)
+
+
+class CleanupStats(TypedDict):
+    """Statistics for cleanup operation."""
+
+    transcripts_deleted: int
+    meetings_deleted: int
+    recordings_deleted: int
+    errors: list[str]
+
+
+async def delete_single_transcript(
+    db: Database, transcript_data: dict, stats: CleanupStats
+):
+    transcript_id = transcript_data["id"]
+    meeting_id = transcript_data["meeting_id"]
+    recording_id = transcript_data["recording_id"]
+
+    try:
+        async with db.transaction(isolation="serializable"):
+            if meeting_id:
+                await db.execute(meetings.delete().where(meetings.c.id == meeting_id))
+                stats["meetings_deleted"] += 1
+                logger.info("Deleted associated meeting", meeting_id=meeting_id)
+
+            if recording_id:
+                recording = await db.fetch_one(
+                    recordings.select().where(recordings.c.id == recording_id)
+                )
+                if recording:
+                    try:
+                        await get_recordings_storage().delete_file(
+                            recording["object_key"]
+                        )
+                    except Exception as storage_error:
+                        logger.warning(
+                            "Failed to delete recording from storage",
+                            recording_id=recording_id,
+                            object_key=recording["object_key"],
+                            error=str(storage_error),
+                        )
+
+                    await db.execute(
+                        recordings.delete().where(recordings.c.id == recording_id)
+                    )
+                    stats["recordings_deleted"] += 1
+                    logger.info(
+                        "Deleted associated recording", recording_id=recording_id
+                    )
+
+            await transcripts_controller.remove_by_id(transcript_id)
+            stats["transcripts_deleted"] += 1
+            logger.info(
+                "Deleted transcript",
+                transcript_id=transcript_id,
+                created_at=transcript_data["created_at"].isoformat(),
+            )
+    except Exception as e:
+        error_msg = f"Failed to delete transcript {transcript_id}: {str(e)}"
+        logger.error(error_msg, exc_info=e)
+        stats["errors"].append(error_msg)
+
+
+async def cleanup_old_transcripts(
+    db: Database, cutoff_date: datetime, stats: CleanupStats
+):
+    """Delete old anonymous transcripts and their associated recordings/meetings."""
+    query = transcripts.select().where(
+        (transcripts.c.created_at < cutoff_date) & (transcripts.c.user_id.is_(None))
+    )
+    old_transcripts = await db.fetch_all(query)
+
+    logger.info(f"Found {len(old_transcripts)} old transcripts to delete")
+
+    for transcript_data in old_transcripts:
+        await delete_single_transcript(db, transcript_data, stats)
+
+
+def log_cleanup_results(stats: CleanupStats):
+    logger.info(
+        "Cleanup completed",
+        transcripts_deleted=stats["transcripts_deleted"],
+        meetings_deleted=stats["meetings_deleted"],
+        recordings_deleted=stats["recordings_deleted"],
+        errors_count=len(stats["errors"]),
+    )
+
+    if stats["errors"]:
+        logger.warning(
+            "Cleanup completed with errors",
+            errors=stats["errors"][:10],
+        )
+
+
+async def cleanup_old_public_data(
+    days: PositiveInt | None = None,
+) -> CleanupStats | None:
+    if days is None:
+        days = settings.PUBLIC_DATA_RETENTION_DAYS
+
+    if not settings.PUBLIC_MODE:
+        logger.info("Skipping cleanup - not a public instance")
+        return None
+
+    cutoff_date = datetime.now(timezone.utc) - timedelta(days=days)
+    logger.info(
+        "Starting cleanup of old public data",
+        cutoff_date=cutoff_date.isoformat(),
+    )
+
+    stats: CleanupStats = {
+        "transcripts_deleted": 0,
+        "meetings_deleted": 0,
+        "recordings_deleted": 0,
+        "errors": [],
+    }
+
+    db = get_database()
+    await cleanup_old_transcripts(db, cutoff_date, stats)
+
+    log_cleanup_results(stats)
+    return stats
+
+
+@shared_task(
+    autoretry_for=(Exception,),
+    retry_kwargs={"max_retries": 3, "countdown": 300},
+)
+@asynctask
+def cleanup_old_public_data_task(days: int | None = None):
+    asyncio.run(cleanup_old_public_data(days=days))

server/reflector/worker/ics_sync.py

@@ -0,0 +1,175 @@
+from datetime import datetime, timedelta, timezone
+
+import structlog
+from celery import shared_task
+from celery.utils.log import get_task_logger
+
+from reflector.asynctask import asynctask
+from reflector.db.calendar_events import calendar_events_controller
+from reflector.db.meetings import meetings_controller
+from reflector.db.rooms import rooms_controller
+from reflector.redis_cache import RedisAsyncLock
+from reflector.services.ics_sync import SyncStatus, ics_sync_service
+from reflector.whereby import create_meeting, upload_logo
+
+logger = structlog.wrap_logger(get_task_logger(__name__))
+
+
+@shared_task
+@asynctask
+async def sync_room_ics(room_id: str):
+    try:
+        room = await rooms_controller.get_by_id(room_id)
+        if not room:
+            logger.warning("Room not found for ICS sync", room_id=room_id)
+            return
+
+        if not room.ics_enabled or not room.ics_url:
+            logger.debug("ICS not enabled for room", room_id=room_id)
+            return
+
+        logger.info("Starting ICS sync for room", room_id=room_id, room_name=room.name)
+        result = await ics_sync_service.sync_room_calendar(room)
+
+        if result["status"] == SyncStatus.SUCCESS:
+            logger.info(
+                "ICS sync completed successfully",
+                room_id=room_id,
+                events_found=result.get("events_found", 0),
+                events_created=result.get("events_created", 0),
+                events_updated=result.get("events_updated", 0),
+                events_deleted=result.get("events_deleted", 0),
+            )
+        elif result["status"] == SyncStatus.UNCHANGED:
+            logger.debug("ICS content unchanged", room_id=room_id)
+        elif result["status"] == SyncStatus.ERROR:
+            logger.error("ICS sync failed", room_id=room_id, error=result.get("error"))
+        else:
+            logger.debug(
+                "ICS sync skipped", room_id=room_id, reason=result.get("reason")
+            )
+
+    except Exception as e:
+        logger.error("Unexpected error during ICS sync", room_id=room_id, error=str(e))
+
+
+@shared_task
+@asynctask
+async def sync_all_ics_calendars():
+    try:
+        logger.info("Starting sync for all ICS-enabled rooms")
+
+        ics_enabled_rooms = await rooms_controller.get_ics_enabled()
+        logger.info(f"Found {len(ics_enabled_rooms)} rooms with ICS enabled")
+
+        for room in ics_enabled_rooms:
+            if not _should_sync(room):
+                logger.debug("Skipping room, not time to sync yet", room_id=room.id)
+                continue
+
+            sync_room_ics.delay(room.id)
+
+        logger.info("Queued sync tasks for all eligible rooms")
+
+    except Exception as e:
+        logger.error("Error in sync_all_ics_calendars", error=str(e))
+
+
+def _should_sync(room) -> bool:
+    if not room.ics_last_sync:
+        return True
+
+    time_since_sync = datetime.now(timezone.utc) - room.ics_last_sync
+    return time_since_sync.total_seconds() >= room.ics_fetch_interval
+
+
+MEETING_DEFAULT_DURATION = timedelta(hours=1)
+
+
+async def create_upcoming_meetings_for_event(event, create_window, room_id, room):
+    if event.start_time <= create_window:
+        return
+    existing_meeting = await meetings_controller.get_by_calendar_event(event.id)
+
+    if existing_meeting:
+        return
+
+    logger.info(
+        "Pre-creating meeting for calendar event",
+        room_id=room_id,
+        event_id=event.id,
+        event_title=event.title,
+    )
+
+    try:
+        end_date = event.end_time or (event.start_time + MEETING_DEFAULT_DURATION)
+
+        whereby_meeting = await create_meeting(
+            "",
+            end_date=end_date,
+            room=room,
+        )
+        await upload_logo(whereby_meeting["roomName"], "./images/logo.png")
+
+        meeting = await meetings_controller.create(
+            id=whereby_meeting["meetingId"],
+            room_name=whereby_meeting["roomName"],
+            room_url=whereby_meeting["roomUrl"],
+            host_room_url=whereby_meeting["hostRoomUrl"],
+            start_date=datetime.fromisoformat(whereby_meeting["startDate"]),
+            end_date=datetime.fromisoformat(whereby_meeting["endDate"]),
+            room=room,
+            calendar_event_id=event.id,
+            calendar_metadata={
+                "title": event.title,
+                "description": event.description,
+                "attendees": event.attendees,
+            },
+        )
+
+        logger.info(
+            "Meeting pre-created successfully",
+            meeting_id=meeting.id,
+            event_id=event.id,
+        )
+
+    except Exception as e:
+        logger.error(
+            "Failed to pre-create meeting",
+            room_id=room_id,
+            event_id=event.id,
+            error=str(e),
+        )
+
+
+@shared_task
+@asynctask
+async def create_upcoming_meetings():
+    async with RedisAsyncLock("create_upcoming_meetings", skip_if_locked=True) as lock:
+        if not lock.acquired:
+            logger.warning(
+                "Another worker is already creating upcoming meetings, skipping"
+            )
+            return
+
+        try:
+            logger.info("Starting creation of upcoming meetings")
+
+            ics_enabled_rooms = await rooms_controller.get_ics_enabled()
+            now = datetime.now(timezone.utc)
+            create_window = now - timedelta(minutes=6)
+
+            for room in ics_enabled_rooms:
+                events = await calendar_events_controller.get_upcoming(
+                    room.id,
+                    minutes_ahead=7,
+                )
+
+                for event in events:
+                    await create_upcoming_meetings_for_event(
+                        event, create_window, room.id, room
+                    )
+            logger.info("Completed pre-creation check for upcoming meetings")
+
+        except Exception as e:
+            logger.error("Error in create_upcoming_meetings", error=str(e))

server/reflector/worker/process.py

@@ -9,6 +9,7 @@ import structlog
 from celery import shared_task
 from celery.utils.log import get_task_logger
 from pydantic import ValidationError
+from redis.exceptions import LockError
 
 from reflector.db.meetings import meetings_controller
 from reflector.db.recordings import Recording, recordings_controller
@@ -16,6 +17,7 @@ from reflector.db.rooms import rooms_controller
 from reflector.db.transcripts import SourceKind, transcripts_controller
 from reflector.pipelines.main_file_pipeline import task_pipeline_file_process
 from reflector.pipelines.main_live_pipeline import asynctask
+from reflector.redis_cache import get_redis_client
 from reflector.settings import settings
 from reflector.whereby import get_room_sessions
 
@@ -147,24 +149,94 @@ async def process_recording(bucket_name: str, object_key: str):
 @shared_task
 @asynctask
 async def process_meetings():
+    """
+    Checks which meetings are still active and deactivates those that have ended.
+
+    Deactivation logic:
+    - Active sessions: Keep meeting active regardless of scheduled time
+    - No active sessions:
+      * Calendar meetings:
+        - If previously used (had sessions): Deactivate immediately
+        - If never used: Keep active until scheduled end time, then deactivate
+      * On-the-fly meetings: Deactivate immediately (created when someone joins,
+        so no sessions means everyone left)
+
+    Uses distributed locking to prevent race conditions when multiple workers
+    process the same meeting simultaneously.
+    """
     logger.info("Processing meetings")
     meetings = await meetings_controller.get_all_active()
+    current_time = datetime.now(timezone.utc)
+    redis_client = get_redis_client()
+    processed_count = 0
+    skipped_count = 0
+
     for meeting in meetings:
-        is_active = False
-        end_date = meeting.end_date
-        if end_date.tzinfo is None:
-            end_date = end_date.replace(tzinfo=timezone.utc)
-        if end_date > datetime.now(timezone.utc):
+        logger_ = logger.bind(meeting_id=meeting.id, room_name=meeting.room_name)
+        lock_key = f"meeting_process_lock:{meeting.id}"
+        lock = redis_client.lock(lock_key, timeout=120)
+
+        try:
+            if not lock.acquire(blocking=False):
+                logger_.debug("Meeting is being processed by another worker, skipping")
+                skipped_count += 1
+                continue
+
+            # Process the meeting
+            should_deactivate = False
+            end_date = meeting.end_date
+            if end_date.tzinfo is None:
+                end_date = end_date.replace(tzinfo=timezone.utc)
+
+            # This API call could be slow, extend lock if needed
             response = await get_room_sessions(meeting.room_name)
+
+            try:
+                # Extend lock after slow operation to ensure we still hold it
+                lock.extend(120, replace_ttl=True)
+            except LockError:
+                logger_.warning("Lost lock for meeting, skipping")
+                continue
+
             room_sessions = response.get("results", [])
-            is_active = not room_sessions or any(
+            has_active_sessions = room_sessions and any(
                 rs["endedAt"] is None for rs in room_sessions
             )
-        if not is_active:
-            await meetings_controller.update_meeting(meeting.id, is_active=False)
-            logger.info("Meeting %s is deactivated", meeting.id)
+            has_had_sessions = bool(room_sessions)
 
-    logger.info("Processed meetings")
+            if has_active_sessions:
+                logger_.debug("Meeting still has active sessions, keep it")
+            elif has_had_sessions:
+                should_deactivate = True
+                logger_.info("Meeting ended - all participants left")
+            elif current_time > end_date:
+                should_deactivate = True
+                logger_.info(
+                    "Meeting deactivated - scheduled time ended with no participants",
+                    meeting.id,
+                )
+            else:
+                logger_.debug("Meeting not yet started, keep it")
+
+            if should_deactivate:
+                await meetings_controller.update_meeting(meeting.id, is_active=False)
+                logger_.info("Meeting is deactivated")
+
+            processed_count += 1
+
+        except Exception as e:
+            logger_.error(f"Error processing meeting", exc_info=True)
+        finally:
+            try:
+                lock.release()
+            except LockError:
+                pass  # Lock already released or expired
+
+    logger.info(
+        f"Processed meetings finished",
+        processed_count=processed_count,
+        skipped_count=skipped_count,
+    )
 
 
 @shared_task

server/reflector/worker/webhook.py

@@ -0,0 +1,258 @@
+"""Webhook task for sending transcript notifications."""
+
+import hashlib
+import hmac
+import json
+import uuid
+from datetime import datetime, timezone
+
+import httpx
+import structlog
+from celery import shared_task
+from celery.utils.log import get_task_logger
+
+from reflector.db.rooms import rooms_controller
+from reflector.db.transcripts import transcripts_controller
+from reflector.pipelines.main_live_pipeline import asynctask
+from reflector.settings import settings
+from reflector.utils.webvtt import topics_to_webvtt
+
+logger = structlog.wrap_logger(get_task_logger(__name__))
+
+
+def generate_webhook_signature(payload: bytes, secret: str, timestamp: str) -> str:
+    """Generate HMAC signature for webhook payload."""
+    signed_payload = f"{timestamp}.{payload.decode('utf-8')}"
+    hmac_obj = hmac.new(
+        secret.encode("utf-8"),
+        signed_payload.encode("utf-8"),
+        hashlib.sha256,
+    )
+    return hmac_obj.hexdigest()
+
+
+@shared_task(
+    bind=True,
+    max_retries=30,
+    default_retry_delay=60,
+    retry_backoff=True,
+    retry_backoff_max=3600,  # Max 1 hour between retries
+)
+@asynctask
+async def send_transcript_webhook(
+    self,
+    transcript_id: str,
+    room_id: str,
+    event_id: str,
+):
+    log = logger.bind(
+        transcript_id=transcript_id,
+        room_id=room_id,
+        retry_count=self.request.retries,
+    )
+
+    try:
+        # Fetch transcript and room
+        transcript = await transcripts_controller.get_by_id(transcript_id)
+        if not transcript:
+            log.error("Transcript not found, skipping webhook")
+            return
+
+        room = await rooms_controller.get_by_id(room_id)
+        if not room:
+            log.error("Room not found, skipping webhook")
+            return
+
+        if not room.webhook_url:
+            log.info("No webhook URL configured for room, skipping")
+            return
+
+        # Generate WebVTT content from topics
+        topics_data = []
+
+        if transcript.topics:
+            # Build topics data with diarized content per topic
+            for topic in transcript.topics:
+                topic_webvtt = topics_to_webvtt([topic]) if topic.words else ""
+                topics_data.append(
+                    {
+                        "title": topic.title,
+                        "summary": topic.summary,
+                        "timestamp": topic.timestamp,
+                        "duration": topic.duration,
+                        "webvtt": topic_webvtt,
+                    }
+                )
+
+        # Build webhook payload
+        frontend_url = f"{settings.UI_BASE_URL}/transcripts/{transcript.id}"
+        participants = [
+            {"id": p.id, "name": p.name, "speaker": p.speaker}
+            for p in (transcript.participants or [])
+        ]
+        payload_data = {
+            "event": "transcript.completed",
+            "event_id": event_id,
+            "timestamp": datetime.now(timezone.utc).isoformat(),
+            "transcript": {
+                "id": transcript.id,
+                "room_id": transcript.room_id,
+                "created_at": transcript.created_at.isoformat(),
+                "duration": transcript.duration,
+                "title": transcript.title,
+                "short_summary": transcript.short_summary,
+                "long_summary": transcript.long_summary,
+                "webvtt": transcript.webvtt,
+                "topics": topics_data,
+                "participants": participants,
+                "source_language": transcript.source_language,
+                "target_language": transcript.target_language,
+                "status": transcript.status,
+                "frontend_url": frontend_url,
+            },
+            "room": {
+                "id": room.id,
+                "name": room.name,
+            },
+        }
+
+        # Convert to JSON
+        payload_json = json.dumps(payload_data, separators=(",", ":"))
+        payload_bytes = payload_json.encode("utf-8")
+
+        # Generate signature if secret is configured
+        headers = {
+            "Content-Type": "application/json",
+            "User-Agent": "Reflector-Webhook/1.0",
+            "X-Webhook-Event": "transcript.completed",
+            "X-Webhook-Retry": str(self.request.retries),
+        }
+
+        if room.webhook_secret:
+            timestamp = str(int(datetime.now(timezone.utc).timestamp()))
+            signature = generate_webhook_signature(
+                payload_bytes, room.webhook_secret, timestamp
+            )
+            headers["X-Webhook-Signature"] = f"t={timestamp},v1={signature}"
+
+        # Send webhook with timeout
+        async with httpx.AsyncClient(timeout=30.0) as client:
+            log.info(
+                "Sending webhook",
+                url=room.webhook_url,
+                payload_size=len(payload_bytes),
+            )
+
+            response = await client.post(
+                room.webhook_url,
+                content=payload_bytes,
+                headers=headers,
+            )
+
+            response.raise_for_status()
+
+            log.info(
+                "Webhook sent successfully",
+                status_code=response.status_code,
+                response_size=len(response.content),
+            )
+
+    except httpx.HTTPStatusError as e:
+        log.error(
+            "Webhook failed with HTTP error",
+            status_code=e.response.status_code,
+            response_text=e.response.text[:500],  # First 500 chars
+        )
+
+        # Don't retry on client errors (4xx)
+        if 400 <= e.response.status_code < 500:
+            log.error("Client error, not retrying")
+            return
+
+        # Retry on server errors (5xx)
+        raise self.retry(exc=e)
+
+    except (httpx.ConnectError, httpx.TimeoutException) as e:
+        # Retry on network errors
+        log.error("Webhook failed with connection error", error=str(e))
+        raise self.retry(exc=e)
+
+    except Exception as e:
+        # Retry on unexpected errors
+        log.exception("Unexpected error in webhook task", error=str(e))
+        raise self.retry(exc=e)
+
+
+async def test_webhook(room_id: str) -> dict:
+    """
+    Test webhook configuration by sending a sample payload.
+    Returns immediately with success/failure status.
+    This is the shared implementation used by both the API endpoint and Celery task.
+    """
+    try:
+        room = await rooms_controller.get_by_id(room_id)
+        if not room:
+            return {"success": False, "error": "Room not found"}
+
+        if not room.webhook_url:
+            return {"success": False, "error": "No webhook URL configured"}
+
+        now = (datetime.now(timezone.utc).isoformat(),)
+        payload_data = {
+            "event": "test",
+            "event_id": uuid.uuid4().hex,
+            "timestamp": now,
+            "message": "This is a test webhook from Reflector",
+            "room": {
+                "id": room.id,
+                "name": room.name,
+            },
+        }
+
+        payload_json = json.dumps(payload_data, separators=(",", ":"))
+        payload_bytes = payload_json.encode("utf-8")
+
+        # Generate headers with signature
+        headers = {
+            "Content-Type": "application/json",
+            "User-Agent": "Reflector-Webhook/1.0",
+            "X-Webhook-Event": "test",
+        }
+
+        if room.webhook_secret:
+            timestamp = str(int(datetime.now(timezone.utc).timestamp()))
+            signature = generate_webhook_signature(
+                payload_bytes, room.webhook_secret, timestamp
+            )
+            headers["X-Webhook-Signature"] = f"t={timestamp},v1={signature}"
+
+        # Send test webhook with short timeout
+        async with httpx.AsyncClient(timeout=10.0) as client:
+            response = await client.post(
+                room.webhook_url,
+                content=payload_bytes,
+                headers=headers,
+            )
+
+            return {
+                "success": response.is_success,
+                "status_code": response.status_code,
+                "message": f"Webhook test {'successful' if response.is_success else 'failed'}",
+                "response_preview": response.text if response.text else None,
+            }
+
+    except httpx.TimeoutException:
+        return {
+            "success": False,
+            "error": "Webhook request timed out (10 seconds)",
+        }
+    except httpx.ConnectError as e:
+        return {
+            "success": False,
+            "error": f"Could not connect to webhook URL: {str(e)}",
+        }
+    except Exception as e:
+        return {
+            "success": False,
+            "error": f"Unexpected error: {str(e)}",
+        }

server/runserver.sh

@@ -2,7 +2,7 @@
 
 if [ "${ENTRYPOINT}" = "server" ]; then
     uv run alembic upgrade head
-    uv run -m reflector.app
+    uv run uvicorn reflector.app:app --host 0.0.0.0 --port 1250
 elif [ "${ENTRYPOINT}" = "worker" ]; then
     uv run celery -A reflector.worker.app worker --loglevel=info
 elif [ "${ENTRYPOINT}" = "beat" ]; then

server/test.ics

@@ -0,0 +1,29 @@
+BEGIN:VCALENDAR
+VERSION:2.0
+CALSCALE:GREGORIAN
+METHOD:PUBLISH
+PRODID:-//Fastmail/2020.5/EN
+X-APPLE-CALENDAR-COLOR:#0F6A0F
+X-WR-CALNAME:Test reflector
+X-WR-TIMEZONE:America/Costa_Rica
+BEGIN:VTIMEZONE
+TZID:America/Costa_Rica
+BEGIN:STANDARD
+DTSTART:19700101T000000
+TZOFFSETFROM:-0600
+TZOFFSETTO:-0600
+END:STANDARD
+END:VTIMEZONE
+BEGIN:VEVENT
+ATTENDEE;CN=Mathieu Virbel;PARTSTAT=ACCEPTED:MAILTO:mathieu@monadical.com
+DTEND;TZID=America/Costa_Rica:20250819T143000
+DTSTAMP:20250819T155951Z
+DTSTART;TZID=America/Costa_Rica:20250819T140000
+LOCATION:http://localhost:1250/mathieu
+ORGANIZER;CN=Mathieu Virbel:MAILTO:mathieu@monadical.com
+SEQUENCE:1
+SUMMARY:Checkin
+TRANSP:OPAQUE
+UID:867df50d-8105-4c58-9280-2b5d26cc9cd3
+END:VEVENT
+END:VCALENDAR

server/tests/conftest.py

@@ -178,6 +178,63 @@ async def dummy_diarization():
         yield
 
 
+@pytest.fixture
+async def dummy_file_transcript():
+    from reflector.processors.file_transcript import FileTranscriptProcessor
+    from reflector.processors.types import Transcript, Word
+
+    class TestFileTranscriptProcessor(FileTranscriptProcessor):
+        async def _transcript(self, data):
+            return Transcript(
+                text="Hello world. How are you today?",
+                words=[
+                    Word(start=0.0, end=0.5, text="Hello", speaker=0),
+                    Word(start=0.5, end=0.6, text=" ", speaker=0),
+                    Word(start=0.6, end=1.0, text="world", speaker=0),
+                    Word(start=1.0, end=1.1, text=".", speaker=0),
+                    Word(start=1.1, end=1.2, text=" ", speaker=0),
+                    Word(start=1.2, end=1.5, text="How", speaker=0),
+                    Word(start=1.5, end=1.6, text=" ", speaker=0),
+                    Word(start=1.6, end=1.8, text="are", speaker=0),
+                    Word(start=1.8, end=1.9, text=" ", speaker=0),
+                    Word(start=1.9, end=2.1, text="you", speaker=0),
+                    Word(start=2.1, end=2.2, text=" ", speaker=0),
+                    Word(start=2.2, end=2.5, text="today", speaker=0),
+                    Word(start=2.5, end=2.6, text="?", speaker=0),
+                ],
+            )
+
+    with patch(
+        "reflector.processors.file_transcript_auto.FileTranscriptAutoProcessor.__new__"
+    ) as mock_auto:
+        mock_auto.return_value = TestFileTranscriptProcessor()
+        yield
+
+
+@pytest.fixture
+async def dummy_file_diarization():
+    from reflector.processors.file_diarization import (
+        FileDiarizationOutput,
+        FileDiarizationProcessor,
+    )
+    from reflector.processors.types import DiarizationSegment
+
+    class TestFileDiarizationProcessor(FileDiarizationProcessor):
+        async def _diarize(self, data):
+            return FileDiarizationOutput(
+                diarization=[
+                    DiarizationSegment(start=0.0, end=1.1, speaker=0),
+                    DiarizationSegment(start=1.2, end=2.6, speaker=1),
+                ]
+            )
+
+    with patch(
+        "reflector.processors.file_diarization_auto.FileDiarizationAutoProcessor.__new__"
+    ) as mock_auto:
+        mock_auto.return_value = TestFileDiarizationProcessor()
+        yield
+
+
 @pytest.fixture
 async def dummy_transcript_translator():
     from reflector.processors.transcript_translator import TranscriptTranslatorProcessor
@@ -238,9 +295,13 @@ async def dummy_storage():
     with (
         patch("reflector.storage.base.Storage.get_instance") as mock_storage,
         patch("reflector.storage.get_transcripts_storage") as mock_get_transcripts,
+        patch(
+            "reflector.pipelines.main_file_pipeline.get_transcripts_storage"
+        ) as mock_get_transcripts2,
     ):
         mock_storage.return_value = dummy
         mock_get_transcripts.return_value = dummy
+        mock_get_transcripts2.return_value = dummy
         yield
 
 
@@ -260,7 +321,10 @@ def celery_config():
 
 @pytest.fixture(scope="session")
 def celery_includes():
-    return ["reflector.pipelines.main_live_pipeline"]
+    return [
+        "reflector.pipelines.main_live_pipeline",
+        "reflector.pipelines.main_file_pipeline",
+    ]
 
 
 @pytest.fixture
@@ -302,7 +366,7 @@ async def fake_transcript_with_topics(tmpdir, client):
     transcript = await transcripts_controller.get_by_id(tid)
     assert transcript is not None
 
-    await transcripts_controller.update(transcript, {"status": "finished"})
+    await transcripts_controller.update(transcript, {"status": "ended"})
 
     # manually copy a file at the expected location
     audio_filename = transcript.audio_mp3_filename

server/tests/test_attendee_parsing_bug.ics

@@ -0,0 +1,18 @@
+BEGIN:VCALENDAR
+VERSION:2.0
+CALSCALE:GREGORIAN
+METHOD:PUBLISH
+PRODID:-//Test/1.0/EN
+X-WR-CALNAME:Test Attendee Bug
+BEGIN:VEVENT
+ATTENDEE:MAILTO:alice@example.com,bob@example.com,charlie@example.com,diana@example.com,eve@example.com,frank@example.com,george@example.com,helen@example.com,ivan@example.com,jane@example.com,kevin@example.com,laura@example.com,mike@example.com,nina@example.com,oscar@example.com,paul@example.com,queen@example.com,robert@example.com,sarah@example.com,tom@example.com,ursula@example.com,victor@example.com,wendy@example.com,xavier@example.com,yvonne@example.com,zack@example.com,amy@example.com,bill@example.com,carol@example.com
+DTEND:20250910T190000Z
+DTSTAMP:20250910T174000Z
+DTSTART:20250910T180000Z
+LOCATION:http://localhost:3000/test-room
+ORGANIZER;CN=Test Organizer:MAILTO:organizer@example.com
+SEQUENCE:1
+SUMMARY:Test Meeting with Many Attendees
+UID:test-attendee-bug-event
+END:VEVENT
+END:VCALENDAR

server/tests/test_attendee_parsing_bug.py

@@ -0,0 +1,192 @@
+import os
+from unittest.mock import AsyncMock, patch
+
+import pytest
+
+from reflector.db.rooms import rooms_controller
+from reflector.services.ics_sync import ICSSyncService
+
+
+@pytest.mark.asyncio
+async def test_attendee_parsing_bug():
+    """
+    Test that reproduces the attendee parsing bug where a string with comma-separated
+    emails gets parsed as individual characters instead of separate email addresses.
+
+    The bug manifests as getting 29 attendees with emails like "M", "A", "I", etc.
+    instead of properly parsed email addresses.
+    """
+    # Create a test room
+    room = await rooms_controller.add(
+        name="test-room",
+        user_id="test-user",
+        zulip_auto_post=False,
+        zulip_stream="",
+        zulip_topic="",
+        is_locked=False,
+        room_mode="normal",
+        recording_type="cloud",
+        recording_trigger="automatic-2nd-participant",
+        is_shared=False,
+        ics_url="http://test.com/test.ics",
+        ics_enabled=True,
+    )
+
+    # Read the test ICS file that reproduces the bug and update it with current time
+    from datetime import datetime, timedelta, timezone
+
+    test_ics_path = os.path.join(
+        os.path.dirname(__file__), "test_attendee_parsing_bug.ics"
+    )
+    with open(test_ics_path, "r") as f:
+        ics_content = f.read()
+
+    # Replace the dates with current time + 1 hour to ensure it's within the 24h window
+    now = datetime.now(timezone.utc)
+    future_time = now + timedelta(hours=1)
+    end_time = future_time + timedelta(hours=1)
+
+    # Format dates for ICS format
+    dtstart = future_time.strftime("%Y%m%dT%H%M%SZ")
+    dtend = end_time.strftime("%Y%m%dT%H%M%SZ")
+    dtstamp = now.strftime("%Y%m%dT%H%M%SZ")
+
+    # Update the ICS content with current dates
+    ics_content = ics_content.replace("20250910T180000Z", dtstart)
+    ics_content = ics_content.replace("20250910T190000Z", dtend)
+    ics_content = ics_content.replace("20250910T174000Z", dtstamp)
+
+    # Create sync service and mock the fetch
+    sync_service = ICSSyncService()
+
+    with patch.object(
+        sync_service.fetch_service, "fetch_ics", new_callable=AsyncMock
+    ) as mock_fetch:
+        mock_fetch.return_value = ics_content
+
+        # Debug: Parse the ICS content directly to examine attendee parsing
+        calendar = sync_service.fetch_service.parse_ics(ics_content)
+        from reflector.settings import settings
+
+        room_url = f"{settings.UI_BASE_URL}/{room.name}"
+
+        print(f"Room URL being used for matching: {room_url}")
+        print(f"ICS content:\n{ics_content}")
+
+        events, total_events = sync_service.fetch_service.extract_room_events(
+            calendar, room.name, room_url
+        )
+
+        print(f"Total events in calendar: {total_events}")
+        print(f"Events matching room: {len(events)}")
+
+        # Perform the sync
+        result = await sync_service.sync_room_calendar(room)
+
+        # Check that the sync succeeded
+        assert result.get("status") == "success"
+        assert result.get("events_found", 0) >= 0  # Allow for debugging
+
+        # We already have the matching events from the debug code above
+        assert len(events) == 1
+        event = events[0]
+
+        # This is where the bug manifests - check the attendees
+        attendees = event["attendees"]
+
+        # Print attendee info for debugging
+        print(f"Number of attendees found: {len(attendees)}")
+        for i, attendee in enumerate(attendees):
+            print(
+                f"Attendee {i}: email='{attendee.get('email')}', name='{attendee.get('name')}'"
+            )
+
+        # With the fix, we should now get properly parsed email addresses
+        # Check that no single characters are parsed as emails
+        single_char_emails = [
+            att for att in attendees if att.get("email") and len(att["email"]) == 1
+        ]
+
+        if single_char_emails:
+            print(
+                f"BUG DETECTED: Found {len(single_char_emails)} single-character emails:"
+            )
+            for att in single_char_emails:
+                print(f"  - '{att['email']}'")
+
+        # Should have attendees but not single-character emails
+        assert len(attendees) > 0
+        assert (
+            len(single_char_emails) == 0
+        ), f"Found {len(single_char_emails)} single-character emails, parsing is still buggy"
+
+        # Check that all emails are valid (contain @ symbol)
+        valid_emails = [
+            att for att in attendees if att.get("email") and "@" in att["email"]
+        ]
+        assert len(valid_emails) == len(
+            attendees
+        ), "Some attendees don't have valid email addresses"
+
+        # We expect around 29 attendees (28 from the comma-separated list + 1 organizer)
+        assert (
+            len(attendees) >= 25
+        ), f"Expected around 29 attendees, got {len(attendees)}"
+
+
+@pytest.mark.asyncio
+async def test_correct_attendee_parsing():
+    """
+    Test what correct attendee parsing should look like.
+    """
+    from datetime import datetime, timezone
+
+    from icalendar import Event
+
+    from reflector.services.ics_sync import ICSFetchService
+
+    service = ICSFetchService()
+
+    # Create a properly formatted event with multiple attendees
+    event = Event()
+    event.add("uid", "test-correct-attendees")
+    event.add("summary", "Test Meeting")
+    event.add("location", "http://test.com/test")
+    event.add("dtstart", datetime.now(timezone.utc))
+    event.add("dtend", datetime.now(timezone.utc))
+
+    # Add attendees the correct way (separate ATTENDEE lines)
+    event.add("attendee", "mailto:alice@example.com", parameters={"CN": "Alice"})
+    event.add("attendee", "mailto:bob@example.com", parameters={"CN": "Bob"})
+    event.add("attendee", "mailto:charlie@example.com", parameters={"CN": "Charlie"})
+    event.add(
+        "organizer", "mailto:organizer@example.com", parameters={"CN": "Organizer"}
+    )
+
+    # Parse the event
+    result = service._parse_event(event)
+
+    assert result is not None
+    attendees = result["attendees"]
+
+    # Should have 4 attendees (3 attendees + 1 organizer)
+    assert len(attendees) == 4
+
+    # Check that all emails are valid email addresses
+    emails = [att["email"] for att in attendees if att.get("email")]
+    expected_emails = [
+        "alice@example.com",
+        "bob@example.com",
+        "charlie@example.com",
+        "organizer@example.com",
+    ]
+
+    for email in emails:
+        assert "@" in email, f"Invalid email format: {email}"
+        assert len(email) > 5, f"Email too short: {email}"
+
+    # Check that we have the expected emails
+    assert "alice@example.com" in emails
+    assert "bob@example.com" in emails
+    assert "charlie@example.com" in emails
+    assert "organizer@example.com" in emails

server/tests/test_calendar_event.py

@@ -0,0 +1,424 @@
+"""
+Tests for CalendarEvent model.
+"""
+
+from datetime import datetime, timedelta, timezone
+
+import pytest
+
+from reflector.db.calendar_events import CalendarEvent, calendar_events_controller
+from reflector.db.rooms import rooms_controller
+
+
+@pytest.mark.asyncio
+async def test_calendar_event_create():
+    """Test creating a calendar event."""
+    # Create a room first
+    room = await rooms_controller.add(
+        name="test-room",
+        user_id="test-user",
+        zulip_auto_post=False,
+        zulip_stream="",
+        zulip_topic="",
+        is_locked=False,
+        room_mode="normal",
+        recording_type="cloud",
+        recording_trigger="automatic-2nd-participant",
+        is_shared=False,
+    )
+
+    # Create calendar event
+    now = datetime.now(timezone.utc)
+    event = CalendarEvent(
+        room_id=room.id,
+        ics_uid="test-event-123",
+        title="Team Meeting",
+        description="Weekly team sync",
+        start_time=now + timedelta(hours=1),
+        end_time=now + timedelta(hours=2),
+        location=f"https://example.com/{room.name}",
+        attendees=[
+            {"email": "alice@example.com", "name": "Alice", "status": "ACCEPTED"},
+            {"email": "bob@example.com", "name": "Bob", "status": "TENTATIVE"},
+        ],
+    )
+
+    # Save event
+    saved_event = await calendar_events_controller.upsert(event)
+
+    assert saved_event.ics_uid == "test-event-123"
+    assert saved_event.title == "Team Meeting"
+    assert saved_event.room_id == room.id
+    assert len(saved_event.attendees) == 2
+
+
+@pytest.mark.asyncio
+async def test_calendar_event_get_by_room():
+    """Test getting calendar events for a room."""
+    # Create room
+    room = await rooms_controller.add(
+        name="events-room",
+        user_id="test-user",
+        zulip_auto_post=False,
+        zulip_stream="",
+        zulip_topic="",
+        is_locked=False,
+        room_mode="normal",
+        recording_type="cloud",
+        recording_trigger="automatic-2nd-participant",
+        is_shared=False,
+    )
+
+    now = datetime.now(timezone.utc)
+
+    # Create multiple events
+    for i in range(3):
+        event = CalendarEvent(
+            room_id=room.id,
+            ics_uid=f"event-{i}",
+            title=f"Meeting {i}",
+            start_time=now + timedelta(hours=i),
+            end_time=now + timedelta(hours=i + 1),
+        )
+        await calendar_events_controller.upsert(event)
+
+    # Get events for room
+    events = await calendar_events_controller.get_by_room(room.id)
+
+    assert len(events) == 3
+    assert all(e.room_id == room.id for e in events)
+    assert events[0].title == "Meeting 0"
+    assert events[1].title == "Meeting 1"
+    assert events[2].title == "Meeting 2"
+
+
+@pytest.mark.asyncio
+async def test_calendar_event_get_upcoming():
+    """Test getting upcoming events within time window."""
+    # Create room
+    room = await rooms_controller.add(
+        name="upcoming-room",
+        user_id="test-user",
+        zulip_auto_post=False,
+        zulip_stream="",
+        zulip_topic="",
+        is_locked=False,
+        room_mode="normal",
+        recording_type="cloud",
+        recording_trigger="automatic-2nd-participant",
+        is_shared=False,
+    )
+
+    now = datetime.now(timezone.utc)
+
+    # Create events at different times
+    # Past event (should not be included)
+    past_event = CalendarEvent(
+        room_id=room.id,
+        ics_uid="past-event",
+        title="Past Meeting",
+        start_time=now - timedelta(hours=2),
+        end_time=now - timedelta(hours=1),
+    )
+    await calendar_events_controller.upsert(past_event)
+
+    # Upcoming event within 30 minutes
+    upcoming_event = CalendarEvent(
+        room_id=room.id,
+        ics_uid="upcoming-event",
+        title="Upcoming Meeting",
+        start_time=now + timedelta(minutes=15),
+        end_time=now + timedelta(minutes=45),
+    )
+    await calendar_events_controller.upsert(upcoming_event)
+
+    # Currently happening event (started 10 minutes ago, ends in 20 minutes)
+    current_event = CalendarEvent(
+        room_id=room.id,
+        ics_uid="current-event",
+        title="Current Meeting",
+        start_time=now - timedelta(minutes=10),
+        end_time=now + timedelta(minutes=20),
+    )
+    await calendar_events_controller.upsert(current_event)
+
+    # Future event beyond 30 minutes
+    future_event = CalendarEvent(
+        room_id=room.id,
+        ics_uid="future-event",
+        title="Future Meeting",
+        start_time=now + timedelta(hours=2),
+        end_time=now + timedelta(hours=3),
+    )
+    await calendar_events_controller.upsert(future_event)
+
+    # Get upcoming events (default 120 minutes) - should include current, upcoming, and future
+    upcoming = await calendar_events_controller.get_upcoming(room.id)
+
+    assert len(upcoming) == 3
+    # Events should be sorted by start_time (current event first, then upcoming, then future)
+    assert upcoming[0].ics_uid == "current-event"
+    assert upcoming[1].ics_uid == "upcoming-event"
+    assert upcoming[2].ics_uid == "future-event"
+
+    # Get upcoming with custom window
+    upcoming_extended = await calendar_events_controller.get_upcoming(
+        room.id, minutes_ahead=180
+    )
+
+    assert len(upcoming_extended) == 3
+    # Events should be sorted by start_time
+    assert upcoming_extended[0].ics_uid == "current-event"
+    assert upcoming_extended[1].ics_uid == "upcoming-event"
+    assert upcoming_extended[2].ics_uid == "future-event"
+
+
+@pytest.mark.asyncio
+async def test_calendar_event_get_upcoming_includes_currently_happening():
+    """Test that get_upcoming includes currently happening events but excludes ended events."""
+    # Create room
+    room = await rooms_controller.add(
+        name="current-happening-room",
+        user_id="test-user",
+        zulip_auto_post=False,
+        zulip_stream="",
+        zulip_topic="",
+        is_locked=False,
+        room_mode="normal",
+        recording_type="cloud",
+        recording_trigger="automatic-2nd-participant",
+        is_shared=False,
+    )
+
+    now = datetime.now(timezone.utc)
+
+    # Event that ended in the past (should NOT be included)
+    past_ended_event = CalendarEvent(
+        room_id=room.id,
+        ics_uid="past-ended-event",
+        title="Past Ended Meeting",
+        start_time=now - timedelta(hours=2),
+        end_time=now - timedelta(minutes=30),
+    )
+    await calendar_events_controller.upsert(past_ended_event)
+
+    # Event currently happening (started 10 minutes ago, ends in 20 minutes) - SHOULD be included
+    currently_happening_event = CalendarEvent(
+        room_id=room.id,
+        ics_uid="currently-happening",
+        title="Currently Happening Meeting",
+        start_time=now - timedelta(minutes=10),
+        end_time=now + timedelta(minutes=20),
+    )
+    await calendar_events_controller.upsert(currently_happening_event)
+
+    # Event starting soon (in 5 minutes) - SHOULD be included
+    upcoming_soon_event = CalendarEvent(
+        room_id=room.id,
+        ics_uid="upcoming-soon",
+        title="Upcoming Soon Meeting",
+        start_time=now + timedelta(minutes=5),
+        end_time=now + timedelta(minutes=35),
+    )
+    await calendar_events_controller.upsert(upcoming_soon_event)
+
+    # Get upcoming events
+    upcoming = await calendar_events_controller.get_upcoming(room.id, minutes_ahead=30)
+
+    # Should only include currently happening and upcoming soon events
+    assert len(upcoming) == 2
+    assert upcoming[0].ics_uid == "currently-happening"
+    assert upcoming[1].ics_uid == "upcoming-soon"
+
+
+@pytest.mark.asyncio
+async def test_calendar_event_upsert():
+    """Test upserting (create/update) calendar events."""
+    # Create room
+    room = await rooms_controller.add(
+        name="upsert-room",
+        user_id="test-user",
+        zulip_auto_post=False,
+        zulip_stream="",
+        zulip_topic="",
+        is_locked=False,
+        room_mode="normal",
+        recording_type="cloud",
+        recording_trigger="automatic-2nd-participant",
+        is_shared=False,
+    )
+
+    now = datetime.now(timezone.utc)
+
+    # Create new event
+    event = CalendarEvent(
+        room_id=room.id,
+        ics_uid="upsert-test",
+        title="Original Title",
+        start_time=now,
+        end_time=now + timedelta(hours=1),
+    )
+
+    created = await calendar_events_controller.upsert(event)
+    assert created.title == "Original Title"
+
+    # Update existing event
+    event.title = "Updated Title"
+    event.description = "Added description"
+
+    updated = await calendar_events_controller.upsert(event)
+    assert updated.title == "Updated Title"
+    assert updated.description == "Added description"
+    assert updated.ics_uid == "upsert-test"
+
+    # Verify only one event exists
+    events = await calendar_events_controller.get_by_room(room.id)
+    assert len(events) == 1
+    assert events[0].title == "Updated Title"
+
+
+@pytest.mark.asyncio
+async def test_calendar_event_soft_delete():
+    """Test soft deleting events no longer in calendar."""
+    # Create room
+    room = await rooms_controller.add(
+        name="delete-room",
+        user_id="test-user",
+        zulip_auto_post=False,
+        zulip_stream="",
+        zulip_topic="",
+        is_locked=False,
+        room_mode="normal",
+        recording_type="cloud",
+        recording_trigger="automatic-2nd-participant",
+        is_shared=False,
+    )
+
+    now = datetime.now(timezone.utc)
+
+    # Create multiple events
+    for i in range(4):
+        event = CalendarEvent(
+            room_id=room.id,
+            ics_uid=f"event-{i}",
+            title=f"Meeting {i}",
+            start_time=now + timedelta(hours=i),
+            end_time=now + timedelta(hours=i + 1),
+        )
+        await calendar_events_controller.upsert(event)
+
+    # Soft delete events not in current list
+    current_ids = ["event-0", "event-2"]  # Keep events 0 and 2
+    deleted_count = await calendar_events_controller.soft_delete_missing(
+        room.id, current_ids
+    )
+
+    assert deleted_count == 2  # Should delete events 1 and 3
+
+    # Get non-deleted events
+    events = await calendar_events_controller.get_by_room(
+        room.id, include_deleted=False
+    )
+    assert len(events) == 2
+    assert {e.ics_uid for e in events} == {"event-0", "event-2"}
+
+    # Get all events including deleted
+    all_events = await calendar_events_controller.get_by_room(
+        room.id, include_deleted=True
+    )
+    assert len(all_events) == 4
+
+
+@pytest.mark.asyncio
+async def test_calendar_event_past_events_not_deleted():
+    """Test that past events are not soft deleted."""
+    # Create room
+    room = await rooms_controller.add(
+        name="past-events-room",
+        user_id="test-user",
+        zulip_auto_post=False,
+        zulip_stream="",
+        zulip_topic="",
+        is_locked=False,
+        room_mode="normal",
+        recording_type="cloud",
+        recording_trigger="automatic-2nd-participant",
+        is_shared=False,
+    )
+
+    now = datetime.now(timezone.utc)
+
+    # Create past event
+    past_event = CalendarEvent(
+        room_id=room.id,
+        ics_uid="past-event",
+        title="Past Meeting",
+        start_time=now - timedelta(hours=2),
+        end_time=now - timedelta(hours=1),
+    )
+    await calendar_events_controller.upsert(past_event)
+
+    # Create future event
+    future_event = CalendarEvent(
+        room_id=room.id,
+        ics_uid="future-event",
+        title="Future Meeting",
+        start_time=now + timedelta(hours=1),
+        end_time=now + timedelta(hours=2),
+    )
+    await calendar_events_controller.upsert(future_event)
+
+    # Try to soft delete all events (only future should be deleted)
+    deleted_count = await calendar_events_controller.soft_delete_missing(room.id, [])
+
+    assert deleted_count == 1  # Only future event deleted
+
+    # Verify past event still exists
+    events = await calendar_events_controller.get_by_room(
+        room.id, include_deleted=False
+    )
+    assert len(events) == 1
+    assert events[0].ics_uid == "past-event"
+
+
+@pytest.mark.asyncio
+async def test_calendar_event_with_raw_ics_data():
+    """Test storing raw ICS data with calendar event."""
+    # Create room
+    room = await rooms_controller.add(
+        name="raw-ics-room",
+        user_id="test-user",
+        zulip_auto_post=False,
+        zulip_stream="",
+        zulip_topic="",
+        is_locked=False,
+        room_mode="normal",
+        recording_type="cloud",
+        recording_trigger="automatic-2nd-participant",
+        is_shared=False,
+    )
+
+    raw_ics = """BEGIN:VEVENT
+UID:test-raw-123
+SUMMARY:Test Event
+DTSTART:20240101T100000Z
+DTEND:20240101T110000Z
+END:VEVENT"""
+
+    event = CalendarEvent(
+        room_id=room.id,
+        ics_uid="test-raw-123",
+        title="Test Event",
+        start_time=datetime.now(timezone.utc),
+        end_time=datetime.now(timezone.utc) + timedelta(hours=1),
+        ics_raw_data=raw_ics,
+    )
+
+    saved = await calendar_events_controller.upsert(event)
+
+    assert saved.ics_raw_data == raw_ics
+
+    # Retrieve and verify
+    retrieved = await calendar_events_controller.get_by_ics_uid(room.id, "test-raw-123")
+    assert retrieved is not None
+    assert retrieved.ics_raw_data == raw_ics

server/tests/test_cleanup.py

@@ -0,0 +1,285 @@
+from datetime import datetime, timedelta, timezone
+from unittest.mock import AsyncMock, patch
+
+import pytest
+
+from reflector.db.recordings import Recording, recordings_controller
+from reflector.db.transcripts import SourceKind, transcripts_controller
+from reflector.worker.cleanup import cleanup_old_public_data
+
+
+@pytest.mark.asyncio
+async def test_cleanup_old_public_data_skips_when_not_public():
+    """Test that cleanup is skipped when PUBLIC_MODE is False."""
+    with patch("reflector.worker.cleanup.settings") as mock_settings:
+        mock_settings.PUBLIC_MODE = False
+
+        result = await cleanup_old_public_data()
+
+        # Should return early without doing anything
+        assert result is None
+
+
+@pytest.mark.asyncio
+async def test_cleanup_old_public_data_deletes_old_anonymous_transcripts():
+    """Test that old anonymous transcripts are deleted."""
+    # Create old and new anonymous transcripts
+    old_date = datetime.now(timezone.utc) - timedelta(days=8)
+    new_date = datetime.now(timezone.utc) - timedelta(days=2)
+
+    # Create old anonymous transcript (should be deleted)
+    old_transcript = await transcripts_controller.add(
+        name="Old Anonymous Transcript",
+        source_kind=SourceKind.FILE,
+        user_id=None,  # Anonymous
+    )
+    # Manually update created_at to be old
+    from reflector.db import get_database
+    from reflector.db.transcripts import transcripts
+
+    await get_database().execute(
+        transcripts.update()
+        .where(transcripts.c.id == old_transcript.id)
+        .values(created_at=old_date)
+    )
+
+    # Create new anonymous transcript (should NOT be deleted)
+    new_transcript = await transcripts_controller.add(
+        name="New Anonymous Transcript",
+        source_kind=SourceKind.FILE,
+        user_id=None,  # Anonymous
+    )
+
+    # Create old transcript with user (should NOT be deleted)
+    old_user_transcript = await transcripts_controller.add(
+        name="Old User Transcript",
+        source_kind=SourceKind.FILE,
+        user_id="user123",
+    )
+    await get_database().execute(
+        transcripts.update()
+        .where(transcripts.c.id == old_user_transcript.id)
+        .values(created_at=old_date)
+    )
+
+    with patch("reflector.worker.cleanup.settings") as mock_settings:
+        mock_settings.PUBLIC_MODE = True
+        mock_settings.PUBLIC_DATA_RETENTION_DAYS = 7
+
+        # Mock the storage deletion
+        with patch("reflector.db.transcripts.get_transcripts_storage") as mock_storage:
+            mock_storage.return_value.delete_file = AsyncMock()
+
+            result = await cleanup_old_public_data()
+
+    # Check results
+    assert result["transcripts_deleted"] == 1
+    assert result["errors"] == []
+
+    # Verify old anonymous transcript was deleted
+    assert await transcripts_controller.get_by_id(old_transcript.id) is None
+
+    # Verify new anonymous transcript still exists
+    assert await transcripts_controller.get_by_id(new_transcript.id) is not None
+
+    # Verify user transcript still exists
+    assert await transcripts_controller.get_by_id(old_user_transcript.id) is not None
+
+
+@pytest.mark.asyncio
+async def test_cleanup_deletes_associated_meeting_and_recording():
+    """Test that meetings and recordings associated with old transcripts are deleted."""
+    from reflector.db import get_database
+    from reflector.db.meetings import meetings
+    from reflector.db.transcripts import transcripts
+
+    old_date = datetime.now(timezone.utc) - timedelta(days=8)
+
+    # Create a meeting
+    meeting_id = "test-meeting-for-transcript"
+    await get_database().execute(
+        meetings.insert().values(
+            id=meeting_id,
+            room_name="Meeting with Transcript",
+            room_url="https://example.com/meeting",
+            host_room_url="https://example.com/meeting-host",
+            start_date=old_date,
+            end_date=old_date + timedelta(hours=1),
+            room_id=None,
+        )
+    )
+
+    # Create a recording
+    recording = await recordings_controller.create(
+        Recording(
+            bucket_name="test-bucket",
+            object_key="test-recording.mp4",
+            recorded_at=old_date,
+        )
+    )
+
+    # Create an old transcript with both meeting and recording
+    old_transcript = await transcripts_controller.add(
+        name="Old Transcript with Meeting and Recording",
+        source_kind=SourceKind.ROOM,
+        user_id=None,
+        meeting_id=meeting_id,
+        recording_id=recording.id,
+    )
+
+    # Update created_at to be old
+    await get_database().execute(
+        transcripts.update()
+        .where(transcripts.c.id == old_transcript.id)
+        .values(created_at=old_date)
+    )
+
+    with patch("reflector.worker.cleanup.settings") as mock_settings:
+        mock_settings.PUBLIC_MODE = True
+        mock_settings.PUBLIC_DATA_RETENTION_DAYS = 7
+
+        # Mock storage deletion
+        with patch("reflector.db.transcripts.get_transcripts_storage") as mock_storage:
+            mock_storage.return_value.delete_file = AsyncMock()
+            with patch(
+                "reflector.worker.cleanup.get_recordings_storage"
+            ) as mock_rec_storage:
+                mock_rec_storage.return_value.delete_file = AsyncMock()
+
+                result = await cleanup_old_public_data()
+
+    # Check results
+    assert result["transcripts_deleted"] == 1
+    assert result["meetings_deleted"] == 1
+    assert result["recordings_deleted"] == 1
+    assert result["errors"] == []
+
+    # Verify transcript was deleted
+    assert await transcripts_controller.get_by_id(old_transcript.id) is None
+
+    # Verify meeting was deleted
+    query = meetings.select().where(meetings.c.id == meeting_id)
+    meeting_result = await get_database().fetch_one(query)
+    assert meeting_result is None
+
+    # Verify recording was deleted
+    assert await recordings_controller.get_by_id(recording.id) is None
+
+
+@pytest.mark.asyncio
+async def test_cleanup_handles_errors_gracefully():
+    """Test that cleanup continues even when individual deletions fail."""
+    old_date = datetime.now(timezone.utc) - timedelta(days=8)
+
+    # Create multiple old transcripts
+    transcript1 = await transcripts_controller.add(
+        name="Transcript 1",
+        source_kind=SourceKind.FILE,
+        user_id=None,
+    )
+    transcript2 = await transcripts_controller.add(
+        name="Transcript 2",
+        source_kind=SourceKind.FILE,
+        user_id=None,
+    )
+
+    # Update created_at to be old
+    from reflector.db import get_database
+    from reflector.db.transcripts import transcripts
+
+    for t_id in [transcript1.id, transcript2.id]:
+        await get_database().execute(
+            transcripts.update()
+            .where(transcripts.c.id == t_id)
+            .values(created_at=old_date)
+        )
+
+    with patch("reflector.worker.cleanup.settings") as mock_settings:
+        mock_settings.PUBLIC_MODE = True
+        mock_settings.PUBLIC_DATA_RETENTION_DAYS = 7
+
+        # Mock remove_by_id to fail for the first transcript
+        original_remove = transcripts_controller.remove_by_id
+        call_count = 0
+
+        async def mock_remove_by_id(transcript_id, user_id=None):
+            nonlocal call_count
+            call_count += 1
+            if call_count == 1:
+                raise Exception("Simulated deletion error")
+            return await original_remove(transcript_id, user_id)
+
+        with patch.object(
+            transcripts_controller, "remove_by_id", side_effect=mock_remove_by_id
+        ):
+            result = await cleanup_old_public_data()
+
+    # Should have one successful deletion and one error
+    assert result["transcripts_deleted"] == 1
+    assert len(result["errors"]) == 1
+    assert "Failed to delete transcript" in result["errors"][0]
+
+
+@pytest.mark.asyncio
+async def test_meeting_consent_cascade_delete():
+    """Test that meeting_consent records are automatically deleted when meeting is deleted."""
+    from reflector.db import get_database
+    from reflector.db.meetings import (
+        meeting_consent,
+        meeting_consent_controller,
+        meetings,
+    )
+
+    # Create a meeting
+    meeting_id = "test-cascade-meeting"
+    await get_database().execute(
+        meetings.insert().values(
+            id=meeting_id,
+            room_name="Test Meeting for CASCADE",
+            room_url="https://example.com/cascade-test",
+            host_room_url="https://example.com/cascade-test-host",
+            start_date=datetime.now(timezone.utc),
+            end_date=datetime.now(timezone.utc) + timedelta(hours=1),
+            room_id=None,
+        )
+    )
+
+    # Create consent records for this meeting
+    consent1_id = "consent-1"
+    consent2_id = "consent-2"
+
+    await get_database().execute(
+        meeting_consent.insert().values(
+            id=consent1_id,
+            meeting_id=meeting_id,
+            user_id="user1",
+            consent_given=True,
+            consent_timestamp=datetime.now(timezone.utc),
+        )
+    )
+
+    await get_database().execute(
+        meeting_consent.insert().values(
+            id=consent2_id,
+            meeting_id=meeting_id,
+            user_id="user2",
+            consent_given=False,
+            consent_timestamp=datetime.now(timezone.utc),
+        )
+    )
+
+    # Verify consent records exist
+    consents = await meeting_consent_controller.get_by_meeting_id(meeting_id)
+    assert len(consents) == 2
+
+    # Delete the meeting
+    await get_database().execute(meetings.delete().where(meetings.c.id == meeting_id))
+
+    # Verify meeting is deleted
+    query = meetings.select().where(meetings.c.id == meeting_id)
+    result = await get_database().fetch_one(query)
+    assert result is None
+
+    # Verify consent records are automatically deleted (CASCADE DELETE)
+    consents_after = await meeting_consent_controller.get_by_meeting_id(meeting_id)
+    assert len(consents_after) == 0

server/tests/test_ics_background_tasks.py

@@ -0,0 +1,255 @@
+from datetime import datetime, timedelta, timezone
+from unittest.mock import AsyncMock, MagicMock, patch
+
+import pytest
+from icalendar import Calendar, Event
+
+from reflector.db import get_database
+from reflector.db.calendar_events import calendar_events_controller
+from reflector.db.rooms import rooms, rooms_controller
+from reflector.services.ics_sync import ics_sync_service
+from reflector.worker.ics_sync import (
+    _should_sync,
+    sync_room_ics,
+)
+
+
+@pytest.mark.asyncio
+async def test_sync_room_ics_task():
+    room = await rooms_controller.add(
+        name="task-test-room",
+        user_id="test-user",
+        zulip_auto_post=False,
+        zulip_stream="",
+        zulip_topic="",
+        is_locked=False,
+        room_mode="normal",
+        recording_type="cloud",
+        recording_trigger="automatic-2nd-participant",
+        is_shared=False,
+        ics_url="https://calendar.example.com/task.ics",
+        ics_enabled=True,
+    )
+
+    cal = Calendar()
+    event = Event()
+    event.add("uid", "task-event-1")
+    event.add("summary", "Task Test Meeting")
+    from reflector.settings import settings
+
+    event.add("location", f"{settings.UI_BASE_URL}/{room.name}")
+    now = datetime.now(timezone.utc)
+    event.add("dtstart", now + timedelta(hours=1))
+    event.add("dtend", now + timedelta(hours=2))
+    cal.add_component(event)
+    ics_content = cal.to_ical().decode("utf-8")
+
+    with patch(
+        "reflector.services.ics_sync.ICSFetchService.fetch_ics", new_callable=AsyncMock
+    ) as mock_fetch:
+        mock_fetch.return_value = ics_content
+
+        # Call the service directly instead of the Celery task to avoid event loop issues
+        await ics_sync_service.sync_room_calendar(room)
+
+        events = await calendar_events_controller.get_by_room(room.id)
+        assert len(events) == 1
+        assert events[0].ics_uid == "task-event-1"
+
+
+@pytest.mark.asyncio
+async def test_sync_room_ics_disabled():
+    room = await rooms_controller.add(
+        name="disabled-room",
+        user_id="test-user",
+        zulip_auto_post=False,
+        zulip_stream="",
+        zulip_topic="",
+        is_locked=False,
+        room_mode="normal",
+        recording_type="cloud",
+        recording_trigger="automatic-2nd-participant",
+        is_shared=False,
+        ics_enabled=False,
+    )
+
+    # Test that disabled rooms are skipped by the service
+    result = await ics_sync_service.sync_room_calendar(room)
+
+    events = await calendar_events_controller.get_by_room(room.id)
+    assert len(events) == 0
+
+
+@pytest.mark.asyncio
+async def test_sync_all_ics_calendars():
+    room1 = await rooms_controller.add(
+        name="sync-all-1",
+        user_id="test-user",
+        zulip_auto_post=False,
+        zulip_stream="",
+        zulip_topic="",
+        is_locked=False,
+        room_mode="normal",
+        recording_type="cloud",
+        recording_trigger="automatic-2nd-participant",
+        is_shared=False,
+        ics_url="https://calendar.example.com/1.ics",
+        ics_enabled=True,
+    )
+
+    room2 = await rooms_controller.add(
+        name="sync-all-2",
+        user_id="test-user",
+        zulip_auto_post=False,
+        zulip_stream="",
+        zulip_topic="",
+        is_locked=False,
+        room_mode="normal",
+        recording_type="cloud",
+        recording_trigger="automatic-2nd-participant",
+        is_shared=False,
+        ics_url="https://calendar.example.com/2.ics",
+        ics_enabled=True,
+    )
+
+    room3 = await rooms_controller.add(
+        name="sync-all-3",
+        user_id="test-user",
+        zulip_auto_post=False,
+        zulip_stream="",
+        zulip_topic="",
+        is_locked=False,
+        room_mode="normal",
+        recording_type="cloud",
+        recording_trigger="automatic-2nd-participant",
+        is_shared=False,
+        ics_enabled=False,
+    )
+
+    with patch("reflector.worker.ics_sync.sync_room_ics.delay") as mock_delay:
+        # Directly call the sync_all logic without the Celery wrapper
+        query = rooms.select().where(
+            rooms.c.ics_enabled == True, rooms.c.ics_url != None
+        )
+        all_rooms = await get_database().fetch_all(query)
+
+        for room_data in all_rooms:
+            room_id = room_data["id"]
+            room = await rooms_controller.get_by_id(room_id)
+            if room and _should_sync(room):
+                sync_room_ics.delay(room_id)
+
+        assert mock_delay.call_count == 2
+        called_room_ids = [call.args[0] for call in mock_delay.call_args_list]
+        assert room1.id in called_room_ids
+        assert room2.id in called_room_ids
+        assert room3.id not in called_room_ids
+
+
+@pytest.mark.asyncio
+async def test_should_sync_logic():
+    room = MagicMock()
+
+    room.ics_last_sync = None
+    assert _should_sync(room) is True
+
+    room.ics_last_sync = datetime.now(timezone.utc) - timedelta(seconds=100)
+    room.ics_fetch_interval = 300
+    assert _should_sync(room) is False
+
+    room.ics_last_sync = datetime.now(timezone.utc) - timedelta(seconds=400)
+    room.ics_fetch_interval = 300
+    assert _should_sync(room) is True
+
+
+@pytest.mark.asyncio
+async def test_sync_respects_fetch_interval():
+    now = datetime.now(timezone.utc)
+
+    room1 = await rooms_controller.add(
+        name="interval-test-1",
+        user_id="test-user",
+        zulip_auto_post=False,
+        zulip_stream="",
+        zulip_topic="",
+        is_locked=False,
+        room_mode="normal",
+        recording_type="cloud",
+        recording_trigger="automatic-2nd-participant",
+        is_shared=False,
+        ics_url="https://calendar.example.com/interval.ics",
+        ics_enabled=True,
+        ics_fetch_interval=300,
+    )
+
+    await rooms_controller.update(
+        room1,
+        {"ics_last_sync": now - timedelta(seconds=100)},
+    )
+
+    room2 = await rooms_controller.add(
+        name="interval-test-2",
+        user_id="test-user",
+        zulip_auto_post=False,
+        zulip_stream="",
+        zulip_topic="",
+        is_locked=False,
+        room_mode="normal",
+        recording_type="cloud",
+        recording_trigger="automatic-2nd-participant",
+        is_shared=False,
+        ics_url="https://calendar.example.com/interval2.ics",
+        ics_enabled=True,
+        ics_fetch_interval=60,
+    )
+
+    await rooms_controller.update(
+        room2,
+        {"ics_last_sync": now - timedelta(seconds=100)},
+    )
+
+    with patch("reflector.worker.ics_sync.sync_room_ics.delay") as mock_delay:
+        # Test the sync logic without the Celery wrapper
+        query = rooms.select().where(
+            rooms.c.ics_enabled == True, rooms.c.ics_url != None
+        )
+        all_rooms = await get_database().fetch_all(query)
+
+        for room_data in all_rooms:
+            room_id = room_data["id"]
+            room = await rooms_controller.get_by_id(room_id)
+            if room and _should_sync(room):
+                sync_room_ics.delay(room_id)
+
+        assert mock_delay.call_count == 1
+        assert mock_delay.call_args[0][0] == room2.id
+
+
+@pytest.mark.asyncio
+async def test_sync_handles_errors_gracefully():
+    room = await rooms_controller.add(
+        name="error-task-room",
+        user_id="test-user",
+        zulip_auto_post=False,
+        zulip_stream="",
+        zulip_topic="",
+        is_locked=False,
+        room_mode="normal",
+        recording_type="cloud",
+        recording_trigger="automatic-2nd-participant",
+        is_shared=False,
+        ics_url="https://calendar.example.com/error.ics",
+        ics_enabled=True,
+    )
+
+    with patch(
+        "reflector.services.ics_sync.ICSFetchService.fetch_ics", new_callable=AsyncMock
+    ) as mock_fetch:
+        mock_fetch.side_effect = Exception("Network error")
+
+        # Call the service directly to test error handling
+        result = await ics_sync_service.sync_room_calendar(room)
+        assert result["status"] == "error"
+
+        events = await calendar_events_controller.get_by_room(room.id)
+        assert len(events) == 0

server/tests/test_ics_sync.py

@@ -0,0 +1,290 @@
+from datetime import datetime, timedelta, timezone
+from unittest.mock import AsyncMock, MagicMock, patch
+
+import pytest
+from icalendar import Calendar, Event
+
+from reflector.db.calendar_events import calendar_events_controller
+from reflector.db.rooms import rooms_controller
+from reflector.services.ics_sync import ICSFetchService, ICSSyncService
+
+
+@pytest.mark.asyncio
+async def test_ics_fetch_service_event_matching():
+    service = ICSFetchService()
+    room_name = "test-room"
+    room_url = "https://example.com/test-room"
+
+    # Create test event
+    event = Event()
+    event.add("uid", "test-123")
+    event.add("summary", "Test Meeting")
+
+    # Test matching with full URL in location
+    event.add("location", "https://example.com/test-room")
+    assert service._event_matches_room(event, room_name, room_url) is True
+
+    # Test non-matching with URL without protocol (exact matching only now)
+    event["location"] = "example.com/test-room"
+    assert service._event_matches_room(event, room_name, room_url) is False
+
+    # Test matching in description
+    event["location"] = "Conference Room A"
+    event.add("description", f"Join at {room_url}")
+    assert service._event_matches_room(event, room_name, room_url) is True
+
+    # Test non-matching
+    event["location"] = "Different Room"
+    event["description"] = "No room URL here"
+    assert service._event_matches_room(event, room_name, room_url) is False
+
+    # Test partial paths should NOT match anymore
+    event["location"] = "/test-room"
+    assert service._event_matches_room(event, room_name, room_url) is False
+
+    event["location"] = f"Room: {room_name}"
+    assert service._event_matches_room(event, room_name, room_url) is False
+
+
+@pytest.mark.asyncio
+async def test_ics_fetch_service_parse_event():
+    service = ICSFetchService()
+
+    # Create test event
+    event = Event()
+    event.add("uid", "test-456")
+    event.add("summary", "Team Standup")
+    event.add("description", "Daily team sync")
+    event.add("location", "https://example.com/standup")
+
+    now = datetime.now(timezone.utc)
+    event.add("dtstart", now)
+    event.add("dtend", now + timedelta(hours=1))
+
+    # Add attendees
+    event.add("attendee", "mailto:alice@example.com", parameters={"CN": "Alice"})
+    event.add("attendee", "mailto:bob@example.com", parameters={"CN": "Bob"})
+    event.add("organizer", "mailto:carol@example.com", parameters={"CN": "Carol"})
+
+    # Parse event
+    result = service._parse_event(event)
+
+    assert result is not None
+    assert result["ics_uid"] == "test-456"
+    assert result["title"] == "Team Standup"
+    assert result["description"] == "Daily team sync"
+    assert result["location"] == "https://example.com/standup"
+    assert len(result["attendees"]) == 3  # 2 attendees + 1 organizer
+
+
+@pytest.mark.asyncio
+async def test_ics_fetch_service_extract_room_events():
+    service = ICSFetchService()
+    room_name = "meeting"
+    room_url = "https://example.com/meeting"
+
+    # Create calendar with multiple events
+    cal = Calendar()
+
+    # Event 1: Matches room
+    event1 = Event()
+    event1.add("uid", "match-1")
+    event1.add("summary", "Planning Meeting")
+    event1.add("location", room_url)
+    now = datetime.now(timezone.utc)
+    event1.add("dtstart", now + timedelta(hours=2))
+    event1.add("dtend", now + timedelta(hours=3))
+    cal.add_component(event1)
+
+    # Event 2: Doesn't match room
+    event2 = Event()
+    event2.add("uid", "no-match")
+    event2.add("summary", "Other Meeting")
+    event2.add("location", "https://example.com/other")
+    event2.add("dtstart", now + timedelta(hours=4))
+    event2.add("dtend", now + timedelta(hours=5))
+    cal.add_component(event2)
+
+    # Event 3: Matches room in description
+    event3 = Event()
+    event3.add("uid", "match-2")
+    event3.add("summary", "Review Session")
+    event3.add("description", f"Meeting link: {room_url}")
+    event3.add("dtstart", now + timedelta(hours=6))
+    event3.add("dtend", now + timedelta(hours=7))
+    cal.add_component(event3)
+
+    # Event 4: Cancelled event (should be skipped)
+    event4 = Event()
+    event4.add("uid", "cancelled")
+    event4.add("summary", "Cancelled Meeting")
+    event4.add("location", room_url)
+    event4.add("status", "CANCELLED")
+    event4.add("dtstart", now + timedelta(hours=8))
+    event4.add("dtend", now + timedelta(hours=9))
+    cal.add_component(event4)
+
+    # Extract events
+    events, total_events = service.extract_room_events(cal, room_name, room_url)
+
+    assert len(events) == 2
+    assert total_events == 3  # 3 events in time window (excluding cancelled)
+    assert events[0]["ics_uid"] == "match-1"
+    assert events[1]["ics_uid"] == "match-2"
+
+
+@pytest.mark.asyncio
+async def test_ics_sync_service_sync_room_calendar():
+    # Create room
+    room = await rooms_controller.add(
+        name="sync-test",
+        user_id="test-user",
+        zulip_auto_post=False,
+        zulip_stream="",
+        zulip_topic="",
+        is_locked=False,
+        room_mode="normal",
+        recording_type="cloud",
+        recording_trigger="automatic-2nd-participant",
+        is_shared=False,
+        ics_url="https://calendar.example.com/test.ics",
+        ics_enabled=True,
+    )
+
+    # Mock ICS content
+    cal = Calendar()
+    event = Event()
+    event.add("uid", "sync-event-1")
+    event.add("summary", "Sync Test Meeting")
+    # Use the actual UI_BASE_URL from settings
+    from reflector.settings import settings
+
+    event.add("location", f"{settings.UI_BASE_URL}/{room.name}")
+    now = datetime.now(timezone.utc)
+    event.add("dtstart", now + timedelta(hours=1))
+    event.add("dtend", now + timedelta(hours=2))
+    cal.add_component(event)
+    ics_content = cal.to_ical().decode("utf-8")
+
+    # Create sync service and mock fetch
+    sync_service = ICSSyncService()
+
+    with patch.object(
+        sync_service.fetch_service, "fetch_ics", new_callable=AsyncMock
+    ) as mock_fetch:
+        mock_fetch.return_value = ics_content
+
+        # First sync
+        result = await sync_service.sync_room_calendar(room)
+
+        assert result["status"] == "success"
+        assert result["events_found"] == 1
+        assert result["events_created"] == 1
+        assert result["events_updated"] == 0
+        assert result["events_deleted"] == 0
+
+        # Verify event was created
+        events = await calendar_events_controller.get_by_room(room.id)
+        assert len(events) == 1
+        assert events[0].ics_uid == "sync-event-1"
+        assert events[0].title == "Sync Test Meeting"
+
+        # Second sync with same content (should be unchanged)
+        # Refresh room to get updated etag and force sync by setting old sync time
+        room = await rooms_controller.get_by_id(room.id)
+        await rooms_controller.update(
+            room, {"ics_last_sync": datetime.now(timezone.utc) - timedelta(minutes=10)}
+        )
+        result = await sync_service.sync_room_calendar(room)
+        assert result["status"] == "unchanged"
+
+        # Third sync with updated event
+        event["summary"] = "Updated Meeting Title"
+        cal = Calendar()
+        cal.add_component(event)
+        ics_content = cal.to_ical().decode("utf-8")
+        mock_fetch.return_value = ics_content
+
+        # Force sync by clearing etag
+        await rooms_controller.update(room, {"ics_last_etag": None})
+
+        result = await sync_service.sync_room_calendar(room)
+        assert result["status"] == "success"
+        assert result["events_created"] == 0
+        assert result["events_updated"] == 1
+
+        # Verify event was updated
+        events = await calendar_events_controller.get_by_room(room.id)
+        assert len(events) == 1
+        assert events[0].title == "Updated Meeting Title"
+
+
+@pytest.mark.asyncio
+async def test_ics_sync_service_should_sync():
+    service = ICSSyncService()
+
+    # Room never synced
+    room = MagicMock()
+    room.ics_last_sync = None
+    room.ics_fetch_interval = 300
+    assert service._should_sync(room) is True
+
+    # Room synced recently
+    room.ics_last_sync = datetime.now(timezone.utc) - timedelta(seconds=100)
+    assert service._should_sync(room) is False
+
+    # Room sync due
+    room.ics_last_sync = datetime.now(timezone.utc) - timedelta(seconds=400)
+    assert service._should_sync(room) is True
+
+
+@pytest.mark.asyncio
+async def test_ics_sync_service_skip_disabled():
+    service = ICSSyncService()
+
+    # Room with ICS disabled
+    room = MagicMock()
+    room.ics_enabled = False
+    room.ics_url = "https://calendar.example.com/test.ics"
+
+    result = await service.sync_room_calendar(room)
+    assert result["status"] == "skipped"
+    assert result["reason"] == "ICS not configured"
+
+    # Room without URL
+    room.ics_enabled = True
+    room.ics_url = None
+
+    result = await service.sync_room_calendar(room)
+    assert result["status"] == "skipped"
+    assert result["reason"] == "ICS not configured"
+
+
+@pytest.mark.asyncio
+async def test_ics_sync_service_error_handling():
+    # Create room
+    room = await rooms_controller.add(
+        name="error-test",
+        user_id="test-user",
+        zulip_auto_post=False,
+        zulip_stream="",
+        zulip_topic="",
+        is_locked=False,
+        room_mode="normal",
+        recording_type="cloud",
+        recording_trigger="automatic-2nd-participant",
+        is_shared=False,
+        ics_url="https://calendar.example.com/error.ics",
+        ics_enabled=True,
+    )
+
+    sync_service = ICSSyncService()
+
+    with patch.object(
+        sync_service.fetch_service, "fetch_ics", new_callable=AsyncMock
+    ) as mock_fetch:
+        mock_fetch.side_effect = Exception("Network error")
+
+        result = await sync_service.sync_room_calendar(room)
+        assert result["status"] == "error"
+        assert "Network error" in result["error"]

server/tests/test_model_api_diarization.py

@@ -0,0 +1,63 @@
+"""
+Tests for diarization Model API endpoint (self-hosted service compatible shape).
+
+Marked with the "model_api" marker and skipped unless DIARIZATION_URL is provided.
+
+Run with for local self-hosted server:
+    DIARIZATION_API_KEY=dev-key \
+    DIARIZATION_URL=http://localhost:8000 \
+    uv run -m pytest -m model_api --no-cov tests/test_model_api_diarization.py
+"""
+
+import os
+
+import httpx
+import pytest
+
+# Public test audio file hosted on S3 specifically for reflector pytests
+TEST_AUDIO_URL = (
+    "https://reflector-github-pytest.s3.us-east-1.amazonaws.com/test_mathieu_hello.mp3"
+)
+
+
+def get_modal_diarization_url():
+    url = os.environ.get("DIARIZATION_URL")
+    if not url:
+        pytest.skip(
+            "DIARIZATION_URL environment variable is required for Model API tests"
+        )
+    return url
+
+
+def get_auth_headers():
+    api_key = os.environ.get("DIARIZATION_API_KEY") or os.environ.get(
+        "REFLECTOR_GPU_APIKEY"
+    )
+    return {"Authorization": f"Bearer {api_key}"} if api_key else {}
+
+
+@pytest.mark.model_api
+class TestModelAPIDiarization:
+    def test_diarize_from_url(self):
+        url = get_modal_diarization_url()
+        headers = get_auth_headers()
+
+        with httpx.Client(timeout=60.0) as client:
+            response = client.post(
+                f"{url}/diarize",
+                params={"audio_file_url": TEST_AUDIO_URL, "timestamp": 0.0},
+                headers=headers,
+            )
+
+            assert response.status_code == 200, f"Request failed: {response.text}"
+            result = response.json()
+
+            assert "diarization" in result
+            assert isinstance(result["diarization"], list)
+            assert len(result["diarization"]) > 0
+
+            for seg in result["diarization"]:
+                assert "start" in seg and "end" in seg and "speaker" in seg
+                assert isinstance(seg["start"], (int, float))
+                assert isinstance(seg["end"], (int, float))
+                assert seg["start"] <= seg["end"]

server/tests/test_model_api_transcript.py

@@ -1,21 +1,21 @@
 """
-Tests for GPU Modal transcription endpoints.
+Tests for transcription Model API endpoints.
 
-These tests are marked with the "gpu-modal" group and will not run by default.
-Run them with: pytest -m gpu-modal tests/test_gpu_modal_transcript_parakeet.py
+These tests are marked with the "model_api" group and will not run by default.
+Run them with: pytest -m model_api tests/test_model_api_transcript.py
 
 Required environment variables:
-- TRANSCRIPT_URL: URL to the Modal.com endpoint (required)
-- TRANSCRIPT_MODAL_API_KEY: API key for authentication (optional)
+- TRANSCRIPT_URL: URL to the Model API endpoint (required)
+- TRANSCRIPT_API_KEY: API key for authentication (optional)
 - TRANSCRIPT_MODEL: Model name to use (optional, defaults to nvidia/parakeet-tdt-0.6b-v2)
 
-Example with pytest (override default addopts to run ONLY gpu_modal tests):
+Example with pytest (override default addopts to run ONLY model_api tests):
     TRANSCRIPT_URL=https://monadical-sas--reflector-transcriber-parakeet-web-dev.modal.run \
-    TRANSCRIPT_MODAL_API_KEY=your-api-key \
-    uv run -m pytest -m gpu_modal --no-cov tests/test_gpu_modal_transcript.py
+    TRANSCRIPT_API_KEY=your-api-key \
+    uv run -m pytest -m model_api --no-cov tests/test_model_api_transcript.py
 
     # Or with completely clean options:
-    uv run -m pytest -m gpu_modal -o addopts="" tests/
+    uv run -m pytest -m model_api -o addopts="" tests/
 
 Running Modal locally for testing:
     modal serve gpu/modal_deployments/reflector_transcriber_parakeet.py
@@ -40,14 +40,16 @@ def get_modal_transcript_url():
     url = os.environ.get("TRANSCRIPT_URL")
     if not url:
         pytest.skip(
-            "TRANSCRIPT_URL environment variable is required for GPU Modal tests"
+            "TRANSCRIPT_URL environment variable is required for Model API tests"
         )
     return url
 
 
 def get_auth_headers():
     """Get authentication headers if API key is available."""
-    api_key = os.environ.get("TRANSCRIPT_MODAL_API_KEY")
+    api_key = os.environ.get("TRANSCRIPT_API_KEY") or os.environ.get(
+        "REFLECTOR_GPU_APIKEY"
+    )
     if api_key:
         return {"Authorization": f"Bearer {api_key}"}
     return {}
@@ -58,8 +60,8 @@ def get_model_name():
     return os.environ.get("TRANSCRIPT_MODEL", "nvidia/parakeet-tdt-0.6b-v2")
 
 
-@pytest.mark.gpu_modal
-class TestGPUModalTranscript:
+@pytest.mark.model_api
+class TestModelAPITranscript:
     """Test suite for GPU Modal transcription endpoints."""
 
     def test_transcriptions_from_url(self):
@@ -272,6 +274,9 @@ class TestGPUModalTranscript:
                 for f in temp_files:
                     Path(f).unlink(missing_ok=True)
 
+    @pytest.mark.skipif(
+        not "parakeet" in get_model_name(), reason="Parakeet only supports English"
+    )
     def test_transcriptions_error_handling(self):
         """Test error handling for invalid requests."""
         url = get_modal_transcript_url()

server/tests/test_model_api_translation.py

@@ -0,0 +1,56 @@
+"""
+Tests for translation Model API endpoint (self-hosted service compatible shape).
+
+Marked with the "model_api" marker and skipped unless TRANSLATION_URL is provided
+or we fallback to TRANSCRIPT_URL base (same host for self-hosted).
+
+Run locally against self-hosted server:
+    TRANSLATION_API_KEY=dev-key \
+    TRANSLATION_URL=http://localhost:8000 \
+    uv run -m pytest -m model_api --no-cov tests/test_model_api_translation.py
+"""
+
+import os
+
+import httpx
+import pytest
+
+
+def get_translation_url():
+    url = os.environ.get("TRANSLATION_URL") or os.environ.get("TRANSCRIPT_URL")
+    if not url:
+        pytest.skip(
+            "TRANSLATION_URL or TRANSCRIPT_URL environment variable is required for Model API tests"
+        )
+    return url
+
+
+def get_auth_headers():
+    api_key = os.environ.get("TRANSLATION_API_KEY") or os.environ.get(
+        "REFLECTOR_GPU_APIKEY"
+    )
+    return {"Authorization": f"Bearer {api_key}"} if api_key else {}
+
+
+@pytest.mark.model_api
+class TestModelAPITranslation:
+    def test_translate_text(self):
+        url = get_translation_url()
+        headers = get_auth_headers()
+
+        with httpx.Client(timeout=60.0) as client:
+            response = client.post(
+                f"{url}/translate",
+                params={"text": "The meeting will start in five minutes."},
+                json={"source_language": "en", "target_language": "fr"},
+                headers=headers,
+            )
+
+            assert response.status_code == 200, f"Request failed: {response.text}"
+            data = response.json()
+
+            assert "text" in data and isinstance(data["text"], dict)
+            assert data["text"].get("en") == "The meeting will start in five minutes."
+            assert isinstance(data["text"].get("fr", ""), str)
+            assert len(data["text"]["fr"]) > 0
+            assert data["text"]["fr"] == "La réunion commencera dans cinq minutes."

server/tests/test_multiple_active_meetings.py

@@ -0,0 +1,167 @@
+"""Tests for multiple active meetings per room functionality."""
+
+from datetime import datetime, timedelta, timezone
+
+import pytest
+
+from reflector.db.calendar_events import CalendarEvent, calendar_events_controller
+from reflector.db.meetings import meetings_controller
+from reflector.db.rooms import rooms_controller
+
+
+@pytest.mark.asyncio
+async def test_multiple_active_meetings_per_room():
+    """Test that multiple active meetings can exist for the same room."""
+    # Create a room
+    room = await rooms_controller.add(
+        name="test-room",
+        user_id="test-user",
+        zulip_auto_post=False,
+        zulip_stream="",
+        zulip_topic="",
+        is_locked=False,
+        room_mode="normal",
+        recording_type="cloud",
+        recording_trigger="automatic-2nd-participant",
+        is_shared=False,
+    )
+
+    current_time = datetime.now(timezone.utc)
+    end_time = current_time + timedelta(hours=2)
+
+    # Create first meeting
+    meeting1 = await meetings_controller.create(
+        id="meeting-1",
+        room_name="test-meeting-1",
+        room_url="https://whereby.com/test-1",
+        host_room_url="https://whereby.com/test-1-host",
+        start_date=current_time,
+        end_date=end_time,
+        room=room,
+    )
+
+    # Create second meeting for the same room (should succeed now)
+    meeting2 = await meetings_controller.create(
+        id="meeting-2",
+        room_name="test-meeting-2",
+        room_url="https://whereby.com/test-2",
+        host_room_url="https://whereby.com/test-2-host",
+        start_date=current_time,
+        end_date=end_time,
+        room=room,
+    )
+
+    # Both meetings should be active
+    active_meetings = await meetings_controller.get_all_active_for_room(
+        room=room, current_time=current_time
+    )
+
+    assert len(active_meetings) == 2
+    assert meeting1.id in [m.id for m in active_meetings]
+    assert meeting2.id in [m.id for m in active_meetings]
+
+
+@pytest.mark.asyncio
+async def test_get_active_by_calendar_event():
+    """Test getting active meeting by calendar event ID."""
+    # Create a room
+    room = await rooms_controller.add(
+        name="test-room",
+        user_id="test-user",
+        zulip_auto_post=False,
+        zulip_stream="",
+        zulip_topic="",
+        is_locked=False,
+        room_mode="normal",
+        recording_type="cloud",
+        recording_trigger="automatic-2nd-participant",
+        is_shared=False,
+    )
+
+    # Create a calendar event
+    event = CalendarEvent(
+        room_id=room.id,
+        ics_uid="test-event-uid",
+        title="Test Meeting",
+        start_time=datetime.now(timezone.utc),
+        end_time=datetime.now(timezone.utc) + timedelta(hours=1),
+    )
+    event = await calendar_events_controller.upsert(event)
+
+    current_time = datetime.now(timezone.utc)
+    end_time = current_time + timedelta(hours=2)
+
+    # Create meeting linked to calendar event
+    meeting = await meetings_controller.create(
+        id="meeting-cal-1",
+        room_name="test-meeting-cal",
+        room_url="https://whereby.com/test-cal",
+        host_room_url="https://whereby.com/test-cal-host",
+        start_date=current_time,
+        end_date=end_time,
+        room=room,
+        calendar_event_id=event.id,
+        calendar_metadata={"title": event.title},
+    )
+
+    # Should find the meeting by calendar event
+    found_meeting = await meetings_controller.get_active_by_calendar_event(
+        room=room, calendar_event_id=event.id, current_time=current_time
+    )
+
+    assert found_meeting is not None
+    assert found_meeting.id == meeting.id
+    assert found_meeting.calendar_event_id == event.id
+
+
+@pytest.mark.asyncio
+async def test_calendar_meeting_deactivates_after_scheduled_end():
+    """Test that unused calendar meetings deactivate after scheduled end time."""
+    # Create a room
+    room = await rooms_controller.add(
+        name="test-room",
+        user_id="test-user",
+        zulip_auto_post=False,
+        zulip_stream="",
+        zulip_topic="",
+        is_locked=False,
+        room_mode="normal",
+        recording_type="cloud",
+        recording_trigger="automatic-2nd-participant",
+        is_shared=False,
+    )
+
+    # Create a calendar event that ended 35 minutes ago
+    event = CalendarEvent(
+        room_id=room.id,
+        ics_uid="test-event-unused",
+        title="Test Meeting Unused",
+        start_time=datetime.now(timezone.utc) - timedelta(hours=2),
+        end_time=datetime.now(timezone.utc) - timedelta(minutes=35),
+    )
+    event = await calendar_events_controller.upsert(event)
+
+    current_time = datetime.now(timezone.utc)
+
+    # Create meeting linked to calendar event
+    meeting = await meetings_controller.create(
+        id="meeting-unused",
+        room_name="test-meeting-unused",
+        room_url="https://whereby.com/test-unused",
+        host_room_url="https://whereby.com/test-unused-host",
+        start_date=event.start_time,
+        end_date=event.end_time,
+        room=room,
+        calendar_event_id=event.id,
+    )
+
+    # Test the new logic: unused calendar meetings deactivate after scheduled end
+    # The meeting ended 35 minutes ago and was never used, so it should be deactivated
+
+    # Simulate process_meetings logic for unused calendar meeting past end time
+    if meeting.calendar_event_id and current_time > meeting.end_date:
+        # In real code, we'd check has_had_sessions = False here
+        await meetings_controller.update_meeting(meeting.id, is_active=False)
+
+    updated_meeting = await meetings_controller.get_by_id(meeting.id)
+    assert updated_meeting.is_active is False  # Deactivated after scheduled end

server/tests/test_processors_pipeline.py

@@ -1,61 +0,0 @@
-import pytest
-
-
-@pytest.mark.asyncio
-@pytest.mark.parametrize("enable_diarization", [False, True])
-async def test_basic_process(
-    dummy_transcript,
-    dummy_llm,
-    dummy_processors,
-    enable_diarization,
-    dummy_diarization,
-):
-    # goal is to start the server, and send rtc audio to it
-    # validate the events received
-    from pathlib import Path
-
-    from reflector.settings import settings
-    from reflector.tools.process import process_audio_file
-
-    # LLM_BACKEND no longer exists in settings
-    # settings.LLM_BACKEND = "test"
-    settings.TRANSCRIPT_BACKEND = "whisper"
-
-    # event callback
-    marks = {}
-
-    async def event_callback(event):
-        if event.processor not in marks:
-            marks[event.processor] = 0
-        marks[event.processor] += 1
-
-    # invoke the process and capture events
-    path = Path(__file__).parent / "records" / "test_mathieu_hello.wav"
-
-    if enable_diarization:
-        # Test with diarization - may fail if pyannote.audio is not installed
-        try:
-            await process_audio_file(
-                path.as_posix(), event_callback, enable_diarization=True
-            )
-        except SystemExit:
-            pytest.skip("pyannote.audio not installed - skipping diarization test")
-    else:
-        # Test without diarization - should always work
-        await process_audio_file(
-            path.as_posix(), event_callback, enable_diarization=False
-        )
-
-    print(f"Diarization: {enable_diarization}, Marks: {marks}")
-
-    # validate the events
-    # Each processor should be called for each audio segment processed
-    # The final processors (Topic, Title, Summary) should be called once at the end
-    assert marks["TranscriptLinerProcessor"] > 0
-    assert marks["TranscriptTranslatorPassthroughProcessor"] > 0
-    assert marks["TranscriptTopicDetectorProcessor"] == 1
-    assert marks["TranscriptFinalSummaryProcessor"] == 1
-    assert marks["TranscriptFinalTitleProcessor"] == 1
-
-    if enable_diarization:
-        assert marks["TestAudioDiarizationProcessor"] == 1

server/tests/test_room_ics.py

@@ -0,0 +1,225 @@
+"""
+Tests for Room model ICS calendar integration fields.
+"""
+
+from datetime import datetime, timezone
+
+import pytest
+
+from reflector.db.rooms import rooms_controller
+
+
+@pytest.mark.asyncio
+async def test_room_create_with_ics_fields():
+    """Test creating a room with ICS calendar fields."""
+    room = await rooms_controller.add(
+        name="test-room",
+        user_id="test-user",
+        zulip_auto_post=False,
+        zulip_stream="",
+        zulip_topic="",
+        is_locked=False,
+        room_mode="normal",
+        recording_type="cloud",
+        recording_trigger="automatic-2nd-participant",
+        is_shared=False,
+        ics_url="https://calendar.google.com/calendar/ical/test/private-token/basic.ics",
+        ics_fetch_interval=600,
+        ics_enabled=True,
+    )
+
+    assert room.name == "test-room"
+    assert (
+        room.ics_url
+        == "https://calendar.google.com/calendar/ical/test/private-token/basic.ics"
+    )
+    assert room.ics_fetch_interval == 600
+    assert room.ics_enabled is True
+    assert room.ics_last_sync is None
+    assert room.ics_last_etag is None
+
+
+@pytest.mark.asyncio
+async def test_room_update_ics_configuration():
+    """Test updating room ICS configuration."""
+    # Create room without ICS
+    room = await rooms_controller.add(
+        name="update-test",
+        user_id="test-user",
+        zulip_auto_post=False,
+        zulip_stream="",
+        zulip_topic="",
+        is_locked=False,
+        room_mode="normal",
+        recording_type="cloud",
+        recording_trigger="automatic-2nd-participant",
+        is_shared=False,
+    )
+
+    assert room.ics_enabled is False
+    assert room.ics_url is None
+
+    # Update with ICS configuration
+    await rooms_controller.update(
+        room,
+        {
+            "ics_url": "https://outlook.office365.com/owa/calendar/test/calendar.ics",
+            "ics_fetch_interval": 300,
+            "ics_enabled": True,
+        },
+    )
+
+    assert (
+        room.ics_url == "https://outlook.office365.com/owa/calendar/test/calendar.ics"
+    )
+    assert room.ics_fetch_interval == 300
+    assert room.ics_enabled is True
+
+
+@pytest.mark.asyncio
+async def test_room_ics_sync_metadata():
+    """Test updating room ICS sync metadata."""
+    room = await rooms_controller.add(
+        name="sync-test",
+        user_id="test-user",
+        zulip_auto_post=False,
+        zulip_stream="",
+        zulip_topic="",
+        is_locked=False,
+        room_mode="normal",
+        recording_type="cloud",
+        recording_trigger="automatic-2nd-participant",
+        is_shared=False,
+        ics_url="https://example.com/calendar.ics",
+        ics_enabled=True,
+    )
+
+    # Update sync metadata
+    sync_time = datetime.now(timezone.utc)
+    await rooms_controller.update(
+        room,
+        {
+            "ics_last_sync": sync_time,
+            "ics_last_etag": "abc123hash",
+        },
+    )
+
+    assert room.ics_last_sync == sync_time
+    assert room.ics_last_etag == "abc123hash"
+
+
+@pytest.mark.asyncio
+async def test_room_get_with_ics_fields():
+    """Test retrieving room with ICS fields."""
+    # Create room
+    created_room = await rooms_controller.add(
+        name="get-test",
+        user_id="test-user",
+        zulip_auto_post=False,
+        zulip_stream="",
+        zulip_topic="",
+        is_locked=False,
+        room_mode="normal",
+        recording_type="cloud",
+        recording_trigger="automatic-2nd-participant",
+        is_shared=False,
+        ics_url="webcal://calendar.example.com/feed.ics",
+        ics_fetch_interval=900,
+        ics_enabled=True,
+    )
+
+    # Get by ID
+    room = await rooms_controller.get_by_id(created_room.id)
+    assert room is not None
+    assert room.ics_url == "webcal://calendar.example.com/feed.ics"
+    assert room.ics_fetch_interval == 900
+    assert room.ics_enabled is True
+
+    # Get by name
+    room = await rooms_controller.get_by_name("get-test")
+    assert room is not None
+    assert room.ics_url == "webcal://calendar.example.com/feed.ics"
+    assert room.ics_fetch_interval == 900
+    assert room.ics_enabled is True
+
+
+@pytest.mark.asyncio
+async def test_room_list_with_ics_enabled_filter():
+    """Test listing rooms filtered by ICS enabled status."""
+    # Create rooms with and without ICS
+    room1 = await rooms_controller.add(
+        name="ics-enabled-1",
+        user_id="test-user",
+        zulip_auto_post=False,
+        zulip_stream="",
+        zulip_topic="",
+        is_locked=False,
+        room_mode="normal",
+        recording_type="cloud",
+        recording_trigger="automatic-2nd-participant",
+        is_shared=True,
+        ics_enabled=True,
+        ics_url="https://calendar1.example.com/feed.ics",
+    )
+
+    room2 = await rooms_controller.add(
+        name="ics-disabled",
+        user_id="test-user",
+        zulip_auto_post=False,
+        zulip_stream="",
+        zulip_topic="",
+        is_locked=False,
+        room_mode="normal",
+        recording_type="cloud",
+        recording_trigger="automatic-2nd-participant",
+        is_shared=True,
+        ics_enabled=False,
+    )
+
+    room3 = await rooms_controller.add(
+        name="ics-enabled-2",
+        user_id="test-user",
+        zulip_auto_post=False,
+        zulip_stream="",
+        zulip_topic="",
+        is_locked=False,
+        room_mode="normal",
+        recording_type="cloud",
+        recording_trigger="automatic-2nd-participant",
+        is_shared=True,
+        ics_enabled=True,
+        ics_url="https://calendar2.example.com/feed.ics",
+    )
+
+    # Get all rooms
+    all_rooms = await rooms_controller.get_all()
+    assert len(all_rooms) == 3
+
+    # Filter for ICS-enabled rooms (would need to implement this in controller)
+    ics_rooms = [r for r in all_rooms if r["ics_enabled"]]
+    assert len(ics_rooms) == 2
+    assert all(r["ics_enabled"] for r in ics_rooms)
+
+
+@pytest.mark.asyncio
+async def test_room_default_ics_values():
+    """Test that ICS fields have correct default values."""
+    room = await rooms_controller.add(
+        name="default-test",
+        user_id="test-user",
+        zulip_auto_post=False,
+        zulip_stream="",
+        zulip_topic="",
+        is_locked=False,
+        room_mode="normal",
+        recording_type="cloud",
+        recording_trigger="automatic-2nd-participant",
+        is_shared=False,
+        # Don't specify ICS fields
+    )
+
+    assert room.ics_url is None
+    assert room.ics_fetch_interval == 300  # Default 5 minutes
+    assert room.ics_enabled is False
+    assert room.ics_last_sync is None
+    assert room.ics_last_etag is None

server/tests/test_room_ics_api.py

@@ -0,0 +1,390 @@
+from datetime import datetime, timedelta, timezone
+from unittest.mock import AsyncMock, patch
+
+import pytest
+from icalendar import Calendar, Event
+
+from reflector.db.calendar_events import CalendarEvent, calendar_events_controller
+from reflector.db.rooms import rooms_controller
+
+
+@pytest.fixture
+async def authenticated_client(client):
+    from reflector.app import app
+    from reflector.auth import current_user_optional
+
+    app.dependency_overrides[current_user_optional] = lambda: {
+        "sub": "test-user",
+        "email": "test@example.com",
+    }
+    yield client
+    del app.dependency_overrides[current_user_optional]
+
+
+@pytest.mark.asyncio
+async def test_create_room_with_ics_fields(authenticated_client):
+    client = authenticated_client
+    response = await client.post(
+        "/rooms",
+        json={
+            "name": "test-ics-room",
+            "zulip_auto_post": False,
+            "zulip_stream": "",
+            "zulip_topic": "",
+            "is_locked": False,
+            "room_mode": "normal",
+            "recording_type": "cloud",
+            "recording_trigger": "automatic-2nd-participant",
+            "is_shared": False,
+            "webhook_url": "",
+            "webhook_secret": "",
+            "ics_url": "https://calendar.example.com/test.ics",
+            "ics_fetch_interval": 600,
+            "ics_enabled": True,
+        },
+    )
+    assert response.status_code == 200
+    data = response.json()
+    assert data["name"] == "test-ics-room"
+    assert data["ics_url"] == "https://calendar.example.com/test.ics"
+    assert data["ics_fetch_interval"] == 600
+    assert data["ics_enabled"] is True
+
+
+@pytest.mark.asyncio
+async def test_update_room_ics_configuration(authenticated_client):
+    client = authenticated_client
+    response = await client.post(
+        "/rooms",
+        json={
+            "name": "update-ics-room",
+            "zulip_auto_post": False,
+            "zulip_stream": "",
+            "zulip_topic": "",
+            "is_locked": False,
+            "room_mode": "normal",
+            "recording_type": "cloud",
+            "recording_trigger": "automatic-2nd-participant",
+            "is_shared": False,
+            "webhook_url": "",
+            "webhook_secret": "",
+        },
+    )
+    assert response.status_code == 200
+    room_id = response.json()["id"]
+
+    response = await client.patch(
+        f"/rooms/{room_id}",
+        json={
+            "ics_url": "https://calendar.google.com/updated.ics",
+            "ics_fetch_interval": 300,
+            "ics_enabled": True,
+        },
+    )
+    assert response.status_code == 200
+    data = response.json()
+    assert data["ics_url"] == "https://calendar.google.com/updated.ics"
+    assert data["ics_fetch_interval"] == 300
+    assert data["ics_enabled"] is True
+
+
+@pytest.mark.asyncio
+async def test_trigger_ics_sync(authenticated_client):
+    client = authenticated_client
+    room = await rooms_controller.add(
+        name="sync-api-room",
+        user_id="test-user",
+        zulip_auto_post=False,
+        zulip_stream="",
+        zulip_topic="",
+        is_locked=False,
+        room_mode="normal",
+        recording_type="cloud",
+        recording_trigger="automatic-2nd-participant",
+        is_shared=False,
+        ics_url="https://calendar.example.com/api.ics",
+        ics_enabled=True,
+    )
+
+    cal = Calendar()
+    event = Event()
+    event.add("uid", "api-test-event")
+    event.add("summary", "API Test Meeting")
+    from reflector.settings import settings
+
+    event.add("location", f"{settings.UI_BASE_URL}/{room.name}")
+    now = datetime.now(timezone.utc)
+    event.add("dtstart", now + timedelta(hours=1))
+    event.add("dtend", now + timedelta(hours=2))
+    cal.add_component(event)
+    ics_content = cal.to_ical().decode("utf-8")
+
+    with patch(
+        "reflector.services.ics_sync.ICSFetchService.fetch_ics", new_callable=AsyncMock
+    ) as mock_fetch:
+        mock_fetch.return_value = ics_content
+
+        response = await client.post(f"/rooms/{room.name}/ics/sync")
+        assert response.status_code == 200
+        data = response.json()
+        assert data["status"] == "success"
+        assert data["events_found"] == 1
+        assert data["events_created"] == 1
+
+
+@pytest.mark.asyncio
+async def test_trigger_ics_sync_unauthorized(client):
+    room = await rooms_controller.add(
+        name="sync-unauth-room",
+        user_id="owner-123",
+        zulip_auto_post=False,
+        zulip_stream="",
+        zulip_topic="",
+        is_locked=False,
+        room_mode="normal",
+        recording_type="cloud",
+        recording_trigger="automatic-2nd-participant",
+        is_shared=False,
+        ics_url="https://calendar.example.com/api.ics",
+        ics_enabled=True,
+    )
+
+    response = await client.post(f"/rooms/{room.name}/ics/sync")
+    assert response.status_code == 403
+    assert "Only room owner can trigger ICS sync" in response.json()["detail"]
+
+
+@pytest.mark.asyncio
+async def test_trigger_ics_sync_not_configured(authenticated_client):
+    client = authenticated_client
+    room = await rooms_controller.add(
+        name="sync-not-configured",
+        user_id="test-user",
+        zulip_auto_post=False,
+        zulip_stream="",
+        zulip_topic="",
+        is_locked=False,
+        room_mode="normal",
+        recording_type="cloud",
+        recording_trigger="automatic-2nd-participant",
+        is_shared=False,
+        ics_enabled=False,
+    )
+
+    response = await client.post(f"/rooms/{room.name}/ics/sync")
+    assert response.status_code == 400
+    assert "ICS not configured" in response.json()["detail"]
+
+
+@pytest.mark.asyncio
+async def test_get_ics_status(authenticated_client):
+    client = authenticated_client
+    room = await rooms_controller.add(
+        name="status-room",
+        user_id="test-user",
+        zulip_auto_post=False,
+        zulip_stream="",
+        zulip_topic="",
+        is_locked=False,
+        room_mode="normal",
+        recording_type="cloud",
+        recording_trigger="automatic-2nd-participant",
+        is_shared=False,
+        ics_url="https://calendar.example.com/status.ics",
+        ics_enabled=True,
+        ics_fetch_interval=300,
+    )
+
+    now = datetime.now(timezone.utc)
+    await rooms_controller.update(
+        room,
+        {"ics_last_sync": now, "ics_last_etag": "test-etag"},
+    )
+
+    response = await client.get(f"/rooms/{room.name}/ics/status")
+    assert response.status_code == 200
+    data = response.json()
+    assert data["status"] == "enabled"
+    assert data["last_etag"] == "test-etag"
+    assert data["events_count"] == 0
+
+
+@pytest.mark.asyncio
+async def test_get_ics_status_unauthorized(client):
+    room = await rooms_controller.add(
+        name="status-unauth",
+        user_id="owner-456",
+        zulip_auto_post=False,
+        zulip_stream="",
+        zulip_topic="",
+        is_locked=False,
+        room_mode="normal",
+        recording_type="cloud",
+        recording_trigger="automatic-2nd-participant",
+        is_shared=False,
+        ics_url="https://calendar.example.com/status.ics",
+        ics_enabled=True,
+    )
+
+    response = await client.get(f"/rooms/{room.name}/ics/status")
+    assert response.status_code == 403
+    assert "Only room owner can view ICS status" in response.json()["detail"]
+
+
+@pytest.mark.asyncio
+async def test_list_room_meetings(authenticated_client):
+    client = authenticated_client
+    room = await rooms_controller.add(
+        name="meetings-room",
+        user_id="test-user",
+        zulip_auto_post=False,
+        zulip_stream="",
+        zulip_topic="",
+        is_locked=False,
+        room_mode="normal",
+        recording_type="cloud",
+        recording_trigger="automatic-2nd-participant",
+        is_shared=False,
+    )
+
+    now = datetime.now(timezone.utc)
+    event1 = CalendarEvent(
+        room_id=room.id,
+        ics_uid="meeting-1",
+        title="Past Meeting",
+        start_time=now - timedelta(hours=2),
+        end_time=now - timedelta(hours=1),
+    )
+    await calendar_events_controller.upsert(event1)
+
+    event2 = CalendarEvent(
+        room_id=room.id,
+        ics_uid="meeting-2",
+        title="Future Meeting",
+        description="Team sync",
+        start_time=now + timedelta(hours=1),
+        end_time=now + timedelta(hours=2),
+        attendees=[{"email": "test@example.com"}],
+    )
+    await calendar_events_controller.upsert(event2)
+
+    response = await client.get(f"/rooms/{room.name}/meetings")
+    assert response.status_code == 200
+    data = response.json()
+    assert len(data) == 2
+    assert data[0]["title"] == "Past Meeting"
+    assert data[1]["title"] == "Future Meeting"
+    assert data[1]["description"] == "Team sync"
+    assert data[1]["attendees"] == [{"email": "test@example.com"}]
+
+
+@pytest.mark.asyncio
+async def test_list_room_meetings_non_owner(client):
+    room = await rooms_controller.add(
+        name="meetings-privacy",
+        user_id="owner-789",
+        zulip_auto_post=False,
+        zulip_stream="",
+        zulip_topic="",
+        is_locked=False,
+        room_mode="normal",
+        recording_type="cloud",
+        recording_trigger="automatic-2nd-participant",
+        is_shared=False,
+    )
+
+    event = CalendarEvent(
+        room_id=room.id,
+        ics_uid="private-meeting",
+        title="Meeting Title",
+        description="Sensitive info",
+        start_time=datetime.now(timezone.utc) + timedelta(hours=1),
+        end_time=datetime.now(timezone.utc) + timedelta(hours=2),
+        attendees=[{"email": "private@example.com"}],
+    )
+    await calendar_events_controller.upsert(event)
+
+    response = await client.get(f"/rooms/{room.name}/meetings")
+    assert response.status_code == 200
+    data = response.json()
+    assert len(data) == 1
+    assert data[0]["title"] == "Meeting Title"
+    assert data[0]["description"] is None
+    assert data[0]["attendees"] is None
+
+
+@pytest.mark.asyncio
+async def test_list_upcoming_meetings(authenticated_client):
+    client = authenticated_client
+    room = await rooms_controller.add(
+        name="upcoming-room",
+        user_id="test-user",
+        zulip_auto_post=False,
+        zulip_stream="",
+        zulip_topic="",
+        is_locked=False,
+        room_mode="normal",
+        recording_type="cloud",
+        recording_trigger="automatic-2nd-participant",
+        is_shared=False,
+    )
+
+    now = datetime.now(timezone.utc)
+
+    past_event = CalendarEvent(
+        room_id=room.id,
+        ics_uid="past",
+        title="Past",
+        start_time=now - timedelta(hours=1),
+        end_time=now - timedelta(minutes=30),
+    )
+    await calendar_events_controller.upsert(past_event)
+
+    soon_event = CalendarEvent(
+        room_id=room.id,
+        ics_uid="soon",
+        title="Soon",
+        start_time=now + timedelta(minutes=15),
+        end_time=now + timedelta(minutes=45),
+    )
+    await calendar_events_controller.upsert(soon_event)
+
+    later_event = CalendarEvent(
+        room_id=room.id,
+        ics_uid="later",
+        title="Later",
+        start_time=now + timedelta(hours=2),
+        end_time=now + timedelta(hours=3),
+    )
+    await calendar_events_controller.upsert(later_event)
+
+    response = await client.get(f"/rooms/{room.name}/meetings/upcoming")
+    assert response.status_code == 200
+    data = response.json()
+    assert len(data) == 2
+    assert data[0]["title"] == "Soon"
+    assert data[1]["title"] == "Later"
+
+    response = await client.get(
+        f"/rooms/{room.name}/meetings/upcoming", params={"minutes_ahead": 180}
+    )
+    assert response.status_code == 200
+    data = response.json()
+    assert len(data) == 2
+    assert data[0]["title"] == "Soon"
+    assert data[1]["title"] == "Later"
+
+
+@pytest.mark.asyncio
+async def test_room_not_found_endpoints(client):
+    response = await client.post("/rooms/nonexistent/ics/sync")
+    assert response.status_code == 404
+
+    response = await client.get("/rooms/nonexistent/ics/status")
+    assert response.status_code == 404
+
+    response = await client.get("/rooms/nonexistent/meetings")
+    assert response.status_code == 404
+
+    response = await client.get("/rooms/nonexistent/meetings/upcoming")
+    assert response.status_code == 404