Compare commits

..

46 Commits

Author SHA1 Message Date
e91979abbc feat: use jitsi file system 2025-09-17 15:16:03 -06:00
95e8011975 Merge main into jisti-integration branch
- Resolved conflicts in server/reflector/views/rooms.py to keep platform-agnostic approach
- Resolved conflicts in www/app/[roomName]/page.tsx to keep VideoPlatformEmbed approach
- Accepted main's version of generated API files (schemas.gen.ts, services.gen.ts, types.gen.ts)
- Removed config-template.ts as per main branch changes
2025-09-15 12:53:49 -06:00
c546e69739 fix: zulip stream and topic selection in share dialog (#644)
* fix: zulip stream and topic selection in share dialog

Replace useListCollection with createListCollection to match the working
room edit implementation. This ensures collections update when data loads,
fixing the issue where streams and topics wouldn't appear until navigation.

* fix: wrap createListCollection in useMemo to prevent recreation on every render

Both streamCollection and topicCollection are now memoized to improve performance
and prevent unnecessary re-renders of Combobox components
2025-09-15 12:34:51 -06:00
Igor Monadical
3f1fe8c9bf chore: remove timeout-based auth session logic (#649)
* remove timeout-based auth session logic

* remove timeout-based auth session logic

---------

Co-authored-by: Igor Loskutov <igor.loskutoff@gmail.com>
2025-09-15 14:19:10 -04:00
5f143fe364 fix: zulip and consent handler on the file pipeline (#645) 2025-09-15 10:49:20 -06:00
Igor Monadical
79f161436e chore: meeting user id removal and room id requirement (#635)
* chore: remove meeting user id and make meeting room id required

* meeting room_id optional

* orphaned meeting room ids DATA migration

* ci fix

* fix meeting_room_id_fkey downgrade

* fix migration rollback

* fix: put index back (meeting room id)

* fix: put index back (meeting room id)

* fix: put index back (meeting room id)

* remove noop migrations

---------

Co-authored-by: Igor Loskutov <igor.loskutoff@gmail.com>
2025-09-12 13:07:58 -04:00
Igor Monadical
5cba5d310d chore: sentry and nextjs major bumps (#633)
* chore: remove nextjs-config

* build fix

* sentry update

* nextjs update

* feature flags doc

* update readme

* explicit nextjs env vars + remove feature-unrelated things and obsolete vars from config

* full config removal

* remove force-dynamic from pages

* compile fix

* restore claude-deleted tests

* no sentry backward compat

* better .env.example

* AUTHENTIK_REFRESH_TOKEN_URL not so required

* accommodate auth system to requiredLogin feature

---------

Co-authored-by: Igor Loskutov <igor.loskutoff@gmail.com>
2025-09-12 12:41:44 -04:00
43ea9349f5 chore(main): release 0.10.0 (#616) 2025-09-11 20:57:19 -06:00
Igor Monadical
b3a8e9739d chore: whereby & s3 settings env error reporting (#637)
Co-authored-by: Igor Loskutov <igor.loskutoff@gmail.com>
2025-09-11 17:52:34 -04:00
Igor Monadical
369ecdff13 feat: replace nextjs-config with environment variables (#632)
* chore: remove nextjs-config

* build fix

* update readme

* explicit nextjs env vars + remove feature-unrelated things and obsolete vars from config

* full config removal

* remove force-dynamic from pages

* compile fix

* restore claude-deleted tests

* better .env.example

---------

Co-authored-by: Igor Loskutov <igor.loskutoff@gmail.com>
2025-09-11 11:20:41 -04:00
fc363bd49b fix: missing follow_redirects=True on modal endpoint (#630) 2025-09-10 08:15:47 -06:00
Igor Monadical
962038ee3f fix: auth post (#627)
Co-authored-by: Igor Loskutov <igor.loskutoff@gmail.com>
2025-09-09 16:46:57 -04:00
Igor Monadical
3b85ff3bdf fix: auth post (#626)
Co-authored-by: Igor Loskutov <igor.loskutoff@gmail.com>
2025-09-09 16:27:46 -04:00
Igor Monadical
cde99ca271 fix: auth post (#624)
Co-authored-by: Igor Loskutov <igor.loskutoff@gmail.com>
2025-09-09 15:48:07 -04:00
Igor Monadical
f81fe9948a fix: anonymous users transcript permissions (#621)
* fix: public transcript visibility

* fix: transcript permissions frontend

* dead code removal

* chore: remove unused code

* fix search tests

* fix search tests

---------

Co-authored-by: Igor Loskutov <igor.loskutoff@gmail.com>
2025-09-09 10:50:29 -04:00
Igor Monadical
5a5b323382 fix: sync backend and frontend token refresh logic (#614)
* sync backend and frontend token refresh logic

* return react strict mode

---------

Co-authored-by: Igor Loskutov <igor.loskutoff@gmail.com>
2025-09-08 10:40:18 -04:00
02a3938822 chore(main): release 0.9.0 (#603) 2025-09-05 22:50:10 -06:00
Igor Monadical
7f5a4c9ddc fix: token refresh locking (#613)
* fix: kv use tls explicit

* fix: token refresh locking

* remove logs

* compile fix

* compile fix

---------

Co-authored-by: Igor Loskutov <igor.loskutoff@gmail.com>
2025-09-05 23:03:24 -04:00
Igor Monadical
08d88ec349 fix: kv use tls explicit (#610)
Co-authored-by: Igor Loskutov <igor.loskutoff@gmail.com>
2025-09-05 18:39:32 -04:00
Igor Monadical
c4d2825c81 feat: frontend openapi react query (#606)
* refactor: migrate from @hey-api/openapi-ts to openapi-react-query

- Replace @hey-api/openapi-ts with openapi-typescript and openapi-react-query
- Generate TypeScript types from OpenAPI spec
- Set up React Query infrastructure with QueryClientProvider
- Migrate all API hooks to use React Query patterns
- Maintain backward compatibility for existing components
- Remove old API infrastructure and dependencies

* fix: resolve import errors and add missing api hooks

- Create constants.ts for RECORD_A_MEETING_URL
- Add api-types.ts for backward compatible type exports
- Update all imports from deleted api folder to new locations
- Add missing React Query hooks for rooms and zulip operations
- Create useApi compatibility layer for unmigrated components

* feat: migrate components to React Query hooks

- Add comprehensive API hooks for all operations
- Migrate rooms page to use React Query mutations
- Update transcript title component to use mutation hook
- Refactor share/privacy component with proper error handling
- Remove direct API client usage in favor of hooks

* feat: complete migration from @hey-api/openapi-ts to openapi-react-query

- Migrated all components from useApi compatibility layer to direct React Query hooks
- Added new hooks for participant operations, room meetings, and speaker operations
- Updated all imports from old api module to api-types
- Fixed TypeScript types and API endpoint signatures
- Removed deprecated useApi.ts compatibility layer
- Fixed SourceKind enum values to match OpenAPI spec
- Added @ts-ignore for Zulip endpoints not in OpenAPI spec yet
- Fixed all compilation errors and type issues

* fix: authentication flow with React Query migration

- Fix middleware management in apiClient to properly handle auth tokens
- Update ApiAuthProvider to correctly configure base URL and auth
- Add missing NextAuth API route handler at app/api/auth/[...nextauth]/route.ts
- Remove middleware ejection attempts (not supported by openapi-fetch)
- Use global variables to store current auth token and API URL
- Setup middleware once on initialization instead of repeatedly adding

This fixes the login/logout flow that was broken after migrating from
the useApi compatibility layer to native React Query hooks.

* fix: prevent unauthorized API calls before authentication

- Add global AuthGuard component to handle authentication at layout level
- Make all API query hooks conditional on authentication status
- Define public routes (like /transcripts/new) that don't require auth
- Fix login flow to use NextAuth signIn instead of non-existent /login route
- Prevent 401 errors by waiting for auth token before making API calls

Previously, all routes under (app) were publicly accessible with each page
handling auth individually. Now authentication is enforced globally while
still allowing specific routes to remain public.

* refactor: remove redundant client-side AuthGuard

The authentication is already properly handled by Next.js middleware
in middleware.ts with LOGIN_REQUIRED_PAGES. The middleware approach is
superior as it:
- Provides server-side protection before page loads
- Prevents flash of unauthorized content
- Centralizes auth logic in one place
- Better performance (no client-side JS needed)

Keep the API hooks conditional to prevent 401 errors before token is ready.

* fix: use direct status check for API query authentication

Changed all query hooks to use direct `status === "authenticated"` check
instead of derived `isAuthenticated && !isLoading` to avoid race conditions
where queries might fire before the authentication token is properly set.

This prevents the brief 401 errors that occur on page refresh when the
session is being restored.

* fix: correct content-type header for FormData uploads

Previously, the API client was setting a default Content-Type of application/json
for all requests, which broke file uploads that need multipart/form-data.

Now the client only sets application/json when the body is not FormData,
allowing FormData to automatically set the correct multipart boundary.

* fix: resolve authentication race condition with React Query

Previously, API calls were being made before the auth token was configured,
causing initial 401 errors that would retry with 200 after token setup.

Changes:
- Add global auth readiness tracking in apiClient
- Create useAuthReady hook that checks both session and token state
- Update all API hooks to use isAuthReady instead of just session status
- Add AuthWrapper component at layout level for consistent loading UX
- Show spinner while authentication initializes across all pages

This ensures API calls only fire after authentication is fully configured,
eliminating the 401/retry pattern and improving user experience.

* refactor: clean up api-hooks.ts comments and improve search invalidation

- Remove redundant function category comments (exports are self-explanatory)
- Remove obvious inline comments for query invalidation
- Fix search endpoint invalidation to clear all queries regardless of parameters

* refactor: remove api-types.ts compatibility layer

- Migrated all 29 files from api-types.ts to use reflector-api.d.ts directly
- Removed $SourceKind manual enum in favor of OpenAPI-generated types
- Fixed unrelated Spinner component TypeScript error in AuthWrapper.tsx
- All imports now use: import type { components } from "path/to/reflector-api"
- Deleted api-types.ts file completely

* refactor: rename api-hooks.ts to apiHooks.ts for consistency

- Renamed api-hooks.ts to apiHooks.ts to follow camelCase convention
- Updated all 21 import statements across the codebase
- Maintains consistency with other non-component files (apiClient.tsx, useAuthReady.ts, etc.)
- Follows established naming pattern: PascalCase for components, camelCase for utilities/hooks

* chore: add .playwright-mcp to .gitignore

* refactor: remove SK helper object and use inline type casting in FilterSidebar

Replace the SK (SourceKind) helper object with direct inline type casting
to simplify the code and reduce unnecessary abstraction.

* chore: clean up migration comments from React Query refactoring

- Remove temporary "// Use new React Query hooks" comments
- Remove "// React Query hooks" comments from browse and rooms pages
- Update package.json script name from codegen to openapi for consistency

* refactor: remove Redis dependencies from frontend authentication

- Replace Redis/Redlock with in-memory cache for token management
- Remove @vercel/kv, ioredis, and redlock dependencies from package.json
- Implement simple lock mechanism for concurrent token refresh prevention
- Use Map-based cache with TTL for token storage
- Maintain same authentication flow without external dependencies

This simplifies the infrastructure requirements and removes the need for
Redis while maintaining the same functionality through in-memory caching.

* fix: add staleTime to prevent cross-tab staled data

* fix: remove infinite re-render loop in useSessionAccessToken

The hook was maintaining redundant local state that caused re-renders
on every update, which triggered NextAuth to continuously refetch the
session, resulting in hundreds of POST requests to /api/auth/session.

Simplified the hook to directly return session values without
unnecessary state duplication.

* fix: handle undefined access tokens in auth.ts

Added fallback to empty string for potentially undefined access_token
and refresh_token from NextAuth account object to satisfy
JWTWithAccessToken type requirements.

* Igor/mathieu/frontend openapi react query (#597)

* small typing

* typing fixes

---------

Co-authored-by: Igor Loskutov <igor.loskutoff@gmail.com>

* self-review-fix

* authReady callback simplify

* fix auth

* fix compose

* room detail page fix

* compile fix

* room edit fix

* normalize auth provider

* room edition state granular management

* cover TODOs + cross-tab cache

* session auto refresh blink

* schema generator error type doc

* protect from zombie auth

* clarify access token refresh logic a bit

* remove react-query tab sharing cache

* remove react-query tab sharing cache

* websocket dupe react devmode protection

* invalidate room on room update

* redis cache

* test ts server

* ci randomness

* less edgy config (ci)

* less edgy config (ci)

* less edgy config (ci)

* ci randomness

* ci randomness

* ci randomness

* ci randomness

* less edgy config (ci)

* added vs edited room state cleanup

* file upload real-time state management fix

* prettier auth state ternary

* prettier auth state ternary

* proper api address from env

* INTERVAL_REFRESH_MS

* node version 20 for tests

* github debug

* github debug

* github debug

* github debug

* github debug

* github debug

* github debug

* github debug

* github debug

* github debug

* github debug

* CI debug

* CI debug

* nextjs magic

* nextjs magic

* doc

* client-side stale auth soft safety net

---------

Co-authored-by: Mathieu Virbel <mat@meltingrocks.com>
Co-authored-by: Igor Loskutov <igor.loskutoff@gmail.com>
2025-09-05 16:01:31 -06:00
0663700a61 fix: align whisper transcriber api with parakeet (#602)
* Documents transcriber api

* Update whisper transcriber api to match parakeet

* Update api transcription spec

* Return 400 for unsupported file type

* Add params to api spec

* Update whisper transcriber implementation to match parakeet
2025-09-05 10:52:14 +02:00
293f7d4f1f feat: implement frontend video platform configuration and abstraction
- Add NEXT_PUBLIC_VIDEO_PLATFORM environment variable support
- Create video platform abstraction layer with factory pattern
- Implement Whereby and Jitsi platform providers
- Update room meeting page to use platform-agnostic component
- Add platform display in room management (cards and table views)
- Support single platform per deployment configuration
- Maintain backward compatibility with existing Whereby integration

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-04 12:21:51 -06:00
41224a424c docs: move platform-jitsi.md to docs/ directory 2025-09-02 18:28:50 -06:00
dd0089906f fix: replace datetime.utcnow() with datetime.now(tz=timezone.utc) in Jitsi health check 2025-09-02 18:25:55 -06:00
fa559b1970 feat: update and expand video platform tests
- Update existing tests for StrEnum instead of string literals
- Add comprehensive WherebyClient tests with HTTP mocking
- Add webhook event storage tests for participant and recording events
- Add typing overload tests for create_platform_client factory
- Update webhook test paths to new video_platforms router locations
- Fix mock ordering and parameter issues in async tests
- Test all platform client functionality including signature verification
- Verify webhook event storage with proper timestamp handling

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-02 18:16:41 -06:00
c26ce65083 feat: update Jitsi documentation with webhook events storage system
- Add comprehensive webhook event storage documentation
- Document event structure and JSON storage in meetings table
- Add practical webhook testing examples with proper signature generation
- Include detailed troubleshooting for webhook signature verification issues
- Add webhook event payload examples for all supported event types
- Document event storage verification and database querying methods
- Enhance existing webhook configuration with real-world examples

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-02 18:09:44 -06:00
52eff2acc0 feat: clean up legacy code and remove excessive documentation
- Remove excessive inline comments from meeting creation flow
- Remove verbose docstrings from simple property methods and basic functions
- Clean up obvious comments like 'Generate JWT tokens', 'Build room URLs'
- Remove unnecessary explanatory comments in platform clients
- Keep only essential documentation for complex logic
- Simplify race condition handling comments
- Remove excessive method documentation for simple operations

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-02 18:08:12 -06:00
7875ec3432 feat: move platform routers to video_platforms folders
- Move Jitsi router from views/jitsi.py to video_platforms/jitsi/router.py
- Move Whereby router from views/whereby.py to video_platforms/whereby/router.py
- Update __init__.py files to export routers from platform packages
- Update app.py imports to use video_platforms instead of views
- Remove old view files after successful migration
- Maintain exact same API endpoint paths (/v1/jitsi, /v1/whereby)

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-02 18:05:04 -06:00
398be06fad feat: add typing overloads and clean up platform client factory
- Add typing overloads to get_platform_client for JitsiClient and WherebyClient return types
- Add overloads to create_platform_client in factory for better IDE support
- Remove PyJWT fallback imports from views/rooms.py
- Remove platform defaults from CreateRoom and UpdateRoom models
- Clean up legacy whereby fallback code in meeting creation
- Use direct platform client access instead of conditional fallbacks

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-02 18:02:43 -06:00
da700069d9 Add webhook events storage to meetings model
- Add events column as JSON type to meetings table with default empty array
- Add events: List[Dict[str, Any]] field to Meeting model
- Create migration 2890b5104577 for events column and apply successfully
- Add MeetingController helper methods for event storage:
  - add_event() for generic event storage with timestamps
  - participant_joined(), participant_left() for participant tracking
  - recording_started(), recording_stopped() for recording events
  - get_events() for event retrieval
- Update Jitsi webhook endpoints to store events:
  - Store participant join/leave events with data and timestamps
  - Store recording start/stop events from Prosody webhooks
  - Store recording completion events from Jibri finalize script
- Events stored with type, timestamp, and data for webhook history tracking
- Fix linting and formatting issues

Addresses PR feedback point 12: save webhook events in meetings events field
2025-09-02 17:53:35 -06:00
51229a1790 Fix Jitsi client issues and create typed meeting data
- Remove 'transcription': True from JWT features in _generate_jwt
- Replace int(time.time()) with generate_uuid4() for room naming to avoid conflicts
- Replace datetime.utcnow() with datetime.now(tz=timezone.utc) for proper timezone handling
- Create JitsiMeetingData(MeetingData) class with typed extra_data properties
- Update PLATFORM_NAME = VideoPlatform.JITSI to use enum
- Update create_meeting to return JitsiMeetingData instance with proper typing
- Fix get_room_sessions mock to use timezone-aware datetime
- Export JitsiMeetingData from jitsi module

Addresses PR feedback points 4, 5, 6, 10: remove transcription features, use UUID,
proper datetime handling, and typed meeting data
2025-09-02 17:44:04 -06:00
2d2c23f7cc Create video_platforms/whereby structure and WherebyClient
- Create video_platforms/whereby/ directory with __init__.py, client.py, tasks.py
- Implement WherebyClient inheriting from VideoPlatformClient interface
- Move all functions from whereby.py into WherebyClient methods
- Use VideoPlatform.WHEREBY enum for PLATFORM_NAME
- Register WherebyClient in platform registry
- Update factory.py to include S3 bucket config for whereby
- Update worker process to use platform abstraction for get_room_sessions
- Preserve exact API behavior for meeting activity detection
- Maintain AWS S3 configuration handling in WherebyClient
- Fix linting and formatting issues

Addresses PR feedback point 7: implement video_platforms/whereby structure
Note: whereby.py kept for legacy fallback until task 7 cleanup
2025-09-02 17:40:32 -06:00
0acb9cac79 Replace Literal with VideoPlatform StrEnum for platform field
- Create VideoPlatform StrEnum with WHEREBY and JITSI values
- Update rooms.py and meetings.py to use VideoPlatform enum
- Update views/rooms.py and video_platforms/factory.py to use enum values
- Generate new migration with proper server_default='whereby'
- Apply migration successfully with backward compatibility
- Fix linting and formatting issues

Addresses PR feedback point 1: use StrEnum instead of Literal[]
2025-09-02 17:36:14 -06:00
d861d92cc2 docs: add comprehensive Jitsi Meet integration user guide
- Complete end-user configuration guide for self-hosted Jitsi Meet
- Covers installation, JWT authentication, and Prosody configuration
- Webhook event handling with mod_event_sync setup
- Jibri recording service configuration and finalize script
- Room creation, JWT token management, and security best practices
- Comprehensive troubleshooting with debug commands and solutions
- Performance optimization and scaling considerations
- Migration guidance from Whereby platform

🤖 Generated with Claude Code
2025-09-02 17:07:09 -06:00
24ff83a2ec docs: add comprehensive Whereby integration user guide
- Complete end-user configuration guide for Whereby video platform
- Covers account setup, API key generation, and webhook configuration
- AWS S3 storage setup with IAM permissions and security best practices
- Room creation, recording options, and meeting feature configuration
- Troubleshooting guide with common issues and debug commands
- Security considerations and performance optimization tips
- Migration guidance from other platforms

🤖 Generated with Claude Code
2025-09-02 17:05:40 -06:00
249234238c feat: add comprehensive video platform test suite
- Created complete test coverage for video platform abstraction
- Tests for base classes, JitsiClient implementation, and platform registry
- JWT generation tests with proper mocking and error scenarios
- Webhook signature verification tests (valid/invalid/missing secret)
- Platform factory tests for Jitsi and Whereby configuration
- Registry tests for platform registration and client creation
- Webhook endpoint tests with signature verification and error cases
- Integration tests for rooms endpoint with platform abstraction
- 24 comprehensive test cases covering all video platform functionality
- All tests passing with proper mocking and isolation

🤖 Generated with Claude Code
2025-09-02 16:54:58 -06:00
42a603d5c3 feat: add PyJWT dependency and finalize Jitsi integration
- Added PyJWT>=2.8.0 to pyproject.toml dependencies
- Installed dependency via uv sync successfully
- Verified JWT generation functionality works correctly
- Confirmed platform factory creates JitsiClient instances
- Validated database migrations applied (platform fields available)
- Tested webhook endpoints are registered and functional
- Verified FastAPI app starts without errors with full integration
- All integration tests pass - Jitsi platform fully functional

🤖 Generated with Claude Code
2025-09-02 16:28:44 -06:00
6d2092f950 feat: create comprehensive Jitsi integration documentation
- Added complete end-user configuration guide at server/platform-jitsi.md
- Covers prerequisites, environment setup, and Jitsi Meet configuration
- Includes JWT authentication, Jibri recording, and Prosody event-sync setup
- Provides troubleshooting guide with common issues and solutions
- Documents security best practices and performance optimization
- Includes testing procedures and migration guidance from Whereby
- Ready for production deployment with step-by-step instructions
- Uses environment variable placeholders for security

🤖 Generated with Claude Code
2025-09-02 16:24:47 -06:00
f2bb6aaecb feat: update rooms.py to use video platform abstraction
- Added platform field to Room, CreateRoom, and UpdateRoom models
- Updated rooms_create function to pass platform parameter
- Rewrote rooms_create_meeting to use platform factory pattern
- Added graceful fallback to legacy whereby implementation
- Maintained API compatibility and error handling patterns
- Prepared for multi-platform support (Whereby/Jitsi)

🤖 Generated with Claude Code
2025-09-02 16:21:58 -06:00
2b136ac7b0 feat: create Jitsi webhook endpoints for event handling
- Added comprehensive Jitsi webhook endpoint in views/jitsi.py
- Handles Prosody event-sync events (muc-occupant-joined/left)
- Implements participant counting following whereby.py pattern
- Added Jibri recording completion webhook endpoint
- Includes signature verification with fallback when platform client unavailable
- Registered router in app.py for /v1/jitsi endpoints
- Added health check endpoint for webhook configuration

🤖 Generated with Claude Code
2025-09-02 16:19:54 -06:00
3f4fc26483 feat: register Jitsi platform in video platforms factory and registry
- Added JitsiClient registration to platform registry
- Enables dynamic platform selection through factory pattern
- Factory configuration already supports Jitsi settings
- Platform abstraction layer now supports beide Whereby and Jitsi

🤖 Generated with Claude Code
2025-09-02 16:17:32 -06:00
8e5ef5bca6 feat: implement JitsiClient with JWT authentication
Complete implementation of JitsiClient following VideoPlatformClient interface
with JWT-based room access control and webhook signature verification.

- Add JWT token generation with proper payload structure
- Implement unique room name generation with timestamp
- Create separate user/host JWT tokens with moderator permissions
- Build secure room URLs with embedded JWT parameters
- Add HMAC-SHA256 webhook signature verification for Prosody events
- Implement all abstract methods with Jitsi-specific behavior
- Include comprehensive typing and error handling

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-02 16:15:49 -06:00
d49fdcb38d feat: create video platforms architecture with Jitsi directory structure
Create complete video platforms abstraction layer following daily.co branch
pattern with Jitsi-specific directory structure.

- Add video_platforms base module with abstract classes
- Create VideoPlatformClient, MeetingData, VideoPlatformConfig interfaces
- Add platform registry system for client management
- Create factory pattern for platform client creation
- Add Jitsi directory structure with __init__.py, tasks.py, client.py
- Configure Jitsi platform in factory with JWT-based authentication

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-02 16:14:42 -06:00
d42380abf1 feat: add Jitsi configuration settings
Add comprehensive Jitsi Meet configuration settings to settings.py
following the same pattern as WHEREBY settings.

- Add JITSI_DOMAIN with meet.jit.si default
- Add JITSI_JWT_SECRET for JWT token signing
- Add JITSI_WEBHOOK_SECRET for webhook validation
- Add JITSI_APP_ID, JITSI_JWT_ISSUER, JITSI_JWT_AUDIENCE for JWT configuration
- Follow consistent naming and typing patterns

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-02 16:11:00 -06:00
cf64e1a3d9 feat: add database migration for platform field
Generate Alembic migration to add platform column to rooms and meetings
tables enabling multi-platform video conferencing support.

- Add platform column to meeting table with whereby default
- Add platform column to room table with whereby default
- Migration tested successfully with alembic upgrade head

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-02 16:10:11 -06:00
ea53ca7000 feat: add platform field to Room and Meeting models
Add platform column to rooms and meetings database tables with Literal typing
to support multiple video conferencing platforms (whereby, jitsi).

- Add platform column to rooms/meetings SQLAlchemy tables with whereby default
- Update Room/Meeting Pydantic models with platform field and Literal typing
- Modify RoomController.add() to accept platform parameter
- Update MeetingController.create() to copy platform from room

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-02 16:08:38 -06:00
123 changed files with 10970 additions and 3261 deletions

View File

@@ -1,5 +1,37 @@
# Changelog
## [0.10.0](https://github.com/Monadical-SAS/reflector/compare/v0.9.0...v0.10.0) (2025-09-11)
### Features
* replace nextjs-config with environment variables ([#632](https://github.com/Monadical-SAS/reflector/issues/632)) ([369ecdf](https://github.com/Monadical-SAS/reflector/commit/369ecdff13f3862d926a9c0b87df52c9d94c4dde))
### Bug Fixes
* anonymous users transcript permissions ([#621](https://github.com/Monadical-SAS/reflector/issues/621)) ([f81fe99](https://github.com/Monadical-SAS/reflector/commit/f81fe9948a9237b3e0001b2d8ca84f54d76878f9))
* auth post ([#624](https://github.com/Monadical-SAS/reflector/issues/624)) ([cde99ca](https://github.com/Monadical-SAS/reflector/commit/cde99ca2716f84ba26798f289047732f0448742e))
* auth post ([#626](https://github.com/Monadical-SAS/reflector/issues/626)) ([3b85ff3](https://github.com/Monadical-SAS/reflector/commit/3b85ff3bdf4fb053b103070646811bc990c0e70a))
* auth post ([#627](https://github.com/Monadical-SAS/reflector/issues/627)) ([962038e](https://github.com/Monadical-SAS/reflector/commit/962038ee3f2a555dc3c03856be0e4409456e0996))
* missing follow_redirects=True on modal endpoint ([#630](https://github.com/Monadical-SAS/reflector/issues/630)) ([fc363bd](https://github.com/Monadical-SAS/reflector/commit/fc363bd49b17b075e64f9186e5e0185abc325ea7))
* sync backend and frontend token refresh logic ([#614](https://github.com/Monadical-SAS/reflector/issues/614)) ([5a5b323](https://github.com/Monadical-SAS/reflector/commit/5a5b3233820df9536da75e87ce6184a983d4713a))
## [0.9.0](https://github.com/Monadical-SAS/reflector/compare/v0.8.2...v0.9.0) (2025-09-06)
### Features
* frontend openapi react query ([#606](https://github.com/Monadical-SAS/reflector/issues/606)) ([c4d2825](https://github.com/Monadical-SAS/reflector/commit/c4d2825c81f81ad8835629fbf6ea8c7383f8c31b))
### Bug Fixes
* align whisper transcriber api with parakeet ([#602](https://github.com/Monadical-SAS/reflector/issues/602)) ([0663700](https://github.com/Monadical-SAS/reflector/commit/0663700a615a4af69a03c96c410f049e23ec9443))
* kv use tls explicit ([#610](https://github.com/Monadical-SAS/reflector/issues/610)) ([08d88ec](https://github.com/Monadical-SAS/reflector/commit/08d88ec349f38b0d13e0fa4cb73486c8dfd31836))
* source kind for file processing ([#601](https://github.com/Monadical-SAS/reflector/issues/601)) ([dc82f8b](https://github.com/Monadical-SAS/reflector/commit/dc82f8bb3bdf3ab3d4088e592a30fd63907319e1))
* token refresh locking ([#613](https://github.com/Monadical-SAS/reflector/issues/613)) ([7f5a4c9](https://github.com/Monadical-SAS/reflector/commit/7f5a4c9ddc7fd098860c8bdda2ca3b57f63ded2f))
## [0.8.2](https://github.com/Monadical-SAS/reflector/compare/v0.8.1...v0.8.2) (2025-08-29)

View File

@@ -66,7 +66,6 @@ pnpm install
# Copy configuration templates
cp .env_template .env
cp config-template.ts config.ts
```
**Development:**

View File

@@ -99,11 +99,10 @@ Start with `cd www`.
```bash
pnpm install
cp .env_template .env
cp config-template.ts config.ts
cp .env.example .env
```
Then, fill in the environment variables in `.env` and the configuration in `config.ts` as needed. If you are unsure on how to proceed, ask in Zulip.
Then, fill in the environment variables in `.env` as needed. If you are unsure on how to proceed, ask in Zulip.
**Run in development mode**
@@ -168,3 +167,34 @@ You can manually process an audio file by calling the process tool:
```bash
uv run python -m reflector.tools.process path/to/audio.wav
```
## Feature Flags
Reflector uses environment variable-based feature flags to control application functionality. These flags allow you to enable or disable features without code changes.
### Available Feature Flags
| Feature Flag | Environment Variable |
|-------------|---------------------|
| `requireLogin` | `NEXT_PUBLIC_FEATURE_REQUIRE_LOGIN` |
| `privacy` | `NEXT_PUBLIC_FEATURE_PRIVACY` |
| `browse` | `NEXT_PUBLIC_FEATURE_BROWSE` |
| `sendToZulip` | `NEXT_PUBLIC_FEATURE_SEND_TO_ZULIP` |
| `rooms` | `NEXT_PUBLIC_FEATURE_ROOMS` |
### Setting Feature Flags
Feature flags are controlled via environment variables using the pattern `NEXT_PUBLIC_FEATURE_{FEATURE_NAME}` where `{FEATURE_NAME}` is the SCREAMING_SNAKE_CASE version of the feature name.
**Examples:**
```bash
# Enable user authentication requirement
NEXT_PUBLIC_FEATURE_REQUIRE_LOGIN=true
# Disable browse functionality
NEXT_PUBLIC_FEATURE_BROWSE=false
# Enable Zulip integration
NEXT_PUBLIC_FEATURE_SEND_TO_ZULIP=true
```

369
docs/jitsi.md Normal file
View File

@@ -0,0 +1,369 @@
# Jitsi Integration for Reflector
This document contains research and planning notes for integrating Jitsi Meet as a replacement for Whereby in Reflector.
## Overview
Jitsi Meet is an open-source video conferencing solution that can replace Whereby in Reflector, providing:
- Cost reduction (no per-minute charges)
- Direct recording access via Jibri
- Real-time event webhooks
- Full customization and control
## Current Whereby Integration Analysis
### Architecture
1. **Room Creation**: User creates a "room" template in Reflector DB with settings
2. **Meeting Creation**: `/rooms/{room_name}/meeting` endpoint calls Whereby API to create meeting
3. **Recording**: Whereby handles recording automatically to S3 bucket
4. **Webhooks**: Whereby sends events for participant tracking
### Database Structure
```python
# Room = Template/Configuration
class Room:
id, name, user_id
recording_type, recording_trigger # cloud, automatic-2nd-participant
webhook_url, webhook_secret
# Meeting = Actual Whereby Meeting Instance
class Meeting:
id # Whereby meetingId
room_name # Generated by Whereby
room_url, host_room_url # Whereby URLs
num_clients # Updated via webhooks
```
## Jitsi Components
### Core Architecture
- **Jitsi Meet**: Web frontend (Next.js + React)
- **Prosody**: XMPP server for messaging/rooms
- **Jicofo**: Conference focus (orchestration)
- **JVB**: Videobridge (media routing)
- **Jibri**: Recording service
- **Jigasi**: SIP gateway (optional, for phone dial-in)
### Exposure Requirements
- **Web service**: 443/80 (frontend)
- **JVB**: 10000/UDP (media streams) - **MUST EXPOSE**
- **Prosody**: 5280 (BOSH/WebSocket) - can proxy via web
- **Jicofo, Jibri, Jigasi**: Internal only
## Recording with Jibri
### How Jibri Works
- Each Jibri instance handles **one recording at a time**
- Records mixed audio/video to MP4 format
- Uses Chrome headless + ffmpeg for capture
- Supports finalize scripts for post-processing
### Jibri Pool for Scaling
- Multiple Jibri instances join "jibribrewery" MUC
- Jicofo distributes recording requests to available instances
- Automatic load balancing and failover
```yaml
# Multiple Jibri instances
jibri1:
environment:
- JIBRI_INSTANCE_ID=jibri1
- JIBRI_BREWERY_MUC=jibribrewery
jibri2:
environment:
- JIBRI_INSTANCE_ID=jibri2
- JIBRI_BREWERY_MUC=jibribrewery
```
### Recording Automation Options
1. **Environment Variables**: `ENABLE_RECORDING=1`, `AUTO_RECORDING=1`
2. **URL Parameters**: `?config.autoRecord=true`
3. **JWT Token**: Include recording permissions in JWT
4. **API Control**: `api.executeCommand('startRecording')`
### Post-Processing Integration
```bash
#!/bin/bash
# finalize.sh - runs after recording completion
RECORDING_FILE=$1
MEETING_METADATA=$2
ROOM_NAME=$3
# Copy to Reflector-accessible location
cp "$RECORDING_FILE" /shared/reflector-uploads/
# Trigger Reflector processing
curl -X POST "http://reflector-api:8000/v1/transcripts/process" \
-H "Content-Type: application/json" \
-d "{
\"file_path\": \"/shared/reflector-uploads/$(basename $RECORDING_FILE)\",
\"room_name\": \"$ROOM_NAME\",
\"source\": \"jitsi\"
}"
```
## React Integration
### Official React SDK
```bash
npm i @jitsi/react-sdk
```
```jsx
import { JitsiMeeting } from '@jitsi/react-sdk'
<JitsiMeeting
room="meeting-room"
serverURL="https://your-jitsi.domain"
jwt="your-jwt-token"
config={{
startWithAudioMuted: true,
fileRecordingsEnabled: true,
autoRecord: true
}}
onParticipantJoined={(participant) => {
// Track participant events
}}
onRecordingStatusChanged={(status) => {
// Handle recording events
}}
/>
```
## Authentication & Room Control
### JWT-Based Access Control
```python
def generate_jitsi_jwt(payload):
return jwt.encode({
"aud": "jitsi",
"iss": "reflector",
"sub": "reflector-user",
"room": payload["room"],
"exp": int(payload["exp"].timestamp()),
"context": {
"user": {
"name": payload["user_name"],
"moderator": payload.get("moderator", False)
},
"features": {
"recording": payload.get("recording", True)
}
}
}, JITSI_JWT_SECRET)
```
### Prevent Anonymous Room Creation
```bash
# Environment configuration
ENABLE_AUTH=1
ENABLE_GUESTS=0
AUTH_TYPE=jwt
JWT_APP_ID=reflector
JWT_APP_SECRET=your-secret-key
```
## Webhook Integration
### Real-time Events via Prosody
Custom event-sync module can send webhooks for:
- Participant join/leave
- Recording start/stop
- Room creation/destruction
- Mute/unmute events
```lua
-- mod_event_sync.lua
module:hook("muc-occupant-joined", function(event)
send_event({
type = "participant_joined",
room = event.room.jid,
participant = {
nick = event.occupant.nick,
jid = event.occupant.jid,
},
timestamp = os.time(),
});
end);
```
### Jibri Recording Webhooks
```bash
# Environment variable
JIBRI_WEBHOOK_SUBSCRIBERS=https://your-reflector.com/webhooks/jibri
```
## Proposed Reflector Integration
### Modified Database Schema
```python
class Meeting(BaseModel):
id: str # Our generated meeting ID
room_name: str # Generated: reflector-{room.name}-{timestamp}
room_url: str # https://jitsi.domain/room_name?jwt=token
host_room_url: str # Same but with moderator JWT
# Add Jitsi-specific fields
jitsi_jwt: str # JWT token
jitsi_room_id: str # Internal room identifier
recording_status: str # pending, recording, completed
recording_file_path: Optional[str]
```
### API Replacement
```python
# Replace whereby.py with jitsi.py
async def create_meeting(room_name_prefix: str, end_date: datetime, room: Room):
# Generate unique room name
jitsi_room = f"reflector-{room.name}-{int(time.time())}"
# Generate JWT tokens
user_jwt = generate_jwt(room=jitsi_room, moderator=False, exp=end_date)
host_jwt = generate_jwt(room=jitsi_room, moderator=True, exp=end_date)
return {
"meetingId": generate_uuid4(), # Our ID
"roomName": jitsi_room,
"roomUrl": f"https://jitsi.domain/{jitsi_room}?jwt={user_jwt}",
"hostRoomUrl": f"https://jitsi.domain/{jitsi_room}?jwt={host_jwt}",
"startDate": datetime.now().isoformat(),
"endDate": end_date.isoformat(),
}
```
### Webhook Endpoints
```python
# Replace whereby webhook with jitsi webhooks
@router.post("/jitsi/events")
async def jitsi_events_webhook(event_data: dict):
event_type = event_data.get("event")
room_name = event_data.get("room", "").split("@")[0]
meeting = await Meeting.get_by_room(room_name)
if event_type == "muc-occupant-joined":
# Update participant count
meeting.num_clients += 1
elif event_type == "jibri-recording-on":
meeting.recording_status = "recording"
elif event_type == "jibri-recording-off":
meeting.recording_status = "processing"
await process_meeting_recording.delay(meeting.id)
@router.post("/jibri/recording-complete")
async def recording_complete(data: dict):
# Handle finalize script webhook
room_name = data.get("room_name")
file_path = data.get("file_path")
meeting = await Meeting.get_by_room(room_name)
meeting.recording_file_path = file_path
meeting.recording_status = "completed"
# Start Reflector processing
await process_recording_for_transcription(meeting.id, file_path)
```
## Deployment with Docker
### Official docker-jitsi-meet
```bash
# Download official release
wget $(wget -q -O - https://api.github.com/repos/jitsi/docker-jitsi-meet/releases/latest | grep zip | cut -d\" -f4)
# Setup
mkdir -p ~/.jitsi-meet-cfg/{web,transcripts,prosody/config,prosody/prosody-plugins-custom,jicofo,jvb,jigasi,jibri}
./gen-passwords.sh # Generate secure passwords
docker compose up -d
```
### Coolify Integration
```yaml
services:
web:
ports: ["80:80", "443:443"]
jvb:
ports: ["10000:10000/udp"] # Must expose for media
jibri1:
environment:
- JIBRI_INSTANCE_ID=jibri1
- JIBRI_FINALIZE_RECORDING_SCRIPT_PATH=/config/finalize.sh
jibri2:
environment:
- JIBRI_INSTANCE_ID=jibri2
```
## Benefits vs Whereby
### Cost & Control
**No per-minute charges** - significant cost savings
**Full recording control** - direct file access
**Custom branding** - complete UI control
**Self-hosted** - no vendor lock-in
### Technical Advantages
**Real-time events** - immediate webhook notifications
**Rich participant metadata** - detailed tracking
**JWT security** - token-based access with expiration
**Multiple recording formats** - audio-only options
**Scalable architecture** - horizontal Jibri scaling
### Integration Benefits
**Same API surface** - minimal changes to existing code
**React SDK** - better frontend integration
**Direct processing** - no S3 download delays
**Event-driven architecture** - better real-time capabilities
## Implementation Plan
1. **Deploy Jitsi Stack** - Set up docker-jitsi-meet with multiple Jibri instances
2. **Create jitsi.py** - Replace whereby.py with Jitsi API functions
3. **Update Database** - Add Jitsi-specific fields to Meeting model
4. **Webhook Integration** - Replace Whereby webhooks with Jitsi events
5. **Frontend Updates** - Replace Whereby embed with Jitsi React SDK
6. **Testing & Migration** - Gradual rollout with fallback to Whereby
## Recording Limitations & Considerations
### Current Limitations
- **Mixed audio only** - Jibri doesn't separate participant tracks natively
- **One recording per Jibri** - requires multiple instances for concurrent recordings
- **Chrome dependency** - Jibri uses headless Chrome for recording
### Metadata Capabilities
**Participant join/leave timestamps** - via webhooks
**Speaking time tracking** - via audio level events
**Meeting duration** - precise timing
**Room-specific data** - custom metadata in JWT
### Alternative Recording Methods
- **Local recording** - browser-based, per-participant
- **Custom recording** - lib-jitsi-meet for individual streams
- **Third-party solutions** - Recall.ai, Otter.ai integrations
## Security Considerations
### JWT Configuration
- **Room-specific tokens** - limit access to specific rooms
- **Time-based expiration** - automatic cleanup
- **Feature permissions** - control recording, moderation rights
- **User identification** - embed user metadata in tokens
### Access Control
- **No anonymous rooms** - all rooms require valid JWT
- **API-only creation** - prevent direct room access
- **Webhook verification** - HMAC signature validation
## Next Steps
1. **Deploy test Jitsi instance** - validate recording pipeline
2. **Prototype jitsi.py** - create equivalent API functions
3. **Test webhook integration** - ensure event delivery works
4. **Performance testing** - validate multiple concurrent recordings
5. **Migration strategy** - plan gradual transition from Whereby
---
*This document serves as the comprehensive planning and research notes for Jitsi integration in Reflector. It should be updated as implementation progresses and new insights are discovered.*

720
docs/video-jitsi.md Normal file
View File

@@ -0,0 +1,720 @@
# Jitsi Meet Integration Configuration Guide
This guide explains how to configure Reflector to use your self-hosted Jitsi Meet installation for video meetings, recording, and participant tracking.
## Overview
Jitsi Meet is an open-source video conferencing platform that can be self-hosted. Reflector integrates with Jitsi Meet to:
- Create secure meeting rooms with JWT authentication
- Track participant join/leave events via Prosody webhooks
- Record meetings using Jibri recording service
- Process recordings for transcription and analysis
## Requirements
### Self-Hosted Jitsi Meet
You need a complete Jitsi Meet installation including:
1. **Jitsi Meet Web Interface** - The main meeting interface
2. **Prosody XMPP Server** - Handles room management and authentication
3. **Jicofo (JItsi COnference FOcus)** - Manages media sessions
4. **Jitsi Videobridge (JVB)** - Handles WebRTC media routing
5. **Jibri Recording Service** - Records meetings (optional but recommended)
### System Requirements
- **Domain with SSL Certificate** - Required for WebRTC functionality
- **Prosody mod_event_sync** - For webhook event handling
- **JWT Authentication** - For secure room access control
- **Storage Solution** - For recording files (local or cloud)
## Configuration Variables
Add the following environment variables to your Reflector `.env` file:
### Required Variables
```bash
# Jitsi Meet Domain (without https://)
JITSI_DOMAIN=meet.example.com
# JWT Secret for room authentication (generate with: openssl rand -hex 32)
JITSI_JWT_SECRET=your-64-character-hex-secret-here
# Webhook secret for event handling (generate with: openssl rand -hex 16)
JITSI_WEBHOOK_SECRET=your-32-character-hex-secret-here
```
### Optional Variables
```bash
# Application identifier (should match Jitsi configuration)
JITSI_APP_ID=reflector
# JWT issuer and audience (should match Jitsi configuration)
JITSI_JWT_ISSUER=reflector
JITSI_JWT_AUDIENCE=jitsi
```
## Installation Steps
### 1. Jitsi Meet Server Installation
#### Quick Installation (Ubuntu/Debian)
```bash
# Add Jitsi repository
curl -fsSL https://download.jitsi.org/jitsi-key.gpg.key | sudo gpg --dearmor -o /usr/share/keyrings/jitsi-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/jitsi-keyring.gpg] https://download.jitsi.org stable/" | sudo tee /etc/apt/sources.list.d/jitsi-stable.list
# Install Jitsi Meet
sudo apt update
sudo apt install jitsi-meet
# Configure SSL certificate
sudo /usr/share/jitsi-meet/scripts/install-letsencrypt-cert.sh
```
#### Docker Installation
```bash
# Clone Jitsi Docker repository
git clone https://github.com/jitsi/docker-jitsi-meet
cd docker-jitsi-meet
# Copy environment template
cp env.example .env
# Edit configuration
nano .env
# Start services
docker-compose up -d
```
### 2. JWT Authentication Setup
#### Update Prosody Configuration
Edit `/etc/prosody/conf.d/your-domain.cfg.lua`:
```lua
VirtualHost "meet.example.com"
authentication = "token"
app_id = "reflector"
app_secret = "your-jwt-secret-here"
-- Allow anonymous access for guests
c2s_require_encryption = false
admins = { "focusUser@auth.meet.example.com" }
modules_enabled = {
"bosh";
"pubsub";
"ping";
"roster";
"saslauth";
"tls";
"dialback";
"disco";
"carbons";
"pep";
"private";
"blocklist";
"vcard";
"version";
"uptime";
"time";
"ping";
"register";
"admin_adhoc";
"token_verification";
"event_sync"; -- Required for webhooks
}
```
#### Configure Jitsi Meet Interface
Edit `/etc/jitsi/meet/your-domain-config.js`:
```javascript
var config = {
hosts: {
domain: 'meet.example.com',
muc: 'conference.meet.example.com'
},
// Enable JWT authentication
enableUserRolesBasedOnToken: true,
// Recording configuration
fileRecordingsEnabled: true,
liveStreamingEnabled: false,
// Reflector integration settings
prejoinPageEnabled: true,
requireDisplayName: true
};
```
### 3. Webhook Event Configuration
#### Install Event Sync Module
```bash
# Download the module
cd /usr/share/jitsi-meet/prosody-plugins/
wget https://raw.githubusercontent.com/jitsi-contrib/prosody-plugins/main/mod_event_sync.lua
```
#### Configure Event Sync
Add to your Prosody configuration:
```lua
Component "conference.meet.example.com" "muc"
storage = "memory"
modules_enabled = {
"muc_meeting_id";
"muc_domain_mapper";
"polls";
"event_sync"; -- Enable event sync
}
-- Event sync webhook configuration
event_sync_url = "https://your-reflector-domain.com/v1/jitsi/events"
event_sync_secret = "your-webhook-secret-here"
-- Events to track
event_sync_events = {
"muc-occupant-joined",
"muc-occupant-left",
"jibri-recording-on",
"jibri-recording-off"
}
#### Webhook Event Payload Examples
**Participant Joined Event:**
```json
{
"event": "muc-occupant-joined",
"room": "reflector-my-room-uuid123",
"timestamp": "2025-01-15T10:30:00.000Z",
"data": {
"occupant_id": "participant-456",
"nick": "John Doe",
"role": "participant",
"affiliation": "none"
}
}
```
**Recording Started Event:**
```json
{
"event": "jibri-recording-on",
"room": "reflector-my-room-uuid123",
"timestamp": "2025-01-15T10:32:00.000Z",
"data": {
"recording_id": "rec-789",
"initiator": "moderator-123"
}
}
```
**Recording Completed Event:**
```json
{
"room_name": "reflector-my-room-uuid123",
"recording_file": "/var/recordings/rec-789.mp4",
"recording_status": "completed",
"timestamp": "2025-01-15T11:15:00.000Z"
}
```
### 4. Jibri Recording Setup (Optional)
#### Install Jibri
```bash
# Install Jibri package
sudo apt install jibri
# Create recording directory
sudo mkdir -p /var/recordings
sudo chown jibri:jibri /var/recordings
```
#### Configure Jibri
Edit `/etc/jitsi/jibri/jibri.conf`:
```hocon
jibri {
recording {
recordings-directory = "/var/recordings"
finalize-script = "/opt/jitsi/jibri/finalize.sh"
}
api {
xmpp {
environments = [{
name = "prod environment"
xmpp-server-hosts = ["meet.example.com"]
xmpp-domain = "meet.example.com"
control-muc {
domain = "internal.auth.meet.example.com"
room-name = "JibriBrewery"
nickname = "jibri-nickname"
}
control-login {
domain = "auth.meet.example.com"
username = "jibri"
password = "jibri-password"
}
}]
}
}
}
```
#### Create Finalize Script
Create `/opt/jitsi/jibri/finalize.sh`:
```bash
#!/bin/bash
# Jibri finalize script for Reflector integration
RECORDING_FILE="$1"
ROOM_NAME="$2"
REFLECTOR_API_URL="${REFLECTOR_API_URL:-http://localhost:1250}"
# Prepare webhook payload
TIMESTAMP=$(date -u +%Y-%m-%dT%H:%M:%S.%3NZ)
PAYLOAD=$(cat <<EOF
{
"room_name": "$ROOM_NAME",
"recording_file": "$RECORDING_FILE",
"recording_status": "completed",
"timestamp": "$TIMESTAMP"
}
EOF
)
# Generate signature
SIGNATURE=$(echo -n "$PAYLOAD" | openssl dgst -sha256 -hmac "$JITSI_WEBHOOK_SECRET" | cut -d' ' -f2)
# Send webhook to Reflector
curl -X POST "$REFLECTOR_API_URL/v1/jibri/recording-complete" \
-H "Content-Type: application/json" \
-H "X-Jitsi-Signature: $SIGNATURE" \
-d "$PAYLOAD"
echo "Recording finalization webhook sent for room: $ROOM_NAME"
```
Make executable:
```bash
sudo chmod +x /opt/jitsi/jibri/finalize.sh
```
### 5. Restart Services
After configuration changes:
```bash
sudo systemctl restart prosody
sudo systemctl restart jicofo
sudo systemctl restart jitsi-videobridge2
sudo systemctl restart jibri
sudo systemctl restart nginx
```
## Room Configuration
### Creating Jitsi Rooms
Create rooms with Jitsi platform in Reflector:
```bash
curl -X POST "https://your-reflector-domain.com/v1/rooms" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $AUTH_TOKEN" \
-d '{
"name": "my-jitsi-room",
"platform": "jitsi",
"recording_type": "cloud",
"recording_trigger": "automatic-2nd-participant",
"is_locked": false,
"room_mode": "normal"
}'
```
### Meeting Creation
Meetings automatically use JWT authentication:
```bash
curl -X POST "https://your-reflector-domain.com/v1/rooms/my-jitsi-room/meeting" \
-H "Authorization: Bearer $AUTH_TOKEN"
```
Response includes JWT-authenticated URLs:
```json
{
"id": "meeting-uuid",
"room_name": "reflector-my-jitsi-room-123456",
"room_url": "https://meet.example.com/room?jwt=user-token",
"host_room_url": "https://meet.example.com/room?jwt=moderator-token"
}
```
## Features and Capabilities
### JWT Authentication
Reflector automatically generates JWT tokens with:
- **Room Access Control** - Secure room entry
- **User Roles** - Moderator vs participant permissions
- **Expiration** - Configurable token lifetime (default 8 hours)
- **Custom Claims** - Room-specific metadata
### Recording Options
**Recording Types:**
- `"none"` - No recording
- `"local"` - Local Jibri recording
- `"cloud"` - Cloud recording (requires external storage)
**Recording Triggers:**
- `"none"` - Manual recording only
- `"prompt"` - Prompt users to start
- `"automatic"` - Start immediately
- `"automatic-2nd-participant"` - Start when 2nd person joins
### Event Tracking and Storage
Reflector automatically stores all webhook events in the `meetings` table for comprehensive meeting analytics:
**Supported Event Types:**
- `muc-occupant-joined` - Participant joined the meeting
- `muc-occupant-left` - Participant left the meeting
- `jibri-recording-on` - Recording started
- `jibri-recording-off` - Recording stopped
- `recording_completed` - Recording file ready for processing
**Event Storage Structure:**
Each webhook event is stored as a JSON object in the `meetings.events` column:
```json
{
"type": "muc-occupant-joined",
"timestamp": "2025-01-15T10:30:00.123456Z",
"data": {
"timestamp": "2025-01-15T10:30:00Z",
"user_id": "participant-123",
"display_name": "John Doe"
}
}
```
**Querying Stored Events:**
```sql
-- Get all events for a meeting
SELECT events FROM meeting WHERE id = 'meeting-uuid';
-- Count participant joins
SELECT json_array_length(
json_extract(events, '$[*] ? (@.type == "muc-occupant-joined")')
) as total_joins FROM meeting WHERE id = 'meeting-uuid';
```
## Testing and Verification
### Health Check
Test Jitsi webhook integration:
```bash
curl "https://your-reflector-domain.com/v1/jitsi/health"
```
Expected response:
```json
{
"status": "ok",
"service": "jitsi-webhooks",
"timestamp": "2025-01-15T10:30:00.000Z",
"webhook_secret_configured": true
}
```
### JWT Token Testing
Verify JWT generation works:
```bash
# Create a test meeting
MEETING=$(curl -X POST "https://your-reflector-domain.com/v1/rooms/test-room/meeting" \
-H "Authorization: Bearer $AUTH_TOKEN" | jq -r '.room_url')
echo "Test meeting URL: $MEETING"
```
### Webhook Testing
#### Manual Webhook Event Testing
Test participant join event:
```bash
# Generate proper signature
PAYLOAD='{"event":"muc-occupant-joined","room":"reflector-test-room-uuid","timestamp":"2025-01-15T10:30:00.000Z","data":{"user_id":"test-user","display_name":"Test User"}}'
SIGNATURE=$(echo -n "$PAYLOAD" | openssl dgst -sha256 -hmac "$JITSI_WEBHOOK_SECRET" | cut -d' ' -f2)
curl -X POST "https://your-reflector-domain.com/v1/jitsi/events" \
-H "Content-Type: application/json" \
-H "X-Jitsi-Signature: $SIGNATURE" \
-d "$PAYLOAD"
```
Expected response:
```json
{
"status": "ok",
"event": "muc-occupant-joined",
"room": "reflector-test-room-uuid"
}
```
#### Recording Webhook Testing
Test recording completion event:
```bash
PAYLOAD='{"room_name":"reflector-test-room-uuid","recording_file":"/recordings/test.mp4","recording_status":"completed","timestamp":"2025-01-15T10:30:00.000Z"}'
SIGNATURE=$(echo -n "$PAYLOAD" | openssl dgst -sha256 -hmac "$JITSI_WEBHOOK_SECRET" | cut -d' ' -f2)
curl -X POST "https://your-reflector-domain.com/v1/jibri/recording-complete" \
-H "Content-Type: application/json" \
-H "X-Jitsi-Signature: $SIGNATURE" \
-d "$PAYLOAD"
```
#### Event Storage Verification
Verify events were stored:
```bash
# Check meeting events via API (requires authentication)
curl -H "Authorization: Bearer $AUTH_TOKEN" \
"https://your-reflector-domain.com/v1/meetings/{meeting-id}"
```
## Troubleshooting
### Common Issues
#### JWT Authentication Failures
**Symptoms**: Users cannot join rooms, "Authentication failed" errors
**Solutions**:
1. Verify `JITSI_JWT_SECRET` matches Prosody configuration
2. Check JWT token hasn't expired (default 8 hours)
3. Ensure system clocks are synchronized between servers
4. Validate JWT issuer/audience configuration matches
**Debug JWT tokens**:
```bash
# Decode JWT payload
echo "JWT_TOKEN_HERE" | cut -d'.' -f2 | base64 -d | jq
```
#### Webhook Events Not Received
**Symptoms**: Participant counts not updating, no recording events
**Solutions**:
1. Verify `mod_event_sync` is loaded in Prosody
2. Check webhook URL is accessible from Jitsi server
3. Validate webhook signature generation
4. Review Prosody and Reflector logs
**Debug webhook connectivity**:
```bash
# Test from Jitsi server
curl -v "https://your-reflector-domain.com/v1/jitsi/health"
# Check Prosody logs
sudo tail -f /var/log/prosody/prosody.log
```
#### Webhook Signature Verification Issues
**Symptoms**: HTTP 401 "Invalid webhook signature" errors
**Solutions**:
1. Verify webhook secret matches between Jitsi and Reflector
2. Check payload encoding (no extra whitespace)
3. Ensure proper HMAC-SHA256 signature generation
**Debug signature generation**:
```bash
# Test signature manually
PAYLOAD='{"event":"test","room":"test","timestamp":"2025-01-15T10:30:00.000Z","data":{}}'
SECRET="your-webhook-secret-here"
# Generate signature (should match X-Jitsi-Signature header)
echo -n "$PAYLOAD" | openssl dgst -sha256 -hmac "$SECRET" | cut -d' ' -f2
# Test with curl
curl -X POST "https://your-reflector-domain.com/v1/jitsi/events" \
-H "Content-Type: application/json" \
-H "X-Jitsi-Signature: $(echo -n "$PAYLOAD" | openssl dgst -sha256 -hmac "$SECRET" | cut -d' ' -f2)" \
-d "$PAYLOAD" -v
```
#### Event Storage Problems
**Symptoms**: Events received but not stored in database
**Solutions**:
1. Check database connectivity and permissions
2. Verify meeting exists before event processing
3. Review Reflector application logs
4. Ensure JSON column support in database
**Debug event storage**:
```bash
# Check meeting exists
curl -H "Authorization: Bearer $TOKEN" \
"https://your-reflector-domain.com/v1/meetings/{meeting-id}"
# Monitor database queries (if using PostgreSQL)
sudo -u postgres psql -c "SELECT * FROM pg_stat_activity WHERE query LIKE '%meeting%';"
# Check Reflector logs for event processing
sudo journalctl -u reflector -f | grep -E "(event|webhook|jitsi)"
```
#### Recording Issues
**Symptoms**: Recordings not starting, finalize script errors
**Solutions**:
1. Verify Jibri service status: `sudo systemctl status jibri`
2. Check recording directory permissions: `/var/recordings`
3. Validate finalize script execution permissions
4. Monitor Jibri logs: `sudo journalctl -u jibri -f`
**Test finalize script**:
```bash
sudo -u jibri /opt/jitsi/jibri/finalize.sh "/test/recording.mp4" "test-room"
```
#### Meeting Creation Failures
**Symptoms**: HTTP 500 errors when creating meetings
**Solutions**:
1. Check Reflector logs for JWT generation errors
2. Verify all required environment variables are set
3. Ensure Jitsi domain is accessible from Reflector
4. Test JWT secret configuration
### Debug Commands
```bash
# Verify Prosody configuration
sudo prosodyctl check config
# Check Jitsi services status
sudo systemctl status prosody jicofo jitsi-videobridge2
# Test JWT generation
curl -X POST "https://your-reflector-domain.com/v1/rooms/test/meeting" \
-H "Authorization: Bearer $TOKEN" -v
# Monitor webhook events
sudo tail -f /var/log/reflector/app.log | grep jitsi
# Check SSL certificates
sudo certbot certificates
```
### Performance Optimization
#### Scaling Considerations
**Single Server Limits:**
- ~50 concurrent participants per JVB instance
- ~10 concurrent Jibri recordings
- CPU and bandwidth become bottlenecks
**Multi-Server Setup:**
- Multiple JVB instances for scaling
- Dedicated Jibri recording servers
- Load balancing for high availability
#### Resource Monitoring
```bash
# Monitor JVB performance
sudo systemctl status jitsi-videobridge2
sudo journalctl -u jitsi-videobridge2 -f
# Check Prosody connections
sudo prosodyctl mod_admin_telnet
> c2s:show()
> muc:rooms()
```
## Security Best Practices
### JWT Security
- Use strong, unique secrets (32+ characters)
- Rotate JWT secrets regularly
- Implement proper token expiration
- Never log or expose JWT tokens
### Network Security
- Use HTTPS/WSS for all communications
- Implement proper firewall rules
- Consider VPN for server-to-server communication
- Monitor for unauthorized access attempts
### Recording Security
- Encrypt recordings at rest
- Implement access controls for recording files
- Regular security audits of file permissions
- Comply with data protection regulations
## Migration from Whereby
If migrating from Whereby to Jitsi:
1. **Parallel Setup** - Configure Jitsi alongside existing Whereby
2. **Room Migration** - Update room platform field to "jitsi"
3. **Test Integration** - Verify meeting creation and webhooks
4. **User Training** - Different UI and feature set
5. **Monitor Performance** - Watch for issues during transition
6. **Cleanup** - Remove Whereby configuration when stable
## Support and Resources
### Jitsi Community Resources
- **Documentation**: [jitsi.github.io/handbook](https://jitsi.github.io/handbook/)
- **Community Forum**: [community.jitsi.org](https://community.jitsi.org/)
- **GitHub Issues**: [github.com/jitsi/jitsi-meet](https://github.com/jitsi/jitsi-meet)
### Professional Support
- **8x8 Commercial Support** - Professional Jitsi hosting and support
- **Community Consulting** - Third-party Jitsi implementation services
### Monitoring and Maintenance
- Monitor system resources (CPU, memory, bandwidth)
- Regular security updates for all components
- Backup configuration files and certificates
- Test disaster recovery procedures

276
docs/video-whereby.md Normal file
View File

@@ -0,0 +1,276 @@
# Whereby Integration Configuration Guide
This guide explains how to configure Reflector to use Whereby as your video meeting platform for room creation, recording, and participant tracking.
## Overview
Whereby is a browser-based video meeting platform that provides hosted meeting rooms with recording capabilities. Reflector integrates with Whereby's API to:
- Create secure meeting rooms with custom branding
- Handle participant join/leave events via webhooks
- Automatically record meetings to AWS S3 storage
- Track meeting sessions and participant counts
## Requirements
### Whereby Account Setup
1. **Whereby Account**: Sign up for a Whereby business account at [whereby.com](https://whereby.com/business)
2. **API Access**: Request API access from Whereby support (required for programmatic room creation)
3. **Webhook Configuration**: Configure webhooks in your Whereby dashboard to point to your Reflector instance
### AWS S3 Storage
Whereby requires AWS S3 for recording storage. You need:
- AWS account with S3 access
- Dedicated S3 bucket for Whereby recordings
- AWS IAM credentials with S3 write permissions
## Configuration Variables
Add the following environment variables to your Reflector `.env` file:
### Required Variables
```bash
# Whereby API Configuration
WHEREBY_API_KEY=your-whereby-jwt-api-key
WHEREBY_WEBHOOK_SECRET=your-webhook-secret-from-whereby
# AWS S3 Storage for Recordings
AWS_WHEREBY_ACCESS_KEY_ID=your-aws-access-key
AWS_WHEREBY_ACCESS_KEY_SECRET=your-aws-secret-key
RECORDING_STORAGE_AWS_BUCKET_NAME=your-s3-bucket-name
```
### Optional Variables
```bash
# Whereby API URL (defaults to production)
WHEREBY_API_URL=https://api.whereby.dev/v1
# SQS Configuration (for recording processing)
AWS_PROCESS_RECORDING_QUEUE_URL=https://sqs.region.amazonaws.com/account/queue
SQS_POLLING_TIMEOUT_SECONDS=60
```
## Configuration Steps
### 1. Whereby API Key Setup
1. **Contact Whereby Support** to request API access for your account
2. **Generate JWT Token** in your Whereby dashboard under API settings
3. **Copy the JWT token** and set it as `WHEREBY_API_KEY` in your environment
The API key is a JWT token that looks like:
```
eyJ[...truncated JWT token...]
```
### 2. Webhook Configuration
1. **Access Whereby Dashboard** and navigate to webhook settings
2. **Set Webhook URL** to your Reflector instance:
```
https://your-reflector-domain.com/v1/whereby
```
3. **Configure Events** to send the following event types:
- `room.client.joined` - When participants join
- `room.client.left` - When participants leave
4. **Generate Webhook Secret** and set it as `WHEREBY_WEBHOOK_SECRET`
5. **Save Configuration** in your Whereby dashboard
### 3. AWS S3 Storage Setup
1. **Create S3 Bucket** dedicated for Whereby recordings
2. **Create IAM User** with programmatic access
3. **Attach S3 Policy** with the following permissions:
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:PutObjectAcl",
"s3:GetObject"
],
"Resource": "arn:aws:s3:::your-bucket-name/*"
}
]
}
```
4. **Configure Environment Variables** with the IAM credentials
### 4. Room Configuration
When creating rooms in Reflector, set the platform to use Whereby:
```bash
curl -X POST "https://your-reflector-domain.com/v1/rooms" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $AUTH_TOKEN" \
-d '{
"name": "my-whereby-room",
"platform": "whereby",
"recording_type": "cloud",
"recording_trigger": "automatic-2nd-participant",
"is_locked": false,
"room_mode": "normal"
}'
```
## Meeting Features
### Recording Options
Whereby supports three recording types:
- **`none`**: No recording
- **`local`**: Local recording (not recommended for production)
- **`cloud`**: Cloud recording to S3 (recommended)
### Recording Triggers
Control when recordings start:
- **`none`**: No automatic recording
- **`prompt`**: Prompt users to start recording
- **`automatic`**: Start immediately when meeting begins
- **`automatic-2nd-participant`**: Start when second participant joins
### Room Modes
- **`normal`**: Standard meeting room
- **`group`**: Group meeting with advanced features
## Webhook Event Handling
Reflector automatically handles these Whereby webhook events:
### Participant Tracking
```json
{
"type": "room.client.joined",
"data": {
"meetingId": "room-uuid",
"numClients": 2
}
}
```
### Recording Events
Whereby sends recording completion events that trigger Reflector's processing pipeline:
- Audio transcription
- Speaker diarization
- Summary generation
## Troubleshooting
### Common Issues
#### API Authentication Errors
**Symptoms**: 401 Unauthorized errors when creating meetings
**Solutions**:
1. Verify your `WHEREBY_API_KEY` is correct and not expired
2. Ensure you have API access enabled on your Whereby account
3. Contact Whereby support if API access is not available
#### Webhook Signature Validation Failed
**Symptoms**: Webhook events rejected with 401 errors
**Solutions**:
1. Verify `WHEREBY_WEBHOOK_SECRET` matches your Whereby dashboard configuration
2. Check webhook URL is correctly configured in Whereby dashboard
3. Ensure webhook endpoint is accessible from Whereby servers
#### Recording Upload Failures
**Symptoms**: Recordings not appearing in S3 bucket
**Solutions**:
1. Verify AWS credentials have S3 write permissions
2. Check S3 bucket name is correct and accessible
3. Ensure AWS region settings match your bucket location
4. Review AWS CloudTrail logs for permission issues
#### Participant Count Not Updating
**Symptoms**: Meeting participant counts remain at 0
**Solutions**:
1. Verify webhook events are being received at `/v1/whereby`
2. Check webhook signature validation is passing
3. Ensure meeting IDs match between Whereby and Reflector database
### Debug Commands
```bash
# Test Whereby API connectivity
curl -H "Authorization: Bearer $WHEREBY_API_KEY" \
https://api.whereby.dev/v1/meetings
# Check webhook endpoint health
curl https://your-reflector-domain.com/v1/whereby/health
# Verify S3 bucket access
aws s3 ls s3://your-bucket-name --profile whereby-user
```
## Security Considerations
### API Key Security
- Store API keys securely using environment variables
- Rotate API keys regularly
- Never commit API keys to version control
- Use separate keys for development and production
### Webhook Security
- Always validate webhook signatures using HMAC-SHA256
- Use HTTPS for all webhook endpoints
- Implement rate limiting on webhook endpoints
- Monitor webhook events for suspicious activity
### Recording Privacy
- Ensure S3 bucket access is restricted to authorized users
- Consider encryption at rest for sensitive recordings
- Implement retention policies for recorded content
- Comply with data protection regulations (GDPR, etc.)
## Performance Optimization
### Meeting Scaling
- Monitor concurrent meeting limits on your Whereby plan
- Implement meeting cleanup for expired sessions
- Use appropriate room modes for different use cases
### Recording Processing
- Configure SQS for asynchronous recording processing
- Monitor S3 storage usage and costs
- Implement automatic cleanup of processed recordings
### Webhook Reliability
- Implement webhook retry mechanisms
- Monitor webhook delivery success rates
- Log webhook events for debugging and auditing
## Migration from Other Platforms
If migrating from another video platform:
1. **Update Room Configuration**: Change existing rooms to use `"platform": "whereby"`
2. **Configure Webhooks**: Set up Whereby webhook endpoints
3. **Test Integration**: Verify meeting creation and event handling
4. **Monitor Performance**: Watch for any issues during transition
5. **Update Documentation**: Inform users of any workflow changes
## Support
For Whereby-specific issues:
- **Whereby Support**: [whereby.com/support](https://whereby.com/support)
- **API Documentation**: [whereby.dev](https://whereby.dev)
- **Status Page**: [status.whereby.com](https://status.whereby.com)
For Reflector integration issues:
- Check application logs for error details
- Verify environment variable configuration
- Test webhook connectivity and authentication
- Review AWS permissions and S3 access

474
docs/video_platforms.md Normal file
View File

@@ -0,0 +1,474 @@
# Video Platforms Architecture (PR #529 Analysis)
This document analyzes the video platforms refactoring implemented in PR #529 for daily.co integration, providing a blueprint for extending support to Jitsi and other video conferencing platforms.
## Overview
The video platforms refactoring introduces a clean abstraction layer that allows Reflector to support multiple video conferencing providers (Whereby, Daily.co, etc.) without changing core application logic. This architecture enables:
- Seamless switching between video platforms
- Platform-specific feature support
- Isolated platform code organization
- Consistent API surface across platforms
- Feature flags for gradual migration
## Architecture Components
### 1. **Directory Structure**
```
server/reflector/video_platforms/
├── __init__.py # Public API exports
├── base.py # Abstract base classes
├── factory.py # Platform client factory
├── registry.py # Platform registration system
├── whereby.py # Whereby implementation
├── daily.py # Daily.co implementation
└── mock.py # Testing implementation
```
### 2. **Core Abstract Classes**
#### `VideoPlatformClient` (base.py)
Abstract base class defining the interface all platforms must implement:
```python
class VideoPlatformClient(ABC):
PLATFORM_NAME: str = ""
@abstractmethod
async def create_meeting(self, room_name_prefix: str, end_date: datetime, room: Room) -> MeetingData
@abstractmethod
async def get_room_sessions(self, room_name: str) -> Dict[str, Any]
@abstractmethod
async def delete_room(self, room_name: str) -> bool
@abstractmethod
async def upload_logo(self, room_name: str, logo_path: str) -> bool
@abstractmethod
def verify_webhook_signature(self, body: bytes, signature: str, timestamp: Optional[str] = None) -> bool
```
#### `MeetingData` (base.py)
Standardized meeting data structure returned by all platforms:
```python
class MeetingData(BaseModel):
meeting_id: str
room_name: str
room_url: str
host_room_url: str
platform: str
extra_data: Dict[str, Any] = {} # Platform-specific data
```
#### `VideoPlatformConfig` (base.py)
Unified configuration structure for all platforms:
```python
class VideoPlatformConfig(BaseModel):
api_key: str
webhook_secret: str
api_url: Optional[str] = None
subdomain: Optional[str] = None
s3_bucket: Optional[str] = None
s3_region: Optional[str] = None
aws_role_arn: Optional[str] = None
aws_access_key_id: Optional[str] = None
aws_access_key_secret: Optional[str] = None
```
### 3. **Platform Registration System**
#### Registry Pattern (registry.py)
- Automatic registration of built-in platforms
- Runtime platform discovery
- Type-safe client instantiation
```python
# Auto-registration of platforms
_PLATFORMS: Dict[str, Type[VideoPlatformClient]] = {}
def register_platform(name: str, client_class: Type[VideoPlatformClient])
def get_platform_client(platform: str, config: VideoPlatformConfig) -> VideoPlatformClient
```
#### Factory System (factory.py)
- Configuration management per platform
- Platform selection logic
- Feature flag integration
```python
def get_platform_for_room(room_id: Optional[str] = None) -> str:
"""Determine which platform to use based on feature flags."""
if not settings.DAILY_MIGRATION_ENABLED:
return "whereby"
if room_id and room_id in settings.DAILY_MIGRATION_ROOM_IDS:
return "daily"
return settings.DEFAULT_VIDEO_PLATFORM
```
### 4. **Database Schema Changes**
#### Room Model Updates
Added `platform` field to track which video platform each room uses:
```python
# Database Schema
platform_column = sqlalchemy.Column(
"platform",
sqlalchemy.String,
nullable=False,
server_default="whereby"
)
# Pydantic Model
class Room(BaseModel):
platform: Literal["whereby", "daily"] = "whereby"
```
#### Meeting Model Updates
Added `platform` field to meetings for tracking and debugging:
```python
# Database Schema
platform_column = sqlalchemy.Column(
"platform",
sqlalchemy.String,
nullable=False,
server_default="whereby"
)
# Pydantic Model
class Meeting(BaseModel):
platform: Literal["whereby", "daily"] = "whereby"
```
**Key Decision**: No platform-specific fields were added to models. Instead, the `extra_data` field in `MeetingData` handles platform-specific information, following the user's rule of using generic `provider_data` as JSON if needed.
### 5. **Settings Configuration**
#### Feature Flags
```python
# Migration control
DAILY_MIGRATION_ENABLED: bool = True
DAILY_MIGRATION_ROOM_IDS: list[str] = []
DEFAULT_VIDEO_PLATFORM: str = "daily"
# Daily.co specific settings
DAILY_API_KEY: str | None = None
DAILY_WEBHOOK_SECRET: str | None = None
DAILY_SUBDOMAIN: str | None = None
AWS_DAILY_S3_BUCKET: str | None = None
AWS_DAILY_S3_REGION: str = "us-west-2"
AWS_DAILY_ROLE_ARN: str | None = None
```
#### Configuration Pattern
Each platform gets its own configuration namespace while sharing common patterns:
```python
def get_platform_config(platform: str) -> VideoPlatformConfig:
if platform == "whereby":
return VideoPlatformConfig(
api_key=settings.WHEREBY_API_KEY or "",
webhook_secret=settings.WHEREBY_WEBHOOK_SECRET or "",
# ... whereby-specific config
)
elif platform == "daily":
return VideoPlatformConfig(
api_key=settings.DAILY_API_KEY or "",
webhook_secret=settings.DAILY_WEBHOOK_SECRET or "",
# ... daily-specific config
)
```
### 6. **API Integration Updates**
#### Room Creation (views/rooms.py)
Updated to use platform factory instead of direct Whereby calls:
```python
@router.post("/rooms/{room_name}/meeting")
async def rooms_create_meeting(room_name: str, user: UserInfo):
# OLD: Direct Whereby integration
# whereby_meeting = await create_meeting("", end_date=end_date, room=room)
# NEW: Platform abstraction
platform = get_platform_for_room(room.id)
client = create_platform_client(platform)
meeting_data = await client.create_meeting(
room_name_prefix=room.name, end_date=end_date, room=room
)
await client.upload_logo(meeting_data.room_name, "./images/logo.png")
```
### 7. **Webhook Handling**
#### Separate Webhook Endpoints
Each platform gets its own webhook endpoint with platform-specific signature verification:
```python
# views/daily.py
@router.post("/daily_webhook")
async def daily_webhook(event: DailyWebhookEvent, request: Request):
# Verify Daily.co signature
body = await request.body()
signature = request.headers.get("X-Daily-Signature", "")
if not verify_daily_webhook_signature(body, signature):
raise HTTPException(status_code=401)
# Handle platform-specific events
if event.type == "participant.joined":
await _handle_participant_joined(event)
```
#### Consistent Event Handling
Despite different event formats, the core business logic remains the same:
```python
async def _handle_participant_joined(event):
room_name = event.data.get("room", {}).get("name") # Daily.co format
meeting = await meetings_controller.get_by_room_name(room_name)
if meeting:
current_count = getattr(meeting, "num_clients", 0)
await meetings_controller.update_meeting(
meeting.id, num_clients=current_count + 1
)
```
### 8. **Worker Task Integration**
#### New Task for Daily.co Recording Processing
Added platform-specific recording processing while maintaining the same pipeline:
```python
@shared_task
@asynctask
async def process_recording_from_url(recording_url: str, meeting_id: str, recording_id: str):
"""Process recording from Direct URL (Daily.co webhook)."""
logger.info("Processing recording from URL for meeting: %s", meeting_id)
# Uses same processing pipeline as Whereby S3 recordings
```
**Key Decision**: Worker tasks remain in main worker module but could be moved to platform-specific folders as suggested by the user.
### 9. **Testing Infrastructure**
#### Comprehensive Test Suite
- Unit tests for each platform client
- Integration tests for platform switching
- Mock platform for testing without external dependencies
- Webhook signature verification tests
```python
class TestPlatformIntegration:
"""Integration tests for platform switching."""
async def test_platform_switching_preserves_interface(self):
"""Test that different platforms provide consistent interface."""
# Test both Mock and Daily platforms return MeetingData objects
# with consistent fields
```
## Implementation Patterns for Jitsi Integration
Based on the daily.co implementation, here's how Jitsi should be integrated:
### 1. **Jitsi Client Implementation**
```python
# video_platforms/jitsi.py
class JitsiClient(VideoPlatformClient):
PLATFORM_NAME = "jitsi"
async def create_meeting(self, room_name_prefix: str, end_date: datetime, room: Room) -> MeetingData:
# Generate unique room name
jitsi_room = f"reflector-{room.name}-{int(time.time())}"
# Generate JWT tokens
user_jwt = self._generate_jwt(room=jitsi_room, moderator=False, exp=end_date)
host_jwt = self._generate_jwt(room=jitsi_room, moderator=True, exp=end_date)
return MeetingData(
meeting_id=generate_uuid4(),
room_name=jitsi_room,
room_url=f"https://jitsi.domain/{jitsi_room}?jwt={user_jwt}",
host_room_url=f"https://jitsi.domain/{jitsi_room}?jwt={host_jwt}",
platform=self.PLATFORM_NAME,
extra_data={"user_jwt": user_jwt, "host_jwt": host_jwt}
)
```
### 2. **Settings Integration**
```python
# settings.py
JITSI_DOMAIN: str = "meet.jit.si"
JITSI_JWT_SECRET: str | None = None
JITSI_WEBHOOK_SECRET: str | None = None
JITSI_API_URL: str | None = None # If using Jitsi API
```
### 3. **Factory Registration**
```python
# registry.py
def _register_builtin_platforms():
from .jitsi import JitsiClient
register_platform("jitsi", JitsiClient)
# factory.py
def get_platform_config(platform: str) -> VideoPlatformConfig:
elif platform == "jitsi":
return VideoPlatformConfig(
api_key="", # Jitsi may not need API key
webhook_secret=settings.JITSI_WEBHOOK_SECRET or "",
api_url=settings.JITSI_API_URL,
)
```
### 4. **Webhook Integration**
```python
# views/jitsi.py
@router.post("/jitsi/events")
async def jitsi_events_webhook(event_data: dict):
# Handle Prosody event-sync webhook format
event_type = event_data.get("event")
room_name = event_data.get("room", "").split("@")[0]
if event_type == "muc-occupant-joined":
# Same participant handling logic as other platforms
```
## Key Benefits of This Architecture
### 1. **Isolation and Organization**
- Platform-specific code contained in separate modules
- No platform logic leaking into core application
- Easy to add/remove platforms without affecting others
### 2. **Consistent Interface**
- All platforms implement the same abstract methods
- Standardized `MeetingData` structure
- Uniform error handling and logging
### 3. **Gradual Migration Support**
- Feature flags for controlled rollouts
- Room-specific platform selection
- Fallback mechanisms for platform failures
### 4. **Configuration Management**
- Centralized settings per platform
- Consistent naming patterns
- Environment-based configuration
### 5. **Testing and Quality**
- Mock platform for testing
- Comprehensive test coverage
- Platform-specific test utilities
## Migration Strategy Applied
The daily.co implementation demonstrates a careful migration approach:
### 1. **Backward Compatibility**
- Default platform remains "whereby"
- Existing rooms continue using Whereby unless explicitly migrated
- Same API endpoints and response formats
### 2. **Feature Flag Control**
```python
# Gradual rollout control
DAILY_MIGRATION_ENABLED: bool = True
DAILY_MIGRATION_ROOM_IDS: list[str] = [] # Specific rooms to migrate
DEFAULT_VIDEO_PLATFORM: str = "daily" # New rooms default
```
### 3. **Data Integrity**
- Platform field tracks which service each room/meeting uses
- No data loss during migration
- Platform-specific data preserved in `extra_data`
### 4. **Monitoring and Rollback**
- Comprehensive logging of platform selection
- Easy rollback by changing feature flags
- Platform-specific error tracking
## Recommendations for Jitsi Integration
Based on this analysis and the user's requirements:
### 1. **Follow the Pattern**
- Create `video_platforms/jitsi/` directory with:
- `client.py` - Main JitsiClient implementation
- `tasks.py` - Jitsi-specific worker tasks
- `__init__.py` - Module exports
### 2. **Settings Organization**
- Use `JITSI_*` prefix for all Jitsi settings
- Follow the same configuration pattern as Daily.co
- Support both environment variables and config files
### 3. **Generic Database Fields**
- Avoid platform-specific columns in database
- Use `provider_data` JSON field if platform-specific data needed
- Keep `platform` field as simple string identifier
### 4. **Worker Task Migration**
According to user requirements, migrate platform-specific tasks:
```
video_platforms/
├── whereby/
│ ├── client.py (moved from whereby.py)
│ └── tasks.py (moved from worker/whereby_tasks.py)
├── daily/
│ ├── client.py (moved from daily.py)
│ └── tasks.py (moved from worker/daily_tasks.py)
└── jitsi/
├── client.py (new JitsiClient)
└── tasks.py (new Jitsi recording tasks)
```
### 5. **Webhook Architecture**
- Create `views/jitsi.py` for Jitsi-specific webhooks
- Follow the same signature verification pattern
- Reuse existing participant tracking logic
## Implementation Checklist for Jitsi
- [ ] Create `video_platforms/jitsi/` directory structure
- [ ] Implement `JitsiClient` following the abstract interface
- [ ] Add Jitsi settings to configuration
- [ ] Register Jitsi platform in factory/registry
- [ ] Create Jitsi webhook endpoint
- [ ] Implement JWT token generation for room access
- [ ] Add Jitsi recording processing tasks
- [ ] Create comprehensive test suite
- [ ] Update database migrations for platform field
- [ ] Document Jitsi-specific configuration
## Conclusion
The video platforms refactoring in PR #529 provides an excellent foundation for adding Jitsi support. The architecture is well-designed with clear separation of concerns, consistent interfaces, and excellent extensibility. The daily.co implementation demonstrates how to add a new platform while maintaining backward compatibility and providing gradual migration capabilities.
The pattern should be directly applicable to Jitsi integration, with the main differences being:
- JWT-based authentication instead of API keys
- Different webhook event formats
- Jibri recording pipeline integration
- Self-hosted deployment considerations
This architecture successfully achieves the user's goals of:
1. Settings-based configuration
2. Generic database fields (no provider-specific columns)
3. Platform isolation in separate directories
4. Worker task organization within platform folders

View File

@@ -0,0 +1,212 @@
# Event Logger for Docker-Jitsi-Meet
A Prosody module that logs Jitsi meeting events to JSONL files alongside recordings, enabling complete participant tracking and speaker statistics.
## Prerequisites
- Running docker-jitsi-meet installation
- Jibri configured for recording
## Installation
### Step 1: Copy the Module
Copy the Prosody module to your custom plugins directory:
```bash
# Create the directory if it doesn't exist
mkdir -p ~/.jitsi-meet-cfg/prosody/prosody-plugins-custom
# Copy the module
cp mod_event_logger.lua ~/.jitsi-meet-cfg/prosody/prosody-plugins-custom/
```
### Step 2: Update Your .env File
Add or modify these variables in your `.env` file:
```bash
# If XMPP_MUC_MODULES already exists, append event_logger
# Example: XMPP_MUC_MODULES=existing_module,event_logger
XMPP_MUC_MODULES=event_logger
# Optional: Configure the module (these are defaults)
JIBRI_RECORDINGS_PATH=/config/recordings
JIBRI_LOG_SPEAKER_STATS=true
JIBRI_SPEAKER_STATS_INTERVAL=10
```
**Important**: If you already have `XMPP_MUC_MODULES` defined, add `event_logger` to the comma-separated list:
```bash
# Existing modules + our module
XMPP_MUC_MODULES=mod_info,mod_alert,event_logger
```
### Step 3: Modify docker-compose.yml
Add a shared recordings volume so Prosody can write events alongside Jibri recordings:
```yaml
services:
prosody:
# ... existing configuration ...
volumes:
- ${CONFIG}/prosody/config:/config:Z
- ${CONFIG}/prosody/prosody-plugins-custom:/prosody-plugins-custom:Z
- ${CONFIG}/recordings:/config/recordings:Z # Add this line
environment:
# Add if not using .env file
- XMPP_MUC_MODULES=${XMPP_MUC_MODULES:-event_logger}
- JIBRI_RECORDINGS_PATH=/config/recordings
jibri:
# ... existing configuration ...
volumes:
- ${CONFIG}/jibri:/config:Z
- ${CONFIG}/recordings:/config/recordings:Z # Add this line
environment:
# For Reflector webhook integration (optional)
- REFLECTOR_WEBHOOK_URL=${REFLECTOR_WEBHOOK_URL:-}
- JIBRI_FINALIZE_RECORDING_SCRIPT_PATH=/config/finalize.sh
```
### Step 4: Add Finalize Script (Optional - For Reflector Integration)
If you want to notify Reflector when recordings complete:
```bash
# Copy the finalize script
cp finalize.sh ~/.jitsi-meet-cfg/jibri/finalize.sh
chmod +x ~/.jitsi-meet-cfg/jibri/finalize.sh
# Add to .env
REFLECTOR_WEBHOOK_URL=http://your-reflector-api:8000
```
### Step 5: Restart Services
```bash
docker-compose down
docker-compose up -d
```
## What Gets Created
After a recording, you'll find in `~/.jitsi-meet-cfg/recordings/{session-id}/`:
- `recording.mp4` - The video recording (created by Jibri)
- `metadata.json` - Basic metadata (created by Jibri)
- `events.jsonl` - Complete participant timeline (created by this module)
## Event Format
Each line in `events.jsonl` is a JSON object:
```json
{"type":"room_created","timestamp":1234567890,"room_name":"TestRoom","room_jid":"testroom@conference.meet.jitsi","meeting_url":"https://meet.jitsi/TestRoom"}
{"type":"recording_started","timestamp":1234567891,"room_name":"TestRoom","session_id":"20240115120000_TestRoom","jibri_jid":"jibri@recorder.meet.jitsi"}
{"type":"participant_joined","timestamp":1234567892,"room_name":"TestRoom","participant":{"jid":"user1@meet.jitsi/web","nick":"John Doe","id":"user1@meet.jitsi","is_moderator":false}}
{"type":"speaker_active","timestamp":1234567895,"room_name":"TestRoom","speaker_jid":"user1@meet.jitsi","speaker_nick":"John Doe","duration":10}
{"type":"participant_left","timestamp":1234567920,"room_name":"TestRoom","participant":{"jid":"user1@meet.jitsi/web","nick":"John Doe","duration_seconds":28}}
{"type":"recording_stopped","timestamp":1234567950,"room_name":"TestRoom","session_id":"20240115120000_TestRoom","meeting_url":"https://meet.jitsi/TestRoom"}
```
## Configuration Options
All configuration can be done via environment variables:
| Environment Variable | Default | Description |
|---------------------|---------|-------------|
| `JIBRI_RECORDINGS_PATH` | `/config/recordings` | Path where recordings are stored |
| `JIBRI_LOG_SPEAKER_STATS` | `true` | Enable speaker statistics logging |
| `JIBRI_SPEAKER_STATS_INTERVAL` | `10` | Seconds between speaker stats updates |
## Verifying Installation
Check that the module is loaded:
```bash
docker-compose logs prosody | grep "Event Logger"
# Should see: "Event Logger loaded - writing to /config/recordings"
```
Check for events after a recording:
```bash
ls -la ~/.jitsi-meet-cfg/recordings/*/events.jsonl
cat ~/.jitsi-meet-cfg/recordings/*/events.jsonl | jq .
```
## Troubleshooting
### No events.jsonl file created
1. **Check module is enabled**:
```bash
docker-compose exec prosody grep -r "event_logger" /config
```
2. **Verify volume permissions**:
```bash
docker-compose exec prosody ls -la /config/recordings
```
3. **Check Prosody logs for errors**:
```bash
docker-compose logs prosody | grep -i error
```
### Module not loading
1. **Verify file exists in container**:
```bash
docker-compose exec prosody ls -la /prosody-plugins-custom/
```
2. **Check XMPP_MUC_MODULES format** (must be comma-separated, no spaces):
- ✅ Correct: `XMPP_MUC_MODULES=mod1,mod2,event_logger`
- ❌ Wrong: `XMPP_MUC_MODULES=mod1, mod2, event_logger`
## Common docker-compose.yml Patterns
### Minimal Addition (if you trust defaults)
```yaml
services:
prosody:
volumes:
- ${CONFIG}/recordings:/config/recordings:Z # Just add this
```
### Full Configuration
```yaml
services:
prosody:
volumes:
- ${CONFIG}/prosody/config:/config:Z
- ${CONFIG}/prosody/prosody-plugins-custom:/prosody-plugins-custom:Z
- ${CONFIG}/recordings:/config/recordings:Z
environment:
- XMPP_MUC_MODULES=event_logger
- JIBRI_RECORDINGS_PATH=/config/recordings
- JIBRI_LOG_SPEAKER_STATS=true
- JIBRI_SPEAKER_STATS_INTERVAL=10
jibri:
volumes:
- ${CONFIG}/jibri:/config:Z
- ${CONFIG}/recordings:/config/recordings:Z
environment:
- JIBRI_RECORDING_DIR=/config/recordings
- JIBRI_FINALIZE_RECORDING_SCRIPT_PATH=/config/finalize.sh
```
## Integration with Reflector
The finalize.sh script will automatically notify Reflector when a recording completes if `REFLECTOR_WEBHOOK_URL` is set. Reflector will receive:
```json
{
"session_id": "20240115120000_TestRoom",
"path": "20240115120000_TestRoom",
"meeting_url": "https://meet.jitsi/TestRoom"
}
```
Reflector then processes the recording along with the complete participant timeline from `events.jsonl`.

View File

@@ -0,0 +1,49 @@
#!/bin/bash
# Jibri finalize script to notify Reflector when recording is complete
# This script is called by Jibri with the recording directory as argument
RECORDING_PATH="$1"
SESSION_ID=$(basename "$RECORDING_PATH")
METADATA_FILE="$RECORDING_PATH/metadata.json"
# Extract meeting URL from Jibri's metadata
MEETING_URL=""
if [ -f "$METADATA_FILE" ]; then
MEETING_URL=$(jq -r '.meeting_url' "$METADATA_FILE" 2>/dev/null || echo "")
fi
echo "[$(date)] Recording finalized: $RECORDING_PATH"
echo "[$(date)] Session ID: $SESSION_ID"
echo "[$(date)] Meeting URL: $MEETING_URL"
# Check if events.jsonl was created by our Prosody module
if [ -f "$RECORDING_PATH/events.jsonl" ]; then
EVENT_COUNT=$(wc -l < "$RECORDING_PATH/events.jsonl")
echo "[$(date)] Found events.jsonl with $EVENT_COUNT events"
else
echo "[$(date)] Warning: No events.jsonl found"
fi
# Notify Reflector if webhook URL is configured
if [ -n "$REFLECTOR_WEBHOOK_URL" ]; then
echo "[$(date)] Notifying Reflector at: $REFLECTOR_WEBHOOK_URL"
RESPONSE=$(curl -s -w "\n%{http_code}" -X POST "$REFLECTOR_WEBHOOK_URL/api/v1/jibri/recording-ready" \
-H "Content-Type: application/json" \
-d "{\"session_id\":\"$SESSION_ID\",\"path\":\"$SESSION_ID\",\"meeting_url\":\"$MEETING_URL\"}")
HTTP_CODE=$(echo "$RESPONSE" | tail -n1)
BODY=$(echo "$RESPONSE" | sed '$d')
if [ "$HTTP_CODE" = "200" ]; then
echo "[$(date)] Reflector notified successfully"
echo "[$(date)] Response: $BODY"
else
echo "[$(date)] Failed to notify Reflector. HTTP code: $HTTP_CODE"
echo "[$(date)] Response: $BODY"
fi
else
echo "[$(date)] No REFLECTOR_WEBHOOK_URL configured, skipping notification"
fi
echo "[$(date)] Finalize script completed"

View File

@@ -0,0 +1,372 @@
local json = require "util.json"
local st = require "util.stanza"
local jid_bare = require "util.jid".bare
local recordings_path = os.getenv("JIBRI_RECORDINGS_PATH") or
module:get_option_string("jibri_recordings_path", "/recordings")
-- room_jid -> { session_id, participants = {jid -> info} }
local active_recordings = {}
-- room_jid -> { participants = {jid -> info}, created_at }
local room_states = {}
local function get_timestamp()
return os.time()
end
local function write_event(session_id, event)
if not session_id then
module:log("warn", "No session_id for event: %s", event.type)
return
end
local session_dir = string.format("%s/%s", recordings_path, session_id)
local event_file = string.format("%s/events.jsonl", session_dir)
module:log("info", "Writing event %s to %s", event.type, event_file)
-- Create directory
local mkdir_cmd = string.format("mkdir -p '%s' 2>&1", session_dir)
local mkdir_result = os.execute(mkdir_cmd)
module:log("debug", "mkdir result: %s", tostring(mkdir_result))
local file, err = io.open(event_file, "a")
if file then
local json_str = json.encode(event)
file:write(json_str .. "\n")
file:close()
module:log("info", "Successfully wrote event %s", event.type)
else
module:log("error", "Failed to write event to %s: %s", event_file, err)
end
end
local function extract_participant_info(occupant)
local info = {
jid = occupant.jid,
bare_jid = occupant.bare_jid,
nick = occupant.nick,
display_name = nil,
role = occupant.role
}
local presence = occupant:get_presence()
if presence then
local nick_element = presence:get_child("nick", "http://jabber.org/protocol/nick")
if nick_element then
info.display_name = nick_element:get_text()
end
local identity = presence:get_child("identity")
if identity then
local user = identity:get_child("user")
if user then
local name = user:get_child("name")
if name then
info.display_name = name:get_text()
end
local id_element = user:get_child("id")
if id_element then
info.id = id_element:get_text()
end
end
end
if not info.display_name and occupant.nick then
local _, _, resource = occupant.nick:match("([^@]+)@([^/]+)/(.+)")
if resource then
info.display_name = resource
end
end
end
return info
end
local function get_room_participant_count(room)
local count = 0
for _ in room:each_occupant() do
count = count + 1
end
return count
end
local function snapshot_room_participants(room)
local participants = {}
local total = 0
local skipped = 0
module:log("info", "Snapshotting room participants")
for _, occupant in room:each_occupant() do
total = total + 1
-- Skip recorders (Jibri)
if occupant.bare_jid and (occupant.bare_jid:match("^recorder@") or
occupant.bare_jid:match("^jibri@")) then
skipped = skipped + 1
else
local info = extract_participant_info(occupant)
participants[occupant.jid] = info
module:log("debug", "Added participant: %s", info.display_name or info.bare_jid)
end
end
module:log("info", "Snapshot: %d total, %d participants", total, total - skipped)
return participants
end
-- Import utility functions if available
local util = module:require "util";
local get_room_from_jid = util.get_room_from_jid;
local room_jid_match_rewrite = util.room_jid_match_rewrite;
-- Main IQ handler for Jibri stanzas
module:hook("pre-iq/full", function(event)
local stanza = event.stanza
if stanza.name ~= "iq" then
return
end
local jibri = stanza:get_child('jibri', 'http://jitsi.org/protocol/jibri')
if not jibri then
return
end
module:log("info", "=== Jibri IQ intercepted ===")
local action = jibri.attr.action
local session_id = jibri.attr.session_id
local room_jid = jibri.attr.room
local recording_mode = jibri.attr.recording_mode
local app_data = jibri.attr.app_data
module:log("info", "Jibri %s - session: %s, room: %s, mode: %s",
action or "?", session_id or "?", room_jid or "?", recording_mode or "?")
if not room_jid or not session_id then
module:log("warn", "Missing room_jid or session_id")
return
end
-- Get the room using util function
local room = get_room_from_jid(room_jid_match_rewrite(jid_bare(stanza.attr.to)))
if not room then
-- Try with the room_jid directly
room = get_room_from_jid(room_jid)
end
if not room then
module:log("error", "Room not found for jid: %s", room_jid)
return
end
module:log("info", "Room found: %s", room:get_name() or room_jid)
if action == "start" then
module:log("info", "Recording START for session %s", session_id)
-- Count and snapshot participants
local participant_count = 0
for _ in room:each_occupant() do
participant_count = participant_count + 1
end
local participants = snapshot_room_participants(room)
local participant_list = {}
for jid, info in pairs(participants) do
table.insert(participant_list, info)
end
active_recordings[room_jid] = {
session_id = session_id,
participants = participants,
started_at = get_timestamp()
}
write_event(session_id, {
type = "recording_started",
timestamp = get_timestamp(),
room_jid = room_jid,
room_name = room:get_name(),
session_id = session_id,
recording_mode = recording_mode,
app_data = app_data,
participant_count = participant_count,
participants_at_start = participant_list
})
elseif action == "stop" then
module:log("info", "Recording STOP for session %s", session_id)
local recording = active_recordings[room_jid]
if recording and recording.session_id == session_id then
write_event(session_id, {
type = "recording_stopped",
timestamp = get_timestamp(),
room_jid = room_jid,
room_name = room:get_name(),
session_id = session_id,
duration = get_timestamp() - recording.started_at,
participant_count = get_room_participant_count(room)
})
active_recordings[room_jid] = nil
else
module:log("warn", "No active recording found for room %s", room_jid)
end
end
end);
-- Room and participant event hooks
local function setup_room_hooks(host_module)
module:log("info", "Setting up room hooks on %s", host_module.host or "unknown")
-- Room created
host_module:hook("muc-room-created", function(event)
local room = event.room
local room_jid = room.jid
room_states[room_jid] = {
participants = {},
created_at = get_timestamp()
}
module:log("info", "Room created: %s", room_jid)
end)
-- Room destroyed
host_module:hook("muc-room-destroyed", function(event)
local room = event.room
local room_jid = room.jid
room_states[room_jid] = nil
active_recordings[room_jid] = nil
module:log("info", "Room destroyed: %s", room_jid)
end)
-- Occupant joined
host_module:hook("muc-occupant-joined", function(event)
local room = event.room
local occupant = event.occupant
local room_jid = room.jid
-- Skip recorders
if occupant.bare_jid and (occupant.bare_jid:match("^recorder@") or
occupant.bare_jid:match("^jibri@")) then
return
end
local participant_info = extract_participant_info(occupant)
-- Update room state
if room_states[room_jid] then
room_states[room_jid].participants[occupant.jid] = participant_info
end
-- Log to active recording if exists
local recording = active_recordings[room_jid]
if recording then
recording.participants[occupant.jid] = participant_info
write_event(recording.session_id, {
type = "participant_joined",
timestamp = get_timestamp(),
room_jid = room_jid,
room_name = room:get_name(),
participant = participant_info,
participant_count = get_room_participant_count(room)
})
end
module:log("info", "Participant joined %s: %s (%d total)",
room:get_name() or room_jid,
participant_info.display_name or participant_info.bare_jid,
get_room_participant_count(room))
end)
-- Occupant left
host_module:hook("muc-occupant-left", function(event)
local room = event.room
local occupant = event.occupant
local room_jid = room.jid
-- Skip recorders
if occupant.bare_jid and (occupant.bare_jid:match("^recorder@") or
occupant.bare_jid:match("^jibri@")) then
return
end
local participant_info = extract_participant_info(occupant)
-- Update room state
if room_states[room_jid] then
room_states[room_jid].participants[occupant.jid] = nil
end
-- Log to active recording if exists
local recording = active_recordings[room_jid]
if recording then
if recording.participants[occupant.jid] then
recording.participants[occupant.jid] = nil
end
write_event(recording.session_id, {
type = "participant_left",
timestamp = get_timestamp(),
room_jid = room_jid,
room_name = room:get_name(),
participant = participant_info,
participant_count = get_room_participant_count(room)
})
end
module:log("info", "Participant left %s: %s (%d remaining)",
room:get_name() or room_jid,
participant_info.display_name or participant_info.bare_jid,
get_room_participant_count(room))
end)
end
-- Module initialization
local current_host = module:get_host()
local host_type = module:get_host_type()
module:log("info", "Event Logger loading on %s (type: %s)", current_host, host_type or "unknown")
module:log("info", "Recording path: %s", recordings_path)
-- Setup room hooks based on host type
if host_type == "component" and current_host:match("^[^.]+%.") then
setup_room_hooks(module)
else
-- Try to find and hook to MUC component
local process_host_module = util.process_host_module
local muc_component_host = module:get_option_string("muc_component") or
module:get_option_string("main_muc")
if not muc_component_host then
local possible_hosts = {
"muc." .. current_host,
"conference." .. current_host,
"rooms." .. current_host
}
for _, host in ipairs(possible_hosts) do
if prosody.hosts[host] then
muc_component_host = host
module:log("info", "Auto-detected MUC component: %s", muc_component_host)
break
end
end
end
if muc_component_host then
process_host_module(muc_component_host, function(host_module, host)
module:log("info", "Hooking to MUC events on %s", host)
setup_room_hooks(host_module)
end)
else
module:log("error", "Could not find MUC component")
end
end

View File

@@ -0,0 +1,194 @@
## Reflector GPU Transcription API (Specification)
This document defines the Reflector GPU transcription API that all implementations must adhere to. Current implementations include NVIDIA Parakeet (NeMo) and Whisper (faster-whisper), both deployed on Modal.com. The API surface and response shapes are OpenAI/Whisper-compatible, so clients can switch implementations by changing only the base URL.
### Base URL and Authentication
- Example base URLs (Modal web endpoints):
- Parakeet: `https://<account>--reflector-transcriber-parakeet-web.modal.run`
- Whisper: `https://<account>--reflector-transcriber-web.modal.run`
- All endpoints are served under `/v1` and require a Bearer token:
```
Authorization: Bearer <REFLECTOR_GPU_APIKEY>
```
Note: To switch implementations, deploy the desired variant and point `TRANSCRIPT_URL` to its base URL. The API is identical.
### Supported file types
`mp3, mp4, mpeg, mpga, m4a, wav, webm`
### Models and languages
- Parakeet (NVIDIA NeMo): default `nvidia/parakeet-tdt-0.6b-v2`
- Language support: only `en`. Other languages return HTTP 400.
- Whisper (faster-whisper): default `large-v2` (or deployment-specific)
- Language support: multilingual (per Whisper model capabilities).
Note: The `model` parameter is accepted by all implementations for interface parity. Some backends may treat it as informational.
### Endpoints
#### POST /v1/audio/transcriptions
Transcribe one or more uploaded audio files.
Request: multipart/form-data
- `file` (File) — optional. Single file to transcribe.
- `files` (File[]) — optional. One or more files to transcribe.
- `model` (string) — optional. Defaults to the implementation-specific model (see above).
- `language` (string) — optional, defaults to `en`.
- Parakeet: only `en` is accepted; other values return HTTP 400
- Whisper: model-dependent; typically multilingual
- `batch` (boolean) — optional, defaults to `false`.
Notes:
- Provide either `file` or `files`, not both. If neither is provided, HTTP 400.
- `batch` requires `files`; using `batch=true` without `files` returns HTTP 400.
- Response shape for multiple files is the same regardless of `batch`.
- Files sent to this endpoint are processed in a single pass (no VAD/chunking). This is intended for short clips (roughly ≤ 30s; depends on GPU memory/model). For longer audio, prefer `/v1/audio/transcriptions-from-url` which supports VAD-based chunking.
Responses
Single file response:
```json
{
"text": "transcribed text",
"words": [
{ "word": "hello", "start": 0.0, "end": 0.5 },
{ "word": "world", "start": 0.5, "end": 1.0 }
],
"filename": "audio.mp3"
}
```
Multiple files response:
```json
{
"results": [
{"filename": "a1.mp3", "text": "...", "words": [...]},
{"filename": "a2.mp3", "text": "...", "words": [...]}]
}
```
Notes:
- Word objects always include keys: `word`, `start`, `end`.
- Some implementations may include a trailing space in `word` to match Whisper tokenization behavior; clients should trim if needed.
Example curl (single file):
```bash
curl -X POST \
-H "Authorization: Bearer $REFLECTOR_GPU_APIKEY" \
-F "file=@/path/to/audio.mp3" \
-F "language=en" \
"$BASE_URL/v1/audio/transcriptions"
```
Example curl (multiple files, batch):
```bash
curl -X POST \
-H "Authorization: Bearer $REFLECTOR_GPU_APIKEY" \
-F "files=@/path/a1.mp3" -F "files=@/path/a2.mp3" \
-F "batch=true" -F "language=en" \
"$BASE_URL/v1/audio/transcriptions"
```
#### POST /v1/audio/transcriptions-from-url
Transcribe a single remote audio file by URL.
Request: application/json
Body parameters:
- `audio_file_url` (string) — required. URL of the audio file to transcribe.
- `model` (string) — optional. Defaults to the implementation-specific model (see above).
- `language` (string) — optional, defaults to `en`. Parakeet only accepts `en`.
- `timestamp_offset` (number) — optional, defaults to `0.0`. Added to each word's `start`/`end` in the response.
```json
{
"audio_file_url": "https://example.com/audio.mp3",
"model": "nvidia/parakeet-tdt-0.6b-v2",
"language": "en",
"timestamp_offset": 0.0
}
```
Response:
```json
{
"text": "transcribed text",
"words": [
{ "word": "hello", "start": 10.0, "end": 10.5 },
{ "word": "world", "start": 10.5, "end": 11.0 }
]
}
```
Notes:
- `timestamp_offset` is added to each words `start`/`end` in the response.
- Implementations may perform VAD-based chunking and batching for long-form audio; word timings are adjusted accordingly.
Example curl:
```bash
curl -X POST \
-H "Authorization: Bearer $REFLECTOR_GPU_APIKEY" \
-H "Content-Type: application/json" \
-d '{
"audio_file_url": "https://example.com/audio.mp3",
"language": "en",
"timestamp_offset": 0
}' \
"$BASE_URL/v1/audio/transcriptions-from-url"
```
### Error handling
- 400 Bad Request
- Parakeet: `language` other than `en`
- Missing required parameters (`file`/`files` for upload; `audio_file_url` for URL endpoint)
- Unsupported file extension
- 401 Unauthorized
- Missing or invalid Bearer token
- 404 Not Found
- `audio_file_url` does not exist
### Implementation details
- GPUs: A10G for small-file/live, L40S for large-file URL transcription (subject to deployment)
- VAD chunking and segment batching; word timings adjusted and overlapping ends constrained
- Pads very short segments (< 0.5s) to avoid model crashes on some backends
### Server configuration (Reflector API)
Set the Reflector server to use the Modal backend and point `TRANSCRIPT_URL` to your chosen deployment:
```
TRANSCRIPT_BACKEND=modal
TRANSCRIPT_URL=https://<account>--reflector-transcriber-parakeet-web.modal.run
TRANSCRIPT_MODAL_API_KEY=<REFLECTOR_GPU_APIKEY>
```
### Conformance tests
Use the pytest-based conformance tests to validate any new implementation (including self-hosted) against this spec:
```
TRANSCRIPT_URL=https://<your-deployment-base> \
TRANSCRIPT_MODAL_API_KEY=your-api-key \
uv run -m pytest -m gpu_modal --no-cov server/tests/test_gpu_modal_transcript.py
```

View File

@@ -0,0 +1,493 @@
# Jitsi Integration Configuration Guide
This guide provides step-by-step instructions for configuring Reflector to work with a self-hosted Jitsi Meet installation for video meetings and recording.
## Prerequisites
Before configuring Jitsi integration, ensure you have:
- **Self-hosted Jitsi Meet installation** (version 2.0.8922 or later recommended)
- **Jibri recording service** configured and running
- **Prosody XMPP server** with mod_event_sync module installed
- **Docker or system deployment** of Reflector with access to environment variables
- **SSL certificates** for secure communication between services
## Environment Configuration
Add the following environment variables to your Reflector deployment:
### Required Settings
```bash
# Jitsi Meet domain (without https://)
JITSI_DOMAIN=meet.example.com
# JWT secret for room authentication (generate with: openssl rand -hex 32)
JITSI_JWT_SECRET=your-64-character-hex-secret-here
# Webhook secret for secure event handling (generate with: openssl rand -hex 16)
JITSI_WEBHOOK_SECRET=your-32-character-hex-secret-here
# Application identifier (should match Jitsi configuration)
JITSI_APP_ID=reflector
# JWT issuer and audience (should match Jitsi configuration)
JITSI_JWT_ISSUER=reflector
JITSI_JWT_AUDIENCE=jitsi
```
### Example .env Configuration
```bash
# Add to your server/.env file
JITSI_DOMAIN=meet.mycompany.com
JITSI_JWT_SECRET=$(openssl rand -hex 32)
JITSI_WEBHOOK_SECRET=$(openssl rand -hex 16)
JITSI_APP_ID=reflector
JITSI_JWT_ISSUER=reflector
JITSI_JWT_AUDIENCE=jitsi
```
## Jitsi Meet Server Configuration
### 1. JWT Authentication Setup
Edit `/etc/prosody/conf.d/[YOUR_DOMAIN].cfg.lua`:
```lua
VirtualHost "meet.example.com"
authentication = "token"
app_id = "reflector"
app_secret = "your-jwt-secret-here"
-- Allow anonymous access for non-authenticated users
c2s_require_encryption = false
admins = { "focusUser@auth.meet.example.com" }
modules_enabled = {
"bosh";
"pubsub";
"ping";
"roster";
"saslauth";
"tls";
"dialback";
"disco";
"carbons";
"pep";
"private";
"blocklist";
"vcard";
"version";
"uptime";
"time";
"ping";
"register";
"admin_adhoc";
"token_verification";
"event_sync"; -- Required for webhook events
}
```
### 2. Room Access Control
Edit `/etc/jitsi/meet/meet.example.com-config.js`:
```javascript
var config = {
hosts: {
domain: 'meet.example.com',
muc: 'conference.meet.example.com'
},
// Enable JWT authentication
enableUserRolesBasedOnToken: true,
// Recording configuration
fileRecordingsEnabled: true,
liveStreamingEnabled: false,
// Reflector-specific settings
prejoinPageEnabled: true,
requireDisplayName: true,
};
```
### 3. Interface Configuration
Edit `/usr/share/jitsi-meet/interface_config.js`:
```javascript
var interfaceConfig = {
// Customize for Reflector branding
APP_NAME: 'Reflector Meeting',
DEFAULT_WELCOME_PAGE_LOGO_URL: 'https://your-domain.com/logo.png',
// Hide unnecessary buttons
TOOLBAR_BUTTONS: [
'microphone', 'camera', 'closedcaptions', 'desktop',
'fullscreen', 'fodeviceselection', 'hangup',
'chat', 'recording', 'livestreaming', 'etherpad',
'sharedvideo', 'settings', 'raisehand', 'videoquality',
'filmstrip', 'invite', 'feedback', 'stats', 'shortcuts',
'tileview', 'videobackgroundblur', 'download', 'help',
'mute-everyone'
]
};
```
## Jibri Configuration
### 1. Recording Service Setup
Edit `/etc/jitsi/jibri/jibri.conf`:
```hocon
jibri {
recording {
recordings-directory = "/var/recordings"
finalize-script = "/opt/jitsi/jibri/finalize.sh"
}
api {
xmpp {
environments = [{
name = "prod environment"
xmpp-server-hosts = ["meet.example.com"]
xmpp-domain = "meet.example.com"
control-muc {
domain = "internal.auth.meet.example.com"
room-name = "JibriBrewery"
nickname = "jibri-nickname"
}
control-login {
domain = "auth.meet.example.com"
username = "jibri"
password = "jibri-password"
}
}]
}
}
}
```
### 2. Finalize Script Setup
Create `/opt/jitsi/jibri/finalize.sh`:
```bash
#!/bin/bash
# Jibri finalize script for Reflector integration
RECORDING_FILE="$1"
ROOM_NAME="$2"
REFLECTOR_API_URL="${REFLECTOR_API_URL:-http://localhost:1250}"
WEBHOOK_SECRET="${JITSI_WEBHOOK_SECRET}"
# Generate webhook signature
generate_signature() {
local payload="$1"
echo -n "$payload" | openssl dgst -sha256 -hmac "$WEBHOOK_SECRET" | cut -d' ' -f2
}
# Prepare webhook payload
TIMESTAMP=$(date -u +%Y-%m-%dT%H:%M:%S.%3NZ)
PAYLOAD=$(cat <<EOF
{
"room_name": "$ROOM_NAME",
"recording_file": "$RECORDING_FILE",
"recording_status": "completed",
"timestamp": "$TIMESTAMP"
}
EOF
)
# Generate signature
SIGNATURE=$(generate_signature "$PAYLOAD")
# Send webhook to Reflector
curl -X POST "$REFLECTOR_API_URL/v1/jibri/recording-complete" \
-H "Content-Type: application/json" \
-H "X-Jitsi-Signature: $SIGNATURE" \
-d "$PAYLOAD" \
--max-time 30
echo "Recording finalization webhook sent for room: $ROOM_NAME"
```
Make the script executable:
```bash
chmod +x /opt/jitsi/jibri/finalize.sh
```
## Prosody Event Configuration
### 1. Event-Sync Module Installation
Install the mod_event_sync module:
```bash
# Download the module
cd /usr/share/jitsi-meet/prosody-plugins/
wget https://raw.githubusercontent.com/jitsi-contrib/prosody-plugins/main/mod_event_sync.lua
# Or if using git
git clone https://github.com/jitsi-contrib/prosody-plugins.git
cp prosody-plugins/mod_event_sync.lua /usr/share/jitsi-meet/prosody-plugins/
```
### 2. Webhook Configuration
Add to `/etc/prosody/conf.d/[YOUR_DOMAIN].cfg.lua`:
```lua
Component "conference.meet.example.com" "muc"
storage = "memory"
modules_enabled = {
"muc_meeting_id";
"muc_domain_mapper";
"polls";
"event_sync"; -- Enable event sync
}
-- Event sync webhook configuration
event_sync_url = "https://your-reflector-domain.com/v1/jitsi/events"
event_sync_secret = "your-webhook-secret-here"
-- Events to track
event_sync_events = {
"muc-occupant-joined",
"muc-occupant-left",
"jibri-recording-on",
"jibri-recording-off"
}
```
### 3. Restart Services
After configuration changes, restart all services:
```bash
systemctl restart prosody
systemctl restart jicofo
systemctl restart jitsi-videobridge2
systemctl restart jibri
systemctl restart nginx
```
## Reflector Room Configuration
### 1. Create Jitsi Room
When creating rooms in Reflector, set the platform field:
```bash
curl -X POST "https://your-reflector-domain.com/v1/rooms" \
-H "Authorization: Bearer $AUTH_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"name": "my-jitsi-room",
"platform": "jitsi",
"recording_type": "cloud",
"recording_trigger": "automatic-2nd-participant",
"is_locked": false,
"room_mode": "normal"
}'
```
### 2. Meeting Creation
Meetings will automatically use Jitsi when the room platform is set to "jitsi":
```bash
curl -X POST "https://your-reflector-domain.com/v1/rooms/my-jitsi-room/meeting" \
-H "Authorization: Bearer $AUTH_TOKEN"
```
## Testing the Integration
### 1. Health Check
Verify Jitsi webhook configuration:
```bash
curl "https://your-reflector-domain.com/v1/jitsi/health"
```
Expected response:
```json
{
"status": "ok",
"service": "jitsi-webhooks",
"timestamp": "2025-01-15T10:30:00.000Z",
"webhook_secret_configured": true
}
```
### 2. Room Creation Test
1. Create a Jitsi room via Reflector API
2. Start a meeting - should generate Jitsi Meet URL with JWT token
3. Join with multiple participants - should trigger participant events
4. Start recording - should trigger Jibri recording workflow
### 3. Webhook Event Test
Monitor Reflector logs for incoming webhook events:
```bash
# Check for participant events
curl -X POST "https://your-reflector-domain.com/v1/jitsi/events" \
-H "Content-Type: application/json" \
-H "X-Jitsi-Signature: test-signature" \
-d '{
"event": "muc-occupant-joined",
"room": "test-room-name",
"timestamp": "2025-01-15T10:30:00.000Z",
"data": {}
}'
```
## Troubleshooting
### Common Issues
#### JWT Authentication Failures
**Symptoms:** Users can't join rooms, "Authentication failed" errors
**Solutions:**
1. Verify JWT secret matches between Jitsi and Reflector
2. Check JWT token expiration (default 8 hours)
3. Ensure system clocks are synchronized
4. Validate JWT issuer/audience configuration
```bash
# Debug JWT tokens
echo "JWT_TOKEN_HERE" | cut -d'.' -f2 | base64 -d | jq
```
#### Webhook Events Not Received
**Symptoms:** Participant counts not updating, recording events missing
**Solutions:**
1. Verify event_sync module is loaded in Prosody
2. Check webhook URL accessibility from Jitsi server
3. Validate webhook signature generation
4. Review Prosody and Reflector logs
```bash
# Test webhook connectivity
curl -v "https://your-reflector-domain.com/v1/jitsi/health"
# Check Prosody logs
tail -f /var/log/prosody/prosody.log
# Check Reflector logs
docker logs your-reflector-container
```
#### Recording Issues
**Symptoms:** Recordings not starting, finalize script errors
**Solutions:**
1. Verify Jibri service status and configuration
2. Check recording directory permissions
3. Validate finalize script execution permissions
4. Monitor Jibri logs for errors
```bash
# Check Jibri status
systemctl status jibri
# Test finalize script
sudo -u jibri /opt/jitsi/jibri/finalize.sh "/test/recording.mp4" "test-room"
# Check Jibri logs
journalctl -u jibri -f
```
### Debug Commands
```bash
# Verify Jitsi configuration
prosodyctl check config
# Test JWT generation
curl -X POST "https://your-reflector-domain.com/v1/rooms/test/meeting" \
-H "Authorization: Bearer $TOKEN" -v
# Monitor webhook events
tail -f /var/log/reflector/app.log | grep jitsi
# Check room participant counts
curl "https://your-reflector-domain.com/v1/rooms" \
-H "Authorization: Bearer $TOKEN" | jq '.data[].num_clients'
```
### Performance Optimization
#### For High-Concurrent Usage
1. **Jitsi Videobridge Tuning:**
```bash
# /etc/jitsi/videobridge/sip-communicator.properties
org.jitsi.videobridge.STATISTICS_INTERVAL=5000
org.jitsi.videobridge.load.INITIAL_STREAM_LIMIT=50
```
2. **Database Connection Pooling:**
```python
# In your Reflector settings
DATABASE_POOL_SIZE=20
DATABASE_MAX_OVERFLOW=30
```
3. **Redis Configuration:**
```bash
# For webhook event caching
REDIS_URL=redis://localhost:6379/1
WEBHOOK_EVENT_TTL=3600
```
## Security Considerations
### Network Security
- Use HTTPS/WSS for all communications
- Implement proper firewall rules
- Consider VPN for server-to-server communication
### Authentication Security
- Rotate JWT secrets regularly
- Use strong webhook secrets (32+ characters)
- Implement rate limiting on webhook endpoints
### Recording Security
- Encrypt recordings at rest
- Implement access controls for recording files
- Regular security audits of file permissions
## Support
For additional support:
1. **Reflector Issues:** Check GitHub issues or create new ones
2. **Jitsi Community:** [Community Forum](https://community.jitsi.org/)
3. **Documentation:** [Jitsi Developer Guide](https://jitsi.github.io/handbook/)
## Migration from Whereby
If migrating from Whereby integration:
1. Update existing rooms to use "jitsi" platform
2. Verify webhook configurations are updated
3. Test recording workflows thoroughly
4. Monitor participant event accuracy
5. Update any custom integrations using meeting APIs
The platform abstraction layer ensures smooth migration with minimal API changes.

View File

@@ -1,41 +1,78 @@
import os
import tempfile
import sys
import threading
import uuid
from typing import Generator, Mapping, NamedTuple, NewType, TypedDict
from urllib.parse import urlparse
import modal
from pydantic import BaseModel
MODELS_DIR = "/models"
MODEL_NAME = "large-v2"
MODEL_COMPUTE_TYPE: str = "float16"
MODEL_NUM_WORKERS: int = 1
MINUTES = 60 # seconds
SAMPLERATE = 16000
UPLOADS_PATH = "/uploads"
CACHE_PATH = "/models"
SUPPORTED_FILE_EXTENSIONS = ["mp3", "mp4", "mpeg", "mpga", "m4a", "wav", "webm"]
VAD_CONFIG = {
"batch_max_duration": 30.0,
"silence_padding": 0.5,
"window_size": 512,
}
volume = modal.Volume.from_name("models", create_if_missing=True)
WhisperUniqFilename = NewType("WhisperUniqFilename", str)
AudioFileExtension = NewType("AudioFileExtension", str)
app = modal.App("reflector-transcriber")
model_cache = modal.Volume.from_name("models", create_if_missing=True)
upload_volume = modal.Volume.from_name("whisper-uploads", create_if_missing=True)
class TimeSegment(NamedTuple):
"""Represents a time segment with start and end times."""
start: float
end: float
class AudioSegment(NamedTuple):
"""Represents an audio segment with timing and audio data."""
start: float
end: float
audio: any
class TranscriptResult(NamedTuple):
"""Represents a transcription result with text and word timings."""
text: str
words: list["WordTiming"]
class WordTiming(TypedDict):
"""Represents a word with its timing information."""
word: str
start: float
end: float
def download_model():
from faster_whisper import download_model
volume.reload()
model_cache.reload()
download_model(MODEL_NAME, cache_dir=MODELS_DIR)
download_model(MODEL_NAME, cache_dir=CACHE_PATH)
volume.commit()
model_cache.commit()
image = (
modal.Image.debian_slim(python_version="3.12")
.pip_install(
"huggingface_hub==0.27.1",
"hf-transfer==0.1.9",
"torch==2.5.1",
"faster-whisper==1.1.1",
)
.env(
{
"HF_HUB_ENABLE_HF_TRANSFER": "1",
@@ -45,19 +82,98 @@ image = (
),
}
)
.run_function(download_model, volumes={MODELS_DIR: volume})
.apt_install("ffmpeg")
.pip_install(
"huggingface_hub==0.27.1",
"hf-transfer==0.1.9",
"torch==2.5.1",
"faster-whisper==1.1.1",
"fastapi==0.115.12",
"requests",
"librosa==0.10.1",
"numpy<2",
"silero-vad==5.1.0",
)
.run_function(download_model, volumes={CACHE_PATH: model_cache})
)
def detect_audio_format(url: str, headers: Mapping[str, str]) -> AudioFileExtension:
parsed_url = urlparse(url)
url_path = parsed_url.path
for ext in SUPPORTED_FILE_EXTENSIONS:
if url_path.lower().endswith(f".{ext}"):
return AudioFileExtension(ext)
content_type = headers.get("content-type", "").lower()
if "audio/mpeg" in content_type or "audio/mp3" in content_type:
return AudioFileExtension("mp3")
if "audio/wav" in content_type:
return AudioFileExtension("wav")
if "audio/mp4" in content_type:
return AudioFileExtension("mp4")
raise ValueError(
f"Unsupported audio format for URL: {url}. "
f"Supported extensions: {', '.join(SUPPORTED_FILE_EXTENSIONS)}"
)
def download_audio_to_volume(
audio_file_url: str,
) -> tuple[WhisperUniqFilename, AudioFileExtension]:
import requests
from fastapi import HTTPException
response = requests.head(audio_file_url, allow_redirects=True)
if response.status_code == 404:
raise HTTPException(status_code=404, detail="Audio file not found")
response = requests.get(audio_file_url, allow_redirects=True)
response.raise_for_status()
audio_suffix = detect_audio_format(audio_file_url, response.headers)
unique_filename = WhisperUniqFilename(f"{uuid.uuid4()}.{audio_suffix}")
file_path = f"{UPLOADS_PATH}/{unique_filename}"
with open(file_path, "wb") as f:
f.write(response.content)
upload_volume.commit()
return unique_filename, audio_suffix
def pad_audio(audio_array, sample_rate: int = SAMPLERATE):
"""Add 0.5s of silence if audio is shorter than the silence_padding window.
Whisper does not require this strictly, but aligning behavior with Parakeet
avoids edge-case crashes on extremely short inputs and makes comparisons easier.
"""
import numpy as np
audio_duration = len(audio_array) / sample_rate
if audio_duration < VAD_CONFIG["silence_padding"]:
silence_samples = int(sample_rate * VAD_CONFIG["silence_padding"])
silence = np.zeros(silence_samples, dtype=np.float32)
return np.concatenate([audio_array, silence])
return audio_array
@app.cls(
gpu="A10G",
timeout=5 * MINUTES,
scaledown_window=5 * MINUTES,
allow_concurrent_inputs=6,
image=image,
volumes={MODELS_DIR: volume},
volumes={CACHE_PATH: model_cache, UPLOADS_PATH: upload_volume},
)
class Transcriber:
@modal.concurrent(max_inputs=10)
class TranscriberWhisperLive:
"""Live transcriber class for small audio segments (A10G).
Mirrors the Parakeet live class API but uses Faster-Whisper under the hood.
"""
@modal.enter()
def enter(self):
import faster_whisper
@@ -71,23 +187,200 @@ class Transcriber:
device=self.device,
compute_type=MODEL_COMPUTE_TYPE,
num_workers=MODEL_NUM_WORKERS,
download_root=MODELS_DIR,
download_root=CACHE_PATH,
local_files_only=True,
)
print(f"Model is on device: {self.device}")
@modal.method()
def transcribe_segment(
self,
audio_data: str,
audio_suffix: str,
language: str,
filename: str,
language: str = "en",
):
with tempfile.NamedTemporaryFile("wb+", suffix=f".{audio_suffix}") as fp:
fp.write(audio_data)
"""Transcribe a single uploaded audio file by filename."""
upload_volume.reload()
file_path = f"{UPLOADS_PATH}/{filename}"
if not os.path.exists(file_path):
raise FileNotFoundError(f"File not found: {file_path}")
with self.lock:
with NoStdStreams():
segments, _ = self.model.transcribe(
file_path,
language=language,
beam_size=5,
word_timestamps=True,
vad_filter=True,
vad_parameters={"min_silence_duration_ms": 500},
)
segments = list(segments)
text = "".join(segment.text for segment in segments).strip()
words = [
{
"word": word.word,
"start": round(float(word.start), 2),
"end": round(float(word.end), 2),
}
for segment in segments
for word in segment.words
]
return {"text": text, "words": words}
@modal.method()
def transcribe_batch(
self,
filenames: list[str],
language: str = "en",
):
"""Transcribe multiple uploaded audio files and return per-file results."""
upload_volume.reload()
results = []
for filename in filenames:
file_path = f"{UPLOADS_PATH}/{filename}"
if not os.path.exists(file_path):
raise FileNotFoundError(f"Batch file not found: {file_path}")
with self.lock:
with NoStdStreams():
segments, _ = self.model.transcribe(
file_path,
language=language,
beam_size=5,
word_timestamps=True,
vad_filter=True,
vad_parameters={"min_silence_duration_ms": 500},
)
segments = list(segments)
text = "".join(seg.text for seg in segments).strip()
words = [
{
"word": w.word,
"start": round(float(w.start), 2),
"end": round(float(w.end), 2),
}
for seg in segments
for w in seg.words
]
results.append(
{
"filename": filename,
"text": text,
"words": words,
}
)
return results
@app.cls(
gpu="L40S",
timeout=15 * MINUTES,
image=image,
volumes={CACHE_PATH: model_cache, UPLOADS_PATH: upload_volume},
)
class TranscriberWhisperFile:
"""File transcriber for larger/longer audio, using VAD-driven batching (L40S)."""
@modal.enter()
def enter(self):
import faster_whisper
import torch
from silero_vad import load_silero_vad
self.lock = threading.Lock()
self.use_gpu = torch.cuda.is_available()
self.device = "cuda" if self.use_gpu else "cpu"
self.model = faster_whisper.WhisperModel(
MODEL_NAME,
device=self.device,
compute_type=MODEL_COMPUTE_TYPE,
num_workers=MODEL_NUM_WORKERS,
download_root=CACHE_PATH,
local_files_only=True,
)
self.vad_model = load_silero_vad(onnx=False)
@modal.method()
def transcribe_segment(
self, filename: str, timestamp_offset: float = 0.0, language: str = "en"
):
import librosa
import numpy as np
from silero_vad import VADIterator
def vad_segments(
audio_array,
sample_rate: int = SAMPLERATE,
window_size: int = VAD_CONFIG["window_size"],
) -> Generator[TimeSegment, None, None]:
"""Generate speech segments as TimeSegment using Silero VAD."""
iterator = VADIterator(self.vad_model, sampling_rate=sample_rate)
start = None
for i in range(0, len(audio_array), window_size):
chunk = audio_array[i : i + window_size]
if len(chunk) < window_size:
chunk = np.pad(
chunk, (0, window_size - len(chunk)), mode="constant"
)
speech = iterator(chunk)
if not speech:
continue
if "start" in speech:
start = speech["start"]
continue
if "end" in speech and start is not None:
end = speech["end"]
yield TimeSegment(
start / float(SAMPLERATE), end / float(SAMPLERATE)
)
start = None
iterator.reset_states()
upload_volume.reload()
file_path = f"{UPLOADS_PATH}/{filename}"
if not os.path.exists(file_path):
raise FileNotFoundError(f"File not found: {file_path}")
audio_array, _sr = librosa.load(file_path, sr=SAMPLERATE, mono=True)
# Batch segments up to ~30s windows by merging contiguous VAD segments
merged_batches: list[TimeSegment] = []
batch_start = None
batch_end = None
max_duration = VAD_CONFIG["batch_max_duration"]
for segment in vad_segments(audio_array):
seg_start, seg_end = segment.start, segment.end
if batch_start is None:
batch_start, batch_end = seg_start, seg_end
continue
if seg_end - batch_start <= max_duration:
batch_end = seg_end
else:
merged_batches.append(TimeSegment(batch_start, batch_end))
batch_start, batch_end = seg_start, seg_end
if batch_start is not None and batch_end is not None:
merged_batches.append(TimeSegment(batch_start, batch_end))
all_text = []
all_words = []
for segment in merged_batches:
start_time, end_time = segment.start, segment.end
s_idx = int(start_time * SAMPLERATE)
e_idx = int(end_time * SAMPLERATE)
segment = audio_array[s_idx:e_idx]
segment = pad_audio(segment, SAMPLERATE)
with self.lock:
segments, _ = self.model.transcribe(
fp.name,
segment,
language=language,
beam_size=5,
word_timestamps=True,
@@ -96,66 +389,220 @@ class Transcriber:
)
segments = list(segments)
text = "".join(segment.text for segment in segments)
text = "".join(seg.text for seg in segments).strip()
words = [
{"word": word.word, "start": word.start, "end": word.end}
for segment in segments
for word in segment.words
{
"word": w.word,
"start": round(float(w.start) + start_time + timestamp_offset, 2),
"end": round(float(w.end) + start_time + timestamp_offset, 2),
}
for seg in segments
for w in seg.words
]
if text:
all_text.append(text)
all_words.extend(words)
return {"text": text, "words": words}
return {"text": " ".join(all_text), "words": all_words}
def detect_audio_format(url: str, headers: dict) -> str:
from urllib.parse import urlparse
from fastapi import HTTPException
url_path = urlparse(url).path
for ext in SUPPORTED_FILE_EXTENSIONS:
if url_path.lower().endswith(f".{ext}"):
return ext
content_type = headers.get("content-type", "").lower()
if "audio/mpeg" in content_type or "audio/mp3" in content_type:
return "mp3"
if "audio/wav" in content_type:
return "wav"
if "audio/mp4" in content_type:
return "mp4"
raise HTTPException(
status_code=400,
detail=(
f"Unsupported audio format for URL. Supported extensions: {', '.join(SUPPORTED_FILE_EXTENSIONS)}"
),
)
def download_audio_to_volume(audio_file_url: str) -> tuple[str, str]:
import requests
from fastapi import HTTPException
response = requests.head(audio_file_url, allow_redirects=True)
if response.status_code == 404:
raise HTTPException(status_code=404, detail="Audio file not found")
response = requests.get(audio_file_url, allow_redirects=True)
response.raise_for_status()
audio_suffix = detect_audio_format(audio_file_url, response.headers)
unique_filename = f"{uuid.uuid4()}.{audio_suffix}"
file_path = f"{UPLOADS_PATH}/{unique_filename}"
with open(file_path, "wb") as f:
f.write(response.content)
upload_volume.commit()
return unique_filename, audio_suffix
@app.function(
scaledown_window=60,
timeout=60,
allow_concurrent_inputs=40,
timeout=600,
secrets=[
modal.Secret.from_name("reflector-gpu"),
],
volumes={MODELS_DIR: volume},
volumes={CACHE_PATH: model_cache, UPLOADS_PATH: upload_volume},
image=image,
)
@modal.concurrent(max_inputs=40)
@modal.asgi_app()
def web():
from fastapi import Body, Depends, FastAPI, HTTPException, UploadFile, status
from fastapi import (
Body,
Depends,
FastAPI,
Form,
HTTPException,
UploadFile,
status,
)
from fastapi.security import OAuth2PasswordBearer
from typing_extensions import Annotated
transcriber = Transcriber()
transcriber_live = TranscriberWhisperLive()
transcriber_file = TranscriberWhisperFile()
app = FastAPI()
oauth2_scheme = OAuth2PasswordBearer(tokenUrl="token")
supported_file_types = ["mp3", "mp4", "mpeg", "mpga", "m4a", "wav", "webm"]
def apikey_auth(apikey: str = Depends(oauth2_scheme)):
if apikey != os.environ["REFLECTOR_GPU_APIKEY"]:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Invalid API key",
headers={"WWW-Authenticate": "Bearer"},
)
if apikey == os.environ["REFLECTOR_GPU_APIKEY"]:
return
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Invalid API key",
headers={"WWW-Authenticate": "Bearer"},
)
class TranscriptResponse(BaseModel):
result: dict
class TranscriptResponse(dict):
pass
@app.post("/v1/audio/transcriptions", dependencies=[Depends(apikey_auth)])
def transcribe(
file: UploadFile,
model: str = "whisper-1",
language: Annotated[str, Body(...)] = "en",
) -> TranscriptResponse:
audio_data = file.file.read()
audio_suffix = file.filename.split(".")[-1]
assert audio_suffix in supported_file_types
file: UploadFile = None,
files: list[UploadFile] | None = None,
model: str = Form(MODEL_NAME),
language: str = Form("en"),
batch: bool = Form(False),
):
if not file and not files:
raise HTTPException(
status_code=400, detail="Either 'file' or 'files' parameter is required"
)
if batch and not files:
raise HTTPException(
status_code=400, detail="Batch transcription requires 'files'"
)
func = transcriber.transcribe_segment.spawn(
audio_data=audio_data,
audio_suffix=audio_suffix,
language=language,
)
result = func.get()
return result
upload_files = [file] if file else files
uploaded_filenames: list[str] = []
for upload_file in upload_files:
audio_suffix = upload_file.filename.split(".")[-1]
if audio_suffix not in SUPPORTED_FILE_EXTENSIONS:
raise HTTPException(
status_code=400,
detail=(
f"Unsupported audio format. Supported extensions: {', '.join(SUPPORTED_FILE_EXTENSIONS)}"
),
)
unique_filename = f"{uuid.uuid4()}.{audio_suffix}"
file_path = f"{UPLOADS_PATH}/{unique_filename}"
with open(file_path, "wb") as f:
content = upload_file.file.read()
f.write(content)
uploaded_filenames.append(unique_filename)
upload_volume.commit()
try:
if batch and len(upload_files) > 1:
func = transcriber_live.transcribe_batch.spawn(
filenames=uploaded_filenames,
language=language,
)
results = func.get()
return {"results": results}
results = []
for filename in uploaded_filenames:
func = transcriber_live.transcribe_segment.spawn(
filename=filename,
language=language,
)
result = func.get()
result["filename"] = filename
results.append(result)
return {"results": results} if len(results) > 1 else results[0]
finally:
for filename in uploaded_filenames:
try:
file_path = f"{UPLOADS_PATH}/{filename}"
os.remove(file_path)
except Exception:
pass
upload_volume.commit()
@app.post("/v1/audio/transcriptions-from-url", dependencies=[Depends(apikey_auth)])
def transcribe_from_url(
audio_file_url: str = Body(
..., description="URL of the audio file to transcribe"
),
model: str = Body(MODEL_NAME),
language: str = Body("en"),
timestamp_offset: float = Body(0.0),
):
unique_filename, _audio_suffix = download_audio_to_volume(audio_file_url)
try:
func = transcriber_file.transcribe_segment.spawn(
filename=unique_filename,
timestamp_offset=timestamp_offset,
language=language,
)
result = func.get()
return result
finally:
try:
file_path = f"{UPLOADS_PATH}/{unique_filename}"
os.remove(file_path)
upload_volume.commit()
except Exception:
pass
return app
class NoStdStreams:
def __init__(self):
self.devnull = open(os.devnull, "w")
def __enter__(self):
self._stdout, self._stderr = sys.stdout, sys.stderr
self._stdout.flush()
self._stderr.flush()
sys.stdout, sys.stderr = self.devnull, self.devnull
def __exit__(self, exc_type, exc_value, traceback):
sys.stdout, sys.stderr = self._stdout, self._stderr
self.devnull.close()

View File

@@ -0,0 +1,36 @@
"""remove user_id from meeting table
Revision ID: 0ce521cda2ee
Revises: 6dec9fb5b46c
Create Date: 2025-09-10 12:40:55.688899
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "0ce521cda2ee"
down_revision: Union[str, None] = "6dec9fb5b46c"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
with op.batch_alter_table("meeting", schema=None) as batch_op:
batch_op.drop_column("user_id")
# ### end Alembic commands ###
def downgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
with op.batch_alter_table("meeting", schema=None) as batch_op:
batch_op.add_column(
sa.Column("user_id", sa.VARCHAR(), autoincrement=False, nullable=True)
)
# ### end Alembic commands ###

View File

@@ -0,0 +1,38 @@
"""Add events column to meetings table
Revision ID: 2890b5104577
Revises: 6e6ea8e607c5
Create Date: 2025-09-02 17:51:41.620777
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "2890b5104577"
down_revision: Union[str, None] = "6e6ea8e607c5"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
with op.batch_alter_table("meeting", schema=None) as batch_op:
batch_op.add_column(
sa.Column(
"events", sa.JSON(), server_default=sa.text("'[]'"), nullable=False
)
)
# ### end Alembic commands ###
def downgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
with op.batch_alter_table("meeting", schema=None) as batch_op:
batch_op.drop_column("events")
# ### end Alembic commands ###

View File

@@ -0,0 +1,32 @@
"""clean up orphaned room_id references in meeting table
Revision ID: 2ae3db106d4e
Revises: def1b5867d4c
Create Date: 2025-09-11 10:35:15.759967
"""
from typing import Sequence, Union
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "2ae3db106d4e"
down_revision: Union[str, None] = "def1b5867d4c"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# Set room_id to NULL for meetings that reference non-existent rooms
op.execute("""
UPDATE meeting
SET room_id = NULL
WHERE room_id IS NOT NULL
AND room_id NOT IN (SELECT id FROM room WHERE id IS NOT NULL)
""")
def downgrade() -> None:
# Cannot restore orphaned references - no operation needed
pass

View File

@@ -0,0 +1,38 @@
"""make meeting room_id required and add foreign key
Revision ID: 6dec9fb5b46c
Revises: 61882a919591
Create Date: 2025-09-10 10:47:06.006819
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "6dec9fb5b46c"
down_revision: Union[str, None] = "61882a919591"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
with op.batch_alter_table("meeting", schema=None) as batch_op:
batch_op.alter_column("room_id", existing_type=sa.VARCHAR(), nullable=False)
batch_op.create_foreign_key(
None, "room", ["room_id"], ["id"], ondelete="CASCADE"
)
# ### end Alembic commands ###
def downgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
with op.batch_alter_table("meeting", schema=None) as batch_op:
batch_op.drop_constraint("meeting_room_id_fkey", type_="foreignkey")
batch_op.alter_column("room_id", existing_type=sa.VARCHAR(), nullable=True)
# ### end Alembic commands ###

View File

@@ -0,0 +1,44 @@
"""Add VideoPlatform enum for rooms and meetings
Revision ID: 6e6ea8e607c5
Revises: 61882a919591
Create Date: 2025-09-02 17:33:21.022214
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "6e6ea8e607c5"
down_revision: Union[str, None] = "61882a919591"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
with op.batch_alter_table("meeting", schema=None) as batch_op:
batch_op.add_column(
sa.Column("platform", sa.String(), server_default="whereby", nullable=False)
)
with op.batch_alter_table("room", schema=None) as batch_op:
batch_op.add_column(
sa.Column("platform", sa.String(), server_default="whereby", nullable=False)
)
# ### end Alembic commands ###
def downgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
with op.batch_alter_table("room", schema=None) as batch_op:
batch_op.drop_column("platform")
with op.batch_alter_table("meeting", schema=None) as batch_op:
batch_op.drop_column("platform")
# ### end Alembic commands ###

View File

@@ -0,0 +1,34 @@
"""make meeting room_id nullable but keep foreign key
Revision ID: def1b5867d4c
Revises: 0ce521cda2ee
Create Date: 2025-09-11 09:42:18.697264
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "def1b5867d4c"
down_revision: Union[str, None] = "0ce521cda2ee"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
with op.batch_alter_table("meeting", schema=None) as batch_op:
batch_op.alter_column("room_id", existing_type=sa.VARCHAR(), nullable=True)
# ### end Alembic commands ###
def downgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
with op.batch_alter_table("meeting", schema=None) as batch_op:
batch_op.alter_column("room_id", existing_type=sa.VARCHAR(), nullable=False)
# ### end Alembic commands ###

View File

@@ -40,6 +40,7 @@ dependencies = [
"llama-index-llms-openai-like>=0.4.0",
"pytest-env>=1.1.5",
"webvtt-py>=0.5.0",
"PyJWT>=2.8.0",
]
[dependency-groups]

View File

@@ -12,6 +12,9 @@ from reflector.events import subscribers_shutdown, subscribers_startup
from reflector.logger import logger
from reflector.metrics import metrics_init
from reflector.settings import settings
from reflector.video_platforms.jitsi import router as jitsi_router
from reflector.video_platforms.whereby import router as whereby_router
from reflector.views.jibri_webhook import router as jibri_webhook_router
from reflector.views.meetings import router as meetings_router
from reflector.views.rooms import router as rooms_router
from reflector.views.rtc_offer import router as rtc_offer_router
@@ -26,7 +29,6 @@ from reflector.views.transcripts_upload import router as transcripts_upload_rout
from reflector.views.transcripts_webrtc import router as transcripts_webrtc_router
from reflector.views.transcripts_websocket import router as transcripts_websocket_router
from reflector.views.user import router as user_router
from reflector.views.whereby import router as whereby_router
from reflector.views.zulip import router as zulip_router
try:
@@ -86,6 +88,8 @@ app.include_router(transcripts_process_router, prefix="/v1")
app.include_router(user_router, prefix="/v1")
app.include_router(zulip_router, prefix="/v1")
app.include_router(whereby_router, prefix="/v1")
app.include_router(jitsi_router, prefix="/v1")
app.include_router(jibri_webhook_router) # No /v1 prefix, uses /api/v1/jibri
add_pagination(app)
# prepare celery

View File

@@ -1,12 +1,11 @@
from datetime import datetime
from typing import Literal
from datetime import datetime, timezone
from typing import Any, Dict, List, Literal
import sqlalchemy as sa
from fastapi import HTTPException
from pydantic import BaseModel, Field
from reflector.db import get_database, metadata
from reflector.db.rooms import Room
from reflector.db.rooms import Room, VideoPlatform
from reflector.utils import generate_uuid4
meetings = sa.Table(
@@ -18,8 +17,12 @@ meetings = sa.Table(
sa.Column("host_room_url", sa.String),
sa.Column("start_date", sa.DateTime(timezone=True)),
sa.Column("end_date", sa.DateTime(timezone=True)),
sa.Column("user_id", sa.String),
sa.Column("room_id", sa.String),
sa.Column(
"room_id",
sa.String,
sa.ForeignKey("room.id", ondelete="CASCADE"),
nullable=True,
),
sa.Column("is_locked", sa.Boolean, nullable=False, server_default=sa.false()),
sa.Column("room_mode", sa.String, nullable=False, server_default="normal"),
sa.Column("recording_type", sa.String, nullable=False, server_default="cloud"),
@@ -41,6 +44,8 @@ meetings = sa.Table(
nullable=False,
server_default=sa.true(),
),
sa.Column("platform", sa.String, nullable=False, server_default="whereby"),
sa.Column("events", sa.JSON, nullable=False, server_default=sa.text("'[]'")),
sa.Index("idx_meeting_room_id", "room_id"),
sa.Index(
"idx_one_active_meeting_per_room",
@@ -81,8 +86,7 @@ class Meeting(BaseModel):
host_room_url: str
start_date: datetime
end_date: datetime
user_id: str | None = None
room_id: str | None = None
room_id: str | None
is_locked: bool = False
room_mode: Literal["normal", "group"] = "normal"
recording_type: Literal["none", "local", "cloud"] = "cloud"
@@ -90,6 +94,8 @@ class Meeting(BaseModel):
"none", "prompt", "automatic", "automatic-2nd-participant"
] = "automatic-2nd-participant"
num_clients: int = 0
platform: VideoPlatform = VideoPlatform.WHEREBY
events: List[Dict[str, Any]] = Field(default_factory=list)
class MeetingController:
@@ -101,12 +107,8 @@ class MeetingController:
host_room_url: str,
start_date: datetime,
end_date: datetime,
user_id: str,
room: Room,
):
"""
Create a new meeting
"""
meeting = Meeting(
id=id,
room_name=room_name,
@@ -114,31 +116,25 @@ class MeetingController:
host_room_url=host_room_url,
start_date=start_date,
end_date=end_date,
user_id=user_id,
room_id=room.id,
is_locked=room.is_locked,
room_mode=room.room_mode,
recording_type=room.recording_type,
recording_trigger=room.recording_trigger,
platform=room.platform,
)
query = meetings.insert().values(**meeting.model_dump())
await get_database().execute(query)
return meeting
async def get_all_active(self) -> list[Meeting]:
"""
Get active meetings.
"""
query = meetings.select().where(meetings.c.is_active)
return await get_database().fetch_all(query)
async def get_by_room_name(
self,
room_name: str,
) -> Meeting:
"""
Get a meeting by room name.
"""
) -> Meeting | None:
query = meetings.select().where(meetings.c.room_name == room_name)
result = await get_database().fetch_one(query)
if not result:
@@ -146,10 +142,7 @@ class MeetingController:
return Meeting(**result)
async def get_active(self, room: Room, current_time: datetime) -> Meeting:
"""
Get latest active meeting for a room.
"""
async def get_active(self, room: Room, current_time: datetime) -> Meeting | None:
end_date = getattr(meetings.c, "end_date")
query = (
meetings.select()
@@ -169,36 +162,78 @@ class MeetingController:
return Meeting(**result)
async def get_by_id(self, meeting_id: str, **kwargs) -> Meeting | None:
"""
Get a meeting by id
"""
query = meetings.select().where(meetings.c.id == meeting_id)
result = await get_database().fetch_one(query)
if not result:
return None
return Meeting(**result)
async def get_by_id_for_http(self, meeting_id: str, user_id: str | None) -> Meeting:
"""
Get a meeting by ID for HTTP request.
If not found, it will raise a 404 error.
"""
query = meetings.select().where(meetings.c.id == meeting_id)
result = await get_database().fetch_one(query)
if not result:
raise HTTPException(status_code=404, detail="Meeting not found")
meeting = Meeting(**result)
if result["user_id"] != user_id:
meeting.host_room_url = ""
return meeting
async def update_meeting(self, meeting_id: str, **kwargs):
query = meetings.update().where(meetings.c.id == meeting_id).values(**kwargs)
await get_database().execute(query)
async def add_event(
self, meeting_id: str, event_type: str, event_data: Dict[str, Any] = None
):
"""Add an event to a meeting's events list."""
if event_data is None:
event_data = {}
event = {
"type": event_type,
"timestamp": datetime.now(tz=timezone.utc).isoformat(),
"data": event_data,
}
# Get current events
query = meetings.select().where(meetings.c.id == meeting_id)
result = await get_database().fetch_one(query)
if not result:
return
current_events = result["events"] or []
current_events.append(event)
# Update with new events list
update_query = (
meetings.update()
.where(meetings.c.id == meeting_id)
.values(events=current_events)
)
await get_database().execute(update_query)
async def participant_joined(
self, meeting_id: str, participant_data: Dict[str, Any] = None
):
"""Record a participant joined event."""
await self.add_event(meeting_id, "participant_joined", participant_data)
async def participant_left(
self, meeting_id: str, participant_data: Dict[str, Any] = None
):
"""Record a participant left event."""
await self.add_event(meeting_id, "participant_left", participant_data)
async def recording_started(
self, meeting_id: str, recording_data: Dict[str, Any] = None
):
"""Record a recording started event."""
await self.add_event(meeting_id, "recording_started", recording_data)
async def recording_stopped(
self, meeting_id: str, recording_data: Dict[str, Any] = None
):
"""Record a recording stopped event."""
await self.add_event(meeting_id, "recording_stopped", recording_data)
async def get_events(self, meeting_id: str) -> List[Dict[str, Any]]:
"""Get all events for a meeting."""
query = meetings.select().where(meetings.c.id == meeting_id)
result = await get_database().fetch_one(query)
if not result:
return []
return result["events"] or []
class MeetingConsentController:
async def get_by_meeting_id(self, meeting_id: str) -> list[MeetingConsent]:
@@ -219,7 +254,7 @@ class MeetingConsentController:
result = await get_database().fetch_one(query)
if result is None:
return None
return MeetingConsent(**result) if result else None
return MeetingConsent(**result)
async def upsert(self, consent: MeetingConsent) -> MeetingConsent:
"""Create new consent or update existing one for authenticated users"""

View File

@@ -1,5 +1,6 @@
import secrets
from datetime import datetime, timezone
from enum import StrEnum
from sqlite3 import IntegrityError
from typing import Literal
@@ -11,6 +12,12 @@ from sqlalchemy.sql import false, or_
from reflector.db import get_database, metadata
from reflector.utils import generate_uuid4
class VideoPlatform(StrEnum):
WHEREBY = "whereby"
JITSI = "jitsi"
rooms = sqlalchemy.Table(
"room",
metadata,
@@ -43,6 +50,9 @@ rooms = sqlalchemy.Table(
),
sqlalchemy.Column("webhook_url", sqlalchemy.String, nullable=True),
sqlalchemy.Column("webhook_secret", sqlalchemy.String, nullable=True),
sqlalchemy.Column(
"platform", sqlalchemy.String, nullable=False, server_default="whereby"
),
sqlalchemy.Index("idx_room_is_shared", "is_shared"),
)
@@ -64,6 +74,7 @@ class Room(BaseModel):
is_shared: bool = False
webhook_url: str | None = None
webhook_secret: str | None = None
platform: VideoPlatform = VideoPlatform.WHEREBY
class RoomController:
@@ -114,6 +125,7 @@ class RoomController:
is_shared: bool,
webhook_url: str = "",
webhook_secret: str = "",
platform: str = "whereby",
):
"""
Add a new room
@@ -134,6 +146,7 @@ class RoomController:
is_shared=is_shared,
webhook_url=webhook_url,
webhook_secret=webhook_secret,
platform=platform,
)
query = rooms.insert().values(**room.model_dump())
try:

View File

@@ -23,7 +23,7 @@ from pydantic import (
from reflector.db import get_database
from reflector.db.rooms import rooms
from reflector.db.transcripts import SourceKind, transcripts
from reflector.db.transcripts import SourceKind, TranscriptStatus, transcripts
from reflector.db.utils import is_postgresql
from reflector.logger import logger
from reflector.utils.string import NonEmptyString, try_parse_non_empty_string
@@ -161,7 +161,7 @@ class SearchResult(BaseModel):
room_name: str | None = None
source_kind: SourceKind
created_at: datetime
status: str = Field(..., min_length=1)
status: TranscriptStatus = Field(..., min_length=1)
rank: float = Field(..., ge=0, le=1)
duration: NonNegativeFloat | None = Field(..., description="Duration in seconds")
search_snippets: list[str] = Field(

View File

@@ -0,0 +1,227 @@
import json
from pathlib import Path
from typing import Any, Dict, List, Literal, Optional, Union
from pydantic import BaseModel
from typing_extensions import TypedDict
class ParticipantInfo(BaseModel):
jid: str
nick: str
id: str
is_moderator: bool = False
class ParticipantLeftInfo(BaseModel):
jid: str
nick: Optional[str] = None
duration_seconds: Optional[int] = None
class RoomCreatedEvent(BaseModel):
type: Literal["room_created"]
timestamp: int
room_name: str
room_jid: str
meeting_url: str
class RecordingStartedEvent(BaseModel):
type: Literal["recording_started"]
timestamp: int
room_name: str
session_id: str
jibri_jid: str
class RecordingStoppedEvent(BaseModel):
type: Literal["recording_stopped"]
timestamp: int
room_name: str
session_id: str
meeting_url: str
class ParticipantJoinedEvent(BaseModel):
type: Literal["participant_joined"]
timestamp: int
room_name: str
participant: ParticipantInfo
class ParticipantLeftEvent(BaseModel):
type: Literal["participant_left"]
timestamp: int
room_name: str
participant: ParticipantLeftInfo
class SpeakerActiveEvent(BaseModel):
type: Literal["speaker_active"]
timestamp: int
room_name: str
speaker_jid: str
speaker_nick: str
duration: int
class DominantSpeakerChangedEvent(BaseModel):
type: Literal["dominant_speaker_changed"]
timestamp: int
room_name: str
previous: str
current: str
JitsiEvent = Union[
RoomCreatedEvent,
RecordingStartedEvent,
RecordingStoppedEvent,
ParticipantJoinedEvent,
ParticipantLeftEvent,
SpeakerActiveEvent,
DominantSpeakerChangedEvent,
]
class RoomInfo(TypedDict):
name: str
jid: str
created_at: int
meeting_url: str
recording_stopped_at: Optional[int]
class ParticipantData(TypedDict):
jid: str
nick: str
id: str
is_moderator: bool
joined_at: int
left_at: Optional[int]
duration: Optional[int]
events: List[str]
class SpeakerStats(TypedDict):
total_time: int
nick: str
class ParsedMetadata(TypedDict):
room: RoomInfo
participants: List[ParticipantData]
speaker_stats: Dict[str, SpeakerStats]
event_count: int
class JitsiEventParser:
def parse_event(self, event_data: Dict[str, Any]) -> Optional[JitsiEvent]:
event_type = event_data.get("type")
try:
if event_type == "room_created":
return RoomCreatedEvent(**event_data)
elif event_type == "recording_started":
return RecordingStartedEvent(**event_data)
elif event_type == "recording_stopped":
return RecordingStoppedEvent(**event_data)
elif event_type == "participant_joined":
return ParticipantJoinedEvent(**event_data)
elif event_type == "participant_left":
return ParticipantLeftEvent(**event_data)
elif event_type == "speaker_active":
return SpeakerActiveEvent(**event_data)
elif event_type == "dominant_speaker_changed":
return DominantSpeakerChangedEvent(**event_data)
else:
return None
except Exception:
return None
def parse_events_file(self, recording_path: str) -> ParsedMetadata:
events_file = Path(recording_path) / "events.jsonl"
room_info: RoomInfo = {
"name": "",
"jid": "",
"created_at": 0,
"meeting_url": "",
"recording_stopped_at": None,
}
if not events_file.exists():
return ParsedMetadata(
room=room_info, participants=[], speaker_stats={}, event_count=0
)
events: List[JitsiEvent] = []
participants: Dict[str, ParticipantData] = {}
speaker_stats: Dict[str, SpeakerStats] = {}
with open(events_file, "r") as f:
for line in f:
if not line.strip():
continue
try:
event_data = json.loads(line)
event = self.parse_event(event_data)
if event is None:
continue
events.append(event)
if isinstance(event, RoomCreatedEvent):
room_info = {
"name": event.room_name,
"jid": event.room_jid,
"created_at": event.timestamp,
"meeting_url": event.meeting_url,
"recording_stopped_at": None,
}
elif isinstance(event, ParticipantJoinedEvent):
participants[event.participant.id] = {
"jid": event.participant.jid,
"nick": event.participant.nick,
"id": event.participant.id,
"is_moderator": event.participant.is_moderator,
"joined_at": event.timestamp,
"left_at": None,
"duration": None,
"events": ["joined"],
}
elif isinstance(event, ParticipantLeftEvent):
participant_id = event.participant.jid.split("/")[0]
if participant_id in participants:
participants[participant_id]["left_at"] = event.timestamp
participants[participant_id]["duration"] = (
event.participant.duration_seconds
)
participants[participant_id]["events"].append("left")
elif isinstance(event, SpeakerActiveEvent):
if event.speaker_jid not in speaker_stats:
speaker_stats[event.speaker_jid] = {
"total_time": 0,
"nick": event.speaker_nick,
}
speaker_stats[event.speaker_jid]["total_time"] += event.duration
elif isinstance(event, RecordingStoppedEvent):
room_info["recording_stopped_at"] = event.timestamp
room_info["meeting_url"] = event.meeting_url
except (json.JSONDecodeError, Exception):
continue
return ParsedMetadata(
room=room_info,
participants=list(participants.values()),
speaker_stats=speaker_stats,
event_count=len(events),
)

View File

@@ -12,7 +12,7 @@ from pathlib import Path
import av
import structlog
from celery import shared_task
from celery import chain, shared_task
from reflector.asynctask import asynctask
from reflector.db.rooms import rooms_controller
@@ -26,6 +26,8 @@ from reflector.logger import logger
from reflector.pipelines.main_live_pipeline import (
PipelineMainBase,
broadcast_to_sockets,
task_cleanup_consent,
task_pipeline_post_to_zulip,
)
from reflector.processors import (
AudioFileWriterProcessor,
@@ -379,6 +381,28 @@ class PipelineMainFile(PipelineMainBase):
await processor.flush()
@shared_task
@asynctask
async def task_send_webhook_if_needed(*, transcript_id: str):
"""Send webhook if this is a room recording with webhook configured"""
transcript = await transcripts_controller.get_by_id(transcript_id)
if not transcript:
return
if transcript.source_kind == SourceKind.ROOM and transcript.room_id:
room = await rooms_controller.get_by_id(transcript.room_id)
if room and room.webhook_url:
logger.info(
"Dispatching webhook",
transcript_id=transcript_id,
room_id=room.id,
webhook_url=room.webhook_url,
)
send_transcript_webhook.delay(
transcript_id, room.id, event_id=uuid.uuid4().hex
)
@shared_task
@asynctask
async def task_pipeline_file_process(*, transcript_id: str):
@@ -406,16 +430,10 @@ async def task_pipeline_file_process(*, transcript_id: str):
await pipeline.set_status(transcript_id, "error")
raise
# Trigger webhook if this is a room recording with webhook configured
if transcript.source_kind == SourceKind.ROOM and transcript.room_id:
room = await rooms_controller.get_by_id(transcript.room_id)
if room and room.webhook_url:
logger.info(
"Dispatching webhook task",
transcript_id=transcript_id,
room_id=room.id,
webhook_url=room.webhook_url,
)
send_transcript_webhook.delay(
transcript_id, room.id, event_id=uuid.uuid4().hex
)
# Run post-processing chain: consent cleanup -> zulip -> webhook
post_chain = chain(
task_cleanup_consent.si(transcript_id=transcript_id),
task_pipeline_post_to_zulip.si(transcript_id=transcript_id),
task_send_webhook_if_needed.si(transcript_id=transcript_id),
)
post_chain.delay()

View File

@@ -47,6 +47,7 @@ class FileDiarizationModalProcessor(FileDiarizationProcessor):
"audio_file_url": data.audio_url,
"timestamp": 0,
},
follow_redirects=True,
)
response.raise_for_status()
diarization_data = response.json()["diarization"]

View File

@@ -54,6 +54,7 @@ class FileTranscriptModalProcessor(FileTranscriptProcessor):
"language": data.language,
"batch": True,
},
follow_redirects=True,
)
response.raise_for_status()
result = response.json()

View File

@@ -1,6 +1,8 @@
from pydantic.types import PositiveInt
from pydantic_settings import BaseSettings, SettingsConfigDict
from reflector.utils.string import NonEmptyString
class Settings(BaseSettings):
model_config = SettingsConfigDict(
@@ -120,13 +122,24 @@ class Settings(BaseSettings):
# Whereby integration
WHEREBY_API_URL: str = "https://api.whereby.dev/v1"
WHEREBY_API_KEY: str | None = None
WHEREBY_API_KEY: NonEmptyString | None = None
# Jibri integration
JIBRI_RECORDINGS_PATH: str = "/recordings"
WHEREBY_WEBHOOK_SECRET: str | None = None
AWS_WHEREBY_ACCESS_KEY_ID: str | None = None
AWS_WHEREBY_ACCESS_KEY_SECRET: str | None = None
AWS_PROCESS_RECORDING_QUEUE_URL: str | None = None
SQS_POLLING_TIMEOUT_SECONDS: int = 60
# Jitsi Meet
JITSI_DOMAIN: str = "meet.jit.si"
JITSI_JWT_SECRET: str | None = None
JITSI_WEBHOOK_SECRET: str | None = None
JITSI_APP_ID: str = "reflector"
JITSI_JWT_ISSUER: str = "reflector"
JITSI_JWT_AUDIENCE: str = "jitsi"
# Zulip integration
ZULIP_REALM: str | None = None
ZULIP_API_KEY: str | None = None

View File

@@ -10,8 +10,11 @@ NonEmptyString = Annotated[
non_empty_string_adapter = TypeAdapter(NonEmptyString)
def parse_non_empty_string(s: str) -> NonEmptyString:
return non_empty_string_adapter.validate_python(s)
def parse_non_empty_string(s: str, error: str | None = None) -> NonEmptyString:
try:
return non_empty_string_adapter.validate_python(s)
except Exception as e:
raise ValueError(f"{e}: {error}" if error else e) from e
def try_parse_non_empty_string(s: str) -> NonEmptyString | None:

View File

@@ -0,0 +1,17 @@
# Video Platform Abstraction Layer
"""
This module provides an abstraction layer for different video conferencing platforms.
It allows seamless switching between providers (Whereby, Daily.co, etc.) without
changing the core application logic.
"""
from .base import MeetingData, VideoPlatformClient, VideoPlatformConfig
from .registry import get_platform_client, register_platform
__all__ = [
"VideoPlatformClient",
"VideoPlatformConfig",
"MeetingData",
"get_platform_client",
"register_platform",
]

View File

@@ -0,0 +1,82 @@
from abc import ABC, abstractmethod
from datetime import datetime
from typing import Any, Dict, Optional
from pydantic import BaseModel
from reflector.db.rooms import Room
class MeetingData(BaseModel):
"""Standardized meeting data returned by all platforms."""
meeting_id: str
room_name: str
room_url: str
host_room_url: str
platform: str
extra_data: Dict[str, Any] = {} # Platform-specific data
class VideoPlatformConfig(BaseModel):
"""Configuration for a video platform."""
api_key: str
webhook_secret: str
api_url: Optional[str] = None
subdomain: Optional[str] = None
s3_bucket: Optional[str] = None
s3_region: Optional[str] = None
aws_role_arn: Optional[str] = None
aws_access_key_id: Optional[str] = None
aws_access_key_secret: Optional[str] = None
class VideoPlatformClient(ABC):
"""Abstract base class for video platform integrations."""
PLATFORM_NAME: str = ""
def __init__(self, config: VideoPlatformConfig):
self.config = config
@abstractmethod
async def create_meeting(
self, room_name_prefix: str, end_date: datetime, room: Room
) -> MeetingData:
"""Create a new meeting room."""
pass
@abstractmethod
async def get_room_sessions(self, room_name: str) -> Dict[str, Any]:
"""Get session information for a room."""
pass
@abstractmethod
async def delete_room(self, room_name: str) -> bool:
"""Delete a room. Returns True if successful."""
pass
@abstractmethod
async def upload_logo(self, room_name: str, logo_path: str) -> bool:
"""Upload a logo to the room. Returns True if successful."""
pass
@abstractmethod
def verify_webhook_signature(
self, body: bytes, signature: str, timestamp: Optional[str] = None
) -> bool:
"""Verify webhook signature for security."""
pass
def format_recording_config(self, room: Room) -> Dict[str, Any]:
"""Format recording configuration for the platform.
Can be overridden by specific implementations."""
if room.recording_type == "cloud" and self.config.s3_bucket:
return {
"type": room.recording_type,
"bucket": self.config.s3_bucket,
"region": self.config.s3_region,
"trigger": room.recording_trigger,
}
return {"type": room.recording_type}

View File

@@ -0,0 +1,54 @@
"""Factory for creating video platform clients based on configuration."""
from typing import TYPE_CHECKING, Literal, Optional, overload
from reflector.db.rooms import VideoPlatform
from reflector.settings import settings
from .base import VideoPlatformClient, VideoPlatformConfig
from .registry import get_platform_client
if TYPE_CHECKING:
from .jitsi import JitsiClient
from .whereby import WherebyClient
def get_platform_config(platform: str) -> VideoPlatformConfig:
"""Get configuration for a specific platform."""
if platform == VideoPlatform.WHEREBY:
return VideoPlatformConfig(
api_key=settings.WHEREBY_API_KEY or "",
webhook_secret=settings.WHEREBY_WEBHOOK_SECRET or "",
api_url=settings.WHEREBY_API_URL,
s3_bucket=settings.RECORDING_STORAGE_AWS_BUCKET_NAME,
aws_access_key_id=settings.AWS_WHEREBY_ACCESS_KEY_ID,
aws_access_key_secret=settings.AWS_WHEREBY_ACCESS_KEY_SECRET,
)
elif platform == VideoPlatform.JITSI:
return VideoPlatformConfig(
api_key="", # Jitsi uses JWT, no API key
webhook_secret=settings.JITSI_WEBHOOK_SECRET or "",
api_url=f"https://{settings.JITSI_DOMAIN}",
)
else:
raise ValueError(f"Unknown platform: {platform}")
@overload
def create_platform_client(platform: Literal["jitsi"]) -> "JitsiClient": ...
@overload
def create_platform_client(platform: Literal["whereby"]) -> "WherebyClient": ...
def create_platform_client(platform: str) -> VideoPlatformClient:
"""Create a video platform client instance."""
config = get_platform_config(platform)
return get_platform_client(platform, config)
def get_platform_for_room(room_id: Optional[str] = None) -> str:
"""Determine which platform to use for a room based on feature flags."""
# For now, default to whereby since we don't have feature flags yet
return VideoPlatform.WHEREBY

View File

@@ -0,0 +1,4 @@
from .client import JitsiClient, JitsiMeetingData
from .router import router
__all__ = ["JitsiClient", "JitsiMeetingData", "router"]

View File

@@ -0,0 +1,111 @@
import hmac
from datetime import datetime, timezone
from hashlib import sha256
from typing import Any, Dict, Optional
import jwt
from reflector.db.rooms import Room, VideoPlatform
from reflector.settings import settings
from reflector.utils import generate_uuid4
from ..base import MeetingData, VideoPlatformClient
class JitsiMeetingData(MeetingData):
@property
def user_jwt(self) -> str:
return self.extra_data.get("user_jwt", "")
@property
def host_jwt(self) -> str:
return self.extra_data.get("host_jwt", "")
@property
def domain(self) -> str:
return self.extra_data.get("domain", "")
class JitsiClient(VideoPlatformClient):
PLATFORM_NAME = VideoPlatform.JITSI
def _generate_jwt(self, room: str, moderator: bool, exp: datetime) -> str:
if not settings.JITSI_JWT_SECRET:
raise ValueError("JITSI_JWT_SECRET is required for JWT generation")
payload = {
"aud": settings.JITSI_JWT_AUDIENCE,
"iss": settings.JITSI_JWT_ISSUER,
"sub": settings.JITSI_DOMAIN,
"room": room,
"exp": int(exp.timestamp()),
"context": {
"user": {
"name": "Reflector User",
"moderator": moderator,
},
"features": {
"recording": True,
"livestreaming": False,
},
},
}
return jwt.encode(payload, settings.JITSI_JWT_SECRET, algorithm="HS256")
async def create_meeting(
self, room_name_prefix: str, end_date: datetime, room: Room
) -> JitsiMeetingData:
jitsi_room = f"reflector-{room.name}-{generate_uuid4()}"
user_jwt = self._generate_jwt(room=jitsi_room, moderator=False, exp=end_date)
host_jwt = self._generate_jwt(room=jitsi_room, moderator=True, exp=end_date)
room_url = f"https://{settings.JITSI_DOMAIN}/{jitsi_room}?jwt={user_jwt}"
host_room_url = f"https://{settings.JITSI_DOMAIN}/{jitsi_room}?jwt={host_jwt}"
return JitsiMeetingData(
meeting_id=generate_uuid4(),
room_name=jitsi_room,
room_url=room_url,
host_room_url=host_room_url,
platform=self.PLATFORM_NAME,
extra_data={
"user_jwt": user_jwt,
"host_jwt": host_jwt,
"domain": settings.JITSI_DOMAIN,
},
)
async def get_room_sessions(self, room_name: str) -> Dict[str, Any]:
return {
"roomName": room_name,
"sessions": [
{
"sessionId": generate_uuid4(),
"startTime": datetime.now(tz=timezone.utc).isoformat(),
"participants": [],
"isActive": True,
}
],
}
async def delete_room(self, room_name: str) -> bool:
return True
async def upload_logo(self, room_name: str, logo_path: str) -> bool:
return True
def verify_webhook_signature(
self, body: bytes, signature: str, timestamp: Optional[str] = None
) -> bool:
if not signature or not self.config.webhook_secret:
return False
try:
expected = hmac.new(
self.config.webhook_secret.encode(), body, sha256
).hexdigest()
return hmac.compare_digest(expected, signature)
except Exception:
return False

View File

@@ -0,0 +1,165 @@
import hmac
from datetime import datetime
from hashlib import sha256
from typing import Any, Dict
from fastapi import APIRouter, HTTPException, Request
from pydantic import BaseModel
from reflector.db.meetings import meetings_controller
from reflector.settings import settings
try:
from reflector.video_platforms import create_platform_client
except ImportError:
# PyJWT not yet installed, will be added in final task
def create_platform_client(platform: str):
return None
router = APIRouter()
class JitsiWebhookEvent(BaseModel):
event: str
room: str
timestamp: datetime
data: Dict[str, Any] = {}
class JibriRecordingEvent(BaseModel):
room_name: str
recording_file: str
recording_status: str
timestamp: datetime
def verify_jitsi_webhook_signature(body: bytes, signature: str) -> bool:
"""Verify Jitsi webhook signature using HMAC-SHA256."""
if not signature or not settings.JITSI_WEBHOOK_SECRET:
return False
try:
client = create_platform_client("jitsi")
if client is None:
# Fallback verification when platform client not available
expected = hmac.new(
settings.JITSI_WEBHOOK_SECRET.encode(), body, sha256
).hexdigest()
return hmac.compare_digest(expected, signature)
return client.verify_webhook_signature(body, signature)
except Exception:
return False
@router.post("/jitsi/events")
async def jitsi_events_webhook(event: JitsiWebhookEvent, request: Request):
"""
Handle Prosody event-sync webhooks from Jitsi Meet.
Expected event types:
- muc-occupant-joined: participant joined the room
- muc-occupant-left: participant left the room
- jibri-recording-on: recording started
- jibri-recording-off: recording stopped
"""
# Verify webhook signature
body = await request.body()
signature = request.headers.get("x-jitsi-signature", "")
if not verify_jitsi_webhook_signature(body, signature):
raise HTTPException(status_code=401, detail="Invalid webhook signature")
# Find meeting by room name
meeting = await meetings_controller.get_by_room_name(event.room)
if not meeting:
raise HTTPException(status_code=404, detail="Meeting not found")
# Handle participant events
if event.event == "muc-occupant-joined":
# Store event and update participant count
await meetings_controller.participant_joined(
meeting.id, {"timestamp": event.timestamp, "data": event.data}
)
current_count = getattr(meeting, "num_clients", 0)
await meetings_controller.update_meeting(
meeting.id, num_clients=current_count + 1
)
elif event.event == "muc-occupant-left":
# Store event and update participant count
await meetings_controller.participant_left(
meeting.id, {"timestamp": event.timestamp, "data": event.data}
)
current_count = getattr(meeting, "num_clients", 0)
await meetings_controller.update_meeting(
meeting.id, num_clients=max(0, current_count - 1)
)
elif event.event == "jibri-recording-on":
# Store recording started event
await meetings_controller.recording_started(
meeting.id, {"timestamp": event.timestamp, "data": event.data}
)
elif event.event == "jibri-recording-off":
# Store recording stopped event
await meetings_controller.recording_stopped(
meeting.id, {"timestamp": event.timestamp, "data": event.data}
)
return {"status": "ok", "event": event.event, "room": event.room}
@router.post("/jibri/recording-complete")
async def jibri_recording_complete(event: JibriRecordingEvent, request: Request):
"""
Handle Jibri recording completion webhook.
This endpoint is called by the Jibri finalize script when a recording
is completed and uploaded to storage.
"""
# Verify webhook signature
body = await request.body()
signature = request.headers.get("x-jitsi-signature", "")
if not verify_jitsi_webhook_signature(body, signature):
raise HTTPException(status_code=401, detail="Invalid webhook signature")
# Find meeting by room name
meeting = await meetings_controller.get_by_room_name(event.room_name)
if not meeting:
raise HTTPException(status_code=404, detail="Meeting not found")
# Store recording completion event
await meetings_controller.add_event(
meeting.id,
"recording_completed",
{
"recording_file": event.recording_file,
"recording_status": event.recording_status,
"timestamp": event.timestamp,
},
)
# TODO: Trigger recording processing pipeline
# This is where we would:
# 1. Download the recording file from Jibri storage
# 2. Create a transcript record in the database
# 3. Queue the audio processing tasks (chunking, transcription, etc.)
# 4. Update meeting status to indicate recording is being processed
return {
"status": "ok",
"room_name": event.room_name,
"recording_file": event.recording_file,
"message": "Recording processing queued",
}
@router.get("/jitsi/health")
async def jitsi_health_check():
"""Simple health check endpoint for Jitsi webhook configuration."""
return {
"status": "ok",
"service": "jitsi-webhooks",
"timestamp": datetime.now(tz=timezone.utc).isoformat(),
"webhook_secret_configured": bool(settings.JITSI_WEBHOOK_SECRET),
}

View File

@@ -0,0 +1,3 @@
"""Jitsi-specific worker tasks."""
# Placeholder for Jitsi recording tasks

View File

@@ -0,0 +1,56 @@
from typing import TYPE_CHECKING, Dict, Literal, Type, overload
from .base import VideoPlatformClient, VideoPlatformConfig
if TYPE_CHECKING:
from .jitsi import JitsiClient
from .whereby import WherebyClient
# Registry of available video platforms
_PLATFORMS: Dict[str, Type[VideoPlatformClient]] = {}
def register_platform(name: str, client_class: Type[VideoPlatformClient]):
"""Register a video platform implementation."""
_PLATFORMS[name.lower()] = client_class
@overload
def get_platform_client(
platform: Literal["jitsi"], config: VideoPlatformConfig
) -> "JitsiClient": ...
@overload
def get_platform_client(
platform: Literal["whereby"], config: VideoPlatformConfig
) -> "WherebyClient": ...
def get_platform_client(
platform: str, config: VideoPlatformConfig
) -> VideoPlatformClient:
"""Get a video platform client instance."""
platform_lower = platform.lower()
if platform_lower not in _PLATFORMS:
raise ValueError(f"Unknown video platform: {platform}")
client_class = _PLATFORMS[platform_lower]
return client_class(config)
def get_available_platforms() -> list[str]:
"""Get list of available platform names."""
return list(_PLATFORMS.keys())
# Auto-register built-in platforms
def _register_builtin_platforms():
from .jitsi import JitsiClient
from .whereby import WherebyClient
register_platform("jitsi", JitsiClient)
register_platform("whereby", WherebyClient)
_register_builtin_platforms()

View File

@@ -0,0 +1,6 @@
"""Whereby video platform integration."""
from .client import WherebyClient
from .router import router
__all__ = ["WherebyClient", "router"]

View File

@@ -0,0 +1,113 @@
import hmac
from datetime import datetime
from hashlib import sha256
from typing import Any, Dict, Optional
import httpx
from reflector.db.rooms import Room, VideoPlatform
from reflector.settings import settings
from ..base import MeetingData, VideoPlatformClient
class WherebyClient(VideoPlatformClient):
PLATFORM_NAME = VideoPlatform.WHEREBY
def __init__(self, config):
super().__init__(config)
self.headers = {
"Content-Type": "application/json; charset=utf-8",
"Authorization": f"Bearer {self.config.api_key}",
}
self.timeout = 10
async def create_meeting(
self, room_name_prefix: str, end_date: datetime, room: Room
) -> MeetingData:
data = {
"isLocked": room.is_locked,
"roomNamePrefix": room_name_prefix,
"roomNamePattern": "uuid",
"roomMode": room.room_mode,
"endDate": end_date.isoformat(),
"recording": {
"type": room.recording_type,
"destination": {
"provider": "s3",
"bucket": settings.RECORDING_STORAGE_AWS_BUCKET_NAME,
"accessKeyId": self.config.aws_access_key_id,
"accessKeySecret": self.config.aws_access_key_secret,
"fileFormat": "mp4",
},
"startTrigger": room.recording_trigger,
},
"fields": ["hostRoomUrl"],
}
async with httpx.AsyncClient() as client:
response = await client.post(
f"{self.config.api_url}/meetings",
headers=self.headers,
json=data,
timeout=self.timeout,
)
response.raise_for_status()
meeting_data = response.json()
return MeetingData(
meeting_id=meeting_data["meetingId"],
room_name=meeting_data["roomName"],
room_url=meeting_data["roomUrl"],
host_room_url=meeting_data["hostRoomUrl"],
platform=self.PLATFORM_NAME,
extra_data={
"startDate": meeting_data["startDate"],
"endDate": meeting_data["endDate"],
"recording": meeting_data.get("recording", {}),
},
)
async def get_room_sessions(self, room_name: str) -> Dict[str, Any]:
async with httpx.AsyncClient() as client:
response = await client.get(
f"{self.config.api_url}/insights/room-sessions?roomName={room_name}",
headers=self.headers,
timeout=self.timeout,
)
response.raise_for_status()
return response.json()
async def delete_room(self, room_name: str) -> bool:
return True
async def upload_logo(self, room_name: str, logo_path: str) -> bool:
try:
async with httpx.AsyncClient() as client:
with open(logo_path, "rb") as f:
response = await client.put(
f"{self.config.api_url}/rooms{room_name}/theme/logo",
headers={
"Authorization": f"Bearer {self.config.api_key}",
},
timeout=self.timeout,
files={"image": f},
)
response.raise_for_status()
return True
except Exception:
return False
def verify_webhook_signature(
self, body: bytes, signature: str, timestamp: Optional[str] = None
) -> bool:
if not signature or not self.config.webhook_secret:
return False
try:
expected = hmac.new(
self.config.webhook_secret.encode(), body, sha256
).hexdigest()
return hmac.compare_digest(expected, signature)
except Exception:
return False

View File

@@ -0,0 +1,4 @@
"""Whereby-specific worker tasks."""
# Placeholder for Whereby-specific background tasks
# This can be extended with Whereby-specific processing tasks in the future

View File

@@ -0,0 +1,126 @@
from pathlib import Path
from typing import Annotated, Any, Dict, Optional
import structlog
from fastapi import APIRouter, Depends, HTTPException
from pydantic import BaseModel
import reflector.auth as auth
from reflector.db.transcripts import SourceKind, transcripts_controller
from reflector.jibri_events import JitsiEventParser
from reflector.pipelines.main_file_pipeline import task_pipeline_file_process
from reflector.settings import settings
logger = structlog.get_logger(__name__)
router = APIRouter(prefix="/api/v1/jibri", tags=["jibri"])
class RecordingReadyRequest(BaseModel):
session_id: str
path: str # Relative path from recordings directory
meeting_url: str
@router.post("/recording-ready")
async def handle_recording_ready(
request: RecordingReadyRequest,
user: Annotated[Optional[auth.UserInfo], Depends(auth.current_user_optional)],
) -> Dict[str, Any]:
user_id = user["sub"] if user else None
recordings_base = Path(settings.JIBRI_RECORDINGS_PATH or "/recordings")
recording_path = recordings_base / request.path
if not recording_path.exists():
raise HTTPException(
status_code=404, detail=f"Recording path not found: {request.path}"
)
recording_file = recording_path / "recording.mp4"
events_file = recording_path / "events.jsonl"
if not recording_file.exists():
raise HTTPException(status_code=404, detail="Recording file not found")
# Parse events if available
metadata = {}
participant_count = 0
if events_file.exists():
parser = JitsiEventParser()
metadata = parser.parse_events_file(str(recording_path))
participant_count = len(metadata.get("participants", []))
logger.info(
"Parsed Jibri events",
session_id=request.session_id,
event_count=metadata.get("event_count", 0),
participant_count=participant_count,
)
else:
logger.warning("No events file found", session_id=request.session_id)
metadata = {
"room": {"meeting_url": request.meeting_url, "name": request.session_id},
"participants": [],
"speaker_stats": {},
"event_count": 0,
}
# Create transcript using controller
title = f"Meeting: {metadata.get('room', {}).get('name', request.session_id)}"
transcript = await transcripts_controller.add(
name=title,
source_kind=SourceKind.FILE,
source_language="en",
target_language="en",
user_id=user_id,
)
# Store Jitsi data in appropriate fields
update_data = {}
# Store participants if available
if metadata.get("participants"):
update_data["participants"] = metadata["participants"]
# Store events data (room info, speaker stats, etc.)
update_data["events"] = {
"jitsi_metadata": metadata,
"session_id": request.session_id,
"recording_path": str(recording_path),
"meeting_url": request.meeting_url,
}
if update_data:
await transcripts_controller.update(transcript, update_data)
# Copy recording file to transcript data path
# The pipeline expects the file to be in the transcript's data path
upload_file = transcript.data_path / "upload.mp4"
upload_file.parent.mkdir(parents=True, exist_ok=True)
# Create symlink or copy the file
import shutil
shutil.copy2(recording_file, upload_file)
# Update status to uploaded
await transcripts_controller.update(transcript, {"status": "uploaded"})
# Trigger processing pipeline
task_pipeline_file_process.delay(transcript_id=transcript.id)
logger.info(
"Jibri recording ready for processing",
transcript_id=transcript.id,
session_id=request.session_id,
participant_count=participant_count,
)
return {
"status": "accepted",
"transcript_id": transcript.id,
"session_id": request.session_id,
"events_found": events_file.exists(),
"participant_count": participant_count,
}

View File

@@ -12,9 +12,11 @@ from pydantic import BaseModel
import reflector.auth as auth
from reflector.db import get_database
from reflector.db.meetings import meetings_controller
from reflector.db.rooms import rooms_controller
from reflector.db.rooms import VideoPlatform, rooms_controller
from reflector.settings import settings
from reflector.whereby import create_meeting, upload_logo
from reflector.video_platforms.factory import (
create_platform_client,
)
from reflector.worker.webhook import test_webhook
logger = logging.getLogger(__name__)
@@ -23,7 +25,6 @@ router = APIRouter()
def parse_datetime_with_timezone(iso_string: str) -> datetime:
"""Parse ISO datetime string and ensure timezone awareness (defaults to UTC if naive)."""
dt = datetime.fromisoformat(iso_string)
if dt.tzinfo is None:
dt = dt.replace(tzinfo=timezone.utc)
@@ -43,6 +44,7 @@ class Room(BaseModel):
recording_type: str
recording_trigger: str
is_shared: bool
platform: VideoPlatform = VideoPlatform.WHEREBY
class RoomDetails(Room):
@@ -72,6 +74,7 @@ class CreateRoom(BaseModel):
is_shared: bool
webhook_url: str
webhook_secret: str
platform: VideoPlatform
class UpdateRoom(BaseModel):
@@ -86,6 +89,7 @@ class UpdateRoom(BaseModel):
is_shared: bool
webhook_url: str
webhook_secret: str
platform: VideoPlatform
class DeletionStatus(BaseModel):
@@ -149,6 +153,7 @@ async def rooms_create(
is_shared=room.is_shared,
webhook_url=room.webhook_url,
webhook_secret=room.webhook_secret,
platform=room.platform,
)
@@ -196,43 +201,49 @@ async def rooms_create_meeting(
if meeting is None:
end_date = current_time + timedelta(hours=8)
whereby_meeting = await create_meeting("", end_date=end_date, room=room)
platform = room.platform
client = create_platform_client(platform)
await upload_logo(whereby_meeting["roomName"], "./images/logo.png")
platform_meeting = await client.create_meeting("", end_date=end_date, room=room)
await client.upload_logo(platform_meeting.room_name, "./images/logo.png")
# Now try to save to database
meeting_data = {
"meeting_id": platform_meeting.meeting_id,
"room_name": platform_meeting.room_name,
"room_url": platform_meeting.room_url,
"host_room_url": platform_meeting.host_room_url,
"start_date": current_time,
"end_date": end_date,
}
try:
meeting = await meetings_controller.create(
id=whereby_meeting["meetingId"],
room_name=whereby_meeting["roomName"],
room_url=whereby_meeting["roomUrl"],
host_room_url=whereby_meeting["hostRoomUrl"],
start_date=parse_datetime_with_timezone(whereby_meeting["startDate"]),
end_date=parse_datetime_with_timezone(whereby_meeting["endDate"]),
id=meeting_data["meeting_id"],
room_name=meeting_data["room_name"],
room_url=meeting_data["room_url"],
host_room_url=meeting_data["host_room_url"],
start_date=meeting_data["start_date"],
end_date=meeting_data["end_date"],
user_id=user_id,
room=room,
)
except (asyncpg.exceptions.UniqueViolationError, sqlite3.IntegrityError):
# Another request already created a meeting for this room
# Log this race condition occurrence
logger.info(
"Race condition detected for room %s - fetching existing meeting",
room.name,
)
logger.warning(
"Whereby meeting %s was created but not used (resource leak) for room %s",
whereby_meeting["meetingId"],
"Platform meeting %s was created but not used (resource leak) for room %s",
meeting_data["meeting_id"],
room.name,
)
# Fetch the meeting that was created by the other request
meeting = await meetings_controller.get_active(
room=room, current_time=current_time
)
if meeting is None:
# Edge case: meeting was created but expired/deleted between checks
logger.error(
"Meeting disappeared after race condition for room %s", room.name
"Meeting disappeared after race condition for room %s",
room.name,
exc_info=True,
)
raise HTTPException(
status_code=503, detail="Unable to join meeting - please try again"
@@ -249,7 +260,6 @@ async def rooms_test_webhook(
room_id: str,
user: Annotated[Optional[auth.UserInfo], Depends(auth.current_user_optional)],
):
"""Test webhook configuration by sending a sample payload."""
user_id = user["sub"] if user else None
room = await rooms_controller.get_by_id(room_id)

View File

@@ -350,8 +350,6 @@ async def transcript_update(
transcript = await transcripts_controller.get_by_id_for_http(
transcript_id, user_id=user_id
)
if not transcript:
raise HTTPException(status_code=404, detail="Transcript not found")
values = info.dict(exclude_unset=True)
updated_transcript = await transcripts_controller.update(transcript, values)
return updated_transcript

View File

@@ -1,18 +1,60 @@
import logging
from datetime import datetime
import httpx
from reflector.db.rooms import Room
from reflector.settings import settings
from reflector.utils.string import parse_non_empty_string
logger = logging.getLogger(__name__)
def _get_headers():
api_key = parse_non_empty_string(
settings.WHEREBY_API_KEY, "WHEREBY_API_KEY value is required."
)
return {
"Content-Type": "application/json; charset=utf-8",
"Authorization": f"Bearer {api_key}",
}
HEADERS = {
"Content-Type": "application/json; charset=utf-8",
"Authorization": f"Bearer {settings.WHEREBY_API_KEY}",
}
TIMEOUT = 10 # seconds
def _get_whereby_s3_auth():
errors = []
try:
bucket_name = parse_non_empty_string(
settings.RECORDING_STORAGE_AWS_BUCKET_NAME,
"RECORDING_STORAGE_AWS_BUCKET_NAME value is required.",
)
except Exception as e:
errors.append(e)
try:
key_id = parse_non_empty_string(
settings.AWS_WHEREBY_ACCESS_KEY_ID,
"AWS_WHEREBY_ACCESS_KEY_ID value is required.",
)
except Exception as e:
errors.append(e)
try:
key_secret = parse_non_empty_string(
settings.AWS_WHEREBY_ACCESS_KEY_SECRET,
"AWS_WHEREBY_ACCESS_KEY_SECRET value is required.",
)
except Exception as e:
errors.append(e)
if len(errors) > 0:
raise Exception(
f"Failed to get Whereby auth settings: {', '.join(str(e) for e in errors)}"
)
return bucket_name, key_id, key_secret
async def create_meeting(room_name_prefix: str, end_date: datetime, room: Room):
s3_bucket_name, s3_key_id, s3_key_secret = _get_whereby_s3_auth()
data = {
"isLocked": room.is_locked,
"roomNamePrefix": room_name_prefix,
@@ -23,23 +65,26 @@ async def create_meeting(room_name_prefix: str, end_date: datetime, room: Room):
"type": room.recording_type,
"destination": {
"provider": "s3",
"bucket": settings.RECORDING_STORAGE_AWS_BUCKET_NAME,
"accessKeyId": settings.AWS_WHEREBY_ACCESS_KEY_ID,
"accessKeySecret": settings.AWS_WHEREBY_ACCESS_KEY_SECRET,
"bucket": s3_bucket_name,
"accessKeyId": s3_key_id,
"accessKeySecret": s3_key_secret,
"fileFormat": "mp4",
},
"startTrigger": room.recording_trigger,
},
"fields": ["hostRoomUrl"],
}
async with httpx.AsyncClient() as client:
response = await client.post(
f"{settings.WHEREBY_API_URL}/meetings",
headers=HEADERS,
headers=_get_headers(),
json=data,
timeout=TIMEOUT,
)
if response.status_code == 403:
logger.warning(
f"Failed to create meeting: access denied on Whereby: {response.text}"
)
response.raise_for_status()
return response.json()
@@ -48,7 +93,7 @@ async def get_room_sessions(room_name: str):
async with httpx.AsyncClient() as client:
response = await client.get(
f"{settings.WHEREBY_API_URL}/insights/room-sessions?roomName={room_name}",
headers=HEADERS,
headers=_get_headers(),
timeout=TIMEOUT,
)
response.raise_for_status()

View File

@@ -20,6 +20,7 @@ else:
"reflector.worker.healthcheck",
"reflector.worker.process",
"reflector.worker.cleanup",
"reflector.worker.jitsi_events",
]
)
@@ -33,6 +34,10 @@ else:
"task": "reflector.worker.process.process_meetings",
"schedule": float(settings.SQS_POLLING_TIMEOUT_SECONDS),
},
"process_jitsi_events": {
"task": "reflector.worker.jitsi_events.process_jitsi_events",
"schedule": 5.0, # Process every 5 seconds
},
"reprocess_failed_recordings": {
"task": "reflector.worker.process.reprocess_failed_recordings",
"schedule": crontab(hour=5, minute=0), # Midnight EST

View File

@@ -0,0 +1,281 @@
"""
Celery tasks for consuming Jitsi events from Redis queues.
"""
import json
from datetime import datetime
from typing import Any, Dict
import redis
import structlog
from sqlalchemy.orm import Session
from reflector.database import get_db_sync
from reflector.models import Meeting, Transcript
from reflector.settings import settings
from reflector.worker.app import app
logger = structlog.get_logger(__name__)
class JitsiEventProcessor:
"""Process Jitsi events from Redis queues."""
def __init__(self):
self.redis_client = redis.Redis(
host=settings.REDIS_HOST or "redis",
port=settings.REDIS_PORT or 6379,
decode_responses=True,
)
self.participants = {} # room_name -> {jid: participant_info}
self.speaker_stats = {} # room_name -> {jid: stats}
def process_participant_joined(self, data: Dict[str, Any], db: Session):
"""Track participant joining a room."""
room_name = data["room_name"]
participant = {
"jid": data["participant_jid"],
"nick": data["participant_nick"],
"id": data["participant_id"],
"is_moderator": data.get("is_moderator", False),
"joined_at": datetime.now(),
}
if room_name not in self.participants:
self.participants[room_name] = {}
self.participants[room_name][participant["jid"]] = participant
logger.info(
"Participant joined",
room=room_name,
participant=participant["nick"],
total_participants=len(self.participants[room_name]),
)
# Update meeting in database if exists
meeting = (
db.query(Meeting)
.filter(
Meeting.room_name == room_name,
Meeting.status.in_(["active", "pending"]),
)
.first()
)
if meeting:
# Store participant info in meeting metadata
metadata = meeting.metadata or {}
if "participants" not in metadata:
metadata["participants"] = []
metadata["participants"].append(
{
"id": participant["id"],
"name": participant["nick"],
"joined_at": participant["joined_at"].isoformat(),
"is_moderator": participant["is_moderator"],
}
)
meeting.metadata = metadata
db.commit()
def process_participant_left(self, data: Dict[str, Any], db: Session):
"""Track participant leaving a room."""
room_name = data["room_name"]
participant_jid = data["participant_jid"]
if room_name in self.participants:
if participant_jid in self.participants[room_name]:
participant = self.participants[room_name][participant_jid]
participant["left_at"] = datetime.now()
logger.info(
"Participant left",
room=room_name,
participant=participant["nick"],
duration=(
participant["left_at"] - participant["joined_at"]
).total_seconds(),
)
# Update meeting in database
meeting = (
db.query(Meeting)
.filter(
Meeting.room_name == room_name,
Meeting.status.in_(["active", "pending"]),
)
.first()
)
if meeting and meeting.metadata and "participants" in meeting.metadata:
for p in meeting.metadata["participants"]:
if p["id"] == participant["id"]:
p["left_at"] = participant["left_at"].isoformat()
break
db.commit()
def process_speaker_stats(self, data: Dict[str, Any], db: Session):
"""Update speaker statistics."""
room_name = data["room_jid"].split("@")[0]
self.speaker_stats[room_name] = data["stats"]
logger.debug(
"Speaker stats updated", room=room_name, speakers=len(data["stats"])
)
def process_recording_completed(self, data: Dict[str, Any], db: Session):
"""Process completed recording with all metadata."""
room_name = data["room_name"]
meeting_url = data["meeting_url"]
recording_path = data["recording_path"]
recording_file = data["recording_file"]
logger.info(
"Recording completed", room=room_name, url=meeting_url, path=recording_path
)
# Get participant data for this room
participants = self.participants.get(room_name, {})
speaker_stats = self.speaker_stats.get(room_name, {})
# Create transcript record with full metadata
transcript = Transcript(
title=f"Recording: {room_name}",
source_url=meeting_url,
metadata={
"jitsi": {
"room_name": room_name,
"meeting_url": meeting_url,
"recording_path": recording_path,
"participants": [
{
"id": p["id"],
"name": p["nick"],
"joined_at": p["joined_at"].isoformat(),
"left_at": p.get("left_at", datetime.now()).isoformat(),
"is_moderator": p["is_moderator"],
"speaking_time": speaker_stats.get(p["jid"], {}).get(
"total_time", 0
),
}
for p in participants.values()
],
"speaker_stats": speaker_stats,
}
},
status="pending",
)
db.add(transcript)
db.commit()
# Trigger processing pipeline
from reflector.pipelines.main_transcript_pipeline import TranscriptMainPipeline
pipeline = TranscriptMainPipeline()
pipeline.create(transcript.id, recording_file)
# Clean up room data
self.participants.pop(room_name, None)
self.speaker_stats.pop(room_name, None)
logger.info(
"Transcript created",
transcript_id=transcript.id,
participants=len(participants),
has_speaker_stats=bool(speaker_stats),
)
processor = JitsiEventProcessor()
@app.task(name="reflector.worker.jitsi_events.process_jitsi_events")
def process_jitsi_events():
"""
Process Jitsi events from Redis queue.
This should be called periodically by Celery Beat.
"""
db = next(get_db_sync())
processed = 0
try:
# Process up to 100 events per run
for _ in range(100):
# Pop event from queue (blocking with 1 second timeout)
event_data = processor.redis_client.brpop(
["jitsi:events:queue", "jitsi:recordings:queue"], timeout=1
)
if not event_data:
break
queue_name, event_json = event_data
event = json.loads(event_json)
event_type = event["type"]
data = event["data"]
logger.debug(f"Processing event: {event_type}")
# Route to appropriate processor
if event_type == "participant_joined":
processor.process_participant_joined(data, db)
elif event_type == "participant_left":
processor.process_participant_left(data, db)
elif event_type == "speaker_stats_update":
processor.process_speaker_stats(data, db)
elif event_type == "recording_completed":
processor.process_recording_completed(data, db)
else:
logger.warning(f"Unknown event type: {event_type}")
processed += 1
if processed > 0:
logger.info(f"Processed {processed} Jitsi events")
except Exception as e:
logger.error(f"Error processing Jitsi events: {e}")
raise
finally:
db.close()
return processed
@app.task(name="reflector.worker.jitsi_events.consume_jitsi_stream")
def consume_jitsi_stream():
"""
Alternative: Use Redis Streams for more reliable event processing.
Redis Streams provide better guarantees and consumer groups.
"""
db = next(get_db_sync())
try:
# Read from stream with consumer group
events = processor.redis_client.xreadgroup(
"reflector-consumers",
"worker-1",
{"jitsi:events": ">"},
count=10,
block=1000,
)
for stream_name, messages in events:
for message_id, data in messages:
event = json.loads(data[b"event"])
# Process event...
# Acknowledge message
processor.redis_client.xack(
stream_name, "reflector-consumers", message_id
)
except Exception as e:
logger.error(f"Error consuming stream: {e}")
raise
finally:
db.close()

View File

@@ -17,7 +17,7 @@ from reflector.db.transcripts import SourceKind, transcripts_controller
from reflector.pipelines.main_file_pipeline import task_pipeline_file_process
from reflector.pipelines.main_live_pipeline import asynctask
from reflector.settings import settings
from reflector.whereby import get_room_sessions
from reflector.video_platforms.factory import create_platform_client
logger = structlog.wrap_logger(get_task_logger(__name__))
@@ -155,11 +155,18 @@ async def process_meetings():
if end_date.tzinfo is None:
end_date = end_date.replace(tzinfo=timezone.utc)
if end_date > datetime.now(timezone.utc):
response = await get_room_sessions(meeting.room_name)
room_sessions = response.get("results", [])
is_active = not room_sessions or any(
rs["endedAt"] is None for rs in room_sessions
)
# Get room sessions using platform client
platform = getattr(meeting, "platform", "whereby")
client = create_platform_client(platform)
if client:
response = await client.get_room_sessions(meeting.room_name)
room_sessions = response.get("results", [])
is_active = not room_sessions or any(
rs["endedAt"] is None for rs in room_sessions
)
else:
# Fallback: assume meeting is still active if we can't check
is_active = True
if not is_active:
await meetings_controller.update_meeting(meeting.id, is_active=False)
logger.info("Meeting %s is deactivated", meeting.id)

49
server/run_jibri_tests.py Normal file
View File

@@ -0,0 +1,49 @@
#!/usr/bin/env python
"""Simple test runner for Jibri tests that doesn't require Docker."""
import os
import sys
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
# Import test functions after path is set
exec(open("tests/test_jibri_events.py").read(), globals())
def run_tests():
tests = [
("test_parse_room_created_event", test_parse_room_created_event),
("test_parse_participant_joined_event", test_parse_participant_joined_event),
(
"test_parse_unknown_event_returns_none",
test_parse_unknown_event_returns_none,
),
(
"test_parse_events_file_with_complete_session",
test_parse_events_file_with_complete_session,
),
("test_parse_events_file_missing_file", test_parse_events_file_missing_file),
]
passed = 0
failed = 0
for name, test_func in tests:
try:
test_func()
print(f"{name}")
passed += 1
except AssertionError as e:
print(f"{name}: {e}")
failed += 1
except Exception as e:
print(f"{name}: Unexpected error: {e}")
failed += 1
print(f"\nResults: {passed} passed, {failed} failed")
return failed == 0
if __name__ == "__main__":
success = run_tests()
sys.exit(0 if success else 1)

View File

@@ -105,7 +105,6 @@ async def test_cleanup_deletes_associated_meeting_and_recording():
host_room_url="https://example.com/meeting-host",
start_date=old_date,
end_date=old_date + timedelta(hours=1),
user_id=None,
room_id=None,
)
)
@@ -241,7 +240,6 @@ async def test_meeting_consent_cascade_delete():
host_room_url="https://example.com/cascade-test-host",
start_date=datetime.now(timezone.utc),
end_date=datetime.now(timezone.utc) + timedelta(hours=1),
user_id="test-user",
room_id=None,
)
)

View File

@@ -272,6 +272,9 @@ class TestGPUModalTranscript:
for f in temp_files:
Path(f).unlink(missing_ok=True)
@pytest.mark.skipif(
not "parakeet" in get_model_name(), reason="Parakeet only supports English"
)
def test_transcriptions_error_handling(self):
"""Test error handling for invalid requests."""
url = get_modal_transcript_url()

View File

@@ -0,0 +1,122 @@
import json
import tempfile
from pathlib import Path
from reflector.jibri_events import (
JitsiEventParser,
ParticipantJoinedEvent,
RoomCreatedEvent,
)
def test_parse_room_created_event():
parser = JitsiEventParser()
event_data = {
"type": "room_created",
"timestamp": 1234567890,
"room_name": "TestRoom",
"room_jid": "testroom@conference.meet.jitsi",
"meeting_url": "https://meet.jitsi/TestRoom",
}
event = parser.parse_event(event_data)
assert isinstance(event, RoomCreatedEvent)
assert event.room_name == "TestRoom"
assert event.meeting_url == "https://meet.jitsi/TestRoom"
def test_parse_participant_joined_event():
parser = JitsiEventParser()
event_data = {
"type": "participant_joined",
"timestamp": 1234567891,
"room_name": "TestRoom",
"participant": {
"jid": "user1@meet.jitsi/resource",
"nick": "John Doe",
"id": "user1@meet.jitsi",
"is_moderator": False,
},
}
event = parser.parse_event(event_data)
assert isinstance(event, ParticipantJoinedEvent)
assert event.participant.nick == "John Doe"
assert event.participant.is_moderator is False
def test_parse_unknown_event_returns_none():
parser = JitsiEventParser()
event_data = {"type": "unknown_event", "timestamp": 1234567890}
event = parser.parse_event(event_data)
assert event is None
def test_parse_events_file_with_complete_session():
parser = JitsiEventParser()
with tempfile.TemporaryDirectory() as tmpdir:
events_file = Path(tmpdir) / "events.jsonl"
events = [
{
"type": "room_created",
"timestamp": 1234567890,
"room_name": "TestRoom",
"room_jid": "testroom@conference.meet.jitsi",
"meeting_url": "https://meet.jitsi/TestRoom",
},
{
"type": "participant_joined",
"timestamp": 1234567892,
"room_name": "TestRoom",
"participant": {
"jid": "user1@meet.jitsi/resource",
"nick": "John Doe",
"id": "user1@meet.jitsi",
"is_moderator": False,
},
},
{
"type": "speaker_active",
"timestamp": 1234567895,
"room_name": "TestRoom",
"speaker_jid": "user1@meet.jitsi",
"speaker_nick": "John Doe",
"duration": 10,
},
{
"type": "participant_left",
"timestamp": 1234567920,
"room_name": "TestRoom",
"participant": {
"jid": "user1@meet.jitsi/resource",
"duration_seconds": 28,
},
},
]
with open(events_file, "w") as f:
for event in events:
f.write(json.dumps(event) + "\n")
metadata = parser.parse_events_file(tmpdir)
assert metadata["room"]["name"] == "TestRoom"
assert metadata["room"]["meeting_url"] == "https://meet.jitsi/TestRoom"
assert len(metadata["participants"]) == 1
assert metadata["event_count"] == 4
def test_parse_events_file_missing_file():
parser = JitsiEventParser()
with tempfile.TemporaryDirectory() as tmpdir:
metadata = parser.parse_events_file(tmpdir)
assert metadata["room"]["name"] == ""
assert len(metadata["participants"]) == 0
assert metadata["event_count"] == 0

View File

@@ -0,0 +1,254 @@
import json
import tempfile
from pathlib import Path
from unittest.mock import Mock, patch
import pytest
from fastapi.testclient import TestClient
from sqlalchemy.orm import Session
from reflector.api.jibri_webhook import router
from reflector.models import Transcript
@pytest.fixture
def client():
from fastapi import FastAPI
app = FastAPI()
app.include_router(router)
return TestClient(app)
@pytest.fixture
def mock_db():
db = Mock(spec=Session)
db.add = Mock()
db.commit = Mock()
db.refresh = Mock()
return db
@pytest.fixture
def mock_settings():
with patch("reflector.api.jibri_webhook.settings") as mock:
mock.JIBRI_RECORDINGS_PATH = "/recordings"
yield mock
@pytest.fixture
def mock_pipeline():
with patch("reflector.api.jibri_webhook.TranscriptMainPipeline") as mock:
pipeline_instance = Mock()
pipeline_instance.create = Mock()
mock.return_value = pipeline_instance
yield mock
class TestJibriWebhook:
def test_recording_ready_success_with_events(
self, client, mock_db, mock_settings, mock_pipeline
):
with tempfile.TemporaryDirectory() as tmpdir:
mock_settings.JIBRI_RECORDINGS_PATH = tmpdir
# Create recording directory and files
session_id = "test-session-123"
recording_dir = Path(tmpdir) / session_id
recording_dir.mkdir()
recording_file = recording_dir / "recording.mp4"
recording_file.write_text("fake video content")
events_file = recording_dir / "events.jsonl"
events = [
{
"type": "room_created",
"timestamp": 1234567890,
"room_name": "TestRoom",
"room_jid": "testroom@conference.meet.jitsi",
"meeting_url": "https://meet.jitsi/TestRoom",
},
{
"type": "participant_joined",
"timestamp": 1234567892,
"room_name": "TestRoom",
"participant": {
"jid": "user1@meet.jitsi/resource",
"nick": "John Doe",
"id": "user1@meet.jitsi",
"is_moderator": False,
},
},
]
with open(events_file, "w") as f:
for event in events:
f.write(json.dumps(event) + "\n")
# Mock database dependency
with patch("reflector.api.jibri_webhook.get_db") as mock_get_db:
mock_get_db.return_value = mock_db
response = client.post(
"/api/v1/jibri/recording-ready",
json={
"session_id": session_id,
"path": session_id,
"meeting_url": "https://meet.jitsi/TestRoom",
},
)
assert response.status_code == 200
data = response.json()
assert data["status"] == "accepted"
assert data["session_id"] == session_id
assert data["events_found"] is True
assert data["participant_count"] == 1
# Verify transcript was created
mock_db.add.assert_called_once()
transcript_arg = mock_db.add.call_args[0][0]
assert isinstance(transcript_arg, Transcript)
assert "TestRoom" in transcript_arg.title
assert transcript_arg.metadata["jitsi"]["room"]["name"] == "TestRoom"
# Verify pipeline was triggered
mock_pipeline.return_value.create.assert_called_once()
def test_recording_ready_success_without_events(
self, client, mock_db, mock_settings, mock_pipeline
):
with tempfile.TemporaryDirectory() as tmpdir:
mock_settings.JIBRI_RECORDINGS_PATH = tmpdir
session_id = "test-session-456"
recording_dir = Path(tmpdir) / session_id
recording_dir.mkdir()
recording_file = recording_dir / "recording.mp4"
recording_file.write_text("fake video content")
with patch("reflector.api.jibri_webhook.get_db") as mock_get_db:
mock_get_db.return_value = mock_db
response = client.post(
"/api/v1/jibri/recording-ready",
json={
"session_id": session_id,
"path": session_id,
"meeting_url": "https://meet.jitsi/NoEventsRoom",
},
)
assert response.status_code == 200
data = response.json()
assert data["status"] == "accepted"
assert data["events_found"] is False
assert data["participant_count"] == 0
# Verify transcript was created with minimal metadata
mock_db.add.assert_called_once()
transcript_arg = mock_db.add.call_args[0][0]
assert transcript_arg.metadata["jitsi"]["participants"] == []
def test_recording_ready_path_not_found(self, client, mock_settings):
with tempfile.TemporaryDirectory() as tmpdir:
mock_settings.JIBRI_RECORDINGS_PATH = tmpdir
response = client.post(
"/api/v1/jibri/recording-ready",
json={
"session_id": "nonexistent",
"path": "nonexistent",
"meeting_url": "https://meet.jitsi/Test",
},
)
assert response.status_code == 404
assert "Recording path not found" in response.json()["detail"]
def test_recording_ready_recording_file_not_found(self, client, mock_settings):
with tempfile.TemporaryDirectory() as tmpdir:
mock_settings.JIBRI_RECORDINGS_PATH = tmpdir
session_id = "test-no-recording"
recording_dir = Path(tmpdir) / session_id
recording_dir.mkdir()
# No recording.mp4 file created
response = client.post(
"/api/v1/jibri/recording-ready",
json={
"session_id": session_id,
"path": session_id,
"meeting_url": "https://meet.jitsi/Test",
},
)
assert response.status_code == 404
assert "Recording file not found" in response.json()["detail"]
def test_recording_ready_with_relative_path(
self, client, mock_db, mock_settings, mock_pipeline
):
with tempfile.TemporaryDirectory() as tmpdir:
mock_settings.JIBRI_RECORDINGS_PATH = tmpdir
# Create nested directory structure
session_id = "2024/01/15/test-session"
recording_dir = Path(tmpdir) / session_id
recording_dir.mkdir(parents=True)
recording_file = recording_dir / "recording.mp4"
recording_file.write_text("fake video content")
with patch("reflector.api.jibri_webhook.get_db") as mock_get_db:
mock_get_db.return_value = mock_db
response = client.post(
"/api/v1/jibri/recording-ready",
json={
"session_id": "test-session",
"path": session_id, # Relative path with subdirectories
"meeting_url": "https://meet.jitsi/Test",
},
)
assert response.status_code == 200
data = response.json()
assert data["status"] == "accepted"
def test_recording_ready_empty_meeting_url(
self, client, mock_db, mock_settings, mock_pipeline
):
with tempfile.TemporaryDirectory() as tmpdir:
mock_settings.JIBRI_RECORDINGS_PATH = tmpdir
session_id = "test-session"
recording_dir = Path(tmpdir) / session_id
recording_dir.mkdir()
recording_file = recording_dir / "recording.mp4"
recording_file.write_text("fake video content")
with patch("reflector.api.jibri_webhook.get_db") as mock_get_db:
mock_get_db.return_value = mock_db
response = client.post(
"/api/v1/jibri/recording-ready",
json={
"session_id": session_id,
"path": session_id,
"meeting_url": "",
},
)
assert response.status_code == 200
# Verify fallback URL was used
transcript_arg = mock_db.add.call_args[0][0]
assert transcript_arg.source_url == f"jitsi://{session_id}"

View File

@@ -58,7 +58,7 @@ async def test_empty_transcript_title_only_match():
"id": test_id,
"name": "Empty Transcript",
"title": "Empty Meeting",
"status": "completed",
"status": "ended",
"locked": False,
"duration": 0.0,
"created_at": datetime.now(timezone.utc),
@@ -109,7 +109,7 @@ async def test_search_with_long_summary():
"id": test_id,
"name": "Test Long Summary",
"title": "Regular Meeting",
"status": "completed",
"status": "ended",
"locked": False,
"duration": 1800.0,
"created_at": datetime.now(timezone.utc),
@@ -165,7 +165,7 @@ async def test_postgresql_search_with_data():
"id": test_id,
"name": "Test Search Transcript",
"title": "Engineering Planning Meeting Q4 2024",
"status": "completed",
"status": "ended",
"locked": False,
"duration": 1800.0,
"created_at": datetime.now(timezone.utc),
@@ -221,7 +221,7 @@ We need to implement PostgreSQL tsvector for better performance.""",
test_result = next((r for r in results if r.id == test_id), None)
if test_result:
assert test_result.title == "Engineering Planning Meeting Q4 2024"
assert test_result.status == "completed"
assert test_result.status == "ended"
assert test_result.duration == 1800.0
assert 0 <= test_result.rank <= 1, "Rank should be normalized to 0-1"
@@ -268,7 +268,7 @@ def mock_db_result():
"title": "Test Transcript",
"created_at": datetime(2024, 6, 15, tzinfo=timezone.utc),
"duration": 3600.0,
"status": "completed",
"status": "ended",
"user_id": "test-user",
"room_id": "room1",
"source_kind": SourceKind.LIVE,
@@ -433,7 +433,7 @@ class TestSearchResultModel:
room_id="room-456",
source_kind=SourceKind.ROOM,
created_at=datetime(2024, 6, 15, tzinfo=timezone.utc),
status="completed",
status="ended",
rank=0.85,
duration=1800.5,
search_snippets=["snippet 1", "snippet 2"],
@@ -443,7 +443,7 @@ class TestSearchResultModel:
assert result.title == "Test Title"
assert result.user_id == "user-123"
assert result.room_id == "room-456"
assert result.status == "completed"
assert result.status == "ended"
assert result.rank == 0.85
assert result.duration == 1800.5
assert len(result.search_snippets) == 2
@@ -474,7 +474,7 @@ class TestSearchResultModel:
id="test-id",
source_kind=SourceKind.LIVE,
created_at=datetime(2024, 6, 15, 12, 30, 45, tzinfo=timezone.utc),
status="completed",
status="ended",
rank=0.9,
duration=None,
search_snippets=[],

View File

@@ -25,7 +25,7 @@ async def test_long_summary_snippet_prioritization():
"id": test_id,
"name": "Test Snippet Priority",
"title": "Meeting About Projects",
"status": "completed",
"status": "ended",
"locked": False,
"duration": 1800.0,
"created_at": datetime.now(timezone.utc),
@@ -106,7 +106,7 @@ async def test_long_summary_only_search():
"id": test_id,
"name": "Test Long Only",
"title": "Standard Meeting",
"status": "completed",
"status": "ended",
"locked": False,
"duration": 1800.0,
"created_at": datetime.now(timezone.utc),

View File

@@ -0,0 +1,768 @@
"""Tests for video platform abstraction and Jitsi integration."""
from datetime import datetime, timedelta, timezone
from unittest.mock import Mock, patch
import pytest
from fastapi.testclient import TestClient
from reflector.db.rooms import Room, VideoPlatform
from reflector.video_platforms.base import (
MeetingData,
VideoPlatformClient,
VideoPlatformConfig,
)
from reflector.video_platforms.factory import (
create_platform_client,
get_platform_config,
)
from reflector.video_platforms.jitsi import JitsiClient
from reflector.video_platforms.registry import (
get_available_platforms,
get_platform_client,
register_platform,
)
from reflector.video_platforms.whereby import WherebyClient
class TestVideoPlatformBase:
"""Test the video platform base classes and interfaces."""
def test_video_platform_config_creation(self):
"""Test VideoPlatformConfig can be created with required fields."""
config = VideoPlatformConfig(
api_key="test-key",
webhook_secret="test-secret",
api_url="https://test.example.com",
)
assert config.api_key == "test-key"
assert config.webhook_secret == "test-secret"
assert config.api_url == "https://test.example.com"
def test_meeting_data_creation(self):
"""Test MeetingData can be created with all fields."""
meeting_data = MeetingData(
meeting_id="test-123",
room_name="test-room",
room_url="https://test.com/room",
host_room_url="https://test.com/host",
platform=VideoPlatform.JITSI,
extra_data={"jwt": "token123"},
)
assert meeting_data.meeting_id == "test-123"
assert meeting_data.room_name == "test-room"
assert meeting_data.platform == VideoPlatform.JITSI
assert meeting_data.extra_data["jwt"] == "token123"
class TestJitsiClient:
"""Test JitsiClient implementation."""
def setup_method(self):
"""Set up test fixtures."""
self.config = VideoPlatformConfig(
api_key="", # Jitsi doesn't use API key
webhook_secret="test-webhook-secret",
api_url="https://meet.example.com",
)
self.client = JitsiClient(self.config)
self.test_room = Room(
id="test-room-id", name="test-room", user_id="test-user", platform="jitsi"
)
@patch("reflector.settings.settings.JITSI_JWT_SECRET", "test-secret-123")
@patch("reflector.settings.settings.JITSI_DOMAIN", "meet.example.com")
@patch("reflector.settings.settings.JITSI_JWT_ISSUER", "reflector")
@patch("reflector.settings.settings.JITSI_JWT_AUDIENCE", "jitsi")
def test_jwt_generation(self):
"""Test JWT token generation with proper payload."""
exp_time = datetime.now(timezone.utc) + timedelta(hours=1)
jwt_token = self.client._generate_jwt(
room="test-room", moderator=True, exp=exp_time
)
# Verify token is generated
assert jwt_token is not None
assert len(jwt_token) > 50 # JWT tokens are quite long
assert jwt_token.count(".") == 2 # JWT has 3 parts separated by dots
@patch("reflector.settings.settings.JITSI_JWT_SECRET", None)
def test_jwt_generation_without_secret_fails(self):
"""Test JWT generation fails without secret."""
exp_time = datetime.now(timezone.utc) + timedelta(hours=1)
with pytest.raises(ValueError, match="JITSI_JWT_SECRET is required"):
self.client._generate_jwt(room="test-room", moderator=False, exp=exp_time)
@patch(
"reflector.video_platforms.jitsi.client.generate_uuid4",
return_value="test-uuid-123",
)
@patch("reflector.settings.settings.JITSI_JWT_SECRET", "test-secret-123")
@patch("reflector.settings.settings.JITSI_DOMAIN", "meet.example.com")
@patch("reflector.settings.settings.JITSI_JWT_ISSUER", "reflector")
@patch("reflector.settings.settings.JITSI_JWT_AUDIENCE", "jitsi")
async def test_create_meeting(self, mock_uuid):
"""Test meeting creation with JWT tokens."""
end_date = datetime.now(timezone.utc) + timedelta(hours=2)
meeting_data = await self.client.create_meeting(
room_name_prefix="test", end_date=end_date, room=self.test_room
)
# Verify meeting data structure
assert meeting_data.meeting_id == "test-uuid-123"
assert meeting_data.platform == VideoPlatform.JITSI
assert "reflector-test-room" in meeting_data.room_name
assert "meet.example.com" in meeting_data.room_url
assert "jwt=" in meeting_data.room_url
assert "jwt=" in meeting_data.host_room_url
# Verify extra data contains JWT tokens
assert "user_jwt" in meeting_data.extra_data
assert "host_jwt" in meeting_data.extra_data
assert "domain" in meeting_data.extra_data
async def test_get_room_sessions(self):
"""Test room sessions retrieval (mock implementation)."""
sessions = await self.client.get_room_sessions("test-room")
assert "roomName" in sessions
assert "sessions" in sessions
assert sessions["roomName"] == "test-room"
assert len(sessions["sessions"]) > 0
assert sessions["sessions"][0]["isActive"] is True
async def test_delete_room(self):
"""Test room deletion (no-op for Jitsi)."""
result = await self.client.delete_room("test-room")
assert result is True
async def test_upload_logo(self):
"""Test logo upload (no-op for Jitsi)."""
result = await self.client.upload_logo("test-room", "logo.png")
assert result is True
def test_verify_webhook_signature_valid(self):
"""Test webhook signature verification with valid signature."""
body = b'{"event": "test"}'
# Generate expected signature
import hmac
from hashlib import sha256
expected_signature = hmac.new(
self.config.webhook_secret.encode(), body, sha256
).hexdigest()
result = self.client.verify_webhook_signature(body, expected_signature)
assert result is True
def test_verify_webhook_signature_invalid(self):
"""Test webhook signature verification with invalid signature."""
body = b'{"event": "test"}'
invalid_signature = "invalid-signature"
result = self.client.verify_webhook_signature(body, invalid_signature)
assert result is False
def test_verify_webhook_signature_no_secret(self):
"""Test webhook signature verification without secret."""
config = VideoPlatformConfig(
api_key="", webhook_secret="", api_url="https://meet.example.com"
)
client = JitsiClient(config)
result = client.verify_webhook_signature(b'{"event": "test"}', "signature")
assert result is False
class TestWherebyClient:
"""Test WherebyClient implementation."""
def setup_method(self):
"""Set up test fixtures."""
self.config = VideoPlatformConfig(
api_key="test-whereby-api-key",
webhook_secret="test-whereby-webhook-secret",
api_url="https://api.whereby.dev",
s3_bucket="test-recordings-bucket",
aws_access_key_id="test-access-key",
aws_access_key_secret="test-access-secret",
)
self.client = WherebyClient(self.config)
self.test_room = Room(
id="test-room-id",
name="test-room",
user_id="test-user",
platform=VideoPlatform.WHEREBY,
)
@patch("httpx.AsyncClient")
async def test_create_meeting(self, mock_client_class):
"""Test Whereby meeting creation."""
# Mock the HTTP response
mock_client = mock_client_class.return_value.__aenter__.return_value
mock_response = Mock()
mock_response.json.return_value = {
"meetingId": "whereby-meeting-123",
"roomName": "whereby-room-456",
"roomUrl": "https://whereby.com/room",
"hostRoomUrl": "https://whereby.com/host-room",
"startDate": "2025-01-15T10:00:00.000Z",
"endDate": "2025-01-15T18:00:00.000Z",
}
mock_response.raise_for_status.return_value = None
mock_client.post.return_value = mock_response
end_date = datetime.now(timezone.utc) + timedelta(hours=2)
meeting_data = await self.client.create_meeting(
room_name_prefix="test", end_date=end_date, room=self.test_room
)
# Verify meeting data structure
assert meeting_data.meeting_id == "whereby-meeting-123"
assert meeting_data.room_name == "whereby-room-456"
assert meeting_data.platform == VideoPlatform.WHEREBY
assert "whereby.com" in meeting_data.room_url
assert "whereby.com" in meeting_data.host_room_url
# Verify HTTP call was made with correct parameters
mock_client.post.assert_called_once()
call_args = mock_client.post.call_args
assert "whereby.dev" in call_args[0][0] # URL
assert "Bearer test-whereby-api-key" in call_args[1]["headers"]["Authorization"]
@patch("httpx.AsyncClient")
async def test_get_room_sessions(self, mock_client_class):
"""Test Whereby room sessions retrieval."""
mock_client = mock_client_class.return_value.__aenter__.return_value
mock_response = Mock()
mock_response.json.return_value = {
"sessions": [
{
"id": "session-123",
"startTime": "2025-01-15T10:00:00Z",
"participants": [],
}
]
}
mock_response.raise_for_status.return_value = None
mock_client.get.return_value = mock_response
sessions = await self.client.get_room_sessions("test-room")
assert "sessions" in sessions
assert len(sessions["sessions"]) == 1
assert sessions["sessions"][0]["id"] == "session-123"
# Verify HTTP call
mock_client.get.assert_called_once()
async def test_delete_room(self):
"""Test room deletion (no-op for Whereby)."""
result = await self.client.delete_room("test-room")
assert result is True
@patch("httpx.AsyncClient")
async def test_upload_logo_success(self, mock_client_class):
"""Test logo upload success."""
mock_client = mock_client_class.return_value.__aenter__.return_value
mock_response = Mock()
mock_response.raise_for_status.return_value = None
mock_client.put.return_value = mock_response
# Create a temporary file for testing
import tempfile
with tempfile.NamedTemporaryFile(mode="w", suffix=".png", delete=False) as f:
f.write("fake logo content")
temp_file = f.name
result = await self.client.upload_logo("test-room", temp_file)
assert result is True
# Verify HTTP call
mock_client.put.assert_called_once()
# Cleanup
import os
os.unlink(temp_file)
@patch("httpx.AsyncClient")
async def test_upload_logo_failure(self, mock_client_class):
"""Test logo upload handles HTTP errors gracefully."""
mock_client = mock_client_class.return_value.__aenter__.return_value
mock_client.put.side_effect = Exception("HTTP error")
result = await self.client.upload_logo("test-room", "logo.png")
assert result is False
def test_verify_webhook_signature_valid(self):
"""Test Whereby webhook signature verification with valid signature."""
body = b'{"event": "test"}'
import hmac
from hashlib import sha256
expected_signature = hmac.new(
self.config.webhook_secret.encode(), body, sha256
).hexdigest()
result = self.client.verify_webhook_signature(body, expected_signature)
assert result is True
def test_verify_webhook_signature_invalid(self):
"""Test Whereby webhook signature verification with invalid signature."""
body = b'{"event": "test"}'
invalid_signature = "invalid-signature"
result = self.client.verify_webhook_signature(body, invalid_signature)
assert result is False
class TestPlatformRegistry:
"""Test platform registry functionality."""
def test_platform_registration(self):
"""Test platform registration and retrieval."""
# Create mock client class
class MockClient(VideoPlatformClient):
async def create_meeting(self, room_name_prefix, end_date, room):
pass
async def get_room_sessions(self, room_name):
pass
async def delete_room(self, room_name):
pass
async def upload_logo(self, room_name, logo_path):
pass
def verify_webhook_signature(self, body, signature, timestamp=None):
pass
# Register mock platform
register_platform("test-platform", MockClient)
# Verify it's available
available = get_available_platforms()
assert "test-platform" in available
# Test client creation
config = VideoPlatformConfig(
api_key="test", webhook_secret="test", api_url="test"
)
client = get_platform_client("test-platform", config)
assert isinstance(client, MockClient)
def test_get_unknown_platform_raises_error(self):
"""Test that requesting unknown platform raises error."""
config = VideoPlatformConfig(
api_key="test", webhook_secret="test", api_url="test"
)
with pytest.raises(ValueError, match="Unknown video platform: nonexistent"):
get_platform_client("nonexistent", config)
def test_builtin_platforms_registered(self):
"""Test that built-in platforms are registered."""
available = get_available_platforms()
assert "jitsi" in available
assert "whereby" in available
class TestPlatformFactory:
"""Test platform factory functionality."""
@patch("reflector.settings.settings.JITSI_JWT_SECRET", "test-secret")
@patch("reflector.settings.settings.JITSI_WEBHOOK_SECRET", "webhook-secret")
@patch("reflector.settings.settings.JITSI_DOMAIN", "meet.example.com")
def test_get_jitsi_platform_config(self):
"""Test Jitsi platform configuration."""
config = get_platform_config("jitsi")
assert config.api_key == "" # Jitsi uses JWT, no API key
assert config.webhook_secret == "webhook-secret"
assert config.api_url == "https://meet.example.com"
@patch("reflector.settings.settings.WHEREBY_API_KEY", "whereby-key")
@patch("reflector.settings.settings.WHEREBY_WEBHOOK_SECRET", "whereby-secret")
@patch("reflector.settings.settings.WHEREBY_API_URL", "https://api.whereby.dev")
def test_get_whereby_platform_config(self):
"""Test Whereby platform configuration."""
config = get_platform_config("whereby")
assert config.api_key == "whereby-key"
assert config.webhook_secret == "whereby-secret"
assert config.api_url == "https://api.whereby.dev"
def test_get_unknown_platform_config_raises_error(self):
"""Test that unknown platform config raises error."""
with pytest.raises(ValueError, match="Unknown platform: nonexistent"):
get_platform_config("nonexistent")
def test_create_platform_client(self):
"""Test platform client creation via factory."""
with patch(
"reflector.video_platforms.factory.get_platform_config"
) as mock_config:
mock_config.return_value = VideoPlatformConfig(
api_key="",
webhook_secret="test-secret",
api_url="https://meet.example.com",
)
client = create_platform_client("jitsi")
assert isinstance(client, JitsiClient)
def test_create_jitsi_client_typing(self):
"""Test that create_platform_client returns correctly typed JitsiClient."""
with patch(
"reflector.video_platforms.factory.get_platform_config"
) as mock_config:
mock_config.return_value = VideoPlatformConfig(
api_key="",
webhook_secret="test-secret",
api_url="https://meet.example.com",
)
# The typing overload should ensure this returns JitsiClient
client = create_platform_client("jitsi")
assert isinstance(client, JitsiClient)
# Verify it has Jitsi-specific methods
assert hasattr(client, "_generate_jwt")
def test_create_whereby_client_typing(self):
"""Test that create_platform_client returns correctly typed WherebyClient."""
with patch(
"reflector.video_platforms.factory.get_platform_config"
) as mock_config:
mock_config.return_value = VideoPlatformConfig(
api_key="whereby-key",
webhook_secret="whereby-secret",
api_url="https://api.whereby.dev",
)
# The typing overload should ensure this returns WherebyClient
client = create_platform_client("whereby")
assert isinstance(client, WherebyClient)
# Verify it has Whereby-specific attributes
assert hasattr(client, "headers")
assert hasattr(client, "timeout")
class TestWebhookEventStorage:
"""Test webhook event storage functionality."""
def setup_method(self):
"""Set up test fixtures."""
from reflector.app import app
self.client = TestClient(app)
@patch("reflector.db.meetings.meetings_controller.participant_joined")
@patch("reflector.db.meetings.meetings_controller.get_by_room_name")
@patch(
"reflector.video_platforms.jitsi.router.verify_jitsi_webhook_signature",
return_value=True,
)
def test_participant_joined_event_storage(
self, mock_verify, mock_get, mock_participant_joined
):
"""Test that participant joined events are stored correctly."""
# Mock meeting
mock_meeting = Mock()
mock_meeting.id = "test-meeting-id"
mock_meeting.num_clients = 1
mock_get.return_value = mock_meeting
payload = {
"event": "muc-occupant-joined",
"room": "test-room",
"timestamp": "2025-01-15T10:30:00.000Z",
"data": {"user_id": "test-user", "display_name": "John Doe"},
}
response = self.client.post(
"/v1/jitsi/events",
json=payload,
headers={"x-jitsi-signature": "valid-signature"},
)
assert response.status_code == 200
# Verify event was stored with correct data
mock_participant_joined.assert_called_once_with(
"test-meeting-id",
{
"timestamp": datetime.fromisoformat(
"2025-01-15T10:30:00.000Z".replace("Z", "+00:00")
),
"data": {"user_id": "test-user", "display_name": "John Doe"},
},
)
@patch("reflector.db.meetings.meetings_controller.recording_started")
@patch("reflector.db.meetings.meetings_controller.get_by_room_name")
@patch(
"reflector.video_platforms.jitsi.router.verify_jitsi_webhook_signature",
return_value=True,
)
def test_recording_started_event_storage(
self, mock_verify, mock_get, mock_recording_started
):
"""Test that recording started events are stored correctly."""
mock_meeting = Mock()
mock_meeting.id = "test-meeting-id"
mock_meeting.num_clients = 1
mock_get.return_value = mock_meeting
payload = {
"event": "jibri-recording-on",
"room": "test-room",
"timestamp": "2025-01-15T10:32:00.000Z",
"data": {"recording_id": "rec-123"},
}
response = self.client.post(
"/v1/jitsi/events",
json=payload,
headers={"x-jitsi-signature": "valid-signature"},
)
assert response.status_code == 200
mock_recording_started.assert_called_once_with(
"test-meeting-id",
{
"timestamp": datetime.fromisoformat(
"2025-01-15T10:32:00.000Z".replace("Z", "+00:00")
),
"data": {"recording_id": "rec-123"},
},
)
@patch("reflector.db.meetings.meetings_controller.add_event")
@patch("reflector.db.meetings.meetings_controller.get_by_room_name")
@patch(
"reflector.video_platforms.jitsi.router.verify_jitsi_webhook_signature",
return_value=True,
)
def test_recording_complete_event_storage(
self, mock_verify, mock_get, mock_add_event
):
"""Test that recording completion events are stored correctly."""
mock_meeting = Mock()
mock_meeting.id = "test-meeting-id"
mock_meeting.num_clients = 1
mock_get.return_value = mock_meeting
payload = {
"room_name": "test-room",
"recording_file": "/recordings/test.mp4",
"recording_status": "completed",
"timestamp": "2025-01-15T11:15:00.000Z",
}
response = self.client.post(
"/v1/jibri/recording-complete",
json=payload,
headers={"x-jitsi-signature": "valid-signature"},
)
assert response.status_code == 200
mock_add_event.assert_called_once_with(
"test-meeting-id",
"recording_completed",
{
"recording_file": "/recordings/test.mp4",
"recording_status": "completed",
"timestamp": datetime.fromisoformat(
"2025-01-15T11:15:00.000Z".replace("Z", "+00:00")
),
},
)
class TestWebhookEndpoints:
"""Test Jitsi webhook endpoints."""
def setup_method(self):
"""Set up test client."""
from reflector.app import app
self.client = TestClient(app)
def test_health_endpoint(self):
"""Test Jitsi health check endpoint."""
response = self.client.get("/v1/jitsi/health")
assert response.status_code == 200
data = response.json()
assert data["status"] == "ok"
assert data["service"] == "jitsi-webhooks"
assert "timestamp" in data
assert "webhook_secret_configured" in data
@patch(
"reflector.video_platforms.jitsi.router.verify_jitsi_webhook_signature",
return_value=True,
)
@patch("reflector.db.meetings.meetings_controller.get_by_room_name")
@patch("reflector.db.meetings.meetings_controller.participant_joined")
@patch("reflector.db.meetings.meetings_controller.update_meeting")
async def test_jitsi_events_webhook_join(
self, mock_update, mock_participant_joined, mock_get, mock_verify
):
"""Test participant join event webhook."""
# Mock meeting
mock_meeting = Mock()
mock_meeting.id = "test-meeting-id"
mock_meeting.num_clients = 1
mock_get.return_value = mock_meeting
payload = {
"event": "muc-occupant-joined",
"room": "test-room",
"timestamp": "2025-01-15T10:30:00.000Z",
"data": {},
}
response = self.client.post(
"/v1/jitsi/events",
json=payload,
headers={"x-jitsi-signature": "valid-signature"},
)
assert response.status_code == 200
data = response.json()
assert data["status"] == "ok"
assert data["event"] == "muc-occupant-joined"
assert data["room"] == "test-room"
@patch(
"reflector.video_platforms.jitsi.router.verify_jitsi_webhook_signature",
return_value=False,
)
async def test_jitsi_events_webhook_invalid_signature(self, mock_verify):
"""Test webhook with invalid signature returns 401."""
payload = {
"event": "muc-occupant-joined",
"room": "test-room",
"timestamp": "2025-01-15T10:30:00.000Z",
"data": {},
}
response = self.client.post(
"/v1/jitsi/events",
json=payload,
headers={"x-jitsi-signature": "invalid-signature"},
)
assert response.status_code == 401
assert "Invalid webhook signature" in response.text
@patch(
"reflector.video_platforms.jitsi.router.verify_jitsi_webhook_signature",
return_value=True,
)
@patch(
"reflector.db.meetings.meetings_controller.get_by_room_name", return_value=None
)
async def test_jitsi_events_webhook_meeting_not_found(self, mock_get, mock_verify):
"""Test webhook with nonexistent meeting returns 404."""
payload = {
"event": "muc-occupant-joined",
"room": "nonexistent-room",
"timestamp": "2025-01-15T10:30:00.000Z",
"data": {},
}
response = self.client.post(
"/v1/jitsi/events",
json=payload,
headers={"x-jitsi-signature": "valid-signature"},
)
assert response.status_code == 404
assert "Meeting not found" in response.text
class TestRoomsPlatformIntegration:
"""Test rooms endpoint integration with platform abstraction."""
def setup_method(self):
"""Set up test client."""
from reflector.app import app
self.client = TestClient(app)
@patch("reflector.auth.current_user_optional")
@patch("reflector.db.rooms.rooms_controller.add")
def test_create_room_with_jitsi_platform(self, mock_add, mock_auth):
"""Test room creation with Jitsi platform."""
from datetime import datetime, timezone
mock_auth.return_value = {"sub": "test-user"}
# Create a proper Room object for the mock return
from reflector.db.rooms import Room
mock_room = Room(
id="test-room-id",
name="test-jitsi-room",
user_id="test-user",
created_at=datetime.now(timezone.utc),
zulip_auto_post=False,
zulip_stream="",
zulip_topic="",
is_locked=False,
room_mode="normal",
recording_type="cloud",
recording_trigger="automatic-2nd-participant",
is_shared=False,
platform=VideoPlatform.JITSI,
)
mock_add.return_value = mock_room
payload = {
"name": "test-jitsi-room",
"platform": "jitsi",
"zulip_auto_post": False,
"zulip_stream": "",
"zulip_topic": "",
"is_locked": False,
"room_mode": "normal",
"recording_type": "cloud",
"recording_trigger": "automatic-2nd-participant",
"is_shared": False,
"webhook_url": "",
"webhook_secret": "",
}
response = self.client.post("/v1/rooms", json=payload)
# Verify the add method was called with platform parameter
mock_add.assert_called_once()
call_args = mock_add.call_args
assert call_args.kwargs["platform"] == "jitsi"
assert call_args.kwargs["name"] == "test-jitsi-room"
assert response.status_code == 200
def test_create_meeting_with_jitsi_platform_fallback(self):
"""Test that meeting creation falls back to whereby when platform client unavailable."""
# This tests the fallback behavior in rooms.py when platform client returns None
# The actual platform integration test is covered in the unit tests above
# Just verify the endpoint exists and has the right structure
# More detailed integration testing would require a full test database setup
assert hasattr(self.client.app, "routes")
# Find the meeting creation route
meeting_routes = [
r
for r in self.client.app.routes
if hasattr(r, "path") and "meeting" in r.path
]
assert len(meeting_routes) > 0

35
server/uv.lock generated
View File

@@ -2706,6 +2706,15 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/0c/7f/113b16d55e8d2dd9143628eec39b138fd6c52f72dcd11b4dae4a3845da4d/pyinstrument-5.0.3-cp312-cp312-win_amd64.whl", hash = "sha256:88df7e3ab11604ae7cef1f576c097a08752bf8fc13c5755803bd3cd92f15aba3", size = 124314, upload-time = "2025-07-02T14:13:26.708Z" },
]
[[package]]
name = "pyjwt"
version = "2.10.1"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/e7/46/bd74733ff231675599650d3e47f361794b22ef3e3770998dda30d3b63726/pyjwt-2.10.1.tar.gz", hash = "sha256:3cc5772eb20009233caf06e9d8a0577824723b44e6648ee0a2aedb6cf9381953", size = 87785, upload-time = "2024-11-28T03:43:29.933Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/61/ad/689f02752eeec26aed679477e80e632ef1b682313be70793d798c1d5fc8f/PyJWT-2.10.1-py3-none-any.whl", hash = "sha256:dcdd193e30abefd5debf142f9adfcdd2b58004e644f25406ffaebd50bd98dacb", size = 22997, upload-time = "2024-11-28T03:43:27.893Z" },
]
[[package]]
name = "pylibsrtp"
version = "0.12.0"
@@ -3136,6 +3145,7 @@ dependencies = [
{ name = "protobuf" },
{ name = "psycopg2-binary" },
{ name = "pydantic-settings" },
{ name = "pyjwt" },
{ name = "pytest-env" },
{ name = "python-jose", extra = ["cryptography"] },
{ name = "python-multipart" },
@@ -3213,6 +3223,7 @@ requires-dist = [
{ name = "protobuf", specifier = ">=4.24.3" },
{ name = "psycopg2-binary", specifier = ">=2.9.10" },
{ name = "pydantic-settings", specifier = ">=2.0.2" },
{ name = "pyjwt", specifier = ">=2.8.0" },
{ name = "pytest-env", specifier = ">=1.1.5" },
{ name = "python-jose", extras = ["cryptography"], specifier = ">=3.3.0" },
{ name = "python-multipart", specifier = ">=0.0.6" },
@@ -3954,8 +3965,8 @@ dependencies = [
{ name = "typing-extensions", marker = "platform_python_implementation != 'PyPy' and sys_platform == 'darwin'" },
]
wheels = [
{ url = "https://download.pytorch.org/whl/cpu/torch-2.8.0-cp311-none-macosx_11_0_arm64.whl", hash = "sha256:3d05017d19bc99741288e458888283a44b0ee881d53f05f72f8b1cfea8998122" },
{ url = "https://download.pytorch.org/whl/cpu/torch-2.8.0-cp312-none-macosx_11_0_arm64.whl", hash = "sha256:a47b7986bee3f61ad217d8a8ce24605809ab425baf349f97de758815edd2ef54" },
{ url = "https://download.pytorch.org/whl/cpu/torch-2.8.0-cp311-none-macosx_11_0_arm64.whl" },
{ url = "https://download.pytorch.org/whl/cpu/torch-2.8.0-cp312-none-macosx_11_0_arm64.whl" },
]
[[package]]
@@ -3980,16 +3991,16 @@ dependencies = [
{ name = "typing-extensions", marker = "platform_python_implementation == 'PyPy' or sys_platform != 'darwin'" },
]
wheels = [
{ url = "https://download.pytorch.org/whl/cpu/torch-2.8.0%2Bcpu-cp311-cp311-linux_s390x.whl", hash = "sha256:2bfc013dd6efdc8f8223a0241d3529af9f315dffefb53ffa3bf14d3f10127da6" },
{ url = "https://download.pytorch.org/whl/cpu/torch-2.8.0%2Bcpu-cp311-cp311-manylinux_2_28_aarch64.whl", hash = "sha256:680129efdeeec3db5da3f88ee5d28c1b1e103b774aef40f9d638e2cce8f8d8d8" },
{ url = "https://download.pytorch.org/whl/cpu/torch-2.8.0%2Bcpu-cp311-cp311-manylinux_2_28_x86_64.whl", hash = "sha256:cb06175284673a581dd91fb1965662ae4ecaba6e5c357aa0ea7bb8b84b6b7eeb" },
{ url = "https://download.pytorch.org/whl/cpu/torch-2.8.0%2Bcpu-cp311-cp311-win_amd64.whl", hash = "sha256:7631ef49fbd38d382909525b83696dc12a55d68492ade4ace3883c62b9fc140f" },
{ url = "https://download.pytorch.org/whl/cpu/torch-2.8.0%2Bcpu-cp311-cp311-win_arm64.whl", hash = "sha256:41e6fc5ec0914fcdce44ccf338b1d19a441b55cafdd741fd0bf1af3f9e4cfd14" },
{ url = "https://download.pytorch.org/whl/cpu/torch-2.8.0%2Bcpu-cp312-cp312-linux_s390x.whl", hash = "sha256:0e34e276722ab7dd0dffa9e12fe2135a9b34a0e300c456ed7ad6430229404eb5" },
{ url = "https://download.pytorch.org/whl/cpu/torch-2.8.0%2Bcpu-cp312-cp312-manylinux_2_28_aarch64.whl", hash = "sha256:610f600c102386e581327d5efc18c0d6edecb9820b4140d26163354a99cd800d" },
{ url = "https://download.pytorch.org/whl/cpu/torch-2.8.0%2Bcpu-cp312-cp312-manylinux_2_28_x86_64.whl", hash = "sha256:cb9a8ba8137ab24e36bf1742cb79a1294bd374db570f09fc15a5e1318160db4e" },
{ url = "https://download.pytorch.org/whl/cpu/torch-2.8.0%2Bcpu-cp312-cp312-win_amd64.whl", hash = "sha256:2be20b2c05a0cce10430cc25f32b689259640d273232b2de357c35729132256d" },
{ url = "https://download.pytorch.org/whl/cpu/torch-2.8.0%2Bcpu-cp312-cp312-win_arm64.whl", hash = "sha256:99fc421a5d234580e45957a7b02effbf3e1c884a5dd077afc85352c77bf41434" },
{ url = "https://download.pytorch.org/whl/cpu/torch-2.8.0%2Bcpu-cp311-cp311-linux_s390x.whl" },
{ url = "https://download.pytorch.org/whl/cpu/torch-2.8.0%2Bcpu-cp311-cp311-manylinux_2_28_aarch64.whl" },
{ url = "https://download.pytorch.org/whl/cpu/torch-2.8.0%2Bcpu-cp311-cp311-manylinux_2_28_x86_64.whl" },
{ url = "https://download.pytorch.org/whl/cpu/torch-2.8.0%2Bcpu-cp311-cp311-win_amd64.whl" },
{ url = "https://download.pytorch.org/whl/cpu/torch-2.8.0%2Bcpu-cp311-cp311-win_arm64.whl" },
{ url = "https://download.pytorch.org/whl/cpu/torch-2.8.0%2Bcpu-cp312-cp312-linux_s390x.whl" },
{ url = "https://download.pytorch.org/whl/cpu/torch-2.8.0%2Bcpu-cp312-cp312-manylinux_2_28_aarch64.whl" },
{ url = "https://download.pytorch.org/whl/cpu/torch-2.8.0%2Bcpu-cp312-cp312-manylinux_2_28_x86_64.whl" },
{ url = "https://download.pytorch.org/whl/cpu/torch-2.8.0%2Bcpu-cp312-cp312-win_amd64.whl" },
{ url = "https://download.pytorch.org/whl/cpu/torch-2.8.0%2Bcpu-cp312-cp312-win_arm64.whl" },
]
[[package]]

34
www/.env.example Normal file
View File

@@ -0,0 +1,34 @@
# Environment
ENVIRONMENT=development
NEXT_PUBLIC_ENV=development
# Site Configuration
NEXT_PUBLIC_SITE_URL=http://localhost:3000
# Nextauth envs
# not used in app code but in lib code
NEXTAUTH_URL=http://localhost:3000
NEXTAUTH_SECRET=your-nextauth-secret-here
# / Nextauth envs
# Authentication (Authentik OAuth/OIDC)
AUTHENTIK_ISSUER=https://authentik.example.com/application/o/reflector
AUTHENTIK_REFRESH_TOKEN_URL=https://authentik.example.com/application/o/token/
AUTHENTIK_CLIENT_ID=your-client-id-here
AUTHENTIK_CLIENT_SECRET=your-client-secret-here
# Feature Flags
# NEXT_PUBLIC_FEATURE_REQUIRE_LOGIN=true
# NEXT_PUBLIC_FEATURE_PRIVACY=false
# NEXT_PUBLIC_FEATURE_BROWSE=true
# NEXT_PUBLIC_FEATURE_SEND_TO_ZULIP=true
# NEXT_PUBLIC_FEATURE_ROOMS=true
# API URLs
NEXT_PUBLIC_API_URL=http://127.0.0.1:1250
NEXT_PUBLIC_WEBSOCKET_URL=ws://127.0.0.1:1250
NEXT_PUBLIC_AUTH_CALLBACK_URL=http://localhost:3000/auth-callback
# Sentry
# SENTRY_DSN=https://your-dsn@sentry.io/project-id
# SENTRY_IGNORE_API_RESOLUTION_ERROR=1

46
www/.env.template Normal file
View File

@@ -0,0 +1,46 @@
# NextAuth configuration
NEXTAUTH_SECRET="your-secret-key"
NEXTAUTH_URL="http://localhost:3000/"
# API configuration
NEXT_PUBLIC_API_URL="http://127.0.0.1:1250"
NEXT_PUBLIC_WEBSOCKET_URL="ws://127.0.0.1:1250"
NEXT_PUBLIC_AUTH_CALLBACK_URL="http://localhost:3000/auth-callback"
NEXT_PUBLIC_SITE_URL="http://localhost:3000/"
# Environment
NEXT_PUBLIC_ENV="development"
ENVIRONMENT="development"
# Video Platform Configuration
# Options: "whereby" | "jitsi" (default: whereby)
NEXT_PUBLIC_VIDEO_PLATFORM="whereby"
# Features
NEXT_PUBLIC_PROJECTOR_MODE="false"
# Authentication providers (optional)
# Authentik
AUTHENTIK_CLIENT_ID=""
AUTHENTIK_CLIENT_SECRET=""
AUTHENTIK_ISSUER=""
AUTHENTIK_REFRESH_TOKEN_URL=""
# Fief
FIEF_CLIENT_ID=""
FIEF_CLIENT_SECRET=""
FIEF_URL=""
# Zulip integration (optional)
ZULIP_API_KEY=""
ZULIP_BOT_EMAIL=""
ZULIP_REALM=""
# External services (optional)
ZEPHYR_LLM_URL=""
# Redis/KV (optional)
KV_REST_API_TOKEN=""
KV_REST_API_READ_ONLY_TOKEN=""
KV_REST_API_URL=""
KV_URL=""

1
www/.gitignore vendored
View File

@@ -40,7 +40,6 @@ next-env.d.ts
# Sentry Auth Token
.sentryclirc
config.ts
# openapi logs
openapi-ts-error-*.log

View File

@@ -2,6 +2,7 @@
import { Flex, Spinner } from "@chakra-ui/react";
import { useAuth } from "../lib/AuthProvider";
import { useLoginRequiredPages } from "../lib/useLoginRequiredPages";
export default function AuthWrapper({
children,
@@ -9,8 +10,10 @@ export default function AuthWrapper({
children: React.ReactNode;
}) {
const auth = useAuth();
const redirectPath = useLoginRequiredPages();
const redirectHappens = !!redirectPath;
if (auth.status === "loading") {
if (auth.status === "loading" || redirectHappens) {
return (
<Flex
flexDir="column"

View File

@@ -7,9 +7,10 @@ import {
FaMicrophone,
FaGear,
} from "react-icons/fa6";
import { TranscriptStatus } from "../../../lib/transcript";
interface TranscriptStatusIconProps {
status: string;
status: TranscriptStatus;
}
export default function TranscriptStatusIcon({

View File

@@ -1,5 +1,5 @@
import { Container, Flex, Link } from "@chakra-ui/react";
import { getConfig } from "../lib/edgeConfig";
import { featureEnabled } from "../lib/features";
import NextLink from "next/link";
import Image from "next/image";
import UserInfo from "../(auth)/userInfo";
@@ -11,8 +11,6 @@ export default async function AppLayout({
}: {
children: React.ReactNode;
}) {
const config = await getConfig();
const { requireLogin, privacy, browse, rooms } = config.features;
return (
<Container
minW="100vw"
@@ -58,7 +56,7 @@ export default async function AppLayout({
>
Create
</Link>
{browse ? (
{featureEnabled("browse") ? (
<>
&nbsp;·&nbsp;
<Link href="/browse" as={NextLink} className="font-light px-2">
@@ -68,7 +66,7 @@ export default async function AppLayout({
) : (
<></>
)}
{rooms ? (
{featureEnabled("rooms") ? (
<>
&nbsp;·&nbsp;
<Link href="/rooms" as={NextLink} className="font-light px-2">
@@ -78,7 +76,7 @@ export default async function AppLayout({
) : (
<></>
)}
{requireLogin ? (
{featureEnabled("requireLogin") ? (
<>
&nbsp;·&nbsp;
<UserInfo />

View File

@@ -10,12 +10,17 @@ import {
Text,
VStack,
HStack,
Badge,
} from "@chakra-ui/react";
import { LuLink } from "react-icons/lu";
import type { components } from "../../../reflector-api";
type Room = components["schemas"]["Room"];
import { RoomActionsMenu } from "./RoomActionsMenu";
import {
getPlatformDisplayName,
getPlatformColor,
} from "../../../lib/videoPlatforms";
interface RoomCardsProps {
rooms: Room[];
@@ -95,6 +100,15 @@ export function RoomCards({
/>
</Flex>
<VStack align="start" fontSize="sm" gap={0}>
<HStack gap={2}>
<Text fontWeight="500">Platform:</Text>
<Badge
colorPalette={getPlatformColor(room.platform)}
size="sm"
>
{getPlatformDisplayName(room.platform)}
</Badge>
</HStack>
{room.zulip_auto_post && (
<HStack gap={2}>
<Text fontWeight="500">Zulip:</Text>

View File

@@ -7,12 +7,17 @@ import {
IconButton,
Text,
Spinner,
Badge,
} from "@chakra-ui/react";
import { LuLink } from "react-icons/lu";
import type { components } from "../../../reflector-api";
type Room = components["schemas"]["Room"];
import { RoomActionsMenu } from "./RoomActionsMenu";
import {
getPlatformDisplayName,
getPlatformColor,
} from "../../../lib/videoPlatforms";
interface RoomTableProps {
rooms: Room[];
@@ -94,16 +99,19 @@ export function RoomTable({
<Table.Root>
<Table.Header>
<Table.Row>
<Table.ColumnHeader width="250px" fontWeight="600">
<Table.ColumnHeader width="200px" fontWeight="600">
Room Name
</Table.ColumnHeader>
<Table.ColumnHeader width="250px" fontWeight="600">
Zulip
</Table.ColumnHeader>
<Table.ColumnHeader width="150px" fontWeight="600">
Room Size
<Table.ColumnHeader width="120px" fontWeight="600">
Platform
</Table.ColumnHeader>
<Table.ColumnHeader width="200px" fontWeight="600">
Zulip
</Table.ColumnHeader>
<Table.ColumnHeader width="130px" fontWeight="600">
Room Size
</Table.ColumnHeader>
<Table.ColumnHeader width="180px" fontWeight="600">
Recording
</Table.ColumnHeader>
<Table.ColumnHeader
@@ -118,6 +126,14 @@ export function RoomTable({
<Table.Cell>
<Link href={`/${room.name}`}>{room.name}</Link>
</Table.Cell>
<Table.Cell>
<Badge
colorPalette={getPlatformColor(room.platform)}
size="sm"
>
{getPlatformDisplayName(room.platform)}
</Badge>
</Table.Cell>
<Table.Cell>
{getZulipDisplay(
room.zulip_auto_post,

View File

@@ -3,8 +3,10 @@ import ScrollToBottom from "../../scrollToBottom";
import { Topic } from "../../webSocketTypes";
import useParticipants from "../../useParticipants";
import { Box, Flex, Text, Accordion } from "@chakra-ui/react";
import { featureEnabled } from "../../../../domainContext";
import { TopicItem } from "./TopicItem";
import { TranscriptStatus } from "../../../../lib/transcript";
import { featureEnabled } from "../../../../lib/features";
type TopicListProps = {
topics: Topic[];
@@ -14,7 +16,7 @@ type TopicListProps = {
];
autoscroll: boolean;
transcriptId: string;
status: string;
status: TranscriptStatus | null;
currentTranscriptText: any;
};

View File

@@ -1,5 +1,5 @@
"use client";
import { useState } from "react";
import { useState, use } from "react";
import TopicHeader from "./topicHeader";
import TopicWords from "./topicWords";
import TopicPlayer from "./topicPlayer";
@@ -9,23 +9,27 @@ import ParticipantList from "./participantList";
import type { components } from "../../../../reflector-api";
type GetTranscriptTopic = components["schemas"]["GetTranscriptTopic"];
import { SelectedText, selectedTextIsTimeSlice } from "./types";
import { useTranscriptUpdate } from "../../../../lib/apiHooks";
import useTranscript from "../../useTranscript";
import {
useTranscriptGet,
useTranscriptUpdate,
} from "../../../../lib/apiHooks";
import { useError } from "../../../../(errors)/errorContext";
import { useRouter } from "next/navigation";
import { Box, Grid } from "@chakra-ui/react";
export type TranscriptCorrect = {
params: {
params: Promise<{
transcriptId: string;
};
}>;
};
export default function TranscriptCorrect({
params: { transcriptId },
}: TranscriptCorrect) {
export default function TranscriptCorrect(props: TranscriptCorrect) {
const params = use(props.params);
const { transcriptId } = params;
const updateTranscriptMutation = useTranscriptUpdate();
const transcript = useTranscript(transcriptId);
const transcript = useTranscriptGet(transcriptId);
const stateCurrentTopic = useState<GetTranscriptTopic>();
const [currentTopic, _sct] = stateCurrentTopic;
const stateSelectedText = useState<SelectedText>();
@@ -36,7 +40,7 @@ export default function TranscriptCorrect({
const router = useRouter();
const markAsDone = async () => {
if (transcript.response && !transcript.response.reviewed) {
if (transcript.data && !transcript.data.reviewed) {
try {
await updateTranscriptMutation.mutateAsync({
params: {
@@ -114,7 +118,7 @@ export default function TranscriptCorrect({
}}
/>
</Grid>
{transcript.response && !transcript.response?.reviewed && (
{transcript.data && !transcript.data?.reviewed && (
<div className="flex flex-row justify-end">
<button
className="p-2 px-4 rounded bg-green-400"

View File

@@ -1,32 +1,38 @@
"use client";
import Modal from "../modal";
import useTranscript from "../useTranscript";
import useTopics from "../useTopics";
import useWaveform from "../useWaveform";
import useMp3 from "../useMp3";
import { TopicList } from "./_components/TopicList";
import { Topic } from "../webSocketTypes";
import React, { useEffect, useState } from "react";
import React, { useEffect, useState, use } from "react";
import FinalSummary from "./finalSummary";
import TranscriptTitle from "../transcriptTitle";
import Player from "../player";
import { useRouter } from "next/navigation";
import { Box, Flex, Grid, GridItem, Skeleton, Text } from "@chakra-ui/react";
import { useTranscriptGet } from "../../../lib/apiHooks";
import { TranscriptStatus } from "../../../lib/transcript";
type TranscriptDetails = {
params: {
params: Promise<{
transcriptId: string;
};
}>;
};
export default function TranscriptDetails(details: TranscriptDetails) {
const transcriptId = details.params.transcriptId;
const params = use(details.params);
const transcriptId = params.transcriptId;
const router = useRouter();
const statusToRedirect = ["idle", "recording", "processing"];
const statusToRedirect = [
"idle",
"recording",
"processing",
] satisfies TranscriptStatus[] as TranscriptStatus[];
const transcript = useTranscript(transcriptId);
const transcriptStatus = transcript.response?.status;
const waiting = statusToRedirect.includes(transcriptStatus || "");
const transcript = useTranscriptGet(transcriptId);
const waiting =
transcript.data && statusToRedirect.includes(transcript.data.status);
const mp3 = useMp3(transcriptId, waiting);
const topics = useTopics(transcriptId);
@@ -38,7 +44,7 @@ export default function TranscriptDetails(details: TranscriptDetails) {
useEffect(() => {
if (waiting) {
const newUrl = "/transcripts/" + details.params.transcriptId + "/record";
const newUrl = "/transcripts/" + params.transcriptId + "/record";
// Shallow redirection does not work on NextJS 13
// https://github.com/vercel/next.js/discussions/48110
// https://github.com/vercel/next.js/discussions/49540
@@ -56,7 +62,7 @@ export default function TranscriptDetails(details: TranscriptDetails) {
);
}
if (transcript?.loading || topics?.loading) {
if (transcript?.isLoading || topics?.loading) {
return <Modal title="Loading" text={"Loading transcript..."} />;
}
@@ -86,7 +92,7 @@ export default function TranscriptDetails(details: TranscriptDetails) {
useActiveTopic={useActiveTopic}
waveform={waveform.waveform}
media={mp3.media}
mediaDuration={transcript.response?.duration || null}
mediaDuration={transcript.data?.duration || null}
/>
) : !mp3.loading && (waveform.error || mp3.error) ? (
<Box p={4} bg="red.100" borderRadius="md">
@@ -116,10 +122,10 @@ export default function TranscriptDetails(details: TranscriptDetails) {
<Flex direction="column" gap={0}>
<Flex alignItems="center" gap={2}>
<TranscriptTitle
title={transcript.response?.title || "Unnamed Transcript"}
title={transcript.data?.title || "Unnamed Transcript"}
transcriptId={transcriptId}
onUpdate={(newTitle) => {
transcript.reload();
transcript.refetch().then(() => {});
}}
/>
</Flex>
@@ -136,23 +142,23 @@ export default function TranscriptDetails(details: TranscriptDetails) {
useActiveTopic={useActiveTopic}
autoscroll={false}
transcriptId={transcriptId}
status={transcript.response?.status}
status={transcript.data?.status || null}
currentTranscriptText=""
/>
{transcript.response && topics.topics ? (
{transcript.data && topics.topics ? (
<>
<FinalSummary
transcriptResponse={transcript.response}
transcriptResponse={transcript.data}
topicsResponse={topics.topics}
onUpdate={(newSummary) => {
transcript.reload();
onUpdate={() => {
transcript.refetch();
}}
/>
</>
) : (
<Flex justify={"center"} alignItems={"center"} h={"100%"}>
<div className="flex flex-col h-full justify-center content-center">
{transcript.response.status == "processing" ? (
{transcript?.data?.status == "processing" ? (
<Text>Loading Transcript</Text>
) : (
<Text>

View File

@@ -1,8 +1,7 @@
"use client";
import { useEffect, useState } from "react";
import { useEffect, useState, use } from "react";
import Recorder from "../../recorder";
import { TopicList } from "../_components/TopicList";
import useTranscript from "../../useTranscript";
import { useWebSockets } from "../../useWebSockets";
import { Topic } from "../../webSocketTypes";
import { lockWakeState, releaseWakeState } from "../../../../lib/wakeLock";
@@ -11,26 +10,29 @@ import useMp3 from "../../useMp3";
import WaveformLoading from "../../waveformLoading";
import { Box, Text, Grid, Heading, VStack, Flex } from "@chakra-ui/react";
import LiveTrancription from "../../liveTranscription";
import { useTranscriptGet } from "../../../../lib/apiHooks";
import { TranscriptStatus } from "../../../../lib/transcript";
type TranscriptDetails = {
params: {
params: Promise<{
transcriptId: string;
};
}>;
};
const TranscriptRecord = (details: TranscriptDetails) => {
const transcript = useTranscript(details.params.transcriptId);
const params = use(details.params);
const transcript = useTranscriptGet(params.transcriptId);
const [transcriptStarted, setTranscriptStarted] = useState(false);
const useActiveTopic = useState<Topic | null>(null);
const webSockets = useWebSockets(details.params.transcriptId);
const webSockets = useWebSockets(params.transcriptId);
const mp3 = useMp3(details.params.transcriptId, true);
const mp3 = useMp3(params.transcriptId, true);
const router = useRouter();
const [status, setStatus] = useState(
webSockets.status.value || transcript.response?.status || "idle",
const [status, setStatus] = useState<TranscriptStatus>(
webSockets.status?.value || transcript.data?.status || "idle",
);
useEffect(() => {
@@ -41,15 +43,15 @@ const TranscriptRecord = (details: TranscriptDetails) => {
useEffect(() => {
//TODO HANDLE ERROR STATUS BETTER
const newStatus =
webSockets.status.value || transcript.response?.status || "idle";
webSockets.status?.value || transcript.data?.status || "idle";
setStatus(newStatus);
if (newStatus && (newStatus == "ended" || newStatus == "error")) {
console.log(newStatus, "redirecting");
const newUrl = "/transcripts/" + details.params.transcriptId;
const newUrl = "/transcripts/" + params.transcriptId;
router.replace(newUrl);
}
}, [webSockets.status.value, transcript.response?.status]);
}, [webSockets.status?.value, transcript.data?.status]);
useEffect(() => {
if (webSockets.waveform && webSockets.waveform) mp3.getNow();
@@ -74,7 +76,7 @@ const TranscriptRecord = (details: TranscriptDetails) => {
<WaveformLoading />
) : (
// todo: only start recording animation when you get "recorded" status
<Recorder transcriptId={details.params.transcriptId} status={status} />
<Recorder transcriptId={params.transcriptId} status={status} />
)}
<VStack
align={"left"}
@@ -97,7 +99,7 @@ const TranscriptRecord = (details: TranscriptDetails) => {
topics={webSockets.topics}
useActiveTopic={useActiveTopic}
autoscroll={true}
transcriptId={details.params.transcriptId}
transcriptId={params.transcriptId}
status={status}
currentTranscriptText={webSockets.accumulatedText}
/>

View File

@@ -1,37 +1,38 @@
"use client";
import { useEffect, useState } from "react";
import useTranscript from "../../useTranscript";
import { useEffect, useState, use } from "react";
import { useWebSockets } from "../../useWebSockets";
import { lockWakeState, releaseWakeState } from "../../../../lib/wakeLock";
import { useRouter } from "next/navigation";
import useMp3 from "../../useMp3";
import { Center, VStack, Text, Heading, Button } from "@chakra-ui/react";
import FileUploadButton from "../../fileUploadButton";
import { useTranscriptGet } from "../../../../lib/apiHooks";
type TranscriptUpload = {
params: {
params: Promise<{
transcriptId: string;
};
}>;
};
const TranscriptUpload = (details: TranscriptUpload) => {
const transcript = useTranscript(details.params.transcriptId);
const params = use(details.params);
const transcript = useTranscriptGet(params.transcriptId);
const [transcriptStarted, setTranscriptStarted] = useState(false);
const webSockets = useWebSockets(details.params.transcriptId);
const webSockets = useWebSockets(params.transcriptId);
const mp3 = useMp3(details.params.transcriptId, true);
const mp3 = useMp3(params.transcriptId, true);
const router = useRouter();
const [status_, setStatus] = useState(
webSockets.status.value || transcript.response?.status || "idle",
webSockets.status?.value || transcript.data?.status || "idle",
);
// status is obviously done if we have transcript
const status =
!transcript.loading && transcript.response?.status === "ended"
? transcript.response?.status
!transcript.isLoading && transcript.data?.status === "ended"
? transcript.data?.status
: status_;
useEffect(() => {
@@ -43,17 +44,17 @@ const TranscriptUpload = (details: TranscriptUpload) => {
//TODO HANDLE ERROR STATUS BETTER
// TODO deprecate webSockets.status.value / depend on transcript.response?.status from query lib
const newStatus =
transcript.response?.status === "ended"
transcript.data?.status === "ended"
? "ended"
: webSockets.status.value || transcript.response?.status || "idle";
: webSockets.status?.value || transcript.data?.status || "idle";
setStatus(newStatus);
if (newStatus && (newStatus == "ended" || newStatus == "error")) {
console.log(newStatus, "redirecting");
const newUrl = "/transcripts/" + details.params.transcriptId;
const newUrl = "/transcripts/" + params.transcriptId;
router.replace(newUrl);
}
}, [webSockets.status.value, transcript.response?.status]);
}, [webSockets.status?.value, transcript.data?.status]);
useEffect(() => {
if (webSockets.waveform && webSockets.waveform) mp3.getNow();
@@ -84,7 +85,7 @@ const TranscriptUpload = (details: TranscriptUpload) => {
Please select the file, supported formats: .mp3, m4a, .wav,
.mp4, .mov or .webm
</Text>
<FileUploadButton transcriptId={details.params.transcriptId} />
<FileUploadButton transcriptId={params.transcriptId} />
</>
)}
{status && status == "uploaded" && (

View File

@@ -9,7 +9,6 @@ import { useRouter } from "next/navigation";
import useCreateTranscript from "../createTranscript";
import SelectSearch from "react-select-search";
import { supportedLanguages } from "../../../supportedLanguages";
import { featureEnabled } from "../../../domainContext";
import {
Flex,
Box,
@@ -21,10 +20,9 @@ import {
Spacer,
} from "@chakra-ui/react";
import { useAuth } from "../../../lib/AuthProvider";
import type { components } from "../../../reflector-api";
import { featureEnabled } from "../../../lib/features";
const TranscriptCreate = () => {
const isClient = typeof window !== "undefined";
const router = useRouter();
const auth = useAuth();
const isAuthenticated = auth.status === "authenticated";
@@ -176,7 +174,7 @@ const TranscriptCreate = () => {
placeholder="Choose your language"
/>
</Box>
{isClient && !loading ? (
{!loading ? (
permissionOk ? (
<Spacer />
) : permissionDenied ? (

View File

@@ -11,10 +11,11 @@ import useAudioDevice from "./useAudioDevice";
import { Box, Flex, IconButton, Menu, RadioGroup } from "@chakra-ui/react";
import { LuScreenShare, LuMic, LuPlay, LuCircleStop } from "react-icons/lu";
import { RECORD_A_MEETING_URL } from "../../api/urls";
import { TranscriptStatus } from "../../lib/transcript";
type RecorderProps = {
transcriptId: string;
status: string;
status: TranscriptStatus;
};
export default function Recorder(props: RecorderProps) {

View File

@@ -1,5 +1,4 @@
import { useEffect, useState } from "react";
import { featureEnabled } from "../../domainContext";
import { ShareMode, toShareMode } from "../../lib/shareMode";
import type { components } from "../../reflector-api";
@@ -24,6 +23,8 @@ import ShareCopy from "./shareCopy";
import ShareZulip from "./shareZulip";
import { useAuth } from "../../lib/AuthProvider";
import { featureEnabled } from "../../lib/features";
type ShareAndPrivacyProps = {
finalSummaryRef: any;
transcriptResponse: GetTranscript;

View File

@@ -1,8 +1,9 @@
import React, { useState, useRef, useEffect, use } from "react";
import { featureEnabled } from "../../domainContext";
import { Button, Flex, Input, Text } from "@chakra-ui/react";
import QRCode from "react-qr-code";
import { featureEnabled } from "../../lib/features";
type ShareLinkProps = {
transcriptId: string;
};

View File

@@ -1,5 +1,4 @@
import { useState, useEffect, useMemo } from "react";
import { featureEnabled } from "../../domainContext";
import type { components } from "../../reflector-api";
type GetTranscript = components["schemas"]["GetTranscript"];
@@ -15,8 +14,7 @@ import {
Checkbox,
Combobox,
Spinner,
useFilter,
useListCollection,
createListCollection,
} from "@chakra-ui/react";
import { TbBrandZulip } from "react-icons/tb";
import {
@@ -25,6 +23,8 @@ import {
useTranscriptPostToZulip,
} from "../../lib/apiHooks";
import { featureEnabled } from "../../lib/features";
type ShareZulipProps = {
transcriptResponse: GetTranscript;
topicsResponse: GetTranscriptTopic[];
@@ -47,8 +47,6 @@ export default function ShareZulip(props: ShareZulipProps & BoxProps) {
const { data: topics = [] } = useZulipTopics(selectedStreamId);
const postToZulipMutation = useTranscriptPostToZulip();
const { contains } = useFilter({ sensitivity: "base" });
const streamItems = useMemo(() => {
return streams.map((stream: Stream) => ({
label: stream.name,
@@ -63,17 +61,21 @@ export default function ShareZulip(props: ShareZulipProps & BoxProps) {
}));
}, [topics]);
const { collection: streamItemsCollection, filter: streamItemsFilter } =
useListCollection({
initialItems: streamItems,
filter: contains,
});
const streamCollection = useMemo(
() =>
createListCollection({
items: streamItems,
}),
[streamItems],
);
const { collection: topicItemsCollection, filter: topicItemsFilter } =
useListCollection({
initialItems: topicItems,
filter: contains,
});
const topicCollection = useMemo(
() =>
createListCollection({
items: topicItems,
}),
[topicItems],
);
// Update selected stream ID when stream changes
useEffect(() => {
@@ -155,15 +157,12 @@ export default function ShareZulip(props: ShareZulipProps & BoxProps) {
<Flex align="center" gap={2}>
<Text>#</Text>
<Combobox.Root
collection={streamItemsCollection}
collection={streamCollection}
value={stream ? [stream] : []}
onValueChange={(e) => {
setTopic(undefined);
setStream(e.value[0]);
}}
onInputValueChange={(e) =>
streamItemsFilter(e.inputValue)
}
openOnClick={true}
positioning={{
strategy: "fixed",
@@ -180,7 +179,7 @@ export default function ShareZulip(props: ShareZulipProps & BoxProps) {
<Combobox.Positioner>
<Combobox.Content>
<Combobox.Empty>No streams found</Combobox.Empty>
{streamItemsCollection.items.map((item) => (
{streamItems.map((item) => (
<Combobox.Item key={item.value} item={item}>
{item.label}
</Combobox.Item>
@@ -196,12 +195,9 @@ export default function ShareZulip(props: ShareZulipProps & BoxProps) {
<Flex align="center" gap={2}>
<Text visibility="hidden">#</Text>
<Combobox.Root
collection={topicItemsCollection}
collection={topicCollection}
value={topic ? [topic] : []}
onValueChange={(e) => setTopic(e.value[0])}
onInputValueChange={(e) =>
topicItemsFilter(e.inputValue)
}
openOnClick
selectionBehavior="replace"
skipAnimationOnMount={true}
@@ -221,7 +217,7 @@ export default function ShareZulip(props: ShareZulipProps & BoxProps) {
<Combobox.Positioner>
<Combobox.Content>
<Combobox.Empty>No topics found</Combobox.Empty>
{topicItemsCollection.items.map((item) => (
{topicItems.map((item) => (
<Combobox.Item key={item.value} item={item}>
{item.label}
<Combobox.ItemIndicator />

View File

@@ -1,7 +1,7 @@
import { useContext, useEffect, useState } from "react";
import { DomainContext } from "../../domainContext";
import { useEffect, useState } from "react";
import { useTranscriptGet } from "../../lib/apiHooks";
import { useAuth } from "../../lib/AuthProvider";
import { API_URL } from "../../lib/apiClient";
export type Mp3Response = {
media: HTMLMediaElement | null;
@@ -19,7 +19,6 @@ const useMp3 = (transcriptId: string, waiting?: boolean): Mp3Response => {
null,
);
const [audioDeleted, setAudioDeleted] = useState<boolean | null>(null);
const { api_url } = useContext(DomainContext);
const auth = useAuth();
const accessTokenInfo =
auth.status === "authenticated" ? auth.accessToken : null;
@@ -78,7 +77,7 @@ const useMp3 = (transcriptId: string, waiting?: boolean): Mp3Response => {
// Audio is not deleted, proceed to load it
audioElement = document.createElement("audio");
audioElement.src = `${api_url}/v1/transcripts/${transcriptId}/audio/mp3`;
audioElement.src = `${API_URL}/v1/transcripts/${transcriptId}/audio/mp3`;
audioElement.crossOrigin = "anonymous";
audioElement.preload = "auto";
@@ -110,7 +109,7 @@ const useMp3 = (transcriptId: string, waiting?: boolean): Mp3Response => {
if (handleError) audioElement.removeEventListener("error", handleError);
}
};
}, [transcriptId, transcript, later, api_url]);
}, [transcriptId, transcript, later]);
const getNow = () => {
setLater(false);

View File

@@ -1,69 +0,0 @@
import type { components } from "../../reflector-api";
import { useTranscriptGet } from "../../lib/apiHooks";
type GetTranscript = components["schemas"]["GetTranscript"];
type ErrorTranscript = {
error: Error;
loading: false;
response: null;
reload: () => void;
};
type LoadingTranscript = {
response: null;
loading: true;
error: false;
reload: () => void;
};
type SuccessTranscript = {
response: GetTranscript;
loading: false;
error: null;
reload: () => void;
};
const useTranscript = (
id: string | null,
): ErrorTranscript | LoadingTranscript | SuccessTranscript => {
const { data, isLoading, error, refetch } = useTranscriptGet(id);
// Map to the expected return format
if (isLoading) {
return {
response: null,
loading: true,
error: false,
reload: refetch,
};
}
if (error) {
return {
error: error as Error,
loading: false,
response: null,
reload: refetch,
};
}
// Check if data is undefined or null
if (!data) {
return {
response: null,
loading: true,
error: false,
reload: refetch,
};
}
return {
response: data,
loading: false,
error: null,
reload: refetch,
};
};
export default useTranscript;

View File

@@ -1,13 +1,12 @@
import { useContext, useEffect, useState } from "react";
import { useEffect, useState } from "react";
import { Topic, FinalSummary, Status } from "./webSocketTypes";
import { useError } from "../../(errors)/errorContext";
import { DomainContext } from "../../domainContext";
import type { components } from "../../reflector-api";
type AudioWaveform = components["schemas"]["AudioWaveform"];
type GetTranscriptSegmentTopic =
components["schemas"]["GetTranscriptSegmentTopic"];
import { useQueryClient } from "@tanstack/react-query";
import { $api } from "../../lib/apiClient";
import { $api, WEBSOCKET_URL } from "../../lib/apiClient";
export type UseWebSockets = {
transcriptTextLive: string;
@@ -16,7 +15,7 @@ export type UseWebSockets = {
title: string;
topics: Topic[];
finalSummary: FinalSummary;
status: Status;
status: Status | null;
waveform: AudioWaveform | null;
duration: number | null;
};
@@ -34,10 +33,9 @@ export const useWebSockets = (transcriptId: string | null): UseWebSockets => {
const [finalSummary, setFinalSummary] = useState<FinalSummary>({
summary: "",
});
const [status, setStatus] = useState<Status>({ value: "" });
const [status, setStatus] = useState<Status | null>(null);
const { setError } = useError();
const { websocket_url: websocketUrl } = useContext(DomainContext);
const queryClient = useQueryClient();
const [accumulatedText, setAccumulatedText] = useState<string>("");
@@ -328,7 +326,7 @@ export const useWebSockets = (transcriptId: string | null): UseWebSockets => {
if (!transcriptId) return;
const url = `${websocketUrl}/v1/transcripts/${transcriptId}/events`;
const url = `${WEBSOCKET_URL}/v1/transcripts/${transcriptId}/events`;
let ws = new WebSocket(url);
ws.onopen = () => {
@@ -494,7 +492,7 @@ export const useWebSockets = (transcriptId: string | null): UseWebSockets => {
return () => {
ws.close();
};
}, [transcriptId, websocketUrl]);
}, [transcriptId]);
return {
transcriptTextLive,

View File

@@ -1,4 +1,5 @@
import type { components } from "../../reflector-api";
import type { TranscriptStatus } from "../../lib/transcript";
type GetTranscriptTopic = components["schemas"]["GetTranscriptTopic"];
@@ -13,7 +14,7 @@ export type FinalSummary = {
};
export type Status = {
value: string;
value: TranscriptStatus;
};
export type TranslatedTopic = {

View File

@@ -0,0 +1,326 @@
"use client";
import {
useCallback,
useEffect,
useRef,
useState,
useContext,
RefObject,
} from "react";
import {
Box,
Button,
Text,
VStack,
HStack,
Spinner,
Icon,
} from "@chakra-ui/react";
import { toaster } from "../components/ui/toaster";
import useRoomMeeting from "./useRoomMeeting";
import { useRouter } from "next/navigation";
import { notFound } from "next/navigation";
import useSessionStatus from "../lib/useSessionStatus";
import { useRecordingConsent } from "../recordingConsentContext";
import useApi from "../lib/useApi";
import { Meeting } from "../api";
import { FaBars } from "react-icons/fa6";
export type RoomDetails = {
params: {
roomName: string;
};
};
// stages: we focus on the consent, then whereby steals focus, then we focus on the consent again, then return focus to whoever stole it initially
const useConsentWherebyFocusManagement = (
acceptButtonRef: RefObject<HTMLButtonElement>,
wherebyRef: RefObject<HTMLElement>,
) => {
const currentFocusRef = useRef<HTMLElement | null>(null);
useEffect(() => {
if (acceptButtonRef.current) {
acceptButtonRef.current.focus();
} else {
console.error(
"accept button ref not available yet for focus management - seems to be illegal state",
);
}
const handleWherebyReady = () => {
console.log("whereby ready - refocusing consent button");
currentFocusRef.current = document.activeElement as HTMLElement;
if (acceptButtonRef.current) {
acceptButtonRef.current.focus();
}
};
if (wherebyRef.current) {
wherebyRef.current.addEventListener("ready", handleWherebyReady);
} else {
console.warn(
"whereby ref not available yet for focus management - seems to be illegal state. not waiting, focus management off.",
);
}
return () => {
wherebyRef.current?.removeEventListener("ready", handleWherebyReady);
currentFocusRef.current?.focus();
};
}, []);
};
const useConsentDialog = (
meetingId: string,
wherebyRef: RefObject<HTMLElement> /*accessibility*/,
) => {
const { state: consentState, touch, hasConsent } = useRecordingConsent();
const [consentLoading, setConsentLoading] = useState(false);
// toast would open duplicates, even with using "id=" prop
const [modalOpen, setModalOpen] = useState(false);
const api = useApi();
const handleConsent = useCallback(
async (meetingId: string, given: boolean) => {
if (!api) return;
setConsentLoading(true);
try {
await api.v1MeetingAudioConsent({
meetingId,
requestBody: { consent_given: given },
});
touch(meetingId);
} catch (error) {
console.error("Error submitting consent:", error);
} finally {
setConsentLoading(false);
}
},
[api, touch],
);
const showConsentModal = useCallback(() => {
if (modalOpen) return;
setModalOpen(true);
const toastId = toaster.create({
placement: "top",
duration: null,
render: ({ dismiss }) => {
const AcceptButton = () => {
const buttonRef = useRef<HTMLButtonElement>(null);
useConsentWherebyFocusManagement(buttonRef, wherebyRef);
return (
<Button
ref={buttonRef}
colorPalette="primary"
size="sm"
onClick={() => {
handleConsent(meetingId, true).then(() => {
/*signifies it's ok to now wait here.*/
});
dismiss();
}}
>
Yes, store the audio
</Button>
);
};
return (
<Box
p={6}
bg="rgba(255, 255, 255, 0.7)"
borderRadius="lg"
boxShadow="lg"
maxW="md"
mx="auto"
>
<VStack gap={4} alignItems="center">
<Text fontSize="md" textAlign="center" fontWeight="medium">
Can we have your permission to store this meeting's audio
recording on our servers?
</Text>
<HStack gap={4} justifyContent="center">
<Button
variant="ghost"
size="sm"
onClick={() => {
handleConsent(meetingId, false).then(() => {
/*signifies it's ok to now wait here.*/
});
dismiss();
}}
>
No, delete after transcription
</Button>
<AcceptButton />
</HStack>
</VStack>
</Box>
);
},
});
// Set modal state when toast is dismissed
toastId.then((id) => {
const checkToastStatus = setInterval(() => {
if (!toaster.isActive(id)) {
setModalOpen(false);
clearInterval(checkToastStatus);
}
}, 100);
});
// Handle escape key to close the toast
const handleKeyDown = (event: KeyboardEvent) => {
if (event.key === "Escape") {
toastId.then((id) => toaster.dismiss(id));
}
};
document.addEventListener("keydown", handleKeyDown);
const cleanup = () => {
toastId.then((id) => toaster.dismiss(id));
document.removeEventListener("keydown", handleKeyDown);
};
return cleanup;
}, [meetingId, handleConsent, wherebyRef, modalOpen]);
return { showConsentModal, consentState, hasConsent, consentLoading };
};
function ConsentDialogButton({
meetingId,
wherebyRef,
}: {
meetingId: string;
wherebyRef: React.RefObject<HTMLElement>;
}) {
const { showConsentModal, consentState, hasConsent, consentLoading } =
useConsentDialog(meetingId, wherebyRef);
if (!consentState.ready || hasConsent(meetingId) || consentLoading) {
return null;
}
return (
<Button
position="absolute"
top="56px"
left="8px"
zIndex={1000}
colorPalette="blue"
size="sm"
onClick={showConsentModal}
>
Meeting is being recorded
<Icon as={FaBars} ml={2} />
</Button>
);
}
const recordingTypeRequiresConsent = (
recordingType: NonNullable<Meeting["recording_type"]>,
) => {
return recordingType === "cloud";
};
// next throws even with "use client"
const useWhereby = () => {
const [wherebyLoaded, setWherebyLoaded] = useState(false);
useEffect(() => {
if (typeof window !== "undefined") {
import("@whereby.com/browser-sdk/embed")
.then(() => {
setWherebyLoaded(true);
})
.catch(console.error.bind(console));
}
}, []);
return wherebyLoaded;
};
export default function Room(details: RoomDetails) {
const wherebyLoaded = useWhereby();
const wherebyRef = useRef<HTMLElement>(null);
const roomName = details.params.roomName;
const meeting = useRoomMeeting(roomName);
const router = useRouter();
const { isLoading, isAuthenticated } = useSessionStatus();
const roomUrl = meeting?.response?.host_room_url
? meeting?.response?.host_room_url
: meeting?.response?.room_url;
const meetingId = meeting?.response?.id;
const recordingType = meeting?.response?.recording_type;
const handleLeave = useCallback(() => {
router.push("/browse");
}, [router]);
useEffect(() => {
if (
!isLoading &&
meeting?.error &&
"status" in meeting.error &&
meeting.error.status === 404
) {
notFound();
}
}, [isLoading, meeting?.error]);
useEffect(() => {
if (isLoading || !isAuthenticated || !roomUrl || !wherebyLoaded) return;
wherebyRef.current?.addEventListener("leave", handleLeave);
return () => {
wherebyRef.current?.removeEventListener("leave", handleLeave);
};
}, [handleLeave, roomUrl, isLoading, isAuthenticated, wherebyLoaded]);
if (isLoading) {
return (
<Box
display="flex"
justifyContent="center"
alignItems="center"
height="100vh"
bg="gray.50"
p={4}
>
<Spinner color="blue.500" size="xl" />
</Box>
);
}
return (
<>
{roomUrl && meetingId && wherebyLoaded && (
<>
<whereby-embed
ref={wherebyRef}
room={roomUrl}
style={{ width: "100vw", height: "100vh" }}
/>
{recordingType && recordingTypeRequiresConsent(recordingType) && (
<ConsentDialogButton
meetingId={meetingId}
wherebyRef={wherebyRef}
/>
)}
</>
)}
</>
);
}

View File

@@ -1,281 +1,37 @@
"use client";
import {
useCallback,
useEffect,
useRef,
useState,
useContext,
RefObject,
} from "react";
import {
Box,
Button,
Text,
VStack,
HStack,
Spinner,
Icon,
} from "@chakra-ui/react";
import { toaster } from "../components/ui/toaster";
import { useCallback, useEffect, useState, use } from "react";
import { Box, Spinner } from "@chakra-ui/react";
import useRoomMeeting from "./useRoomMeeting";
import { useRouter } from "next/navigation";
import { notFound } from "next/navigation";
import { useRecordingConsent } from "../recordingConsentContext";
import { useMeetingAudioConsent } from "../lib/apiHooks";
import type { components } from "../reflector-api";
type Meeting = components["schemas"]["Meeting"];
import { FaBars } from "react-icons/fa6";
import { useAuth } from "../lib/AuthProvider";
import VideoPlatformEmbed from "../lib/videoPlatforms/VideoPlatformEmbed";
export type RoomDetails = {
params: {
params: Promise<{
roomName: string;
};
};
// stages: we focus on the consent, then whereby steals focus, then we focus on the consent again, then return focus to whoever stole it initially
const useConsentWherebyFocusManagement = (
acceptButtonRef: RefObject<HTMLButtonElement>,
wherebyRef: RefObject<HTMLElement>,
) => {
const currentFocusRef = useRef<HTMLElement | null>(null);
useEffect(() => {
if (acceptButtonRef.current) {
acceptButtonRef.current.focus();
} else {
console.error(
"accept button ref not available yet for focus management - seems to be illegal state",
);
}
const handleWherebyReady = () => {
console.log("whereby ready - refocusing consent button");
currentFocusRef.current = document.activeElement as HTMLElement;
if (acceptButtonRef.current) {
acceptButtonRef.current.focus();
}
};
if (wherebyRef.current) {
wherebyRef.current.addEventListener("ready", handleWherebyReady);
} else {
console.warn(
"whereby ref not available yet for focus management - seems to be illegal state. not waiting, focus management off.",
);
}
return () => {
wherebyRef.current?.removeEventListener("ready", handleWherebyReady);
currentFocusRef.current?.focus();
};
}, []);
};
const useConsentDialog = (
meetingId: string,
wherebyRef: RefObject<HTMLElement> /*accessibility*/,
) => {
const { state: consentState, touch, hasConsent } = useRecordingConsent();
// toast would open duplicates, even with using "id=" prop
const [modalOpen, setModalOpen] = useState(false);
const audioConsentMutation = useMeetingAudioConsent();
const handleConsent = useCallback(
async (meetingId: string, given: boolean) => {
try {
await audioConsentMutation.mutateAsync({
params: {
path: {
meeting_id: meetingId,
},
},
body: {
consent_given: given,
},
});
touch(meetingId);
} catch (error) {
console.error("Error submitting consent:", error);
}
},
[audioConsentMutation, touch],
);
const showConsentModal = useCallback(() => {
if (modalOpen) return;
setModalOpen(true);
const toastId = toaster.create({
placement: "top",
duration: null,
render: ({ dismiss }) => {
const AcceptButton = () => {
const buttonRef = useRef<HTMLButtonElement>(null);
useConsentWherebyFocusManagement(buttonRef, wherebyRef);
return (
<Button
ref={buttonRef}
colorPalette="primary"
size="sm"
onClick={() => {
handleConsent(meetingId, true).then(() => {
/*signifies it's ok to now wait here.*/
});
dismiss();
}}
>
Yes, store the audio
</Button>
);
};
return (
<Box
p={6}
bg="rgba(255, 255, 255, 0.7)"
borderRadius="lg"
boxShadow="lg"
maxW="md"
mx="auto"
>
<VStack gap={4} alignItems="center">
<Text fontSize="md" textAlign="center" fontWeight="medium">
Can we have your permission to store this meeting's audio
recording on our servers?
</Text>
<HStack gap={4} justifyContent="center">
<Button
variant="ghost"
size="sm"
onClick={() => {
handleConsent(meetingId, false).then(() => {
/*signifies it's ok to now wait here.*/
});
dismiss();
}}
>
No, delete after transcription
</Button>
<AcceptButton />
</HStack>
</VStack>
</Box>
);
},
});
// Set modal state when toast is dismissed
toastId.then((id) => {
const checkToastStatus = setInterval(() => {
if (!toaster.isActive(id)) {
setModalOpen(false);
clearInterval(checkToastStatus);
}
}, 100);
});
// Handle escape key to close the toast
const handleKeyDown = (event: KeyboardEvent) => {
if (event.key === "Escape") {
toastId.then((id) => toaster.dismiss(id));
}
};
document.addEventListener("keydown", handleKeyDown);
const cleanup = () => {
toastId.then((id) => toaster.dismiss(id));
document.removeEventListener("keydown", handleKeyDown);
};
return cleanup;
}, [meetingId, handleConsent, wherebyRef, modalOpen]);
return {
showConsentModal,
consentState,
hasConsent,
consentLoading: audioConsentMutation.isPending,
};
};
function ConsentDialogButton({
meetingId,
wherebyRef,
}: {
meetingId: string;
wherebyRef: React.RefObject<HTMLElement>;
}) {
const { showConsentModal, consentState, hasConsent, consentLoading } =
useConsentDialog(meetingId, wherebyRef);
if (!consentState.ready || hasConsent(meetingId) || consentLoading) {
return null;
}
return (
<Button
position="absolute"
top="56px"
left="8px"
zIndex={1000}
colorPalette="blue"
size="sm"
onClick={showConsentModal}
>
Meeting is being recorded
<Icon as={FaBars} ml={2} />
</Button>
);
}
const recordingTypeRequiresConsent = (
recordingType: NonNullable<Meeting["recording_type"]>,
) => {
return recordingType === "cloud";
};
// next throws even with "use client"
const useWhereby = () => {
const [wherebyLoaded, setWherebyLoaded] = useState(false);
useEffect(() => {
if (typeof window !== "undefined") {
import("@whereby.com/browser-sdk/embed")
.then(() => {
setWherebyLoaded(true);
})
.catch(console.error.bind(console));
}
}, []);
return wherebyLoaded;
}>;
};
export default function Room(details: RoomDetails) {
const wherebyLoaded = useWhereby();
const wherebyRef = useRef<HTMLElement>(null);
const roomName = details.params.roomName;
const [platformReady, setPlatformReady] = useState(false);
const params = use(details.params);
const roomName = params.roomName;
const meeting = useRoomMeeting(roomName);
const router = useRouter();
const status = useAuth().status;
const isAuthenticated = status === "authenticated";
const isLoading = status === "loading" || meeting.loading;
const roomUrl = meeting?.response?.host_room_url
? meeting?.response?.host_room_url
: meeting?.response?.room_url;
const meetingId = meeting?.response?.id;
const recordingType = meeting?.response?.recording_type;
const handleLeave = useCallback(() => {
router.push("/browse");
}, [router]);
const handlePlatformReady = useCallback(() => {
setPlatformReady(true);
}, []);
useEffect(() => {
if (
!isLoading &&
@@ -287,16 +43,6 @@ export default function Room(details: RoomDetails) {
}
}, [isLoading, meeting?.error]);
useEffect(() => {
if (isLoading || !isAuthenticated || !roomUrl || !wherebyLoaded) return;
wherebyRef.current?.addEventListener("leave", handleLeave);
return () => {
wherebyRef.current?.removeEventListener("leave", handleLeave);
};
}, [handleLeave, roomUrl, isLoading, isAuthenticated, wherebyLoaded]);
if (isLoading) {
return (
<Box
@@ -312,23 +58,15 @@ export default function Room(details: RoomDetails) {
);
}
if (!meeting?.response || !isAuthenticated) {
return null;
}
return (
<>
{roomUrl && meetingId && wherebyLoaded && (
<>
<whereby-embed
ref={wherebyRef}
room={roomUrl}
style={{ width: "100vw", height: "100vh" }}
/>
{recordingType && recordingTypeRequiresConsent(recordingType) && (
<ConsentDialogButton
meetingId={meetingId}
wherebyRef={wherebyRef}
/>
)}
</>
)}
</>
<VideoPlatformEmbed
meeting={meeting.response}
onLeave={handleLeave}
onReady={handlePlatformReady}
/>
);
}

View File

@@ -1,6 +1,6 @@
import NextAuth from "next-auth";
import { authOptions } from "../../../lib/authBackend";
const handler = NextAuth(authOptions);
const handler = NextAuth(authOptions());
export { handler as GET, handler as POST };

View File

@@ -1,49 +0,0 @@
"use client";
import { createContext, useContext, useEffect, useState } from "react";
import { DomainConfig } from "./lib/edgeConfig";
type DomainContextType = Omit<DomainConfig, "auth_callback_url">;
export const DomainContext = createContext<DomainContextType>({
features: {
requireLogin: false,
privacy: true,
browse: false,
sendToZulip: false,
},
api_url: "",
websocket_url: "",
});
export const DomainContextProvider = ({
config,
children,
}: {
config: DomainConfig;
children: any;
}) => {
const [context, setContext] = useState<DomainContextType>();
useEffect(() => {
if (!config) return;
const { auth_callback_url, ...others } = config;
setContext(others);
}, [config]);
if (!context) return;
return (
<DomainContext.Provider value={context}>{children}</DomainContext.Provider>
);
};
// Get feature config client-side with
export const featureEnabled = (
featureName: "requireLogin" | "privacy" | "browse" | "sendToZulip",
) => {
const context = useContext(DomainContext);
return context.features[featureName] as boolean | undefined;
};
// Get config server-side (out of react) : see lib/edgeConfig.

View File

@@ -3,11 +3,10 @@ import { Metadata, Viewport } from "next";
import { Poppins } from "next/font/google";
import { ErrorProvider } from "./(errors)/errorContext";
import ErrorMessage from "./(errors)/errorMessage";
import { DomainContextProvider } from "./domainContext";
import { RecordingConsentProvider } from "./recordingConsentContext";
import { getConfig } from "./lib/edgeConfig";
import { ErrorBoundary } from "@sentry/nextjs";
import { Providers } from "./providers";
import { assertExistsAndNonEmptyString } from "./lib/utils";
const poppins = Poppins({
subsets: ["latin"],
@@ -22,8 +21,13 @@ export const viewport: Viewport = {
maximumScale: 1,
};
const NEXT_PUBLIC_SITE_URL = assertExistsAndNonEmptyString(
process.env.NEXT_PUBLIC_SITE_URL,
"NEXT_PUBLIC_SITE_URL required",
);
export const metadata: Metadata = {
metadataBase: new URL(process.env.NEXT_PUBLIC_SITE_URL!),
metadataBase: new URL(NEXT_PUBLIC_SITE_URL),
title: {
template: "%s Reflector",
default: "Reflector - AI-Powered Meeting Transcriptions by Monadical",
@@ -68,21 +72,17 @@ export default async function RootLayout({
}: {
children: React.ReactNode;
}) {
const config = await getConfig();
return (
<html lang="en" className={poppins.className} suppressHydrationWarning>
<body className={"h-[100svh] w-[100svw] overflow-x-hidden relative"}>
<DomainContextProvider config={config}>
<RecordingConsentProvider>
<ErrorBoundary fallback={<p>"something went really wrong"</p>}>
<ErrorProvider>
<ErrorMessage />
<Providers>{children}</Providers>
</ErrorProvider>
</ErrorBoundary>
</RecordingConsentProvider>
</DomainContextProvider>
<RecordingConsentProvider>
<ErrorBoundary fallback={<p>"something went really wrong"</p>}>
<ErrorProvider>
<ErrorMessage />
<Providers>{children}</Providers>
</ErrorProvider>
</ErrorBoundary>
</RecordingConsentProvider>
</body>
</html>
);

View File

@@ -1,17 +1,19 @@
"use client";
import { createContext, useContext, useEffect } from "react";
import { createContext, useContext } from "react";
import { useSession as useNextAuthSession } from "next-auth/react";
import { signOut, signIn } from "next-auth/react";
import { configureApiAuth, configureApiAuthRefresh } from "./apiClient";
import { configureApiAuth } from "./apiClient";
import { assertCustomSession, CustomSession } from "./types";
import { Session } from "next-auth";
import { SessionAutoRefresh } from "./SessionAutoRefresh";
import { REFRESH_ACCESS_TOKEN_ERROR } from "./auth";
import { assertExists } from "./utils";
import { featureEnabled } from "./features";
type AuthContextType = (
| { status: "loading" }
| { status: "refreshing" }
| { status: "refreshing"; user: CustomSession["user"] }
| { status: "unauthenticated"; error?: string }
| {
status: "authenticated";
@@ -26,74 +28,94 @@ type AuthContextType = (
};
const AuthContext = createContext<AuthContextType | undefined>(undefined);
const isAuthEnabled = featureEnabled("requireLogin");
const noopAuthContext: AuthContextType = {
status: "unauthenticated",
update: async () => {
return null;
},
signIn: async () => {
throw new Error("signIn not supposed to be called");
},
signOut: async () => {
throw new Error("signOut not supposed to be called");
},
};
export function AuthProvider({ children }: { children: React.ReactNode }) {
const { data: session, status, update } = useNextAuthSession();
const customSession = session ? assertCustomSession(session) : null;
const contextValue: AuthContextType = {
...(() => {
switch (status) {
case "loading": {
const sessionIsHere = !!customSession;
switch (sessionIsHere) {
case false: {
return { status };
const contextValue: AuthContextType = isAuthEnabled
? {
...(() => {
switch (status) {
case "loading": {
const sessionIsHere = !!session;
// actually exists sometimes; nextAuth types are something else
switch (sessionIsHere as boolean) {
case false: {
return { status };
}
case true: {
return {
status: "refreshing" as const,
user: assertCustomSession(
assertExists(session as unknown as Session),
).user,
};
}
default: {
throw new Error("unreachable");
}
}
}
case true: {
return { status: "refreshing" as const };
case "authenticated": {
const customSession = assertCustomSession(session);
if (customSession?.error === REFRESH_ACCESS_TOKEN_ERROR) {
// token had expired but next auth still returns "authenticated" so show user unauthenticated state
return {
status: "unauthenticated" as const,
};
} else if (customSession?.accessToken) {
return {
status,
accessToken: customSession.accessToken,
accessTokenExpires: customSession.accessTokenExpires,
user: customSession.user,
};
} else {
console.warn(
"illegal state: authenticated but have no session/or access token. ignoring",
);
return { status: "unauthenticated" as const };
}
}
case "unauthenticated": {
return { status: "unauthenticated" as const };
}
default: {
const _: never = sessionIsHere;
const _: never = status;
throw new Error("unreachable");
}
}
}
case "authenticated": {
if (customSession?.error === REFRESH_ACCESS_TOKEN_ERROR) {
// token had expired but next auth still returns "authenticated" so show user unauthenticated state
return {
status: "unauthenticated" as const,
};
} else if (customSession?.accessToken) {
return {
status,
accessToken: customSession.accessToken,
accessTokenExpires: customSession.accessTokenExpires,
user: customSession.user,
};
} else {
console.warn(
"illegal state: authenticated but have no session/or access token. ignoring",
);
return { status: "unauthenticated" as const };
}
}
case "unauthenticated": {
return { status: "unauthenticated" as const };
}
default: {
const _: never = status;
throw new Error("unreachable");
}
})(),
update,
signIn,
signOut,
}
})(),
update,
signIn,
signOut,
};
: noopAuthContext;
// not useEffect, we need it ASAP
// apparently, still no guarantee this code runs before mutations are fired
configureApiAuth(
contextValue.status === "authenticated" ? contextValue.accessToken : null,
contextValue.status === "authenticated"
? contextValue.accessToken
: contextValue.status === "loading"
? undefined
: null,
);
useEffect(() => {
configureApiAuthRefresh(
contextValue.status === "authenticated" ? contextValue.update : null,
);
}, [contextValue.status === "authenticated" && contextValue.update]);
return (
<AuthContext.Provider value={contextValue}>
<SessionAutoRefresh>{children}</SessionAutoRefresh>

View File

@@ -9,12 +9,11 @@
import { useEffect } from "react";
import { useAuth } from "./AuthProvider";
import { REFRESH_ACCESS_TOKEN_BEFORE } from "./auth";
const REFRESH_BEFORE = REFRESH_ACCESS_TOKEN_BEFORE;
import { shouldRefreshToken } from "./auth";
export function SessionAutoRefresh({ children }) {
const auth = useAuth();
const accessTokenExpires =
auth.status === "authenticated" ? auth.accessTokenExpires : null;
@@ -23,18 +22,15 @@ export function SessionAutoRefresh({ children }) {
// and not too slow (debuggable)
const INTERVAL_REFRESH_MS = 5000;
const interval = setInterval(() => {
if (accessTokenExpires !== null) {
const timeLeft = accessTokenExpires - Date.now();
console.log("time left", timeLeft);
// if (timeLeft < REFRESH_BEFORE) {
// auth
// .update()
// .then(() => {})
// .catch((e) => {
// // note: 401 won't be considered error here
// console.error("error refreshing auth token", e);
// });
// }
if (accessTokenExpires === null) return;
if (shouldRefreshToken(accessTokenExpires)) {
auth
.update()
.then(() => {})
.catch((e) => {
// note: 401 won't be considered error here
console.error("error refreshing auth token", e);
});
}
}, INTERVAL_REFRESH_MS);

View File

@@ -2,46 +2,51 @@
import createClient from "openapi-fetch";
import type { paths } from "../reflector-api";
import {
queryOptions,
useMutation,
useQuery,
useSuspenseQuery,
} from "@tanstack/react-query";
import createFetchClient from "openapi-react-query";
import { assertExistsAndNonEmptyString } from "./utils";
import { assertExistsAndNonEmptyString, parseNonEmptyString } from "./utils";
import { isBuildPhase } from "./next";
import { Session } from "next-auth";
import { assertCustomSession } from "./types";
import { HttpMethod, PathsWithMethod } from "openapi-typescript-helpers";
import { getSession } from "next-auth/react";
import { assertExtendedToken } from "./types";
const API_URL = !isBuildPhase
? assertExistsAndNonEmptyString(process.env.NEXT_PUBLIC_API_URL)
export const API_URL = !isBuildPhase
? assertExistsAndNonEmptyString(
process.env.NEXT_PUBLIC_API_URL,
"NEXT_PUBLIC_API_URL required",
)
: "http://localhost";
// Create the base openapi-fetch client with a default URL
// The actual URL will be set via middleware in AuthProvider
// TODO decide strict validation or not
export const WEBSOCKET_URL =
process.env.NEXT_PUBLIC_WEBSOCKET_URL || "ws://127.0.0.1:1250";
export const client = createClient<paths>({
baseUrl: API_URL,
});
export const $api = createFetchClient<paths>(client);
// will assert presence/absence of login initially
const initialSessionPromise = getSession();
let currentAuthToken: string | null | undefined = null;
let refreshAuthCallback: (() => Promise<Session | null>) | null = null;
const injectAuth = (request: Request, accessToken: string | null) => {
if (accessToken) {
request.headers.set("Authorization", `Bearer ${currentAuthToken}`);
} else {
request.headers.delete("Authorization");
const waitForAuthTokenDefinitivePresenceOrAbsence = async () => {
const initialSession = await initialSessionPromise;
if (currentAuthToken === undefined) {
currentAuthToken =
initialSession === null
? null
: assertExtendedToken(initialSession).accessToken;
}
return request;
// otherwise already overwritten by external forces
return currentAuthToken;
};
client.use({
onRequest({ request }) {
request = injectAuth(request, currentAuthToken || null);
async onRequest({ request }) {
const token = await waitForAuthTokenDefinitivePresenceOrAbsence();
if (token !== null) {
request.headers.set(
"Authorization",
`Bearer ${parseNonEmptyString(token)}`,
);
}
// XXX Only set Content-Type if not already set (FormData will set its own boundary)
// This is a work around for uploading file, we're passing a formdata
// but the content type was still application/json
@@ -55,46 +60,13 @@ client.use({
},
});
client.use({
async onResponse({ response, request, params, schemaPath }) {
if (response.status === 401) {
console.log(
"response.status is 401!",
refreshAuthCallback,
request,
schemaPath,
);
}
if (response.status === 401 && refreshAuthCallback) {
try {
const session = await refreshAuthCallback();
if (!session) {
console.warn("Token refresh failed, no session returned");
return response;
}
const customSession = assertCustomSession(session);
currentAuthToken = customSession.accessToken;
const r = await client.request(
request.method as HttpMethod,
schemaPath as PathsWithMethod<paths, HttpMethod>,
...params,
);
return r.response;
} catch (error) {
console.error("Token refresh failed during 401 retry:", error);
}
}
return response;
},
});
export const $api = createFetchClient<paths>(client);
let currentAuthToken: string | null | undefined = undefined;
// the function contract: lightweight, idempotent
export const configureApiAuth = (token: string | null | undefined) => {
// watch only for the initial loading; "reloading" state assumes token presence/absence
if (token === undefined && currentAuthToken !== undefined) return;
currentAuthToken = token;
};
export const configureApiAuthRefresh = (
callback: (() => Promise<Session | null>) | null,
) => {
refreshAuthCallback = callback;
};

View File

@@ -96,8 +96,6 @@ export function useTranscriptProcess() {
}
export function useTranscriptGet(transcriptId: string | null) {
const { isAuthenticated } = useAuthReady();
return $api.useQuery(
"get",
"/v1/transcripts/{transcript_id}",
@@ -109,7 +107,7 @@ export function useTranscriptGet(transcriptId: string | null) {
},
},
{
enabled: !!transcriptId && isAuthenticated,
enabled: !!transcriptId,
},
);
}
@@ -292,18 +290,16 @@ export function useTranscriptUploadAudio() {
}
export function useTranscriptWaveform(transcriptId: string | null) {
const { isAuthenticated } = useAuthReady();
return $api.useQuery(
"get",
"/v1/transcripts/{transcript_id}/audio/waveform",
{
params: {
path: { transcript_id: transcriptId || "" },
path: { transcript_id: transcriptId! },
},
},
{
enabled: !!transcriptId && isAuthenticated,
enabled: !!transcriptId,
},
);
}
@@ -316,7 +312,7 @@ export function useTranscriptMP3(transcriptId: string | null) {
"/v1/transcripts/{transcript_id}/audio/mp3",
{
params: {
path: { transcript_id: transcriptId || "" },
path: { transcript_id: transcriptId! },
},
},
{
@@ -326,8 +322,6 @@ export function useTranscriptMP3(transcriptId: string | null) {
}
export function useTranscriptTopics(transcriptId: string | null) {
const { isAuthenticated } = useAuthReady();
return $api.useQuery(
"get",
"/v1/transcripts/{transcript_id}/topics",
@@ -337,7 +331,7 @@ export function useTranscriptTopics(transcriptId: string | null) {
},
},
{
enabled: !!transcriptId && isAuthenticated,
enabled: !!transcriptId,
},
);
}

12
www/app/lib/array.ts Normal file
View File

@@ -0,0 +1,12 @@
export type NonEmptyArray<T> = [T, ...T[]];
export const isNonEmptyArray = <T>(arr: T[]): arr is NonEmptyArray<T> =>
arr.length > 0;
export const assertNonEmptyArray = <T>(
arr: T[],
err?: string,
): NonEmptyArray<T> => {
if (isNonEmptyArray(arr)) {
return arr;
}
throw new Error(err ?? "Expected non-empty array");
};

View File

@@ -1,3 +1,20 @@
import { assertExistsAndNonEmptyString } from "./utils";
export const REFRESH_ACCESS_TOKEN_ERROR = "RefreshAccessTokenError" as const;
// 4 min is 1 min less than default authentic value. here we assume that authentic won't be set to access tokens < 4 min
export const REFRESH_ACCESS_TOKEN_BEFORE = 4 * 60 * 1000;
export const shouldRefreshToken = (accessTokenExpires: number): boolean => {
const timeLeft = accessTokenExpires - Date.now();
return timeLeft < REFRESH_ACCESS_TOKEN_BEFORE;
};
export const LOGIN_REQUIRED_PAGES = [
"/transcripts/[!new]",
"/browse(.*)",
"/rooms(.*)",
];
export const PROTECTED_PAGES = new RegExp(
LOGIN_REQUIRED_PAGES.map((page) => `^${page}$`).join("|"),
);

View File

@@ -2,123 +2,163 @@ import { AuthOptions } from "next-auth";
import AuthentikProvider from "next-auth/providers/authentik";
import type { JWT } from "next-auth/jwt";
import { JWTWithAccessToken, CustomSession } from "./types";
import { assertExists, assertExistsAndNonEmptyString } from "./utils";
import {
assertExists,
assertExistsAndNonEmptyString,
assertNotExists,
} from "./utils";
import {
REFRESH_ACCESS_TOKEN_BEFORE,
REFRESH_ACCESS_TOKEN_ERROR,
shouldRefreshToken,
} from "./auth";
import {
getTokenCache,
setTokenCache,
deleteTokenCache,
} from "./redisTokenCache";
import { tokenCacheRedis } from "./redisClient";
import { tokenCacheRedis, redlock } from "./redisClient";
import { isBuildPhase } from "./next";
import { sequenceThrows } from "./errorUtils";
import { featureEnabled } from "./features";
// REFRESH_ACCESS_TOKEN_BEFORE because refresh is based on access token expiration (imagine we cache it 30 days)
const TOKEN_CACHE_TTL = REFRESH_ACCESS_TOKEN_BEFORE;
const getAuthentikClientId = () =>
assertExistsAndNonEmptyString(
process.env.AUTHENTIK_CLIENT_ID,
"AUTHENTIK_CLIENT_ID required",
);
const getAuthentikClientSecret = () =>
assertExistsAndNonEmptyString(
process.env.AUTHENTIK_CLIENT_SECRET,
"AUTHENTIK_CLIENT_SECRET required",
);
const getAuthentikRefreshTokenUrl = () =>
assertExistsAndNonEmptyString(
process.env.AUTHENTIK_REFRESH_TOKEN_URL,
"AUTHENTIK_REFRESH_TOKEN_URL required",
);
const refreshLocks = new Map<string, Promise<JWTWithAccessToken>>();
const CLIENT_ID = !isBuildPhase
? assertExistsAndNonEmptyString(process.env.AUTHENTIK_CLIENT_ID)
: "noop";
const CLIENT_SECRET = !isBuildPhase
? assertExistsAndNonEmptyString(process.env.AUTHENTIK_CLIENT_SECRET)
: "noop";
export const authOptions: AuthOptions = {
providers: [
AuthentikProvider({
clientId: CLIENT_ID,
clientSecret: CLIENT_SECRET,
issuer: process.env.AUTHENTIK_ISSUER,
authorization: {
params: {
scope: "openid email profile offline_access",
export const authOptions = (): AuthOptions =>
featureEnabled("requireLogin")
? {
providers: [
AuthentikProvider({
...(() => {
const [clientId, clientSecret] = sequenceThrows(
getAuthentikClientId,
getAuthentikClientSecret,
);
return {
clientId,
clientSecret,
};
})(),
issuer: process.env.AUTHENTIK_ISSUER,
authorization: {
params: {
scope: "openid email profile offline_access",
},
},
}),
],
session: {
strategy: "jwt",
},
},
}),
],
session: {
strategy: "jwt",
},
callbacks: {
async jwt({ token, account, user }) {
console.log("token.sub jwt callback", token.sub);
const KEY = `token:${token.sub}`;
callbacks: {
async jwt({ token, account, user }) {
if (account && !account.access_token) {
await deleteTokenCache(tokenCacheRedis, `token:${token.sub}`);
}
if (account && user) {
// called only on first login
// XXX account.expires_in used in example is not defined for authentik backend, but expires_at is
const expiresAtS = assertExists(account.expires_at);
const expiresAtMs = expiresAtS * 1000;
if (!account.access_token) {
await deleteTokenCache(tokenCacheRedis, KEY);
} else {
const jwtToken: JWTWithAccessToken = {
...token,
accessToken: account.access_token,
accessTokenExpires: expiresAtMs,
refreshToken: account.refresh_token,
};
await setTokenCache(tokenCacheRedis, KEY, {
token: jwtToken,
timestamp: Date.now(),
});
return jwtToken;
}
}
if (account && user) {
// called only on first login
// XXX account.expires_in used in example is not defined for authentik backend, but expires_at is
if (account.access_token) {
const expiresAtS = assertExists(account.expires_at);
const expiresAtMs = expiresAtS * 1000;
const jwtToken: JWTWithAccessToken = {
...token,
accessToken: account.access_token,
accessTokenExpires: expiresAtMs,
refreshToken: account.refresh_token,
};
if (jwtToken.error) {
await deleteTokenCache(tokenCacheRedis, `token:${token.sub}`);
} else {
assertNotExists(
jwtToken.error,
`panic! trying to cache token with error in jwt: ${jwtToken.error}`,
);
await setTokenCache(tokenCacheRedis, `token:${token.sub}`, {
token: jwtToken,
timestamp: Date.now(),
});
return jwtToken;
}
}
}
const currentToken = await getTokenCache(tokenCacheRedis, KEY);
console.log(
"currentToken.token.accessTokenExpires",
currentToken?.token?.accessTokenExpires,
currentToken?.token?.accessTokenExpires
? Date.now() < currentToken?.token?.accessTokenExpires
: "?",
);
if (currentToken && Date.now() < currentToken.token.accessTokenExpires) {
return currentToken.token;
}
const currentToken = await getTokenCache(
tokenCacheRedis,
`token:${token.sub}`,
);
console.debug(
"currentToken from cache",
JSON.stringify(currentToken, null, 2),
"will be returned?",
currentToken &&
!shouldRefreshToken(currentToken.token.accessTokenExpires),
);
if (
currentToken &&
!shouldRefreshToken(currentToken.token.accessTokenExpires)
) {
return currentToken.token;
}
// access token has expired, try to update it
return await lockedRefreshAccessToken(token);
},
async session({ session, token }) {
const extendedToken = token as JWTWithAccessToken;
return {
...session,
accessToken: extendedToken.accessToken,
accessTokenExpires: extendedToken.accessTokenExpires,
error: extendedToken.error,
user: {
id: assertExists(extendedToken.sub),
name: extendedToken.name,
email: extendedToken.email,
// access token has expired, try to update it
return await lockedRefreshAccessToken(token);
},
async session({ session, token }) {
const extendedToken = token as JWTWithAccessToken;
return {
...session,
accessToken: extendedToken.accessToken,
accessTokenExpires: extendedToken.accessTokenExpires,
error: extendedToken.error,
user: {
id: assertExists(extendedToken.sub),
name: extendedToken.name,
email: extendedToken.email,
},
} satisfies CustomSession;
},
},
} satisfies CustomSession;
},
},
};
}
: {
providers: [],
};
async function lockedRefreshAccessToken(
token: JWT,
): Promise<JWTWithAccessToken> {
const lockKey = `${token.sub}-refresh`;
const lockKey = `${token.sub}-lock`;
const existingRefresh = refreshLocks.get(lockKey);
if (existingRefresh) {
return await existingRefresh;
}
const refreshPromise = (async () => {
try {
return redlock
.using([lockKey], 10000, async () => {
const cached = await getTokenCache(tokenCacheRedis, `token:${token.sub}`);
if (cached)
console.debug(
"received cached token. to delete?",
Date.now() - cached.timestamp > TOKEN_CACHE_TTL,
);
else console.debug("no cached token received");
if (cached) {
if (Date.now() - cached.timestamp > TOKEN_CACHE_TTL) {
await deleteTokenCache(tokenCacheRedis, `token:${token.sub}`);
} else if (Date.now() < cached.token.accessTokenExpires) {
} else if (!shouldRefreshToken(cached.token.accessTokenExpires)) {
console.debug("returning cached token", cached.token);
return cached.token;
}
}
@@ -126,32 +166,51 @@ async function lockedRefreshAccessToken(
const currentToken = cached?.token || (token as JWTWithAccessToken);
const newToken = await refreshAccessToken(currentToken);
console.debug("current token during refresh", currentToken);
console.debug("new token during refresh", newToken);
if (newToken.error) {
await deleteTokenCache(tokenCacheRedis, `token:${token.sub}`);
return newToken;
}
assertNotExists(
newToken.error,
`panic! trying to cache token with error during refresh: ${newToken.error}`,
);
await setTokenCache(tokenCacheRedis, `token:${token.sub}`, {
token: newToken,
timestamp: Date.now(),
});
return newToken;
} finally {
setTimeout(() => refreshLocks.delete(lockKey), 100);
}
})();
refreshLocks.set(lockKey, refreshPromise);
return refreshPromise;
})
.catch((e) => {
console.error("error refreshing token", e);
deleteTokenCache(tokenCacheRedis, `token:${token.sub}`).catch((e) => {
console.error("error deleting errored token", e);
});
return {
...token,
error: REFRESH_ACCESS_TOKEN_ERROR,
} as JWTWithAccessToken;
});
}
async function refreshAccessToken(token: JWT): Promise<JWTWithAccessToken> {
const [url, clientId, clientSecret] = sequenceThrows(
getAuthentikRefreshTokenUrl,
getAuthentikClientId,
getAuthentikClientSecret,
);
try {
const url = `${process.env.AUTHENTIK_REFRESH_TOKEN_URL}`;
const options = {
headers: {
"Content-Type": "application/x-www-form-urlencoded",
},
body: new URLSearchParams({
client_id: process.env.AUTHENTIK_CLIENT_ID as string,
client_secret: process.env.AUTHENTIK_CLIENT_SECRET as string,
client_id: clientId,
client_secret: clientSecret,
grant_type: "refresh_token",
refresh_token: token.refreshToken as string,
}).toString(),

View File

@@ -1,54 +0,0 @@
import { get } from "@vercel/edge-config";
import { isBuildPhase } from "./next";
type EdgeConfig = {
[domainWithDash: string]: {
features: {
[featureName in
| "requireLogin"
| "privacy"
| "browse"
| "sendToZulip"]: boolean;
};
auth_callback_url: string;
websocket_url: string;
api_url: string;
};
};
export type DomainConfig = EdgeConfig["domainWithDash"];
// Edge config main keys can only be alphanumeric and _ or -
export function edgeKeyToDomain(key: string) {
return key.replaceAll("_", ".");
}
export function edgeDomainToKey(domain: string) {
return domain.replaceAll(".", "_");
}
// get edge config server-side (prefer DomainContext when available), domain is the hostname
export async function getConfig() {
if (process.env.NEXT_PUBLIC_ENV === "development") {
try {
return require("../../config").localConfig;
} catch (e) {
// next build() WILL try to execute the require above even if conditionally protected
// but thank god it at least runs catch{} block properly
if (!isBuildPhase) throw new Error(e);
return require("../../config-template").localConfig;
}
}
const domain = new URL(process.env.NEXT_PUBLIC_SITE_URL!).hostname;
let config = await get(edgeDomainToKey(domain));
if (typeof config !== "object") {
console.warn("No config for this domain, falling back to default");
config = await get(edgeDomainToKey("default"));
}
if (typeof config !== "object") throw Error("Error fetching config");
return config as DomainConfig;
}

View File

@@ -1,4 +1,6 @@
function shouldShowError(error: Error | null | undefined) {
import { isNonEmptyArray, NonEmptyArray } from "./array";
export function shouldShowError(error: Error | null | undefined) {
if (
error?.name == "ResponseError" &&
(error["response"].status == 404 || error["response"].status == 403)
@@ -8,4 +10,40 @@ function shouldShowError(error: Error | null | undefined) {
return true;
}
export { shouldShowError };
const defaultMergeErrors = (ex: NonEmptyArray<unknown>): unknown => {
try {
return new Error(
ex
.map((e) =>
e ? (e.toString ? e.toString() : JSON.stringify(e)) : `${e}`,
)
.join("\n"),
);
} catch (e) {
console.error("Error merging errors:", e);
return ex[0];
}
};
type ReturnTypes<T extends readonly (() => any)[]> = {
[K in keyof T]: T[K] extends () => infer R ? R : never;
};
// sequence semantic for "throws"
// calls functions passed and collects its thrown values
export function sequenceThrows<Fns extends readonly (() => any)[]>(
...fs: Fns
): ReturnTypes<Fns> {
const results: unknown[] = [];
const errors: unknown[] = [];
for (const f of fs) {
try {
results.push(f());
} catch (e) {
errors.push(e);
}
}
if (errors.length) throw defaultMergeErrors(errors as NonEmptyArray<unknown>);
return results as ReturnTypes<Fns>;
}

55
www/app/lib/features.ts Normal file
View File

@@ -0,0 +1,55 @@
export const FEATURES = [
"requireLogin",
"privacy",
"browse",
"sendToZulip",
"rooms",
] as const;
export type FeatureName = (typeof FEATURES)[number];
export type Features = Readonly<Record<FeatureName, boolean>>;
export const DEFAULT_FEATURES: Features = {
requireLogin: true,
privacy: true,
browse: true,
sendToZulip: true,
rooms: true,
} as const;
function parseBooleanEnv(
value: string | undefined,
defaultValue: boolean = false,
): boolean {
if (!value) return defaultValue;
return value.toLowerCase() === "true";
}
// WARNING: keep process.env.* as-is, next.js won't see them if you generate dynamically
const features: Features = {
requireLogin: parseBooleanEnv(
process.env.NEXT_PUBLIC_FEATURE_REQUIRE_LOGIN,
DEFAULT_FEATURES.requireLogin,
),
privacy: parseBooleanEnv(
process.env.NEXT_PUBLIC_FEATURE_PRIVACY,
DEFAULT_FEATURES.privacy,
),
browse: parseBooleanEnv(
process.env.NEXT_PUBLIC_FEATURE_BROWSE,
DEFAULT_FEATURES.browse,
),
sendToZulip: parseBooleanEnv(
process.env.NEXT_PUBLIC_FEATURE_SEND_TO_ZULIP,
DEFAULT_FEATURES.sendToZulip,
),
rooms: parseBooleanEnv(
process.env.NEXT_PUBLIC_FEATURE_ROOMS,
DEFAULT_FEATURES.rooms,
),
};
export const featureEnabled = (featureName: FeatureName): boolean => {
return features[featureName];
};

View File

@@ -1,30 +1,41 @@
import Redis from "ioredis";
import { isBuildPhase } from "./next";
import Redlock, { ResourceLockedError } from "redlock";
export type RedisClient = Pick<Redis, "get" | "setex" | "del">;
export type RedlockClient = {
using: <T>(
keys: string | string[],
ttl: number,
cb: () => Promise<T>,
) => Promise<T>;
};
const KV_USE_TLS = process.env.KV_USE_TLS
? process.env.KV_USE_TLS === "true"
: undefined;
let redisClient: Redis | null = null;
const getRedisClient = (): RedisClient => {
if (redisClient) return redisClient;
const redisUrl = process.env.KV_URL;
if (!redisUrl) {
throw new Error("KV_URL environment variable is required");
}
const redis = new Redis(redisUrl, {
redisClient = new Redis(redisUrl, {
maxRetriesPerRequest: 3,
lazyConnect: true,
...(KV_USE_TLS === true
? {
tls: {},
}
: {}),
});
redis.on("error", (error) => {
redisClient.on("error", (error) => {
console.error("Redis error:", error);
});
// not necessary but will indicate redis config errors by failfast at startup
// happens only once; after that connection is allowed to die and the lib is assumed to be able to restore it eventually
redis.connect().catch((e) => {
console.error("Failed to connect to Redis:", e);
process.exit(1);
});
return redis;
return redisClient;
};
// next.js buildtime usage - we want to isolate next.js "build" time concepts here
@@ -43,4 +54,25 @@ const noopClient: RedisClient = (() => {
del: noopDel,
};
})();
const noopRedlock: RedlockClient = {
using: <T>(resource: string | string[], ttl: number, cb: () => Promise<T>) =>
cb(),
};
export const redlock: RedlockClient = isBuildPhase
? noopRedlock
: (() => {
const r = new Redlock([getRedisClient()], {});
r.on("error", (error) => {
if (error instanceof ResourceLockedError) {
return;
}
// Log all other errors.
console.error(error);
});
return r;
})();
export const tokenCacheRedis = isBuildPhase ? noopClient : getRedisClient();

View File

@@ -9,7 +9,6 @@ const TokenCacheEntrySchema = z.object({
accessToken: z.string(),
accessTokenExpires: z.number(),
refreshToken: z.string().optional(),
error: z.string().optional(),
}),
timestamp: z.number(),
});
@@ -46,14 +45,15 @@ export async function getTokenCache(
}
}
const TTL_SECONDS = 30 * 24 * 60 * 60;
export async function setTokenCache(
redis: KV,
key: string,
value: TokenCacheEntry,
): Promise<void> {
const encodedValue = TokenCacheEntryCodec.encode(value);
const ttlSeconds = Math.floor(REFRESH_ACCESS_TOKEN_BEFORE / 1000);
await redis.setex(key, ttlSeconds, encodedValue);
await redis.setex(key, TTL_SECONDS, encodedValue);
}
export async function deleteTokenCache(redis: KV, key: string): Promise<void> {

View File

@@ -0,0 +1,5 @@
import { components } from "../reflector-api";
type ApiTranscriptStatus = components["schemas"]["GetTranscript"]["status"];
export type TranscriptStatus = ApiTranscriptStatus;

View File

@@ -21,7 +21,7 @@ export interface CustomSession extends Session {
// assumption that JWT is JWTWithAccessToken - we set it in jwt callback of auth; typing isn't strong around there
// but the assumption is crucial to auth working
export const assertExtendedToken = <T>(
t: T,
t: Exclude<T, null | undefined>,
): T & {
accessTokenExpires: number;
accessToken: string;
@@ -45,7 +45,7 @@ export const assertExtendedToken = <T>(
};
export const assertExtendedTokenAndUserId = <U, T extends { user?: U }>(
t: T,
t: Exclude<T, null | undefined>,
): T & {
accessTokenExpires: number;
accessToken: string;
@@ -55,7 +55,7 @@ export const assertExtendedTokenAndUserId = <U, T extends { user?: U }>(
} => {
const extendedToken = assertExtendedToken(t);
if (typeof (extendedToken.user as any)?.id === "string") {
return t as T & {
return t as Exclude<T, null | undefined> & {
accessTokenExpires: number;
accessToken: string;
user: U & {
@@ -67,8 +67,14 @@ export const assertExtendedTokenAndUserId = <U, T extends { user?: U }>(
};
// best attempt to check the session is valid
export const assertCustomSession = <S extends Session>(s: S): CustomSession => {
export const assertCustomSession = <T extends Session>(
s: Exclude<T, null | undefined>,
): CustomSession => {
const r = assertExtendedTokenAndUserId(s);
// no other checks for now
return r as CustomSession;
};
export type Mutable<T> = {
-readonly [P in keyof T]: T[P];
};

View File

@@ -0,0 +1,26 @@
// for paths that are not supposed to be public
import { PROTECTED_PAGES } from "./auth";
import { usePathname } from "next/navigation";
import { useAuth } from "./AuthProvider";
import { useEffect } from "react";
const HOME = "/" as const;
export const useLoginRequiredPages = () => {
const pathname = usePathname();
const isProtected = PROTECTED_PAGES.test(pathname);
const auth = useAuth();
const isNotLoggedIn = auth.status === "unauthenticated";
// safety
const isLastDestination = pathname === HOME;
const shouldRedirect = isNotLoggedIn && isProtected && !isLastDestination;
useEffect(() => {
if (!shouldRedirect) return;
// on the backend, the redirect goes straight to the auth provider, but we don't have it because it's hidden inside next-auth middleware
// so we just "softly" lead the user to the main page
// warning: if HOME redirects somewhere else, we won't be protected by isLastDestination
window.location.href = HOME;
}, [shouldRedirect]);
// optionally save from blink, since window.location.href takes a bit of time
return shouldRedirect ? HOME : null;
};

Some files were not shown because too many files have changed in this diff Show More