Mathieu Virbel
88ed7cfa78
* feat(rooms): add webhook notifications for transcript completion
- Add webhook_url and webhook_secret fields to rooms table
- Create Celery task with 24-hour retry window using exponential backoff
- Send transcript metadata, diarized text, topics, and summaries via webhook
- Add HMAC signature verification for webhook security
- Add test endpoint POST /v1/rooms/{room_id}/webhook/test
- Update frontend with webhook configuration UI and test button
- Auto-generate webhook secret if not provided
- Trigger webhook after successful file pipeline processing for room recordings
* style: linting
* fix: remove unwanted files
* fix: update openapi gen
* fix: self-review
* docs: add comprehensive webhook documentation
- Document webhook configuration, events, and payloads
- Include transcript.completed and test event examples
- Add security considerations and best practices
- Provide example webhook receiver implementation
- Document retry policy and signature verification
* fix: remove audio_mp3_url from webhook payload
- Remove audio download URL generation from webhook
- Update documentation to reflect the change
- Keep only frontend_url for accessing transcripts
* docs: remove unwanted section
* fix: correct API method name and type imports for rooms
- Fix v1RoomsRetrieve to v1RoomsGet
- Update Room type to RoomDetails throughout frontend
- Fix type imports in useRoomList, RoomList, RoomTable, and RoomCards
* feat: add show/hide toggle for webhook secret field
- Add eye icon button to reveal/hide webhook secret when editing
- Show password dots when webhook secret is hidden
- Reset visibility state when opening/closing dialog
- Only show toggle button when editing existing room with secret
* fix: resolve event loop conflict in webhook test endpoint
- Extract webhook test logic into shared async function
- Call async function directly from FastAPI endpoint
- Keep Celery task wrapper for background processing
- Fixes RuntimeError: event loop already running
* refactor: remove unnecessary Celery task for webhook testing
- Webhook testing is synchronous and provides immediate feedback
- No need for background processing via Celery
- Keep only the async function called directly from API endpoint
* feat: improve webhook test error messages and display
- Show HTTP status code in error messages
- Parse JSON error responses to extract meaningful messages
- Improved UI layout for webhook test results
- Added colored background for success/error states
- Better text wrapping for long error messages
* docs: adjust doc
* fix: review
* fix: update attempts to match close 24h
* fix: add event_id
* fix: changed to uuid, to have new event_id when reprocess.
* style: linting
* fix: alembic revision
Reflector
Reflector is an AI-powered audio transcription and meeting analysis platform that provides real-time transcription, speaker diarization, translation and summarization for audio content and live meetings. It works 100% with local models (whisper/parakeet, pyannote, seamless-m4t, and your local llm like phi-4).
|
|
|
|
|
What is Reflector?
Reflector is a web application that utilizes local models to process audio content, providing:
- Real-time Transcription: Convert speech to text using Whisper (multi-language) or Parakeet (English) models
- Speaker Diarization: Identify and label different speakers using Pyannote 3.1
- Live Translation: Translate audio content in real-time to many languages with Facebook Seamless-M4T
- Topic Detection & Summarization: Extract key topics and generate concise summaries using LLMs
- Meeting Recording: Create permanent records of meetings with searchable transcripts
Currently we provide modal.com gpu template to deploy.
Background
The project architecture consists of three primary components:
- Back-End: Python server that offers an API and data persistence, found in
server/. - Front-End: NextJS React project hosted on Vercel, located in
www/. - GPU implementation: Providing services such as speech-to-text transcription, topic generation, automated summaries, and translations.
It also uses authentik for authentication if activated.
Contribution Guidelines
All new contributions should be made in a separate branch, and goes through a Pull Request. Conventional commits must be used for the PR title and commits.
Usage
To record both your voice and the meeting you're taking part in, you need:
- For an in-person meeting, make sure your microphone is in range of all participants.
- If using several microphones, make sure to merge the audio feeds into one with an external tool.
- For an online meeting, if you do not use headphones, your microphone should be able to pick up both your voice and the audio feed of the meeting.
- If you want to use headphones, you need to merge the audio feeds with an external tool.
Permissions:
You may have to add permission for browser's microphone access to record audio in
System Preferences -> Privacy & Security -> Microphone
System Preferences -> Privacy & Security -> Accessibility. You will be prompted to provide these when you try to connect.
How to Install Blackhole (Mac Only)
This is an external tool for merging the audio feeds as explained in the previous section of this document. Note: We currently do not have instructions for Windows users.
- Install Blackhole-2ch (2 ch is enough) by 1 of 2 options listed.
- Setup "Aggregate device" to route web audio and local microphone input.
- Setup Multi-Output device
- Then goto
System Preferences -> Soundand choose the devices created from the Output and Input tabs. - The input from your local microphone, the browser run meeting should be aggregated into one virtual stream to listen to and the output should be fed back to your specified output devices if everything is configured properly.
Installation
Note: we're working toward better installation, theses instructions are not accurate for now
Frontend
Start with cd www.
Installation
pnpm install
cp .env_template .env
cp config-template.ts config.ts
Then, fill in the environment variables in .env and the configuration in config.ts as needed. If you are unsure on how to proceed, ask in Zulip.
Run in development mode
pnpm dev
Then (after completing server setup and starting it) open http://localhost:3000 to view it in the browser.
OpenAPI Code Generation
To generate the TypeScript files from the openapi.json file, make sure the python server is running, then run:
pnpm openapi
Backend
Start with cd server.
Run in development mode
docker compose up -d redis
# on the first run, or if the schemas changed
uv run alembic upgrade head
# start the worker
uv run celery -A reflector.worker.app worker --loglevel=info
# start the app
uv run -m reflector.app --reload
Then fill .env with the omitted values (ask in Zulip).
Crontab (optional)
For crontab (only healthcheck for now), start the celery beat (you don't need it on your local dev environment):
uv run celery -A reflector.worker.app beat
GPU models
Currently, reflector heavily use custom local models, deployed on modal. All the micro services are available in server/gpu/
To deploy llm changes to modal, you need:
- a modal account
- set up the required secret in your modal account (REFLECTOR_GPU_APIKEY)
- install the modal cli
- connect your modal cli to your account if not done previously
modal run path/to/required/llm
Using local files
You can manually process an audio file by calling the process tool:
uv run python -m reflector.tools.process path/to/audio.wav